alp: A Python Workflow Module

[phyllisstein]

PyAl is now alp, a slightly streamlined version of the module that provides much of the same functionality while abandoning some of the strange and non-Pythonic cruft. You can read the documentation at the github page; more has been removed than added, to clarify and simplify things, but there are some new features for decoding text and arguments, as suggested by Jan (nikipore).

 

Currently Includes:

• Functions for finding your bundle ID, cache and storage paths, and query arguments.
• Functions for reading and writing JSON and plist files.
• A class to simplify generating feedback XML for Alfred.
• A class to simplify saving and retrieving settings.
• A class to interface with the OS X Keychain.
• Several bundled modules for working with HTTP requests.
• A Notification Center interface.
• An e-mail interface.

Edited March 19, 2013 by phyllisstein

[phyllisstein]

I spent the rest of the evening adding Keychain support, which the GitHub repository and the README now reflect. The method to modify a saved password may still be a bit shaky, but the rest seems quite solid. Check out the link above for more details.

[phyllisstein]

And this morning, I rejiggered things so that PyAl/Request can be deleted if you don't need to make an HTTP request and it will be transparently ignored. No reason to include the external dependencies if you don't actually depend on them, after all. It's set up so as not to change the implementation of the Request class—if you've already started using PyAl, you can update to this version without changing your code.

[phyllisstein]

The feedback interface has been totally rewritten to reflect my belated discovery of the different icon types and make creating and returning feedback a little more straightforward (I hope). More details at GitHub, in the README.

[mlgill]

I spent the rest of the evening adding Keychain support, which the GitHub repository and the README now reflect. The method to modify a saved password may still be a bit shaky, but the rest seems quite solid. Check out the link above for more details.

 

Awesome. I'll look into using this to fix my Cloud.app uploader workflow so the password isn't stored in plain text anymore.

[phyllisstein]

Awesome. I'll look into using this to fix my Cloud.app uploader workflow so the password isn't stored in plain text anymore.

 

I'd be duly flattered! I'd love to see it getting some use. All of the basic behaviors have tested fine as of the latest commit—I'm adapting my own workflows to the new version to test it—but be aware that the Feedback search stuff hasn't been tested very completely yet.

 

Edit: Oh, and the "basic behaviors" include the Keychain stuff, which is now perfectly reliable. Memory management with ctypes almost broke my brain.

[palobo]

I'd be duly flattered! I'd love to see it getting some use. All of the basic behaviors have tested fine as of the latest commit—I'm adapting my own workflows to the new version to test it—but be aware that the Feedback search stuff hasn't been tested very completely yet.

 

Edit: Oh, and the "basic behaviors" include the Keychain stuff, which is now perfectly reliable. Memory management with ctypes almost broke my brain.

I've got  workflow that I want to delve into (good way of learning python) and I'll definitely be using this! Also going to be updating my App Store Linker to take advantage of this mate!

 

Great work.

 

PS. May have to pick your brain once or twice, rather rookie at python...

[phyllisstein]

I've got  workflow that I want to delve into (good way of learning python) and I'll definitely be using this! Also going to be updating my App Store Linker to take advantage of this mate!

 

Great work.

 

PS. May have to pick your brain once or twice, rather rookie at python...

Cool! And thanks!

 

I'm always happy to answer questions, though I warn you that I myself am still mostly at the Google-the-documentation phase of working with Python. Just couldn't resist the urge to standardize.

 

I've been slowly updating my own stuff to use the latest version of the module, and things seem to be working pretty smoothly. I've corrected a couple of typos, but otherwise I've now used pretty much all of its features without hitches. And since the updated feedback system feels more right, I daresay it's just additions and tweaks from here on in.

[phyllisstein]

I've updated the Feedback Item class to support the new 'type="file"' XML in build b100. If your workflow was already using PyAl, updating it won't require any changes to your code; however, it will allow you to designate results as actionable files if the spirit moves you.

[nikipore]

I've independently whipped up my own Python API (https://github.com/nikipore/alfred-python) before I was aware of yours. For not knowing Python, you did a great job (even if you knew Python better, that's pretty awesome work). Some suggestions though from someone who makes a living as a Pythonista:

• The Feedback.py module alone has over 200 lines (compared to the <90 lines of alfred.py which deliver the full core API functionality). What's the added value of Feedback class over Item? The pythonic way is to just pass an iterable of Items to the result XML compiler and leave the organisation of Items to the user. For most usages, a simple generator or dict should suffice.
• Your Item attribute getters and setters are very unconventional. Have you had a look at the Python property (a built-in concept for getters and setters)?
• from ... import * is a bit of a Java thing and considered evil in Python because you lose control over your namespace.
• Split out the non-core stuff into separate packages. The Keychain API is awesome and probably worth to become a 1st class PyPI member. (Meaning: you might wish to publish that as a PyPI package, cf. http://pythonhosted.org/an_example_pypi_project/index.html). It looks like a no-nonsense super-lightweight alternative to https://pypi.python.org/pypi/keyring. The same applies to the Spotlight search.
• modules and packages should always be lowercase: Keychain.py -> keychain.py etc. Your main package doesn't have a name at all! (only implicitly via the GitHub project name PyAI).
• Have a look at encodings! I haven't tested any of your workflows using PyAI, but I guess Umlauts and their kin will not faithfully arrive (in David's PHP API, they don't either). The trick is to normalize using unicodedata.normalize('NFC', s.decode('utf-8'))
• Many functions do not really add value over direct access to the Python stdlib. E.g., the json.dumps/loads and plistlib stuff would be better off if you extracted the common path logic and leave the call of the stdlib functions (like json.dumps()) to the user.

I'm not saying all that because I'm a know-it-all but rather because I think there should be just one Alfred API, and it should be very lightweight and concise and restrict itself to the single responsibility of communicating with Alfred and the workflow environment (access to standard paths etc.). When you've trimmed some fat (150 lines for the core functionality should be plenty), I'll happily migrate over to PyAI (if it then has a Pythonic name :-) If you're interested in sharing/merging with python-alfred, that's also cool.

[phyllisstein]

Jan,

Thanks so much for your feedback! It's great to hear from someone who actually knows their Pythonic asshole from their Pythonic elbow (i.e., someone other than me) about this stuff. No know-it-all-ism detected whatsoever. I think that your remarks are spot-on, and that your API definitely looks much prettier and clearer—I especially think the args and the unified work are great ideas. I agree that it'd be ideal to try minimizing the number of APIs floating around, so maybe we can collaborate a bit on merging your work and mine? I'm of course happy to trim and slim, but I'm also willing to defer to your knowledge and let you pull the stuff that's useful—the keychain interface and whatever else—and massage it into your code.

[nikipore]

Ok, merging sounds cool.

 

I also believe that it makes more sense to migrate towards the cleaner API (which you seem to agree is alfred-python). Whenever you want to merge something into alfred-python yourself, please fork and make a pull request. If you wish I can also grant you write access (unless I'm unable to work out how that works 8-). Any non-Pythonic "mess" you produce will be cleaned up by me, I'm rather anal about that :-)

 

As I said, the Keychain and Spotlight APIs are great, but not directly related to Alfred (or is it?) and IMHO therefore (following the single responsibility principle) should go to repositories of its own. We certainly may refer to these repositories in the alfred-python README to ease usage for developers. Recipes or links to workflow projects using the API would also be cool.

 

If there is Alfred related stuff you want me to pull, just go ahead and tell me. Some things I already pulled myself, e.g., the prepending of uid's with the bundleid. (Although I'm not sure whether that's really necessary or Alfred has separate uid spaces per workflow @David: can you clarify that?)

 

@David (if you read this): It would be great if you opened a sticky post about existing Alfred APIs in order to increase leverage.

 

[phyllisstein]

My thinking about the Keychain and Spotlight stuff is actually in sort of flagrant violation of the single-responsibility principle. Viz., my thinking is that the best way to get people who may not have a lot of experience with code stuff creating workflows is to manage and make invisible a handful of potentially common tasks—like HTTP requests, bundling BS and requests_cache, and those two. But this could be very misguided.

 

I'll keep in mind that if I come up with any core stuff, I can fork your repo. And in the meantime—when I get some time in the meantime—I'll keep slimming and trimming (and renaming, heh) this guy.

[nikipore]

Ok, sounds like a good plan, let me know if I can be of help.

[phyllisstein]

Taking into account some of Jan's feedback—though probably less than he'd like—I've slimmed down PyAl a bit and turned it into alp. There are more details in the first post in this thread and at the Github page. Currently working on adding a Notification Center interface; a way of interacting with e-mail has been requested and is under consideration.

[greghoch]

Thanks for the great library! It's very helpful for writing workflow.

 

I also like the simplification but it would be great to still have a `PyAl.Item().fromDictionary()` function in `alp`. I think it's a pretty useful function. Alternatively, it would be nice to have a dictionary as an optional argument for `alp.item(**kwargs)` so that the arguments get set right at the beginning. 

 

The other common task for workflows is some kind of fuzzy search algorithm, which takes a list and returns the matches or something.

[phyllisstein]

Thanks! As @nikipore pointed out to me, you can accomplish the same thing as Item.fromDictionary() by passing a dictionary to item.__init__() like this: 

myDictionary = {} # Whatever
i = alp.item(**myDictionary)

 

Fuzzy searching would be an interesting addition, but potentially beyond my ken. It's also built into the way Alfred returns results, I think, so potentially unnecessary?

[greghoch]

great, I didn't know that!

 

About the build in function: Does that mean Alfred applies a fuzzy search to the results I feedback and combines the information from the fuzzy search with the ranking based on the learning algorithm to rank my results (decided position 1, 2...)? Does the fuzzy search only looks at the title or also the subtitle? 

[woofy31]

Hey,

 

Awesome to see this thread, because I've developed an equivalent library for AppleScript (see signature for link if you're an AppleScript lover), so I'll keep my eyes on this thread to see what other users need, what other features we should add to our workflow libraries to make them as similar as possible for all types of Alfred users and programmers, etc.

 

I hope you won't mind that I try to keep my library consistent with libraries written in other languages like yours

[phyllisstein]

E-mail and Notification Center interfaces are tested and working well. See the changes at https://github.com/phyllisstein/alp .

 

great, I didn't know that!

 

About the build in function: Does that mean Alfred applies a fuzzy search to the results I feedback and combines the information from the fuzzy search with the ranking based on the learning algorithm to rank my results (decided position 1, 2...)? Does the fuzzy search only looks at the title or also the subtitle? 

To be honest, I'm not sure. It's possible that David would know a bit more about this; if not, maybe Andrew could help. It occurred to me, though, that it wouldn't be a bad idea to have a fuzzy way of matching arguments to the script, so that'll be added to the list.

 

Hey,

 

Awesome to see this thread, because I've developed an equivalent library for AppleScript (see signature for link if you're an AppleScript lover), so I'll keep my eyes on this thread to see what other users need, what other features we should add to our workflow libraries to make them as similar as possible for all types of Alfred users and programmers, etc.

 

I hope you won't mind that I try to keep my library consistent with libraries written in other languages like yours

Looks terrific! I'll have to play around with it a bit—I've mostly used AppleScript as a sort of second-string language, but you've got some pretty impressive stuff there!

Edited March 19, 2013 by phyllisstein

[johnjddoe]

Hi! I'm just a workflow developer excited by all these cool new libraries. I have some questions about alp:

 

What is `alp.args()` for? Alfred by default escapes everything for you, so you can just use `sys.argv[1]`. Shell escaping is very poorly understood by most people, so perhaps I should explain more clearly:

In Alfred, run your python script, for example, by using language `/bin/bash`, and run your script with `python myscript.py {query}`. Note that there are no quotes around `{query}`. Make sure everything is ticked under the Escaping section. Then, in your script, `sys.argv[1]` will contain the exact arguments that the user passed to your script.

 

Also, what is `alp.decode(s)` for? I read the Python docs, but I'm not sure why you would need to normalize UTF-8 strings.

 

Finally, you should probably change the Github repo name to alp. Nice name, BTW.

[johnjddoe]

And IMHO `alp.timestamp(format=None)` should not be part of the library. If anyone needs the current date, they can just call the relevant Python function themselves, or at most create a wrapper for it that returns a string in the format they need.

[phyllisstein]

Hi! I'm just a workflow developer excited by all these cool new libraries. I have some questions about alp:

 

What is `alp.args()` for? Alfred by default escapes everything for you, so you can just use `sys.argv[1]`. Shell escaping is very poorly understood by most people, so perhaps I should explain more clearly:

In Alfred, run your python script, for example, by using language `/bin/bash`, and run your script with `python myscript.py {query}`. Note that there are no quotes around `{query}`. Make sure everything is ticked under the Escaping section. Then, in your script, `sys.argv[1]` will contain the exact arguments that the user passed to your script.

 

Also, what is `alp.decode(s)` for? I read the Python docs, but I'm not sure why you would need to normalize UTF-8 strings.

 

Finally, you should probably change the Github repo name to alp. Nice name, BTW.

I understand the way that escaping and arguments work. I think the hope there, and with string normalization, is to make alp behave a little better when there are special characters thrown into the mix, as I've been advised will happen with one of my own workflows. I'm not sure that it's the best technique, and I haven't done much testing yet, but it's a tilt at the ole windmill. The other element of the logic is that I'd like to reduce the amount of docs-Googling brand-new workflow developers have to engage in. Thanks for the reminder about changing the repo name.

 

And IMHO `alp.timestamp(format=None)` should not be part of the library. If anyone needs the current date, they can just call the relevant Python function themselves, or at most create a wrapper for it that returns a string in the format they need.

Again, it eliminates the need to search the docs for the right formatting codes, which I can personally never remember, and standardizes the way timestamps are used as UIDs for the benefit of Alfred's sorting mechanism. Most of the module boils down to things that anyone could do on his or her own—in fact, all of it does; there's nothing totally "new" in here, as far as I can tell, just a collection of useful "old" tasks made simpler. E.g., yes, anyone could write the ctypes interface to the Keychain functions; the point is that now no one has to.

[nikipore]

Hi! I'm just a workflow developer excited by all these cool new libraries. I have some questions about alp:

 

What is `alp.args()` for? Alfred by default escapes everything for you, so you can just use `sys.argv[1]`. Shell escaping is very poorly understood by most people, so perhaps I should explain more clearly:

In Alfred, run your python script, for example, by using language `/bin/bash`, and run your script with `python myscript.py {query}`. Note that there are no quotes around `{query}`. Make sure everything is ticked under the Escaping section. Then, in your script, `sys.argv[1]` will contain the exact arguments that the user passed to your script.

 

Also, what is `alp.decode(s)` for? I read the Python docs, but I'm not sure why you would need to normalize UTF-8 strings.

 

Finally, you should probably change the Github repo name to alp. Nice name, BTW.

 

Both, unescaping and decoding were features I introduced in python-alfred and which Daniel pulled trusting in my Python skills (and I do understand shell escaping as well, just didn't test it thoroughly enough). I've now tested that you don't need to (and hence shouldn't, so I'll remove that feature from alfred-python) unescape sys.argv when calling a Python script from the command line (choose /bin/bash and tick all escape options) via

python script.py "{query}"

and not, as you write, without the quotation marks. As to decoding, you definitely should move from UTF-8 to unicode to have an internally consistent handling of characters, and you should use u'string' literals throughout, and finally encode everything to UTF-8 again. The normalizing is necessary if you try and feed Python such strange things as my surname (Müller). The ü will end up basically as a UTF-8 version of the points followed by a u which is a Mac OS X thing and by no means a representation Python understands without normalizing.

 

I agree on your opinion about putting a lot of Alfred unrelated things into alp.py, and I still hope that Daniel moves that stuff into a module (or, better, package) of its own, but I see his point of setting the hurdle for script kids as low as possible.

 

Jan

[nikipore]

I understand the way that escaping and arguments work. I think the hope there, and with string normalization, is to make alp behave a little better when there are special characters thrown into the mix, as I've been advised will happen with one of my own workflows. I'm not sure that it's the best technique, and I haven't done much testing yet, but it's a tilt at the ole windmill. The other element of the logic is that I'd like to reduce the amount of docs-Googling brand-new workflow developers have to engage in. Thanks for the reminder about changing the repo name.

 

Again, it eliminates the need to search the docs for the right formatting codes, which I can personally never remember, and standardizes the way timestamps are used as UIDs for the benefit of Alfred's sorting mechanism. Most of the module boils down to things that anyone could do on his or her own—in fact, all of it does; there's nothing totally "new" in here, as far as I can tell, just a collection of useful "old" tasks made simpler. E.g., yes, anyone could write the ctypes interface to the Keychain functions; the point is that now no one has to.

And no one did so far to the best of my knowledge. One more nit-picking thing: Python classes are usually written uppercase.