[Help] calling dependencies in a Python workflow

[Cedric]

I'm trying to create a Python workflow that would work with the API of https://genderize.io

 

In my Python code I use the "requests" and "collections" modules. The code works as intended when ran through the Python interpreter, but now I don't know how to actually finalize the worfklow and include the dependencies.

 

• I tried using the Python dependency bundler, but it didn't seem to work.
• I tried to pip install inside the workflow folder, but I have the Homebrew related bug with the pip install --target=url.

I used a technique which did install requests in the workflow, but I'm not sure how to point to it now. I'm very confused despite my best efforts, and the workflow folders reflect that. Hopefully someone can walk me though the correct steps to do once the code is written and working.

 

Thanks!

 

Here is the workflow: link

 

And the code:

#!/usr/bin/env python
# encoding: utf-8

#Code under MIT lience, modified from Stack Overflow answers, as listed below:
#orlp http://stackoverflow.com/a/19560097
#Aaron Hall http://stackoverflow.com/a/33737067
#metatoaster http://stackoverflow.com/a/25480206
#Ultimate Zero http://stackoverflow.com/a/13224079

import requests
from collections import OrderedDict

import bundler
bundler.init()



#1.PREPARING THE QUERY

#import Alfred's query
query = '{query}'

#transform query into a list
values = query.split()

#format the keys as expect by genderize.io
keys = ['name' + '[' + str(i) + ']' for i in range(len(values))]

#create the parameters as an ordered dictionary of keys and values, as expected by requests.get()
names = OrderedDict([(k, v) for k, v in zip(keys, values)])

#2. SENDING THE QUERY AND FORMATTING THE RESULTS

#query the API with the GET request
r = requests.get('https://api.genderize.io/', params=names)

#define a function to easily access probability.
proba = [float(r.json()[p]['probability']) for p in range(len(values))]

#define a function to easily access count.
count = [int(r.json()[c]['count']) for c in range(len(values))]

#define a function to easily access gender.
gender = [str(r.json()[g]['gender']) for g in range(len(values))]

#3. PROCESSING THE RESULTS

#print the results of the query, with some variations linked to the strenght of the guess
for r in range(len(values)):
    if proba[r] >= 0.9:
        print values[r], 'is most likely', gender[r]
    elif 0.7 <= proba[r] < 0.9:
        print values[r], 'is probably', gender[r]
    elif 0.5 < probar[r] < 0.7:
        print values[r], 'is maybe', gender[r]
    else:
        'the gender of', values[r], 'is unknown'

[deanishe]

Don't use the bundler. It's abandonware, AFAIK.

Secondly, your script won't be run with the Homebrew Python, but the system one (/usr/bin/python). That's because Alfred doesn't use your shell environment and thus doesn't use your PATH.

Also, don't paste your code in Alfred's Script box. That's why your imports aren't working. When you call import XYZ, Python first looks in the directory your script is in. But there isn't a script file if your code is in Alfred's Script box. import will then only work for libraries installed in the system Python.

Put your Python code in a separate script in your workflow directory (next to info.plist) and treat the Script box like a shell, like this.

It's astonishing that the pip bug still hasn't been fixed, tbh.

The simplest solution is to install a "user" version of pip with pip install --user -U pip

This will install it in your user Python site-packages, which is accessible to both Homebrew and system Pythons.

Then you can use /path/to/python -m pip <pip> <options> <here>

So, to install requests in your workflow, use: /usr/bin/python -m pip install --target /path/to/workflow/directory requests

That's a bit long-winded, so I've created a few aliases:

alias pip-here="/usr/bin/python -m pip install --target . -U"
alias pip-user="/usr/bin/python -m pip install --user"
alias pip-system="/usr/bin/python -m pip"

[Cedric]

 

Don't use the bundler. It's abandonware, AFAIK.

Secondly, your script won't be run with the Homebrew Python, but the system one (/usr/bin/python). That's because Alfred doesn't use your shell environment and thus doesn't use your PATH.

Also, don't paste your code in Alfred's Script box. That's why your imports aren't working. When you call import XYZ, Python first looks in the directory your script is in. But there isn't a script file if your code is in Alfred's Script box. import will then only work for libraries installed in the system Python.

Put your Python code in a separate script in your workflow directory (next to info.plist) and treat the Script box like a shell, like this.

It's astonishing that the pip bug still hasn't been fixed, tbh.

The simplest solution is to install a "user" version of pip with pip install --user -U pip

This will install it in your user Python site-packages, which is accessible to both Homebrew and system Pythons.

Then you can use /path/to/python -m pip <pip> <options> <here>

So, to install requests in your workflow, use: /usr/bin/python -m pip install --target /path/to/workflow/directory requests

That's a bit long-winded, so I've created a few aliases:

alias pip-here="/usr/bin/python -m pip install --target . -U"
alias pip-user="/usr/bin/python -m pip install --user"
alias pip-system="/usr/bin/python -m pip"

 

Thanks a lot for the tips, and especially for the aliases. pip now works as it should and I successfully installed Requests in the worfklow. I do have a message about pip not being up to date, should I try to update it? It seems to be in the system folder.

 

That said, my workflow still does not work. I tried using the script filter, but Alfred would display fallback options (Google...) before I'm able to finish typing the names.

I then tried with the other script object, but nothing happens.

 

If you could give it a look and tell me what's wrong, that would be really helpful. Here is the link.

 

Thanks!

[deanishe]

Alfred has a debugger. Your error message is in there.

 

But there are all kinds of things wrong with the code. You can't just assume that the API response is correct and of the length you need. You have to check that first.

 

You're also not even passing the query to your script.

 

If you're using Alfred-Workflow, you don't need requests. workflow.web is intended for exactly what you're trying to do (simple web requests where requests would be overkill).

 

Please read the tutorial for Alfred-Workflow. Most of what you need to know is in there.

[Cedric]

 

Alfred has a debugger. Your error message is in there.

 

But there are all kinds of things wrong with the code. You can't just assume that the API response is correct and of the length you need. You have to check that first.

 

You're also not even passing the query to your script.

 

If you're using Alfred-Workflow, you don't need requests. workflow.web is intended for exactly what you're trying to do (simple web requests where requests would be overkill).

 

Please read the tutorial for Alfred-Workflow. Most of what you need to know is in there.

 

 

Thanks for the quick reply!

 

About the error in the code

I'm not sure I understand what's happening:

I tested the code in iPython, and it works.

I then tested the worfklow while using fixed values ('Alice Nicolas' instead of '{query}') and it works.

When I'm using it with {query} it doesn't work. Here is the error message, for reference:

[ERROR: alfred.workflow.action.script] Traceback (most recent call last):
  File "genderize.py", line 36, in <module>
    proba = [float(r.json()[p]['probability']) for p in range(len(values))]
KeyError: 0

About passing the query

I'm not sure I understand exactly how do I pass the query. In the other workflow that I wrote in bash, I knew that

scrip.sh word

  makes 'word' a positional parameter that I can grab in the code.

 

For python, I don't know how to tell Alfred "replace the {query} in the code with my input". I thought it was managed automatically.

 

About the problems with the code

 

I know that my code doesn't work if the API returns errors, I went for a first version that would allow me to test creating a working workflow in Alfred. Once i can do that, I will focus on making the code stronger. I'm a beginner at Python, so I don't have the best practices yet in regards to querying API.

 

About alfred-wofklow

 

It's included in the folder at the moment, but not used. I plan to review the tutorials later to see how I can adapt the code to my needs if I want to complexify the workflow. Working with requests first was a good way to learn a popular Python module that I will probably use in other projects.

Edited April 6, 2016 by Cedric

[deanishe]

Like I already said, most of what you need to know is in the tutorial.

If you don't want to read it, that's fine, but I'm not going to waste my time rewriting it in this thread.

[Cedric]

Like I already said, most of what you need to know is in the tutorial.

If you don't want to read it, that's fine, but I'm not going to waste my time rewriting it in this thread.

 

I've already read it twice before posting (along with Alfred-workflow documentation and the bundler code comments and documentation, and other forum posts), I do try to do my homework before asking for help!

 

The tutorial does not do exactly what I'm trying to do, as I'm not trying to show a list of posts. It might be why the answer didn't seem so obvious to me. But I guess I'll take a small break and re-read it again, maybe taking a step back will help seeing the answers there.

 

Thanks a lot for your help, sorry if it seemed like I was trying to make your write the tutorial again.

[deanishe]

Sorry if you have already read the docs. I got the impression that you hadn't because:
 

Working with requests first was a good way to learn a popular Python module that I will probably use in other projects.

From http://www.deanishe.net/alfred-workflow/api/web.html#web

workflow.web provides a simple API for retrieving data from the Web modelled on the excellent requests library.

The purpose of workflow.web is to cover trivial cases at just 0.5% of the size of requests.

You can literally replace requests.get() with workflow.web.get() and it will work exactly the same.
 

The tutorial does not do exactly what I'm trying to do, as I'm not trying to show a list of posts.

The posts part is largely irrelevant. The tutorial is really about showing lists of things in Alfred using Script Filters, which is what you're trying to do.
 
And it covers important stuff like how to pass the query to your script:
 
 
 
And (after a fashion) how to get the query in your script:
 

# Get query from Alfred
if len(wf.args):
    query = wf.args[0]
else:
    query = None

If you're not using Alfred-Workflow, you need to do query = sys.argv[1] instead of wf.args[0]

(But that will fail as soon as you try a non-ASCII query.)

What you don't appear to be aware of is that it you want to show results in Alfred, you can't just print them. Alfred requires the results in a specific XML format.

That is why you should probably use Alfred-Workflow: it can generate the XML very easily for you.

On top of that, as the docs state, it will catch errors for you and show you a proper error message instead of failing silently, like your workflow currently does.

It will also largely handle text encoding for you, which means you're workflow won't fail when you use a non-ASCII query, such as "Cédric"

It's a great idea to figure out how Alfred workflows work by writing all your own stuff. For someone new to programming, however, there are quite a lot of things you need to learn to write a Script Filter.

I literally wrote Alfred-Workflow for people like you. The goal was to make it as easy as possible for people who aren't programmers to write Script Filters for Alfred.

 

You would do well to write your workflow with Alfred-Workflow first. When it's working, replace Alfred-Workflow with your own code as you figure out what each bit is doing.

Edited April 6, 2016 by deanishe

[deanishe]

Regarding the Python dependency bundler: Its goal was to make it possible to share libraries and tools between different workflows (so you don't need to include, say, requests with every workflow you write). It doesn't help much with actually writing workflows.

 

The demo workflow you looked at uses Alfred-Workflow for all workflow-related things. The bundler parts are for generating icons, showing dialogs and notification (which Alfred-Workflow can do on its own now).

 

The demo is based on the original Python version of the bundler. Somebody else rewrote the Python bundler after I quit the project, so it almost certainly doesn't work anymore.

 

AFAIK, the entire bundler project has been abandoned, at least for now.

Edited April 6, 2016 by deanishe

[Cedric]

Thanks so much for the response, it's so much clearer now.

 

A lot of things weren't obvious to me, like the part about "modelled on the requests library" meaning that it worked exactly the same way.

Because I didn't understand how Alfred worked, the part of the code about grabbing the query went over my head, as I didn't understand what the code was doing.

 

Here is why I'm using the print function: my goal with the v0.1 of the workflow was to be able to display (print) very simply the answer to the question "male or female?"

 

I naively thought that I could write a v0.1 my Python script and easily integrate it into Alfred, and iterate from there. Obviously, there are too many hurdles in doing that, so I will:

- finalise the script (by making sure my code works when the API returns errors or the like).

- integrate it into Alfred by using your library

 

Thanks for the clarification, really. I'm going to post again here when I'm done rewriting the code.

[deanishe]

Here is why I'm using the print function: my goal with the v0.1 of the workflow was to be able to display (print) very simply the answer to the question "male or female?"

I understand what your goal is, but not how you want to implement it. print is the correct way to send some output to the next action: what you print is the next action's {query}. So print "x is probably y" is correct for the Keyword->Run Script->Large Type combination in your workflow.

But you've also got a Script Filter in your workflow, and they do not work that way. A Script Filter requires you to use Alfred's XML format for the output. If you print anything but Alfred's XML in a Script Filter, it's broken.

If you want the result to show up immediately in Alfred's main window, like this, you have to use a Script Filter (and XML).

 

I naively thought that I could write a v0.1 my Python script and easily integrate it into Alfred, and iterate from there. Obviously, there are too many hurdles in doing that, so I will:

That's not naïve, that's absolutely the right way to do it. What is not good is writing all your code in the top-level of the script. You're just making your own life harder that way.

What you should try to do is to split up your code into useful, self-contained units that you can use like building blocks. Break the program down into a set of smaller problems, then solve them one at a time.

So, you'd want to put your code that calls the API in its own function with very specific behaviour, for example: 

def gender_of_name(name):
    url = 'www.example.com/api'
    response = requests.get(url, params={'name': name})
    # check for valid response here
    if not is_valid(response):
        return None
    # parse response
    data = response.json()
    if data['gender'] == 'm':
        return 'male'
    elif data['gender'] == 'f':
        return 'female'
    else:
        return 'unisex'

Once you have that working, you can do what you want with it (an Alfred workflow, a command-line program) and you don't have to think about the API any more.

Look at the way this script is organised.

Each function has a very specific job and knows nothing about anything else, so when it comes to executing the workflow in main(), it's just a case of plugging them into each other. It'd be super easy to update that script to, say, get filepaths from the clipboard instead of Alfred/command-line arguments, or to do something else with the output.