defunct ZotQuery: an Alfred workflow for Zotero

[smarg19]

Also, actually I just updated to version 5.6 with fixes for BibTeX export. Please upgrade before you start testing the BibTeX capabilities. 

[kithairon]

As to the RTF/Scannable Cites format, I actually started a script to generate these items. Its basically done, but since I don't use this feature I wasn't able to test it, and so left it out. Tho it remains in the code. You can see it in the GitHub repo here: https://github.com/s...cites-export.py . If you want this feature, add it the workflow and test it out, if it works well for you, I can add it as a permanent feature.

 

 

Happy to add it and test – just let me know where. (RTF/ODT is a .js function in Zotero rather than a CSL like rtf-scan, does that matter?)

Edited February 20, 2014 by kithairon

[smarg19]

That shouldn't matter. The idea is to have ZotQuery export the Scannable Cite keys in the proper format, then when you finish writing you run the paper through the .js script to format those cite keys. 

 

Here's how you can set it up to start testing. Open Alfred Preferences and go to Workflows. Find ZotQuery. In the top right corner is a small + button. This is how you will add elements to the workflow. You will need three things:

 

* Input -> Script Filter [python filter_zot-general.py "{query}" 2>&1 | tee filter.log]

 

* Actions -> Run Script [python test_scannable-cites-export.py "{query}" 2>&1 | tee action.log]

 

* Outputs -> Copy to Clipboard [{query}]

 

When you press the + button, you will see these initial options, when you hover over one of them, you will see the sub-options. When you create each item, put the code which is in the brackets into the window. Make sure that the Language for the code is  /bin/bash. Now, I would recommend `z:test` as the keyword for the script filter, but you can use whatever. Once you create all three elements, make sure that they are connected/piped together. 

 

If you get all three elements piped together with this code in each, you can start exporting a Scannable CiteKey when you press return on an item found using the script filter. You should try to generate a whole range of different ones, put them in a document, then run that document through the Zotero script that formats things. Let me know how it all goes. 

 

P.S. Things should look something like this: { | Zetzel, et al., 2002 | | |zu:1140739:6W6TEKZW}

 

And you should double check that the python script test_scannable-cites-export.py has the input line uncommented. That is, it should look like this

item_key = alp.args()[0]
#item_key = 'KI66IFPT'

Let me know if you have any problems or questions.

[kithairon]

Thanks for helping me. Created the three elements with your code and did as suggested ( adding a "Post notification" for good measure...). Your scannable-cites-export seems to work quite well. I played with a dozen different entries, scanned the .odt afterwards and created a bibliography  – all looks good. One exception: the script doesn't handle umlauts or other diacritica very well when they happen to be in the author's name – and I have lots of those. Here the output when the author has diacritica in his/her name:

Traceback (most recent call last):

File "test_scannable-cites-export.py", line 74, in <module>

print scannable_cite

UnicodeEncodeError: 'ascii' codec can't encode character u'\u012b' in position 9: ordinal not in range(128)

 

I am very delighted that you have done all this already – for the uninitiated it looks as if you are fairly close already to have this working.

[smarg19]

Ah, good catch. I'll update. I'll plug, on a related note, that learning about text encoding is actually quite interesting. But, I'm glad to hear that it's already working well. I'll update this weekend with proper text encoding. 

 

Question: I have commented out two AppleScript dialog boxes to dynamically allow you to insert a prefix and suffix. I removed them because i thought that it would be faster for users to just add them in the paper. Since I imagine most people don't *always* have a prefix and a suffix. Is the current set-up, which is very fast and simple, good?

[kithairon]

As to your question: I believe the current set-up may be preferable since the pre- & suffix case scenario is a lot less common than the straight citation – at least it is for me. As pre- & suffixes are not difficult to insert into the .odt document and the dialogue boxes in ZotQery would need responding to *every* time it may be better as is. There is something very lean about the current flow. On the other hand: why don't you put them in if they only need uncommenting, I give it a whirl and come back with an informed opinion?

Your workflow has already changed my workflow...

[smarg19]

To try it out, you will need to open the workflow folder and find the `test_scannable-cites-export.py` script. Open it in a text editor or something like that. 

 

You will see this code near the bottom:

# See if user wishes to insert prefix
#icon_path = re.sub('/', ':', alp.local(join='icon.png'))
#a_script = """
#	set icon_ to "Macintosh HD%s" as alias
#	set prefix to display dialog "Insert prefix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
#	text returned of prefix
#""" % icon_path
#prefix = applescript.asrun(a_script)[0:-1]

# See if user wishes to insert suffix
#a_script = """
#	set icon_ to "Macintosh HD%s" as alias
#	set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
#	text returned of suffix
#""" % icon_path
#suffix = applescript.asrun(a_script)[0:-1]

prefix = ''
suffix = ''

This needs to be changed to this:

# See if user wishes to insert prefix
icon_path = re.sub('/', ':', alp.local(join='icon.png'))
a_script = """
	set icon_ to "Macintosh HD%s" as alias
	set prefix to display dialog "Insert prefix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
	text returned of prefix
""" % icon_path
prefix = applescript.asrun(a_script)[0:-1]

# See if user wishes to insert suffix
a_script = """
	set icon_ to "Macintosh HD%s" as alias
	set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
	text returned of suffix
""" % icon_path
suffix = applescript.asrun(a_script)[0:-1]

#prefix = ''
#suffix = ''

Change the code, then save it. Then run the z:test testing again. 

 

Like you said, two dialog boxes will pop up *every* time. 

 

And I'm glad to know that ZotQuery has helped you. 

[kithairon]

Done. Getting this error from the action.log - it looks like we're missing an icon.  (The last part is obviously my cite.)

execution error: File Macintosh HD:Users:amw:Library:Application Support:Alfred 2:Alfred.alfredpreferences:workflows:user.workflow.6408C3E2-F979-4DED-8EEA-7353EA0B12B8:icon.png wasn’t found. (-43)

execution error: File Macintosh HD:Users:amw:Library:Application Support:Alfred 2:Alfred.alfredpreferences:workflows:user.workflow.6408C3E2-F979-4DED-8EEA-7353EA0B12B8:icon.png wasn’t found. (-43)

{ | Back, et al., 2006 | | |zu:17490:XBMSCFBJ}

[deanishe]

I don't know about your system, but that fails on mine because my boot drive is called "Boot", not "Macintosh HD".

 

@smarg19: You shouldn't hardcode stuff like drive names.

 

Try this instead:

icon_path = alp.local(join='icon.png')
a_script = """
set icon_ to POSIX file "%s" as alias
…
""" % icon_path

[smarg19]

*hand-to-face*. Simple mistake. Thanks for the fix. Luckily, the primary place such dialogs existed with the old install dependencies script, which is no longer a part of the workflow. But thanks for the reminder not to really hard-code anything. 

 

@kithairon, you can use deanishe's code to fix the issue currently. I will have it resolved when I push the next update.

[kithairon]

Pasted deanishe's code: On running the amended z:test I get this:

179:180: syntax error: Expected end of line, etc. but found unknown token. (-2741)
execution error: File Macintosh HD/Users/amw/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.6408C3E2-F979-4DED-8EEA-7353EA0B12B8/icon.png wasn’t found. (-43)
{ | Back, et al., 2006 | | |zu:17490:XBMSCFBJ}

 

Thanks.

[deanishe]

That wasn't meant for cut-and-pasting. It was just to show the "POSIX file" line with a bit of context.

 

There's a '…' in there standing in for the lines that didn't need changing.

 

Also, it might not even fix your problem.

[kithairon]

@deanishe: Thanks for your help. Admittedly my cut&paste was not a very bright move. Have since re-installed ZQ5.6 and fiddled with the test_scannable_cites-export.py – changing what I believe to be the lines you meant to alter. Before I give you the errror-message, here what my test_scannable-cites-export.py actually looks like:

 

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import alp
import json
import re
from dependencies import applescript

"""
EXAMPLES:
{ | Morrison, 1956 | | |zu:1140739:FGBSNPS8}

{ | Jauss, et al., 1990 | | |zu:1140739:KI66IFPT}

{ | Wittern, & Pellegrin, 1996 | | |zu:1140739:JQHKJ4NZ}
"""

# Get Zotero data from JSON cache
with open(alp.storage(join='zotero_db.json'), 'r') as f:
    zot_data = json.load(f)
    f.close()

# Get the User ID from the settings file
with open(alp.storage(join="settings.json"), 'r') as f:
    data = json.load(f)
    f.close()
uid = data['user_id']


item_key = alp.args()[0]
#item_key = 'KI66IFPT'

for item in zot_data:
    if item['key'] == item_key:
        year = item['data']['issued']

        if len(item['creators']) == 1:
            last = item['creators'][0]['family']
        elif len(item['creators']) == 2:
            last1 = item['creators'][0]['family']
            last2 = item['creators'][1]['family']
            last = last1 + ', & ' + last2
        elif len(item['creators']) > 2:
            for i in item['creators']:
                if i['type'] == 'author':
                    last = i['family'] + ', et al.'
            try:
                last
            except:
                last = item['creators'][0]['family'] + ', et al.'


# See if user wishes to insert prefix

icon_path = alp.local(join='icon.png')
a_script = """
set icon_ to POSIX file "%s" as alias
    set prefix to display dialog "Insert prefix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
    text returned of prefix
""" % icon_path
prefix = applescript.asrun(a_script)[0:-1]

# See if user wishes to insert suffix
a_script = """
set icon_ to POSIX file "%s" as alias
    set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
    text returned of suffix
""" % icon_path
suffix = applescript.asrun(a_script)[0:-1]

prefix = ''
suffix = ''

scannable_cite = '{' + prefix + ' | ' + last + ', ' + year + ' | | ' + suffix + '|zu:' + uid + ':' + item_key + '}'

print scannable_cite

 

---------------

And here the error it produces:

194:310: execution error: No user interaction allowed. (-1713)
194:310: execution error: No user interaction allowed. (-1713)
{ | Back, et al., 2006 | | |zu:17490:XBMSCFBJ}
 

It's quite possible I botched something in the test....py – but this should be obvious to either of you after a glance at the script.

Thanks!

[deanishe]

That's an AppleScript error. It can usually be fixed by wrapping your script in a tell block:

tell Application "Alfred 2"
    set icon_ to POSIX file "%s" as alias
    set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
    text returned of suffix
end tell

It's also possible it doesn't like your input (seeing as the third citation seems to have worked).

 

When posting an error report, it's always a good idea to also post what input you passed to the script that caused the error.

Edited February 21, 2014 by deanishe

[kithairon]

Here again, my test...py script with what believe you suggested doing – I paste it in full so you may spot things I missed:

 

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import alp
import json
import re
from dependencies import applescript

"""
EXAMPLES:
{ | Morrison, 1956 | | |zu:1140739:FGBSNPS8}

{ | Jauss, et al., 1990 | | |zu:1140739:KI66IFPT}

{ | Wittern, & Pellegrin, 1996 | | |zu:1140739:JQHKJ4NZ}
"""

# Get Zotero data from JSON cache
with open(alp.storage(join='zotero_db.json'), 'r') as f:
    zot_data = json.load(f)
    f.close()

# Get the User ID from the settings file
with open(alp.storage(join="settings.json"), 'r') as f:
    data = json.load(f)
    f.close()
uid = data['user_id']


item_key = alp.args()[0]
#item_key = 'KI66IFPT'

for item in zot_data:
    if item['key'] == item_key:
        year = item['data']['issued']

        if len(item['creators']) == 1:
            last = item['creators'][0]['family']
        elif len(item['creators']) == 2:
            last1 = item['creators'][0]['family']
            last2 = item['creators'][1]['family']
            last = last1 + ', & ' + last2
        elif len(item['creators']) > 2:
            for i in item['creators']:
                if i['type'] == 'author':
                    last = i['family'] + ', et al.'
            try:
                last
            except:
                last = item['creators'][0]['family'] + ', et al.'


# See if user wishes to insert prefix

icon_path = alp.local(join='icon.png')
a_script = """
tell Application "Alfred 2"
    set icon_ to POSIX file "%s" as alias
    set prefix to display dialog "Insert prefix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
    text returned of prefix
end tell
""" % icon_path
prefix = applescript.asrun(a_script)[0:-1]

# See if user wishes to insert suffix
a_script = """
tell Application "Alfred 2"
    set icon_ to POSIX file "%s" as alias
    set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export" with icon icon_
    text returned of suffix
end tell
""" % icon_path
suffix = applescript.asrun(a_script)[0:-1]

prefix = ''
suffix = ''

scannable_cite = '{' + prefix + ' | ' + last + ', ' + year + ' | | ' + suffix + '|zu:' + uid + ':' + item_key + '}'

print scannable_cite
------------------------

 

The input that throws the error below is a single one of my zotero entries (I tried a few, with and without diacritics - all of them produce the same error). The Zotero item's cite going to the clipboard looks ok, so it does get through, albeit with the error. The pre- respectively suffix dialogue boxes never actually appear.

 

223:339: execution error: Alfred 2 got an error: File does not contain an icon. (-50)
223:339: execution error: Alfred 2 got an error: File does not contain an icon. (-50)
{ | Back, et al., 2006 | | |zu:17490:XBMSCFBJ}

 

Let me know what to fix – I'm willing but this is way over my head.

Edited February 21, 2014 by kithairon

[smarg19]

That's a truly odd error. I don't know it's origin, but the more I think about it, the more I think any such dialog boxes are unnecessary and add bloat. So if the format of the Scannable Cites is consistently being handled by the Zotero plugin, then I'll just add the simple form of the feature.

That seems best

[deanishe]

If you delete with icon icon_ that should fix that error.

[kithairon]

We're there – that last bit of trimming did the job. Delighted to say the workflow exports the rtf/odf-scan cites and in my latest test...py all the dialogues are coming up. Great! Chapeau to Stephan; and thanks to deanishe.

 

The pre- and suffix dialogues seem easily moved along empty with two returns; for my own needs I will probably keep a version of the scannable-cites-export.py with them in place. I understand if you want to avoid them in your next version to keep the workflow free of what others might see as superfluous.

 

Leaves the diacritics issue in the author's name.

 

Thanks to both of you for your patient help.

Kithairon

 

Here, for what it's worth, the latest version of the test-scannable-cites-export.py that worked:

 

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import alp
import json
import re
from dependencies import applescript

"""
EXAMPLES:
{ | Morrison, 1956 | | |zu:1140739:FGBSNPS8}

{ | Jauss, et al., 1990 | | |zu:1140739:KI66IFPT}

{ | Wittern, & Pellegrin, 1996 | | |zu:1140739:JQHKJ4NZ}
"""

# Get Zotero data from JSON cache
with open(alp.storage(join='zotero_db.json'), 'r') as f:
    zot_data = json.load(f)
    f.close()

# Get the User ID from the settings file
with open(alp.storage(join="settings.json"), 'r') as f:
    data = json.load(f)
    f.close()
uid = data['user_id']


item_key = alp.args()[0]
#item_key = 'KI66IFPT'

for item in zot_data:
    if item['key'] == item_key:
        year = item['data']['issued']

        if len(item['creators']) == 1:
            last = item['creators'][0]['family']
        elif len(item['creators']) == 2:
            last1 = item['creators'][0]['family']
            last2 = item['creators'][1]['family']
            last = last1 + ', & ' + last2
        elif len(item['creators']) > 2:
            for i in item['creators']:
                if i['type'] == 'author':
                    last = i['family'] + ', et al.'
            try:
                last
            except:
                last = item['creators'][0]['family'] + ', et al.'


# See if user wishes to insert prefix

icon_path = alp.local(join='icon.png')
a_script = """
tell Application "Alfred 2"
    set icon_ to POSIX file "%s" as alias
    set prefix to display dialog "Insert prefix for citation?" default answer "" with title "ZotQuery Citation Export"
    text returned of prefix
end tell
""" % icon_path
prefix = applescript.asrun(a_script)[0:-1]

# See if user wishes to insert suffix
a_script = """
tell Application "Alfred 2"
    set icon_ to POSIX file "%s" as alias
    set suffix to display dialog "Insert suffix for citation?" default answer "" with title "ZotQuery Citation Export"
    text returned of suffix
end tell
""" % icon_path
suffix = applescript.asrun(a_script)[0:-1]

prefix = ''
suffix = ''

scannable_cite = '{' + prefix + ' | ' + last + ', ' + year + ' | | ' + suffix + '|zu:' + uid + ':' + item_key + '}'

print scannable_cite

 

--------------------

Edited February 21, 2014 by kithairon

[kithairon]

After some more testing: Sorry to say that the script doesn't actually add any of the pre- or suffixes (entered into the dialog-boxes) into the rtf/odf-scan-cites . Tried a number of authors and a number of possible pre- or suffixes with no success – the cites come out as clean rtf/odf-cites but with none of the altered pre- respectively suffixes. (The action.log only has the correct cite and nothing else.)

Edited February 22, 2014 by kithairon

[smarg19]

It's a small problem. Give me the weekend and I'll push out an update with everything in place. If you want to change it manually, move these two lines:

prefix = ''
suffix = ''

above the dialog box lines. That will fix the issue.

[kithairon]

Thanks. Take your time for the good work.

Moved the two lines as suggested and the pre- & suffixes are actually working – they appear as expected in the cite. All seems to be working – great!

[rafaelcampos]

Ok. I have uploaded a new version of the workflow that I believe will fix the problem. Please go re-install. 

 

For posterity's sake, I will briefly explain the issue. In short, I made a rookie mistake (which makes sense, since I'm a Python rookie). I like organization, so I wanted to put all of the code's dependencies and modules within a sub-directory in the workflow folder. So I put the alp module, the pyzotero module, the applescript script, and the html2md scripts in there. At various points in the workflow all of these scripts and modules are called, but none of this is my code. Well, I didn't fully understand (and still don't *fully*) how Python treats scripts that are imported in other scripts. When I was importing the alp module, my local workflow was actual importing the alp module that I have download to my PATH (/Library/Python/2.7/site-packages), not the alp module within the dependencies directory. This was allowing my local workflow to function properly. BUT, for those of you (most of you I would imagine) who don't have the alp module within your system, the import call could only find the module in the dependencies directory. Well, for Python, nesting modules and such is not exactly the same as nesting directories in the file system. Your "folders" need to be properly configured Packages, not simply directories. So by putting alp within the dependencies folder, I was actually putting it within a Python Package without knowing it, and thus without configuring the dependencies directory accordingly. This is probably rambling at this point, and I'm not even 100% clear on all of this, but the short point is, I've fixed it (I believe).

 

I have also deleted these modules from my PATH, so that I can better debug the average user's problems. I'm sorry for the issues, but thanks a ton to @culb0743 and @kithairon for bringing this major bug to light. 

 

Please let me know if this fixes your problems.

 

stephen

Hi Stephen, I had the same problem…

 

I just can't search on Zotero.

So, if you could help me on this. Answering your first similar ask:

 

I have a 'six.py' folder. And also in "dependencies/alp/core_dependencies" there are the three things you told: a folder called biplist, a scipt called __init__.py, and a script called six.py.

 

my Python version is Python 2.7.2 (default, Oct 11 2012, 20:14:37)  [GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin

 

& My OS X is 10.8.3.

 

I hope you could help me.

Thank you

[deanishe]

It sounds like you're using an old version of the workflow. Try downloading the current version.

[rafaelcampos]

It sounds like you're using an old version of the workflow. Try downloading the current version.

humm thanks Deanishe, but my version is 5.6.

I'm new with workflow, I really don't know what the problem is.

[smarg19]

Ok. Can you please post the results of the action and filter logs. Those files are in the workflow folder. Also, can you tell me the exact error returned?