defunct ZotQuery: an Alfred workflow for Zotero

[Vero]

@lutefish Would you consider sharing your workflow so that fellow users can help you with the little bits you're unsure about, and benefit from the features that work well in the latest version?  A workflow that's available to others is a workflow that can be kept alive by the community.

 

Cheers,
Vero

[lutefish]

Sure - I’ve been hesitant only because I’m not the workflow’s creator, and don’t want to take away from his accomplishment, nor take credit for his work. I forked his project on github and uploaded both the modified python script and the modified workflow (alas, to the top level, as I’m new to github). Download here -https://github.com/lutefiasco/alfred_zotquery

[deanishe]

1 hour ago, lutefish said:

I’ve been hesitant only because I’m not the workflow’s creator, and don’t want to take away from his accomplishment, nor take credit for his work.

 

Just put Stephen's name in the README as the original creator, and add your own name to the exiting copyrights in file headers, rather than replacing his.

 

Don't bother commenting out old code: just delete it. Stephen's not maintaining his workflows any more, so I don't think he would object to your forking and continuing ZotQuery. Everything he did is preserved for posterity via git.

 

FWIW, the workflow is configured to self update. You should change the settings to point to your repo, and attach the .alfredworkflow file to each release if you want that mechanism to work properly.

 

Also, I strongly recommend updating the version of Alfred-Workflow bundled with the workflow, instead of using Fixum: Alfred will disable the workflow for new users if it recognises the old version.

[lutefish]

Deanishe - 

All of this seems reasonable, but it's essentially at the edge of or past my ability to make these changes. I'm not sure, for example, what goes into to fixing the last two issues you've identified (self-update/fixum), though I see you've linked to where the python script calls its own GitHub source.  But, I didn't make any changes to the file/source you're pointing to, only made changes to the workflow after it was already installed with Alfred 2. 

 

I retained the existing SQL calls just so I could identify what was changed, and in case there's any reason for someone to go back to them - particularly as I suspect in use-case scenarios other than mine (with multiple Zotero user accounts accessing a single library), I've just broken things.

 

Frankly, I don't think it will work as a newly installed workflow. I was able to get it running only because I had it running as an Alfred 2 workflow, and it had created the various json files it needed, so I was able to copy/move them around as needed. I'm happy to keep tinkering with what's up on Github, and answer any questions I can, but this is at the edge of my technical abilities (and time, alas).

[deanishe]

@lutefish I'm afraid I'm not very familiar with this workflow, as I don't use Zotero (I think that's what it's called…).

 

18 minutes ago, lutefish said:

it's essentially at the edge of or past my ability to make these changes.

 

Don't worry about that: this whole workflow was beyond @smarg19's ability when he started it. Didn't stop him at all  

 

And as with Stephen, I'm happy to help with any Alfred or Python-related questions.

 

21 minutes ago, lutefish said:

But, I didn't make any changes to the file/source you're pointing to, only made changes to the workflow after it was already installed with Alfred 2. 

 

I see you already forked the repo. What you should do it change the files in the source directory, which directly corresponds to the workflow's contents, and then use this script to symlink the source directory to your workflows directory (i.e. workflow-install.py -s source from the workflow root). You can also use this script to build your .alfredworkflow file (i.e. workflow-build.py -v source).

 

29 minutes ago, lutefish said:

what goes into to fixing the last two issues you've identified (self-update/fixum)

 

As regards self-updating, change smargh/alfred_zotquery to lutefiasco/alfred_zotquery (i.e. your repo) and be sure to add the .alfredworkflow file to any release. Here are the relevant docs for the Alfred-Workflow library (which does the updating).

Regarding updating Alfred-Workflow (the fixum thing): That can be a little involved, as it requires installing pip. Once you've done that, change to the source directory in Terminal or iTerm and run /usr/bin/python -m pip install --target . --upgrade Alfred-Workflow

[Damoeire]

Hey @lutefish . Thanks for getting this up and going again. I'm having trouble (as usual) with some basics. The opening config questions.

 

The zotero library I presume is library/applicationsupport/zotero ?

Which file is the sqlite file?

The attachments directory I presume isn't too important if I don't use links, and I have the API and user codes.

 

Also since I last reinstalled zotero, it has sort of broken my dropbox a little. My alfred library is in dropbox, and there are four files that won't sync, related to the zotero workflow I'm fairly sure. I'm tried deleting the workflow, restarting mac. I'm not too worried at this stage.

 

Having taken guesses, I get a no such file or directory error.

Cheers,

d.

Edited November 10, 2017 by Damoeire

[lutefish]

Damoeire - 

 

Zotero (as of v. 5) now uses your home directory by default, so the database is located at  ~/Zotero/zotero.sqlite

I'm not sure about the attachments directory, as I define a custom path in Zotero's preferences.

 

This is all hanging together with duct tape and hope, but I'll try to tinker with it and follow some of the instructions above to get the GitHub repository in order. Time is a bit of an issue at the moment.

 

[Damoeire]

On 10/11/2017 at 6:52 PM, lutefish said:

 

Zotero (as of v. 5) now uses your home directory by default, so the database is located at  ~/Zotero/zotero.sqlite

 

Excuse my ignorance - but are you saying that zotero is in the same directory as say my ~/Documents and ~/Downloads ? That's what I understand my home directory to be. Whereas my zotero is in ~/library/applicationsupport/zotero

 

In order to find zotero.sqlite, I then need to go to:

~/library/ applicationsupport /zotero/Profiles/8ioy240u.default/zotero 

and there is zotero.sqlite

So I've tried a few different things, but the error log from setting that directory as both Zotero storage and same for the zotero.sqlite is below.

 

What little I know is that:

 

IOError: [Errno 2] No such file or directory: u'/Users/damianevans/Library/Application Support/Alfred 3/Workflow Data/com.hackademic.zotquery/zotero_db.json'

 

Is correct. But in the workflow directory there is both a com.hackademic.zotquery directory and a com.hackademic.zotquery2 directory. Neither have a zotero_db.json file, but the second (zotquery2) has both a zotquery.db file and zotquery.json file!

 

Thanks for your help! If I could get zotquery running again, it would be great.

 

ERROR LOG:

 

Starting debug for 'ZotQuery'

[2017-11-13 10:42:17][STDERR: input.scriptfilter] 10:42:17 workflow.py:2225 DEBUG    Workflow version : .91.0
10:42:17 workflow.py:2267 DEBUG    Workflow finished in 0.001 seconds.
[2017-11-13 10:42:17][ERROR: input.scriptfilter] Code 1: 10:42:17 workflow.py:2225 DEBUG    Workflow version : .91.0
10:42:17 workflow.py:2243 ERROR    [Errno 2] No such file or directory: u'/Users/damianevans/Library/Application Support/Alfred 3/Workflow Data/com.hackademic.zotquery/zotero_db.json'
Traceback (most recent call last):
  File "/Users/damianevans/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.A8341E82-B839-4DD7-B0BC-264C5209D319/workflow/workflow.py", line 2236, in run
    func(self)
  File "zotquery.py", line 1525, in main
    z_filter = ZotFilter(_query, _scope, personal_only)
  File "zotquery.py", line 734, in __init__
    with open(self.wf_obj.datafile("zotero_db.json"), 'r') as file_obj:
IOError: [Errno 2] No such file or directory: u'/Users/damianevans/Library/Application Support/Alfred 3/Workflow Data/com.hackademic.zotquery/zotero_db.json'
10:42:17 workflow.py:2267 DEBUG    Workflow finished in 0.002 seconds.
[2017-11-13 10:42:18][ERROR: input.scriptfilter] Code 1: 10:42:18 workflow.py:2225 DEBUG    Workflow version : .91.0
10:42:18 workflow.py:2243 ERROR    [Errno 2] No such file or directory: u'/Users/damianevans/Library/Application Support/Alfred 3/Workflow Data/com.hackademic.zotquery/zotero_db.json'
Traceback (most recent call last):
  File "/Users/damianevans/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.A8341E82-B839-4DD7-B0BC-264C5209D319/workflow/workflow.py", line 2236, in run
    func(self)
  File "zotquery.py", line 1525, in main
    z_filter = ZotFilter(_query, _scope, personal_only)
  File "zotquery.py", line 734, in __init__
    with open(self.wf_obj.datafile("zotero_db.json"), 'r') as file_obj:
IOError: [Errno 2] No such file or directory: u'/Users/damianevans/Library/Application Support/Alfred 3/Workflow Data/com.hackademic.zotquery/zotero_db.json'
10:42:18 workflow.py:2267 DEBUG    Workflow finished in 0.002 seconds.

[lutefish]

Hmm. Two things. One, are you using Zotero 5? The most recent Zotero 5 moves the data directory from Application Support to /Documents/Zotero.  You can check by opening the Zotero app, choosing preferences, and then clicking on the "Files and Folders" tab. The "data directory location." will tell you where it's storing the database (and indicate where the default is). I'm not sure how to reconfigure zotquery, so I'm afraid the default Z5 data directory location (/Users/YOURUSERNAME/Zotero) is the only place that will work.

 

The .json file is a different issue. there are some permissions issues when attempting to configure/update zotquery, and I'm not sure it's able to write to the necessary paths. I seem to recall I just copied the .json files from an earlier version to the necessary directory, and ZotQuery was able to update them (though not create them in the first instance).

 

My apologies this is such an unstable semi-solution.

[Damoeire]

@lutefish Sorry I didn't reply to the above. I got caught up with other things and haven't had the chance to spend the time trying to work it out. However, I subscribed to a conversation on github where fractaledmind (the author) recently said this:

 

fractaledmind commented 19 hours ago

I am no longer in the academy and thus no longer use Zotero regularly. Sadly, as a result, I no longer use this workflow and have no plans on updating it. However, if you or someone else or even a group of people wanted to take control of the workflow, I would be happy to walk them thru the codebase as it is now and hand over the codebase for further development.

Here's in link:

https://github.com/fractaledmind/alfred_zotquery/issues/9

 

So Lutefish, allow me to introduce you to fractaledmind, if in an indirect and inefficient manner! If you're into it.  

[dfay]

Glad to see you've made the connection. For all people complain about his unmaintained workflows, Stephen did a lot of great work on automation.

[deanishe]

2 hours ago, dfay said:

For all people complain about his unmaintained workflows, Stephen did a lot of great work on automation.

 

@smarg19 was straight-up epic in his workflow development.

 

He always bit off more than he could chew, and by the gods, he chewed it all the same.

 

I admire his determination immensely.

[Jason_xiao]

 

I am a PhD student that use Alfred and Zotero everyday. To enhance the productivity, I download the workflow named zotquery (https://github.com/lutefiasco/alfred_zotquery). After I import this workflow, I just cannot use it. Everytime I use this workflow to search things, it always shows "'Workflow libraries are being updated',. 'Your workflow will continue momentarily'"! So, anybody can help me? 

[Jason_xiao]

@lutefish Thanks for your updating this workflow. It is very important for us academic people, because zotquery really helps me a lot. As I mentioned before, I download this workflow from the link you provided but encountered some unknown errors? Do you know how to resolve it? Thanks a lot

[lutefish]

Have you checked the get pip edit (somewhere above) has been made? I seem to recall the library updating process freezing if that wasn't sorted. 

[Jason_xiao]

@lutefish sorry, I am not familiar with coding, workflow writing, or software development. Could you please make it clear? I really want to use this workflow... I have spent several days in figuring this issue out (T_T)

[deanishe]

@Jason_xiao Could you please read the stickied notice on reporting workflow issues.

 

It’s basically impossible to help someone with “unknown errors”.

 

You’ll have to provide more information than that in order for anybody to be able to help you. 

 

 

@lutefish Does the workflow itself call pip to install dependencies? It would be much better to include them with the workflow if at all possible.

[Jason_xiao1]

I am so sorry that I did not make my question clear before. Thanks for @deanishe constructive suggestions, and I read through the stickied notices. According to  @lutefish  and  beakmachine's  aforesaid advice, I checked the "get-pip" issue,  and found that the bundler.py file (line 161) is right. 

 

My question is no matter which I try to do with zotquery (e.g., I enter zot:t), it shows in the top right corner:

Worflow libraries are bieng updated
Your workflow will continue momentarily

My Mac OS is EI Capitan 10.11.6, Alfred 3, Zotero5, the workflow zotquery is v 8.5 (download from the link @lutefish provided, many thanks!).

 

Then, I try to use the de-bug to find the errors. First, I enter zot:t in the Alfred, it shows:

 

• [2017-12-14 09:20:19][ERROR: input.scriptfilter] Code 1: Traceback (most recent call last):
•   File "zotquery.py", line 86, in <module>
•     bundler.init()
•   File "/Users/xiaozhenxin/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.A38CF440-D2B6-4638-A067-9AF7CF0C151E/bundler.py", line 508, in init
•     _update()
•   File "/Users/xiaozhenxin/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.A38CF440-D2B6-4638-A067-9AF7CF0C151E/bundler.py", line 294, in _update
•     response = urllib2.urlopen(PIP_INSTALLER_URL)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
•     return opener.open(url, data, timeout)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
•     response = self._open(req, data)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
•     '_open', req)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
•     result = func(*args)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
•     context=self._context)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
•     raise URLError(err)
• urllib2.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>

or, I enter z:c (i.e., database catching) in the Alfred, it shows:

• [2017-12-14 09:23:58][ERROR: action.script] Traceback (most recent call last):
•   File "zotquery.py", line 86, in <module>
•     bundler.init()
•   File "/Users/xiaozhenxin/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.A38CF440-D2B6-4638-A067-9AF7CF0C151E/bundler.py", line 508, in init
•     _update()
•   File "/Users/xiaozhenxin/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.A38CF440-D2B6-4638-A067-9AF7CF0C151E/bundler.py", line 294, in _update
•     response = urllib2.urlopen(PIP_INSTALLER_URL)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen
•     return opener.open(url, data, timeout)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open
•     response = self._open(req, data)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open
•     '_open', req)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain
•     result = func(*args)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open
•     context=self._context)
•   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open
•     raise URLError(err)
• urllib2.URLError: <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>

I cannot understand these coding things. Please let me know if anybody knows how to solve this problem. Many many thanks in advance!!!

[deanishe]

That error usually means trying to use https:// on a URL that doesn’t support HTTPS. Presumably, PIP_INSTALLER_URL is no longer set to the correct URL.

[Jason_xiao1]

Thanks, deanishe! So, in your opinion, I need to change the  PIP_INSTALLER_URL to make it fit the correct URL? Do you mean by change codes? If so, could you please give me some tips? 

 

Best,

Jason

[deanishe]

I can’t give you a concrete solution, as I don’t use Zotero or the workflow.

 

The best solution would be for @lutefish to rip out all the “bundler” crap and include the libraries and assets with the workflow.

 

The bundler was abandoned before it was even stable and hasn’t been updated in years.

 

If Zotero is free, and someone can give me a sample database to run the workflow against, I can have a look at it (I think I wrote the bundler code it’s using).

[deanishe]

This URL seems to be the right one for PIP_INSTALLER_URL

 

https://bootstrap.pypa.io/get-pip.py

 

Using that might fix it.

[Damoeire]

@deanish Well that's too good an offer to refuse. 

 

https://www.dropbox.com/s/py9fqpx445mgm4i/Example Library.json?dl=0

 

That's a link to an exported library as CSL json.

 

Hopefully you can import that into zotero, which is indeed free.

 

I have currently put getting this workflow to work on the backburner as I got halfway through - got confused, stopped and now don't know what part of the process I'm at and don't have the time to try to fix it, which, based upon previous attempts, would probably be two to three hours with no guarantee of getting it working. So any offer to make it easier, I'm there.

 

If that library doesn't work, let me know, I'll try another way. 

[deanishe]

1 hour ago, Damoeire said:

I have currently put getting this workflow to work on the backburner as I got halfway through - got confused, stopped and now don't know what part of the process I'm at and don't have the time to try to fix it

 

Where should I grab the "best" version of the code from? Your GitHub repo?

[Damoeire]

I think @lutefish would have the best answer to that question. I'm not even sure how a GitHub repo works and didn't know I had one.