defunct ZotQuery: an Alfred workflow for Zotero

smarg19 · May 28, 2014

Run the configuration (z:config) with Alfred's debugger on. Paste here the code from the debugger.

Sebastian Daza · May 28, 2014

Starting debug for 'ZotQuery'

[ERROR: alfred.workflow.action.script] Code 1: 15:27:09 workflow.py:1263 ERROR [Errno 2] No such file or directory: u'None/prefs.js'

Traceback (most recent call last):

File "/Users/sdaza/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.89B1EB42-50D8-44BC-A99F-0FCAA2F6D384/workflow/workflow.py", line 1261, in run

func(self)

File "zotquery.py", line 1496, in main

print z_configurator.config()

File "zotquery.py", line 520, in config

return self.set_zot_paths()

File "zotquery.py", line 550, in set_zot_paths

if self._get_paths(zf_pref_path)[0]:

File "zotquery.py", line 636, in _get_paths

with open(prefs, 'r') as file_obj:

IOError: [Errno 2] No such file or directory: u'None/prefs.js'

15:27:18 workflow.py:1263 ERROR [Errno 2] No such file or directory: u'/Users/sdaza/Library/Application Support/Alfred 2/Workflow Data/com.hackademic.zotquery/paths.json'

Traceback (most recent call last):

File "/Users/sdaza/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.89B1EB42-50D8-44BC-A99F-0FCAA2F6D384/workflow/workflow.py", line 1261, in run

func(self)

File "zotquery.py", line 1490, in main

z_cacher = ZotCache(_force, _personal_only)

File "zotquery.py", line 170, in __init__

self.zot_db = self.get_path('database_path')

File "zotquery.py", line 207, in get_path

with open(self.wf_obj.datafile("paths.json"), 'r') as file_obj:

IOError: [Errno 2] No such file or directory: u'/Users/sdaza/Library/Application Support/Alfred 2/Workflow Data/com.hackademic.zotquery/paths.json'

smarg19 · May 28, 2014

Ok. That's helpful. I just pushed version 8.0.6 to Packal. I think this should fix the problem. Update and report back.

stephen

Sebastian Daza · May 28, 2014

It still doesn't work for me. Configuration seems to be right...

I don't know the difference between storage and linked attachment folder...

Starting debug for 'ZotQuery'

[ERROR: alfred.workflow.input.scriptfilter] Code 1: 16:29:37 workflow.py:1263 ERROR [Errno 2] No such file or directory: u'/Users/sdaza/Library/Application Support/Alfred 2/Workflow Data/com.hackademic.zotquery/zotero_db.json'

Traceback (most recent call last):

File "/Users/sdaza/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.89B1EB42-50D8-44BC-A99F-0FCAA2F6D384/workflow/workflow.py", line 1261, in run

func(self)

File "zotquery.py", line 1510, in main

z_filter = ZotFilter(_query, _scope)

File "zotquery.py", line 723, in __init__

with open(self.wf_obj.datafile("zotero_db.json"), 'r') as file_obj:

IOError: [Errno 2] No such file or directory: u'/Users/sdaza/Library/Application Support/Alfred 2/Workflow Data/com.hackademic.zotquery/zotero_db.json'

smarg19 · May 28, 2014

The "storage" directory/folder is where Zotero stores all internal attachments. If you have Standalone, it should be at ~/Library/Application Support/Zotero/Profiles/[something].default/zotero/storage. Can you open the paths.json file in ZotQuery's data directory (use z:bug then open the "Storage" folder) and tell me whats in there? Also, can you run z:cache with the debugger on and paste the results?

Sebastian Daza · May 28, 2014

paths.json:

{

"database_path": "alias Macintosh HD:Users:sdaza:Library:Application Support:Zotero:Profiles:fkkyo13j.default:zotero:zotero.sqlite",

"link-attachments_path": "alias Macintosh HD:Users:sdaza:Library:Application Support:Zotero:",

"storage_path": "alias Macintosh HD:Users:sdaza:Library:Application Support:Zotero:Profiles:fkkyo13j.default:zotero:storage:"

}

z:cache

Starting debug for 'ZotQuery'

[ERROR: alfred.workflow.action.script] Code 1: 16:39:48 workflow.py:1263 ERROR no such table: items

Traceback (most recent call last):

File "/Users/sdaza/Dropbox/Alfred/Alfred.alfredpreferences/workflows/user.workflow.89B1EB42-50D8-44BC-A99F-0FCAA2F6D384/workflow/workflow.py", line 1261, in run

func(self)

File "zotquery.py", line 1500, in main

print z_cacher.cache()

File "zotquery.py", line 308, in cache

json_cache = self.get_cache()

File "zotquery.py", line 323, in get_cache

basic_info = self.info_query()

File "zotquery.py", line 264, in info_query

return self.sqlite_get(info_sql)

File "zotquery.py", line 236, in sqlite_get

_info = cur.execute(_sql)

OperationalError: no such table: items

smarg19 · May 28, 2014

Ok. That's super helpful. There was a bug in a helper script I was using which was saving Applescript-style alias paths instead of standard POSIX paths in the paths.json file. I just uploaded version 8.0.7 to Packal. It will fix this issue. So run z:config again once you update.

Sebastian Daza · May 28, 2014

Now it is working, thanks! I will let you know if I have any other issue.

Sebastian Daza · May 30, 2014

Hello,

When I add a new reference to zotero, the cache should update after a query, right?

The update doesn't work... the only way is using database caching every time I add a reference.

Is that correct?

smarg19 · May 31, 2014

To be specific, ZotQuery will automatically try to update the cache after you perform an action on a query; for example, export a short reference. ZotQuery does not check the cache merely after a query.

If this is a major nuisance, it may be possible to alter the functionality, but please try updating using z:cache or performing an action on query. This should always update the cache. As far as I can tell, that code is solid, so I don't think there is a bug there.

stephen

greghoch · June 2, 2014

The recent update fixed my problem (reported a while ago) as well. Thanks!

In an earlier post, you described a way to limit the 'zot' search to certain data fields (which also made it much faster):

http://www.alfredforum.com/topic/3708-zotquery-an-alfred-workflow-for-zotero/?p=25417

With the recent updates, I can do that in the 'zot_filters.json' file, right?

Also, is it possible to restrict the search in two ways:

1) I would like to exclude Zotero items that are in group libraries. Many items are both in my personal and in group libraries so that I see them twice in ZotQuery searches.

2) Is it possible to exclude certain item types? I am, for example, not interested in finding web-pages, notes and some other stuff.

Thanks!

smarg19 · June 2, 2014

In an earlier post, you described a way to limit the 'zot' search to certain data fields (which also made it much faster):

http://www.alfredforum.com/topic/3708-zotquery-an-alfred-workflow-for-zotero/?p=25417

With the recent updates, I can do that in the 'zot_filters.json' file, right?

Correct. If you edit this file, it changes what ZotQuery looks at for the various filters. The file is (obviously) in JSON format. The keys are the various types of filters (general, titles, in-collection, etc). For each key, there is a list of items that it will search. This items are themselves lists with two items (except for notes, which is only one item). To remove an item, be sure to remove its entirety:

[
"data",
"title"
],

You could also use a scripting language to read the JSON, manipulate the dictionary, then overwrite the file with new JSON.

Also, is it possible to restrict the search in two ways:

1) I would like to exclude Zotero items that are in group libraries. Many items are both in my personal and in group libraries so that I see them twice in ZotQuery searches.

2) Is it possible to exclude certain item types? I am, for example, not interested in finding web-pages, notes and some other stuff.

To number 1: I have added this feature, just not really documented it. If you open Alfred and ZotQuery, and you open the Run Script action connected to z:cache you will find this:

python zotquery.py --cache True False

These three arguments tell zotquery to use the caching object (--cache), to force a cache update (True) and to not limit the scope to only the user's personal library (False). As detailed in the description at the top of zotquery.py, the last argument is a Boolean value for whether or not to limit the scope of the cache to the user's personal library. By default, it is set to False, which means that group libraries are included in ZotQuery's cache. If you change this to True, ZotQuery will only cache your personal library, and thus will only search your personal library.

ZotQuery also has a cache updater that runs each time you perform an action. This is the Run Script action beneath the "Citation Copied!" notification for each filter type. If you open this Run Script action you will see:

python zotquery.py --cache False False.

This will check to see if the cache needs updating (thus the False force argument), and will include all libraries. To exclude group libraries, change each of these Run Script actions to:

python zotquery.py --cache False True.

To number 2: You would have to alter the code in zotquery.py to achieve this, but it is totally possible. The section that would need to change is the info_sql method under the ZotCache object. This should start at line 256. It will look like this:

def info_query(self):
"""Retrieve (key, id, type id) from item"""
info_sql = """SELECT key, itemID, itemTypeID, libraryID
    FROM items
    WHERE 
        itemTypeID != 14
        and itemTypeID != 1
    ORDER BY dateAdded DESC"""
return self.sqlite_get(info_sql)

This is a sqlite query that begins the caching the process by getting all of the items that ZotQuery will cache. If you want to exclude certain item types, this is where you will do it. There are two ways you could do this inclusive or exclusive, that is, you can say get all items that are of these types (inclusive) or say get all items that aren't of these types (exclusive). To determine the itemTypeID, you will need to explore the .sqlite file. This is simple tho. Use the z:bug keyword, then open "Storage". In this directory will be the file zotquery.sqlite. Open this will something that can read .sqlite databases (I use a free app called SQLite Database Browser) If you explore the table titled "itemTypes", this will show you every possible item type and its corresponding id number. These are the numbers you will need.

Let's say you want to only include Books, Chapters, Articles, Theses, Manuscripts, and Conference Papers. This is inclusive. You would change the info_sql method to this:

def info_query(self):
"""Retrieve (key, id, type id) from item"""
info_sql = """SELECT key, itemID, itemTypeID, libraryID
    FROM items
    WHERE 
        itemTypeID IN (2, 3, 4, 7, 9, 33)
    ORDER BY dateAdded DESC"""
return self.sqlite_get(info_sql)

You can add whatever numbers correspond to items that you want to have searchable.

Conversely, you could simply exclude the items you don't want cached. Let's say you don't want Notes, Webpages, or Attachments, then you will use something like this:

def info_query(self):
"""Retrieve (key, id, type id) from item"""
info_sql = """SELECT key, itemID, itemTypeID, libraryID
    FROM items
    WHERE 
        itemTypeID not IN (1, 13, 14)
    ORDER BY dateAdded DESC"""
return self.sqlite_get(info_sql)

Using either of these methods will achieve your goal.

Hope this all helps to get ZotQuery working exactly as you would like it.

stephen

greghoch · June 3, 2014

Great! Thanks! Everything is working now!

greghoch · June 3, 2014

Actually, I am not sure whether the personal_only argument works. I changed the syntax for the z:config, z:cache, and the script below citation copied so that the second argument is always True. Then I updated the cache with z:cache but items group libraries still show up.

smarg19 · June 3, 2014

Actually, I am not sure whether the personal_only argument works. I changed the syntax for the z:config, z:cache, and the script below citation copied so that the second argument is always True. Then I updated the cache with z:cache but items group libraries still show up.

Good catch. In the upgrading, that functionality got a bit shoddy, and since I don't use it, I didn't spot it. I've just pushed version 8.1 to Packal which will fix this issue. Unfortunately, you will have to change the second argument in all those places again (Sorry). But at least it will work now.

stephen

greghoch · June 4, 2014

Awesome, thanks!

smarg19 · June 9, 2014

UPDATE: Version 8.5 now uses the Alfred Bundler to store all external dependencies, well, externally. This greatly lessens the size of the workflow (now 303.74 KB), which is great for those of us who back up to Dropbox. The way bundler works, on the very first run of version 8.5, you will have to wait as it downloads everything that's necessary (should take around 15 seconds, but after that, ZotQuery will be as fast as ever).

Please update and start saving space now

stephen

Sukh · June 20, 2014

Hi

Many congrats on this workflow: is exceptionally useful. I've not been through the entire forum so not sure if this has been solved previously (forgive me) but I've never managed to get ZotQuery to open up the attached PDFs in Zotero. Shift-enter should do it, but nothing happens. Enter alone switches to that entry in Zotero.

I provide the Debug results below. I tried a couple of different things - hence its a bit a long, but it should demonstrate that the other bits work.

I presume its something in the way I've set it up? Maybe its not looking in the right place?

Thanks again.

Starting debug for 'ZotQuery'

[iNFO: alfred.workflow.input.keyword] Processing output 'alfred.workflow.action.script' with arg ''

[iNFO: alfred.workflow.action.script] Processing output 'alfred.workflow.output.notification' with arg 'Cache Updated!

'