defunct ZotQuery: an Alfred workflow for Zotero

[dfay]

In re: the output format, I wouldn't trouble yourself to writing RTF output code unless you've really got it in the bag already.  The only formatting that's going to be needed are italics, bold, maybe underlining in a few edge cases.  This can be done in HTML and piped to textutil without risk, I'd think.  

 

Also I'd agree that collections aren't necessary for the workflow.  In my experience, when I would be working with a collection (organizing readings, finalizing references for a paper, etc.), I'd also have my bibliographic software open and be working in there directly.  The times I'd use the workflow are when I'm writing and don't want to be interrupted but just want to quickly grab a reference or open a linked PDF.

 

Hope that helps -- can't wait to see the finished product.

Edited December 20, 2017 by dfay

[deanishe]

6 hours ago, dfay said:

This can be done in HTML and piped to textutil without risk

 

I don't like the way textutil works. It doesn't just convert your HTML snippet to an RTF snippet, it outputs a complete RTF document with the font set to Times Roman, which makes it crap for pasting into other texts. It may be possible to strip those bits from the output, but it's just as easy, and more reliable, to write a proper RTF formatter.

 

6 hours ago, dfay said:

The times I'd use the workflow are when I'm writing and don't want to be interrupted but just want to quickly grab a reference or open a linked PDF.

 

Thanks. I'll focus on that, then.

[Damoeire]

Got this reply from zotero forums. Any use to you? I can ask him how to get them into zotero if not. 

 

emilianoeheyns commented on Sample Library for testing

I do have quite a few test references (138 test cases with 1589 references in all) and you're more than welcome to use them, but they're not in a format that Zotero can import without help -- they're straight JSON dumps of what the importer expects, produced and consumed by one of the BBT-bundled translators. 

I chose JSON dumps because I couldn't find an existing format that roundtripped cleanly; Zotero RDF came closest but wasn't (at the time at least) enough for my tests.

[deanishe]

Thanks @Damoeire, but I think I'm good for test data at the moment.

[deanishe]

Here's a beta version. Would anyone like to give it a try?

[lutefish]

@deanishe  @Damoeire  @dfay Jeez - I step away for a few days, and this!

 

@deanishe - running it, first thing I see when searching is an error - ValueError: year=1854 is before 1900; the datetime strftime() methods require year >= 1900

I have lots of pre 1900, even free 1800 entries in my Zotero library. The error doesn't stop the script from returning items with year values before that time, or from opening the attachments

 

 

 

 

[deanishe]

@lutefish Thanks for spotting that. The datetime is unused, so I've binned it. Should be fixed in v0.1.1-beta.

You'll have to run zot workflow:delcache to clear the old cached data first, though.

 

Hmm. Now I come to think of it, I added a switch to auto-clear the cache when the format changes, but I forgot to use it…

[lutefish]

OK. Downloaded new, ran delcache and I'm now getting duplicate results returned in the Alfred display. Looking at the debug logs I see  "[index] 2 result(s) for u'1975'" but the workflow displays a list of item 1, item 2, item 2, item 1.

 

Gonna be away again through the holidays, but we're up to two round of beers on me.

 

Cheers.

[deanishe]

Do you perhaps have two copies installed?

 

That log line is written right before the results are returned, so there really are only 2 results.

[Damoeire]

@deanishe Amazing! Beers coming from me too. It's working great, both opening the entry in zotero and opening attachments. I'm getting no citation style set when trying to copy citation, is that my end or is that not working yet.

 

Can't say how great this is to have a working version. Also, love the name and the icon.

 

Congratulations and thanks so much. It's late here, I'm just home and only have had 5 mins to look at it, will give it a better going over tomorrow.

 

[deanishe]

1 minute ago, Damoeire said:

I'm getting no citation style set when trying to copy citation, is that my end or is that not working yet.

 

How are you trying to copy citations, and what does the debugger say?

[Damoeire]

Was just editing my post. I got it to work, not sure what I was doing wrong. A list of citation style came up this time, I must have done something wrong.

 

It gave me this:

 

<i>The Cultural Politics of Jazz Collectives: This Is Our Music</i>, eds. Gebhardt, N and Whyton, T (Routledge, 2015).

 

That's my own citation style, I'll have to look at it again tomorrow though unfortunately.

THanks again!

[deanishe]

Don't forget to read the instructions.

[Damoeire]

5 minutes ago, deanishe said:

Don't forget to read the instructions.

I was so excited I didn't look. Bad beta tester! Im away from the computer now but looking forward to tomorrow. 

[mosb]

Enormous thanks to everyone, particularly @deanishe, for your efforts: I used to use ZotQuery daily, and would love to have a replacement.

 

I have installed the new ZotHero workflow, and am using the Zotero 5.0.32 with the default data directory. When I attempt to do anything with the workflow, I get the following error:

 

```

Error in workflow 'ZotHero'

sequence item 0: expected string or Unicode, NoneType found. 

```

 

Any help at all appreciated!

[deanishe]

Could you post the output from Alfred's debugger?

[mosb]

Thanks for the super prompt response! Output below for `zot a`.

 

[2017-12-21 14:46:20][input.scriptfilter] Queuing argument 'a'
[2017-12-21 14:46:21][input.scriptfilter] Script with argument 'a' finished
[2017-12-21 14:46:21][STDERR: input.scriptfilter] .
14:46:21 workflow.py:2055 DEBUG    ---------- ZotHero (0.1.1) ----------
14:46:21 zh:280 DEBUG    args={'--help': False,
 '<key>': None,
 '<query>': 'a',
 '<style>': None,
 'attachments': False,
 'citations': False,
 'copy': False,
 'fields': False,
 'search': True}
14:46:21 core.py:35 DEBUG    [core] cachedir=u'~/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero'
14:46:21 core.py:145 INFO     [core] searching for "a" ...
14:46:21 util.py:123 DEBUG    [util] copying u'/Users/mosb/Zotero/zotero.sqlite' to u'/Users/mosb/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero/zotero.sqlite' ...
14:46:21 core.py:93 DEBUG    [core] storage=u'~/Zotero/storage'
14:46:21 core.py:94 DEBUG    [core] styles=u'~/Zotero/styles'
14:46:21 zotero.py:188 DEBUG    [zotero] database last modified 0.5 secs ago
14:46:21 index.py:184 DEBUG    [index] not yet initialised
14:46:21 index.py:253 DEBUG    [index] updating u'~/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero/search.sqlite' ...
14:46:21 index.py:131 DEBUG    [index] initialising u'~/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero/search.sqlite' ...
14:46:21 index.py:139 DEBUG    [index] opened u'~/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero/search.sqlite'
14:46:21 zotero.py:180 DEBUG    [zotero] opened database u'~/Library/Caches/com.runningwithcrayons.Alfred-3/Workflow Data/net.deanishe.alfred.zothero/zotero.sqlite'
14:46:21 workflow.py:2074 ERROR    sequence item 0: expected string or Unicode, NoneType found
Traceback (most recent call last):
  File "./lib/workflow/workflow.py", line 2067, in run
    func(self)
  File "./zh", line 289, in main
    return do_search(query)
  File "./zh", line 84, in do_search
    entries = app.search(query)
  File "./lib/zothero/core.py", line 146, in search
    return self.index.search(query)
  File "./lib/zothero/core.py", line 110, in index
    self._index.update(self.zotero)
  File "./lib/zothero/index.py", line 281, in update
    u' '.join([d.name for d in e.attachments]),
TypeError: sequence item 0: expected string or Unicode, NoneType found
14:46:21 workflow.py:2097 DEBUG    ---------- finished in 0.559s ----------
[2017-12-21 14:46:21][input.scriptfilter] {"items": [{"valid": false, "subtitle": "sequence item 0: expected string or Unicode, NoneType found", "icon": {"path": "/System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/AlertStopIcon.icns"}, "title": "Error in workflow 'ZotHero'"}]}

 

[deanishe]

Could you run zot workflow:delcache to clear the cached data and try again?

 

I'll publish a new version that should help track down which item is causing the issue.

[lutefish]

Another issue - I can't open attachments. This may be because I store attachments in a different directory than I store the Zotero database (which uses the default data directory ~/Zotero/)

 

Disabled the other version, so I'm getting the proper results. No issues configuring citation styles or accessing the database successfully.

 

But when trying to shift-enter to view attachments, the script fails (silently from the user perspective). Debug shows: "[ERROR: action.script] The file /Users/lutefish/Box Sync/AlfredSync/Alfred.alfredpreferences/workflows/user.workflow.3E466815-B937-4C74-B823-C1F211FCAE3D/attachments:2017/Fisher Chancery Standard Written English.pdf does not exist." 

 

Is there anyway to specify a distinct attachment top-level directory from database directory - what Zotero calls "Linked Attachment Base Directory" vs "Data Directory Location"? 

[deanishe]

1 hour ago, lutefish said:

Is there anyway to specify a distinct attachment top-level directory from database directory

 

There is now. Grab v0.1.2-beta, and set ATTACHMENTS_DIR in the configuration sheet.

If it isn't set, the workflow writes a warning message to the log and doesn't show the attachments.

 

 

[lutefish]

That did the trick. 

 

Seriously - is there some way I can compensate you in gratitude? Or, alternately, some way I can contribute to the larger Alfred community, as a thanks? You've made life better for the tiny number of academic users who use both (Alfred, Zotero), and have my profoundest thanks.

 

[dfay]

This is great - search is blazing fast.  I'm running into issues with some CSL styles though -- it doesn't copy successfully and gives the same error message:

 

22:11:00 util.py:223 INFO     [0.03s] updated search index
22:11:00 cache.py:301 DEBUG    [cache] opening store 'styles'...
22:11:00 styles.py:140 DEBUG    [styles] style=CSLStyle(name=u'The Tort Law Review', path=u'/Users/username/Zotero/styles/the-tort-law-review.csl')
22:11:00 styles.py:141 DEBUG    [styles] csl=[{u'type': u'article-journal', u'title': u'A New Ecosystems Ecology for Anthropology', u'issued': {u'date-parts': [[2003, 0, 0]]}, u'author': [{u'given': u'Thomas', u'family': u'Abel'}, {u'given': u'John Richard', u'family': u'Stepp'}], u'volume': u'7', u'container-title': u'Conservation Ecology', u'issue': u'3', u'id': u'YBE2PQP8'}]
22:11:00 workflow.py:2074 ERROR    'NoneType' object has no attribute 'render'
Traceback (most recent call last):
  File "./lib/workflow/workflow.py", line 2067, in run
    func(self)
  File "./zh", line 493, in main
    return do_copy(args['<style>'], args['<key>'])
  File "./zh", line 289, in do_copy
    data = app.styles.cite(e, s)
  File "./lib/zothero/styles.py", line 157, in cite
    html = u''.join(bib_html.cite(cite, warn))
  File "./lib/citeproc/frontend.py", line 102, in cite
    return self.style.render_citation(citation, self._cites, callback)
  File "./lib/citeproc/frontend.py", line 69, in render_citation
    return self.root.citation.render(citation, cites, callback)
AttributeError: 'NoneType' object has no attribute 'render'
22:11:00 workflow.py:2076 INFO     for assistance, see: https://github.com/deanishe/zothero/issues
22:11:00 workflow.py:2097 DEBUG    ---------- finished in 0.517s ----------

Looks like the citeproc is expecting something it's not getting maybe?

[Damoeire]

@deanishe

Again, this is fantastic. I'm not sure the best way to report possible bugs, or features that I'm expecting to work that aren't, so bear with me.

 

Web pages aren't showing up in results. I've tested that by finding specific webpages in my library, and can't find them using zothero

 

Another issue:

I have an entry for a Bill Evans Trio recording. 

Bill Evans Trio, How My Heart Sings!, Audio CD (Riverside, 1962), Amazon.com.

 

I can get it to show by doing an all fields search using the title but not using 'zot bill evans trio'

The entry doesn't show at 'bill evans' but possibly there are too many entires at that point. Zothero gives me 11 before going back to the top. I would expect that behaviour.

'zot bill evans t' shows me 7 listings, but not the above mentioned one. 

'zot bill evans tr' gives me no matches found.

 

I'm not sure what the issue is. The 'performer' is a single entry 'Bill Evans Trio'.

 

Furthermore, I have another recording entry by bill evans, this time the performer field is 'Evans, Bill' (Lastname, firstname). If I search by field 'creator' it appears if I search for 'Evans' but not 'Bill Evans' or 'Evans, Bill'. If I change the creator field in zotero to a single field 'Bill Evans' (rather than a two field (Lastname, firstname))  then it appears however I search for it.

 

If you still have a copy of my library you may be able to replicate that.

 

 

I love the two different ways of citation by the way.

Yesterday the italics were showing as html code <i> </i> when I posted in this forum but today when I pasted in this post the title was actually italics.

 

Pasting into textedit or notes (mac) gives me the codes. I would prefer if it was italicized in the paste. I'm sure I can do a keyboard maestro macro that does that easily though. Perhaps I'm missing something?

 

I'll keep looking for issues if that helps.

 

2 minutes ago, lutefish said:

Seriously - is there some way I can compensate you in gratitude? Or, alternately, some way I can contribute to the larger Alfred community, as a thanks? You've made life better for the tiny number of academic users who use both (Alfred, Zotero), and have my profoundest thanks.

 

Seriously, I second that completely. And I think that the number of people who use alfred AND zotero will increase as a result of this workflow. I don't think the number will remain tiny.

I will be buying beers. Today is rent day and 4 days before Christmas but it will be done. I'd also like to help spread the word of this workflow when you are ready for release.

 

 

[deanishe]

8 minutes ago, dfay said:

I'm running into issues with some CSL styles though

 

Yeah. It seems that citeproc doesn't like styles that define an independent-parent (or perhaps styles that only define an independent-parent). The style you're using is, as far as I can tell, empty:

<?xml version="1.0" encoding="utf-8"?>
<style xmlns="http://purl.org/net/xbiblio/csl" version="1.0" default-locale="en-GB">
  <!-- Generated with https://github.com/citation-style-language/utilities/tree/master/generate_dependent_styles/data/tr-law-australia -->
  <info>
    <title>The Tort Law Review</title>
    <title-short>Tort L Rev</title-short>
    <id>http://www.zotero.org/styles/the-tort-law-review</id>
    <link href="http://www.zotero.org/styles/the-tort-law-review" rel="self"/>
    <link href="http://www.zotero.org/styles/thomson-reuters-legal-tax-and-accounting-australia" rel="independent-parent"/>
    <category citation-format="note"/>
    <category field="law"/>
    <issn>1039-3285</issn>
    <updated>2014-05-29T23:47:57+00:00</updated>
    <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights>
  </info>
</style>

What does the parent style (called "Thomson Reuters - Legal, Tax & Accounting Australia") give you? For me, it looks like this:

 

Quote

Aballéa M, “Face à berlin. l’ambassade de france en allemagne. expérience singulière d’une diplomatie en terrain hostile (1871-1933)” (2016) Relations internationales 103, http://www.cairn.info/resume.php?ID_ARTICLE=RI_166_0103 viewed 4 August 2016.

 

 

[dfay]

Two other thoughts on the search by field:

 

1 - zot:creators - it would be helpful but not essential to allow search for author vs. editor 

2 - it would be great to be able to search the abstract field as well