ZotHero — Generate Zotero citations in Alfred

[justice]

Like @damagekko I also signed up because this workflow is vital to my academic work. I get the very same error about 'pasteboard'. While it was working before with the updated version, in 12.3.1. the workflow does not seem to paste the results to the clipboard (loading data, browsing all literature works, however). Is it likely that this is something that could be fixed? 

[damagekko]

On 4/29/2022 at 7:44 PM, justice said:

Like @damagekko I also signed up because this workflow is vital to my academic work. I get the very same error about 'pasteboard'. While it was working before with the updated version, in 12.3.1. the workflow does not seem to paste the results to the clipboard (loading data, browsing all literature works, however). Is it likely that this is something that could be fixed? 

 

Not fixable, I guess the author would need to rewrite the whole workflow. Shame it's completely abandoned.

 

Honestly though I see the workflows being essential part of the Alfred's ecosystem and them not working is slowly driving me away from Alfred to elsewhere...

[vitor]

On 4/29/2022 at 5:44 PM, justice said:

Is it likely that this is something that could be fixed?

 

If you are on Intel, "${HOME}/.pyenv/versions/2.7.18/bin/python2.7" -m pip install PyObjC on a Terminal may suffice. No one has ever given me confirmation either way on that solution.

 

1 hour ago, damagekko said:

Not fixable, I guess the author would need to rewrite the whole workflow.

 

That is incorrect. As noted by @justice, everything but the pasting works. The crux of the issue is a single file, lib/pasteboard.py, which tries to use the Python Objective-C bridge to store multiple data formats to the clipboard at once. There are a few ways to go about fixing it, from rewriting that file (and the code which calls it, which is short), to only caring about plain text, to getting PyObjC to install on Apple Silicon.

[Sorehead]

I've worked in academia for several years and just discovered this workflow - it's incredible. Thanks.

[justice]

On 5/11/2022 at 3:58 PM, vitor said:

No one has ever given me confirmation either way on that solution.

The workflow was working in Monterey on M1 (and I'm sure I installed Python via that terminal command), but 12.3.1 broke it for me, unfortunately. If the code "is short", maybe someone with the knowledge of these things will find a solution. Here's to hoping! ☺️

[giovanni]

although I switched years ago to Paperpile (for which I created a workflow) ZotHero is a great workflow and I'd like to try and help migrate it to Python3, so that fellow academics can continue to use it.

 

However I have not used ZotHero in the past and I would need some help troubleshooting, as for some features I don't know how they used to work. For example, when copying a citation with a numeric format, e.g. a superscript number, I just get a superscript 1, is that the intended behavior? 

 

A first version is here. I was able to import a ~6k Zotero library, and run all the other major functions, but again I am not familiar with all of them. I changed the bundle-id for now so it doesn't interfere with existing installations.  

 

If you have the patience and time (@damagekko, @lutefish, @justice?) I would appreciate some help with troubleshooting!

 

[lutefish]

@giovanni If I get a chance, I'll take a look, but I've mostly moved over to Raycast now (particularly given the just-released Zotero extension that works on the local database, rather than using the Web API).

 

But I appreciate the labor and the community that keeps projects like this alive.

[lutefish]

Installed it, but it's not returning any results, and getting stuck in a loop, displaying "Initializing Search Index".

 

Forcing a zotconf reload doesn't resolve the issue.

 

11:13:21 workflow.py:2093 DEBUG    ---------- ZotHero (1.99.0) ----------

...

11:13:21 index.py:153 DEBUG    [index] opened '~/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/giovanni-zothero/search.sqlite'

11:13:21 background.py:233 DEBUG    [update] command cached: /Users/matthew/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/giovanni-zothero/update.argcache

11:13:21 background.py:237 DEBUG    [update] passing job to background runner: ['/Library/Developer/CommandLineTools/usr/bin/python3', '-m', 'workflow.background', 'update']

.

11:13:21 workflow.py:2093 DEBUG    ---------- ZotHero (1.99.0) ----------

11:13:21 background.py:243 DEBUG    [update] background job started

11:13:21 zh.py:133 DEBUG    RUNNING BACKGROUND

11:13:21 core.py:197 INFO     [core] searching for "mannyng" ...

11:13:21 index.py:245 INFO     [index] 0 result(s) for 'mannyng'

11:13:22 workflow.py:2314 DEBUG    set last run version: 1.99.0

11:13:22 workflow.py:2136 DEBUG    ---------- finished in 0.138s ----------

[11:13:22.090] ZotHero[Script Filter] {

  "items": [

    {

      "title": "Initialising Search Index \u2026",

      "subtitle": "Your results will appear momentarily",

      "valid": false,

      "icon": {

        "path": "icons/reload.png"

      }

    }

  ],

  "rerun": 0.2

}

[giovanni]

thanks @lutefish how big is your library? It does take a bit to build and I don't see errors (yet). 

[lutefish]

Huh. Only 3500 items. I'll give it a longer chance.

[lutefish]

Letting it run for several minutes didn't resolve...

[lutefish]

Ah, here's the error: 11:29:45 workflow.py:2114 ERROR    [Errno 2] No such file or directory: 'zh.py'

 

(I also tried to change the default style, and it didn't "take" the change until I had run it several times.

[giovanni]

thank you @lutefish! The good news: no errors (yet!) from the migrated Workflow library, which (for me!) is huge progress.

 

I also noticed that the update of default style was not immediate (but I could see it updated the next time I ran the workflow). Was that not the case before? 

 

Also, do you happen to remember whether styles with numeric citations only showed `1` if you select them and then hit 'copy citation'? 

 

bad news: it's not working . Do you know how to access the cache folder? It should be in ~/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/giovanni-zothero  and it should contain a log file. Could you share that? Or, you can choose "Open Log File" from the configuration dialog. 

 

thanks for your help!

 

 

 

 

[lutefish]

Hmm. Can't attach text files. The log in the cache directory you specificy seems to repeat the same error:

 

11:28:18 workflow.py:2114 ERROR    [Errno 2] No such file or directory: 'zh.py'
Traceback (most recent call last):
  File "/Users/USERNAMEREDACTED/Box Sync/AlfredSync/Alfred.alfredpreferences/workflows/user.workflow.FC919AE5-620A-46E9-9D2D-D8DBD4F2580A/lib/workflow/workflow.py", line 2107, in run
    func(self)
  File "/Users/USERNAMEREDACTED/Box Sync/AlfredSync/Alfred.alfredpreferences/workflows/user.workflow.FC919AE5-620A-46E9-9D2D-D8DBD4F2580A/lib/workflow/background.py", line 282, in main
    retcode = subprocess.call(args, **kwargs)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 340, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'zh.py'
11:28:18 core.py:197 INFO     [core] searching for "mannyng" ...

[giovanni]

this must be related to different python installations because it works well for me.

Let me investigate by trying on other machines. 

[giovanni]

@lutefish could you give it a try now? Changing style will give you an error, but should still change it. I was hoping you could check if the database gets created before I try to fix the rest. thanks!

[lutefish]

First I got a "database locked" error, so I closed Zotero. Now I'm getting a "name 'unicode' is not defined" error.

 

16:07:51 workflow.py:2114 ERROR    name 'unicode' is not defined
Traceback (most recent call last):
  File "lib/workflow/workflow.py", line 2107, in run
    func(self)
  File "zh.py", line 721, in main
    return do_search(query)
  File "zh.py", line 131, in do_search
    if app.stale and not running:
  File "lib/zothero/core.py", line 159, in stale
    if self.index.empty:
  File "lib/zothero/core.py", line 152, in index
    self._index.update(self.zotero)
  File "lib/zothero/index.py", line 277, in update
    if not self._update(zot, force):
  File "lib/zothero/index.py", line 325, in _update
    for e in it:
  File "lib/zothero/zotero.py", line 268, in all_entries
    yield self._load_entry(row)
  File "lib/zothero/zotero.py", line 320, in _load_entry
    e.citekey = self.bbt.citekey('{}_{}'.format(e.library, e.key))
  File "lib/zothero/zotero.py", line 195, in bbt
    self._bbt = BetterBibTex(self.datadir)
  File "lib/zothero/betterbibtex.py", line 50, in __init__
    self._refkeys = {
  File "lib/zothero/betterbibtex.py", line 51, in <dictcomp>
    unicode(ck['libraryID']) + '_' + ck['itemKey']: ck['citekey']
NameError: name 'unicode' is not defined
16:07:51 workflow.py:2116 INFO     for assistance, see: https://github.com/deanishe/zothero/issues
16:07:51 workflow.py:2136 DEBUG    ---------- finished in 0.185s ----------
[16:07:51.785] ZotHero[Script Filter] {
  "items": [

[giovanni]

thanks @lutefish this is helpful. I don't want to waste more of your time, I added Vitor's OneUpdater, so it should update automatically with new versions, please feel free to try again whenever you have time. 

[giovanni]

most recent version is here cc @lutefish

Here is the current status of the pasteboard  issue (summarized below by @vitor)

• it works with minor changes (e.g. in my version v1.99.2) but the user needs to install the PyObjC library
• bundling the whole PyObjC library with the Workflow causes an error that I could not figure out: ImportError: cannot import name '_objc' from partially initialized module 'objc ' (most likely due to a circular import) (lib/objc/__init__.py)
• I could not bundle (or install) AppKit only: Building wheel for pycairo (pyproject.toml) did not run successfully.
• Zothero is currently copying to the clipboard the RTF citation, which I think is useful to most users. It is pasted as plain text in some applications (e.g. TextEdit) but not others (e.g. Visual Studio Code). I could not figure out how to copy to both plain text and RTF clipboard in Python without using PyObjC, it might be possible in Applescript. Perhaps we can add a setting so that the user can choose the type of text to copy to the clipboard.

 

 

 

[vitor]

On 5/21/2022 at 4:49 PM, giovanni said:

it might be possible in Applescript

 

It is (JXA).

[justice]

@giovanni, thank you for putting your time into this 👍. I do not have any knowledge of programming, but if it helps, here is what your latest updated workflow (1.99.3) on M1, 12.4 tells me (I can no longer search the database):

 

 

From the debugging console (just from where the error occurs -- I don't know if this is of any help):

Quote

17:17:34 workflow.py:2114 ERROR    __str__ returned non-string (type NoneType)
Traceback (most recent call last):
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/workflow/workflow.py", line 2107, in run
    func(self)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/zh.py", line 736, in main
    return do_search(query)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/zh.py", line 133, in do_search
    if app.stale and not running:
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/core.py", line 159, in stale
    if self.index.empty:
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/core.py", line 152, in index
    self._index.update(self.zotero)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/index.py", line 277, in update
    if not self._update(zot, force):
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/index.py", line 325, in _update
    for e in it:
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/zotero.py", line 268, in all_entries
    yield self._load_entry(row)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/zotero.py", line 316, in _load_entry
    e.notes = self._entry_notes(e.id)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/zotero.py", line 366, in _entry_notes
    return [strip_tags(row['note']) for row in rows]
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/zotero.py", line 366, in <listcomp>
    return [strip_tags(row['note']) for row in rows]
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/util.py", line 131, in strip_tags
    return HTMLText.strip(html)
  File "/Users/XXX/Library/Application Support/Alfred/Alfred.alfredpreferences/workflows/user.workflow.8DDC5A16-0398-43FC-AA23-1FF4FA425EB7/lib/zothero/util.py", line 91, in strip
    return str(p)
TypeError: __str__ returned non-string (type NoneType)
17:17:34 workflow.py:2116 INFO     for assistance, see: https://github.com/giovannicoppola/zothero/issues
17:17:34 workflow.py:2136 DEBUG    ---------- finished in 0.034s ----------

 

[giovanni]

hi @justicethanks for the feedback. did it work before and then stopped, or it never worked? 

[justice]

2 minutes ago, giovanni said:

hi @justicethanks for the feedback. did it work before and then stopped, or it never worked? 

It was working at one point, then it stopped pasting to clipboard (I could still browse everything, see my earlier post). Seeing that you took matters in your own hands, I tried the latest version you updated, which gives me the above error.

[giovanni]

would you be able to export and share your library or a small part of it so I can troubleshoot? I stopped using Zotero years ago and my library even if big might not be using all the features (eg. tags?) which seem to be the problem here. When I first imported my collection Zothero was tripped by some duplicate records, but I assumed it was a problem specific to my library. 

[giovanni]

On 5/23/2022 at 2:29 PM, vitor said:

It is (JXA).

thanks @vitor for pointing me to this function. I implemented the multiple clipboard formats in the most recent version. 

 

The remaining issue I am aware of is the one from html.parser raised by @justice and a github user, but I don't see it with my library and I am not sure how to troubleshoot. If somebody with this problem is willing to work with me I think we can figure it out. 

 

Edited June 14, 2022 by giovanni