BibQuery: Search BibDesk from Alfred

[smarg19]

BibQuery
Search BibDesk from Alfred
Version: 1.0.1
Download from Packal

BibQuery is essentially a visual clone of ZotQuery for the Mac app BibDesk, which is a citation manager for BibTeX. Users of BibDesk can now enjoy the clean search interface found in ZotQuery, which clear icons for publication type, and clean presentation of publication data.



Users can also search their .bib databases with the same variety of queries:

• general (keywords: bib or b)
• titles (keywords: bib:t or bt)
• creators (keywords: bib:a or ba)
• in-keyword (keywords: bib:nk or bnk)
• in-group (keywords: bib:ng or bng)
• for keyword (keywords: bib:k or bk)
• for group (keywords: bib:g or bg)

Also of note, BibQuery works without BibDesk being open and even functions if you have multiple .bib databases that BibDesk manages. In short, BibQuery brings all of your citations to you.

Once you find the item you're looking for, BibQuery currently has 3 possible actions:

• you can open up BibDesk to that item (simply press return)
• you can copy a LaTeX cite command for that item (simply press control+return)
• you can open that item's PDF attachment, if it has one (simply press shift+return)

For those of you interested, the source code can be found on my GitHub.

Enjoy!
stephen

[schoeps]

Hi stephen, 

 

first, your workflow looks great and judging on its complexity it seems that it requires more time than I can afford for bibsearch. Hence I am very pragmatic and once your feature set is greater than bibsearch's, I will most probably stop maintaining bibsearch. In particular because my time does hardly allow it.

 

Let me give some general remarks:

• it would be nice to display the bibtype because there might be preprints and papers with the same name, date and authors in your database
• users asked me to display the year before the authors list because it might be very long (e.g. in medicine or physics, ..) and then the date information is not shown
• I think for many users it is convenient to copy a complete citation string of a publication, e.g. if they have to work with MS Word and they want to add a single reference in some footnote. This is ongoing work in bibsearch. However, you can take whatever I have added there.

 

However, I tried it today and I got the following error 

Starting debug for 'BibQuery'


[ERROR: alfred.workflow.input.scriptfilter] Code 1: 09:44:22 workflow.py:1199 ERROR    Bad file header
Traceback (most recent call last):
  File "/Users/xxx/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.CF861A51-8CB7-4408-93CA-D51684C77953/workflow/workflow.py", line 1197, in run
    func(self)
  File "bibquery.py", line 442, in main
    filter(queries, scope, wf)
  File "bibquery.py", line 420, in filter
    simple_filter(query, scope, wf)
  File "bibquery.py", line 310, in simple_filter
    data = read_cachedir()
  File "bibquery.py", line 83, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/xxx/Library/Application Support/Alfred 2/Alfred.alfredpreferences/workflows/user.workflow.CF861A51-8CB7-4408-93CA-D51684C77953/ccl_bplist.py", line 258, in load
    raise BplistError("Bad file header")
BplistError: Bad file header

[smarg19]

Thanks for the kind words and the suggestions. As to the suggestions specifically:

• what do you mean by "bibtype"? If you mean publication type, that's what the different icons represent.
  

• I might add a config file, but I don't plan on changing the default. It's simple enough to alter in the code anyway.
  

• I'm working on a number of things in this regard currently...
  

 

As to the error, there appears to be something wrong with your BibDesk cache files. They ought to be byte plist files. The error comes from the byte plist parser. I'm not certain what the problem is yet, I just got that error, but then it went away (all I did was change one thing in BibDesk then save). I will investigate.

Edited May 8, 2014 by smarg19

[dfay]

Hi Stephen....nice to see your automation work continuing (I had given you some feedback on the Skim / Markdown citation scripts last fall).

 

Unfortunately I am also getting errors with this - as soon as I start typing a query, I get the message 

 

Error in workflow 'com.hackademic.bibquery'

days=-1259767745; must have magnitude <= 999999999

 

Here is the full error log from Alfred - I didn't get a chance to look much into it but it looks like the error is coming from one of the libraries you're using, not your code itself.

[ERROR: alfred.workflow.input.scriptfilter] Code 1: 15:06:18 workflow.py:1199 ERROR    days=-1259767745; must have magnitude <= 999999999
Traceback (most recent call last):
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/workflow/workflow.py", line 1197, in run
    func(self)
  File "bibquery.py", line 442, in main
    filter(queries, scope, wf)
  File "bibquery.py", line 420, in filter
    simple_filter(query, scope, wf)
  File "bibquery.py", line 310, in simple_filter
    data = read_cachedir()
  File "bibquery.py", line 83, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 271, in load
    return __decode_object(f, offset_table[top_level_object_index], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 245, in __decode_object
    val = __decode_object(f, offset_table[value_refs[i]], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 126, in __decode_object
    return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)
OverflowError: days=-1259767745; must have magnitude <= 999999999
[ERROR: alfred.workflow.input.scriptfilter] Code 1: 15:06:19 workflow.py:1199 ERROR    days=-1259767745; must have magnitude <= 999999999
Traceback (most recent call last):
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/workflow/workflow.py", line 1197, in run
    func(self)
  File "bibquery.py", line 442, in main
    filter(queries, scope, wf)
  File "bibquery.py", line 420, in filter
    simple_filter(query, scope, wf)
  File "bibquery.py", line 310, in simple_filter
    data = read_cachedir()
  File "bibquery.py", line 83, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 271, in load
    return __decode_object(f, offset_table[top_level_object_index], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 245, in __decode_object
    val = __decode_object(f, offset_table[value_refs[i]], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 126, in __decode_object
    return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)
OverflowError: days=-1259767745; must have magnitude <= 999999999
[ERROR: alfred.workflow.input.scriptfilter] Code 1: 15:06:19 workflow.py:1199 ERROR    days=-1259767745; must have magnitude <= 999999999
Traceback (most recent call last):
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/workflow/workflow.py", line 1197, in run
    func(self)
  File "bibquery.py", line 442, in main
    filter(queries, scope, wf)
  File "bibquery.py", line 420, in filter
    simple_filter(query, scope, wf)
  File "bibquery.py", line 310, in simple_filter
    data = read_cachedir()
  File "bibquery.py", line 83, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 271, in load
    return __decode_object(f, offset_table[top_level_object_index], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 245, in __decode_object
    val = __decode_object(f, offset_table[value_refs[i]], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 126, in __decode_object
    return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)
OverflowError: days=-1259767745; must have magnitude <= 999999999
[ERROR: alfred.workflow.input.scriptfilter] Code 1: 15:06:20 workflow.py:1199 ERROR    days=-1259767745; must have magnitude <= 999999999
Traceback (most recent call last):
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/workflow/workflow.py", line 1197, in run
    func(self)
  File "bibquery.py", line 442, in main
    filter(queries, scope, wf)
  File "bibquery.py", line 420, in filter
    simple_filter(query, scope, wf)
  File "bibquery.py", line 310, in simple_filter
    data = read_cachedir()
  File "bibquery.py", line 83, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 271, in load
    return __decode_object(f, offset_table[top_level_object_index], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 245, in __decode_object
    val = __decode_object(f, offset_table[value_refs[i]], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/alfred/alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 126, in __decode_object
    return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)
OverflowError: days=-1259767745; must have magnitude <= 999999999

[schoeps]

As to the error, there appears to be something wrong with your BibDesk cache files. They ought to be byte plist files. The error comes from the byte plist parser. I'm not certain what the problem is yet, I just got that error, but then it went away (all I did was change one thing in BibDesk then save). I will investigate.

Thanks. I deleted my bibdesk cache and recreated it; now it's working. Your parsing seems to be more sensible than my approach (but it seems to be faster too...).  

 

what do you mean by "bibtype"? If you mean publication type, that's what the different icons represent.

 

 

 

the icons are really nice and some meanings are obvious but some are not so clear. Furthermore there are probably more types than icons ("electronic", "misc" and unpublished",...). I don't know about others but I use all of them. However it's probably easy to modify your workflow to do that.

 

 

I might add a config file, but I don't plan on changing the default. It's simple enough to alter in the code anyway.

yes, see above.

 

 

I'm working on a number of things in this regard currently...

with bibdesk it's not too difficult as you might have seen. Alternatives are other external parsing engines (bibtex2html...) but this seemed to be overkill for me.

[smarg19]

 

Hi Stephen....nice to see your automation work continuing (I had given you some feedback on the Skim / Markdown citation scripts last fall).

 

Unfortunately I am also getting errors with this - as soon as I start typing a query, I get the message 

 

Error in workflow 'com.hackademic.bibquery'

days=-1259767745; must have magnitude <= 999999999

 

Here is the full error log from Alfred - I didn't get a chance to look much into it but it looks like the error is coming from one of the libraries you're using, not your code itself.

 

I'm honestly not sure what the problem here might be, but there are two possible solutions. The first is to update your BibDesk cache. I would try that first. Other than that, I will need to look into perhaps another bplist parser (tho the problem may be in the BibDesk plist for all I know)...

 

Sorry for the half-guesses.

[Januz]

Thanks for the workflow!

 

I was previously using Sebastian's workflow, and I still like some things better (some of which Sebastian mentioned):

 

- I think it would be better not to copy a LaTeX cite-command, but only the cite key as one might want to use a different cite command (citep, parencite, textcite, etc.) or one might want to add several cite keys into the same citation [e.g., xxxx (Smith et al, 2000; Frost, 2001)]

- I was one of the users requesting that the year should be in front, because I (and most natural scientists) often have citations including several authors

 

Generally:

 

- it would be great to be able to select which attachment to open if there exist more than one

 

Thanks!

[smarg19]

- I think it would be better not to copy a LaTeX cite-command, but only the cite key as one might want to use a different cite command (citep, parencite, textcite, etc.) or one might want to add several cite keys into the same citation [e.g., xxxx (Smith et al, 2000; Frost, 2001)]

It is an unfortunate aspect of Alfred workflows that you can't really create a Settings menu to allow for fine-grained control of the setup. But you can jump in and alter any of the code, so I will walk you thru that. In order to alter the behavior of the cite action, you will need to open the actions.py script. Around line 78, you should find this function:

def export_cite_command(cite_key):
"""Return LaTeX cite command"""
cmd = "\\cite{{{0}}}".format(cite_key)
set_clipboard(cmd)
return "Cite Command"

To achieve your desired behavior (merely exporting the cite key), delete the cmd variable line and change the next line to set_clipboard(cite_key). So, the new function would look like this:

def export_cite_command(cite_key):
"""Return cite key"""
set_clipboard(cite_key)
return "Cite Key"

- I was one of the users requesting that the year should be in front, because I (and most natural scientists) often have citations including several authors

In order to change the order of creators and date, you will need to open the bibquery.py file. Near the bottom is a function called info_format. At the bottom of this function (should be around line 230), you will find what it returns: return [creator_ref, date_final, title_final]. To get your desired result, simply change this to: return [date_final, creator_ref, title_final]. This will ensure that the date is always put first in the results sub-title.

- it would be great to be able to select which attachment to open if there exist more than one.

This is a feature that I am still working on. Unfortunately, it's not simple to implement. I'm still trying to figure out the simplest UI to achieve this. Sorry that it's not done yet.

Hope this helps some tho,

stephen

[Januz]

Hey Stephen,

 

thank you very much for your instructions and all your great work! Much appreciated!

 

Best,

 

Janosch

[Dellu]

Brilliant. thank you very much

[schoeps]

I had problems with opening attachments on Yosemite. However, it could be easily fixed by replacing line 96 of "actions.py" by 

_file = clean_file.replace('file://', '')

Maybe this helps someone...

[smarg19]

Could you post the before and after of line 96, so that I can see exactly what needs to be changed, and hopefully can see why.

[schoeps]

Could you post the before and after of line 96, so that I can see exactly what needs to be changed, and hopefully can see why.

                 

Sure. I guess Yosemite stores paths differently....

if 'pdf' in source:    
    if not 'http' in source:
        clean_file = urllib.unquote(source)
        _file = clean_file.replace('file://', '')
        if os.path.isfile(_file):
            subprocess.Popen(
                ['open', _file], 
                shell=False, 
                stdout=subprocess.PIPE)

Edited October 19, 2014 by Sebastian

[dfay]

By the way, I figured out that weird error I had above....I had a pub with year set (by some database's downloaded BibTeX) to 20070904 instead of 2007 

[dfay]

Ok....another one, in case this is useful for someone else.  The workflow stopped working again and was throwing an error after flashing the first few results:

 

PCDATA invalid Char value 28

 

This is caused by an invisible character which is legal in BibTeX but not in XML .  I found it by opening the XML in TextWrangler and searching for \x1C (hexidecimal equivalent of 28), and deleting the character from the corresponding BibDesk entry.

 

Now the workflow works again.

 

Again, downloaded BibTeX has all kinds of dirt scattered in it.

 

Perhaps this would be a nice addition to the workflow:

 

http://lsimons.wordpress.com/2011/03/17/stripping-illegal-characters-out-of-xml-in-python/

Edited November 11, 2014 by dfay

[smarg19]

@dfay, these are great notices. If you've altered your version of the workflow's code, you could throw a GitHub pull request and I could easily add it. Right now, I'm working on my rewrite of ZotQuery, so I can't fiddle with this right now, but I will eventually get back to it.

[dfay]

I haven't changed the code in the workflow at all -- just cleaned up my .bib files.  If I get a chance over the weekend I'll give it a shot - it's pretty obvious from your well-documented code how to hook the XML cleaner into bibquery.py .  

[dfay]

Well....I've ended up switching back to Sebastian's BibDesk search workflow....BibQuery was just running too slow with my biblio of 5000+ entries...I could start BibDesk and run the search there in the time it was taking to give me results.  I'm sure this is the price of working in python....

[smarg19]

TBH, I haven't looked at this code in a long time, but the speed is almost certainly not a Python issue. It's likely a toolchain issue. I believe that each time, the workflow reads BibDesks cache files. It would probably be better to have a background process that updates a SQLite database against the cache files, and then the workflow itself reads and queries against the db.

Poke me sometime during the summer and I may be able to do that.

[rory.mullen]

Hi smarg19, first thanks so much for the workflow, I use it often. I just updated my BibDesk installation to version 1.6.5, and now when I search my library with your workflow, I am no longer able to use the cmd+return shortcut to open files attached to BibDesk records. Any idea what might be causing this?

 

Thanks in advance!

[dfay]

Hey....I don't know that smarg19 is around much these days but I don't see anything in the 1.6.5 release notes that should affect this -- unless you have turned on the new

• Hidden preference to save only the relative path for linked files

 

But...after a bit of testing...the other functions of the workflow work but this one doesn't work for me either.  Here's why:

 

The open_attachment function is the only one that calls read_cachedir() which in turn calls ccl_bplist.load, which fails.

 

Here's the full log:

18:06:42 workflow.py:1199 ERROR    date value out of range
Traceback (most recent call last):
  File "/Users/username/Dropbox/Sync/Alfred/Alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/workflow/workflow.py", line 1197, in run
    func(self)
  File "actions.py", line 168, in main
    print act(cite_key, action, wf)
  File "actions.py", line 143, in act
    open_attachment(cite_key)
  File "actions.py", line 86, in open_attachment
    data = read_cachedir()
  File "actions.py", line 69, in read_cachedir
    bib_data = ccl_bplist.load(_file)
  File "/Users/username/Dropbox/Sync/Alfred/Alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 271, in load
    return __decode_object(f, offset_table[top_level_object_index], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/Alfred/Alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 245, in __decode_object
    val = __decode_object(f, offset_table[value_refs[i]], collection_offset_size, offset_table)
  File "/Users/username/Dropbox/Sync/Alfred/Alfred.alfredpreferences/workflows/user.workflow.34B64201-8299-4E7C-85A1-F7736E15E97E/ccl_bplist.py", line 126, in __decode_object
    return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)
OverflowError: date value out of range

If you look at my post above, I had a similar error related to bad dates in imported BibTeX -- maybe you imported some problematic BibTeX (of which there is plenty available....) which is causing the crash?

[Alexander_G]

I had the same problem with the workflow. It stopped working once I have added an almost empty Bibtex entry in Bibdesk (just attached the pdf without filling in the fields). Since I do that quite often I thought it would be a good idea to make the workflow a bit more failsafe by giving the date value a standard value of 0 if none exists (ccl_bplist.py, line 123--127):

 

elif type_byte & 0xFF == 0x33: # Date   0011 0011
        date_bytes = f.read(8)
        date_value = __decode_float(date_bytes)
        if date_value < 0: date_value = 0
        return datetime.datetime(2001,1,1) + datetime.timedelta(seconds = date_value)