Bookends Tools — A curated toolset for Bookends reference manager

[deanishe]

 

6 minutes ago, iandol said:

I worry about the performance hit as it has to call many applescript events to pull data from Bookends

 

The performance issues are down to the sheer slowness of Apple Events and inter-app communication. If your AppleScript is doing a substantial amount of communication with an application, that's 90+% of the bottleneck right there.

 

If you're trying to talk to an app, the extra overhead of running an AppleScript as a subprocess via Ruby/Python/whatever is entirely trivial compared to the slowness of pulling stuff out of the app via Apple Events.

 

The AppleScript JSON lib you linked to is, AFAIK, absolutely fine. But you're still writing in AppleScript…

 

There are a few loonies who like to program in AppleScript, but the rest of us either call the smallest-possible AppleScripts from another language (and do all the interesting stuff there), or use JXA (JavaScript) instead. JXA supports JSON natively.

 

[swilcox]

Thanks for much for the help on this.

[iandol]

@deanishe — I translated the applescript into Ruby, but there is substantial overhead of firing up osascript to run an apple event. The same query in Applescript takes substantially longer in Ruby (scales with the number of records I need to return)

 

./findReferences.applescript Lamme  0.14s user 0.15s system 66% cpu 0.448 total

./findReferences.rb Lamme  2.67s user 2.19s system 65% cpu 7.436 total

 

This is what takes the time, it is the same loop for each database record in Applescript and Ruby (showing the ruby version):

 

	def getRecords()
		return unless @cansearch
		@list.each_with_index do |uuid, i|
			ti = osascript <<-EOT
			tell application "Bookends"
				return «event RubyRFLD» #{uuid} given string:"title"
			end tell
			EOT
			@title[i] = ti.chomp.strip

			au = osascript <<-EOT
			tell application "Bookends"
				return «event RubyRFLD» #{uuid} given string:"authors"
			end tell
			EOT
			@authors[i] = au.split(',')[0]

			da = osascript <<-EOT
			tell application "Bookends"
				return «event RubyRFLD» #{uuid} given string:"thedate"
			end tell
			EOT
			@date[i] = da.chomp.strip.split(' ')[0]
		end
	end

	# this converts to -e line format so osascript can run, pass in a heredoc
	# beware this splits on \n so can't include them in the applescript itself
	def osascript(script)
		cmd = ['osascript'] + script.split(/\n/).map { |line| ['-e', line] }.flatten
		IO.popen(cmd) { |io| return io.read }
	end

 

I looked at some micro-optimisations (thinking perhaps IO.popen could be improved, see https://gist.github.com/koffeinfrei/3f8b88fb5b80ee665761) but nothing made much of a difference. 

 

Anyway, I can group the database items together to minimise the number of times I call osascript which should be a major win (I can do this for applescript also), but as utterly terrible as Applescript is, for very few specific tasks (like this one that calls many apple events and returns quickly back to Alfred), it still has its place.

 

I really wish Apple took automation seriously enough to have kept rubyOSA, pythonOSA and Javascript moving forwards in the OSA architecture , but these are all in various states of bitrot and abandonment... :-(

 

Edited February 4, 2018 by iandol

[deanishe]

58 minutes ago, iandol said:

but there is substantial overhead of firing up osascript to run an apple event

 

There is if you're running osascript three times for every single item.

 

Smarter to call it once for all the items and print them as tab-delimited lines, or better yet, use JXA instead of AppleScript and output JSON.

 

Edited February 4, 2018 by deanishe

[iandol]

I managed to get my Ruby rewrite nice and fast by optimising the number of times I must call bookends (I use an ASCII code 30 [\u001E] record separator). It now takes around 0.3secs, and it doesn't matter how many results are returned; for example for this large search (returns 471 records) we are clearly faster than before, and the Ruby code is much easier to maintain and modify.

 

./findReferencesTitle.rb "V1"  0.15s user 0.09s system 70% cpu 0.334 total

./findReferencesTitle.applescript "V1"  0.80s user 0.73s system 66% cpu 2.307 total

 

findReferences.rb

 

---------------------------------------------------------------------------

 

I've released a beta version of the workflow with the new faster becite/betitle/bebib tools. I've added multiple author search and an option YEAR to refine the search. So for example [friston hobson 2014] will search for references with authors "Friston" AND "Hobson" published in 2014. If someone tries this can you please let me know if it works for you...

 

bookends-tools-beta.alfredworkflow

 

Edited February 5, 2018 by iandol

[taja]

15 hours ago, iandol said:

I've released a beta version of the workflow with the new faster becite/betitle/bebib tools. I've added multiple author search and an option YEAR to refine the search. So for example [friston hobson 2014] will search for references with authors "Friston" AND "Hobson" published in 2014. If someone tries this can you please let me know if it works for you...

 

Works fantastically well, and faster!

[iandol]

No longer a beta, V1.2.1 released with a new tool beall that searches in any database field (we are now fast so it seems ok). ⇧ pressed for becite/betitle/beall will open ref directly in Bookends...

[iandol]

V1.2.2 released, with only a small fix for the new API of the Scopus Search tool...

[taja]

Thanks again for this fantastic tool. A request: at the moment you can go to the selected reference in Bookends from the results list, and then open its attachment using the relevant keyboard shortcut in Bookends. But would it be possible to open the reference attachment directly from the search results using a modifer key?

Edited April 29, 2018 by taja

[iandol]

On 4/29/2018 at 6:01 PM, taja said:

Thanks again for this fantastic tool. A request: at the moment you can go to the selected reference in Bookends from the results list, and then open its attachment using the relevant keyboard shortcut in Bookends. But would it be possible to open the reference attachment directly from the search results using a modifer key?

 

Hi @taja, do you normally use ⌘⇧O to do this? I tried to make it automatic (use ⇧ and ref opens and attachment if present otherwise nothing) but the problem is if there is no attachment it opens an attachment from the previously selected ref that does have an attachment even though a reference without an attachment is selected... I added it using another modifier [fn] — can you test if V1.2.3 does what you want it to? I've now used up all available modifiers (⌘, ⌥, ⌃, ⇧ and [fn])!

[taja]

Works great, thank you!

[iandol]

Just released V1.2.4 — becite/betitle/beall now shows if a reference has an attachment, and for Bookends 13.x users use the new applescript JSON events that are slightly more efficient.

[Cassady]

This workflow is ridiculously helpful - thank you for all the time and effort put into creating (and updating) it. Makes the interaction between Bookends and Scrivener seamless.

[iandol]

I've just released V1.2.5, which presents the first AND last author names with initials to provide a bit more context for the search results.

 

And I realised that I can get Quicklook for attachments working in Alfred (thanks to input from vitor and deanishe). This is really cool: so you search, focus a results item with an attachment (denoted by the ?) then press [shift] or [⌘][Y] and you can quicklook the PDF directly without Alfred losing focus so you can check the paper before inserting the citation.

 

https://github.com/iandol/bookends-tools/releases/tag/V1.2.5 — or wait for OneUpdater to update for you.

[Cassady]

@iandol Have you seen Jon's offer for a beta-test in regards to the new floating window, over at the SonnySoftware forum? Been playing around with it, hope you will share your thoughts! 

[iandol]

Haven't had a chance to play with it yet, but hopefully I'll get some time this week!!! ?

[taja]

1.2.5 doesn’t work for me for some reason: searches don't come up with any results. Odd because 1.2.4 (which I have reverted to) works fine. Any thoughts on what might be happening?

[deanishe]

@taja: "doesn't work" isn't an actionable description of an issue. Could you provide more information, as described in the "reporting problems" thread?

[taja]

Sorry - I'm quite new to Alfred. Here's what the workflow debugger threw up. Apologies if this isn't the right thing to be providing here, but this is what comes up  in the debugger (with 'interesting information' only selected) when I use (1) beall and (2) betitle.

 

(1)
Starting debug for 'Bookends Tools'

[2018-07-10 11:04:07][ERROR: input.scriptfilter] Code 1: /Users/Was/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.C38A9670-F1D2-4A70-9F49-35850C922B4F/findReferencesAll.rb:245:in `<main>': uninitialized constant FindReferences (NameError)
Did you mean?  FindReferencesAll

(2)
[2018-07-10 11:05:15][ERROR: input.scriptfilter] Code 1: /Users/Was/Library/Application Support/Alfred 3/Alfred.alfredpreferences/workflows/user.workflow.C38A9670-F1D2-4A70-9F49-35850C922B4F/findReferencesTitle.rb:245:in `<main>': uninitialized constant FindReferences (NameError)
Did you mean?  FindReferencesTitle

[Cassady]

Hoping someone who has the OneUpdater script attached to the BE workflow, will be kind enough to check whether I have the URL settings correct?

 

I attached the OneUpadter script to my most frequently used BE trigger, and set the update frequency to "0" while testing, but it is still running v1.2.4, as opposed to having updated to the newest version.

 

Am I pointing OneUpdater at the right place?

 

# THESE VARIABLES MUST BE SET. SEE THE ONEUPDATER README FOR AN EXPLANATION OF EACH.
readonly remote_info_plist="https://raw.githubusercontent.com/iandol/bookends-tools/master/source/info.plist"
readonly workflow_url="https://github.com/iandol/bookends-tools/blob/master/bookends-tools.alfredworkflow"
readonly download_type='direct'
readonly frequency_check='0'

 

[iandol]

Hi taja, OK that is my fault, I'll make an updated V1.2.5 soon.

 

EDIT: V1.2.6 is released! https://github.com/iandol/bookends-tools/releases 

 

Cassady: actually I've never tested OneUpdater, so possibly I'm doing something wrong. @vitor — do these setting look OK (I normally set frequency to 4)?

Edited July 11, 2018 by iandol

[vitor]

7 hours ago, iandol said:

@vitor — do these setting look OK (I normally set frequency to 4)?

 

The settings on the .alfredworkflow of the latest release look correct, yes.

 

22 hours ago, Cassady said:

Am I pointing OneUpdater at the right place?

# THESE VARIABLES MUST BE SET. SEE THE ONEUPDATER README FOR AN EXPLANATION OF EACH.
readonly remote_info_plist="https://raw.githubusercontent.com/iandol/bookends-tools/master/source/info.plist"
readonly workflow_url="https://github.com/iandol/bookends-tools/blob/master/bookends-tools.alfredworkflow"
readonly download_type='direct'
readonly frequency_check='0'

 

 

You are not. You are pointing workflow_url at a webpage, not a file. The correct URL would be https://raw.githubusercontent.com/iandol/bookends-tools/master/bookends-tools.alfredworkflow.

[taja]

@iandol no fault about it!  And I'm very glad to know how easy it is to debug a workflow using Alfred. Thank you for fixing it so quickly!

[Cassady]

2 hours ago, vitor said:

 

The settings on the .alfredworkflow of the latest release look correct, yes.

 

 

You are not. You are pointing workflow_url at a webpage, not a file. The correct URL would be https://raw.githubusercontent.com/iandol/bookends-tools/master/bookends-tools.alfredworkflow.

 

Appreciate the reply and assistance @vitor – I have made the change, wouldn't have spotted that.

 

I just want to be clear – so I copy the appropriate Run Script node, paste it into the BE workflow, and link the node to my most frequently used BE workflow 'action'.

Once the URLs are then in place, is the idea that upon me firing the BE trigger a few times, the Script will periodically check the GitHub repository of the BE workflow, and if there is a newer version available, install it in the background? Or will I be prompted? 

 

[vitor]

27 minutes ago, Cassady said:

I just want to be clear – so I copy the appropriate Run Script node, paste it into the BE workflow, and link the node to my most frequently used BE workflow 'action'.

 

Yes.

 

27 minutes ago, Cassady said:

Once the URLs are then in place, is the idea that upon me firing the BE trigger a few times, the Script will periodically check the GitHub repository of the BE workflow

 

Every time it fires, it’ll check how long it was since the last check. If it has been longer than the defined number of days, it’ll check online to see if there’s a new version and reset the clock.

 

29 minutes ago, Cassady said:

if there is a newer version available, install it in the background? Or will I be prompted? 

 

It will download the new version in the background, but it will warn you with a notification. After it downloads the update it’ll prompt you to install, so you can accept or refuse. The downloaded Workflow will be in your Downloads directory.