Jump to content
robwalton

Ulysses workflow

Recommended Posts

It is a URL callback and that looks like a good library, thanks. I'll give it a go. If it worked it would bypass a lot of cruft (I have an old school fifo pipe in the mix too)---it depends, I guess, on wether ulysses is happy sending its callback's to an http server rather than a locally registered url application scheme tied by OSX to a specific application (described here).

Share this post


Link to post

Wow this is looking great.  I was writing some AppleScript to clean up a bunch of Ulysses sheets (by adding BibDesk keywords as native Ulysses keywords rather than hashtags, now that Ulysses keywords are exposed through the new URL scheme update) and found myself thinking someone should write a nice tidy API for the URL scheme....

 

One quirk I've noticed with the Ulysses Alfred workflow -- even though i have "On My Mac" turned off in Ulysses prefs., and all my sheets are in iCloud, the subtitle for the workflow always shows the path as /On My Mac/correct path from here on out .  Everything else works, it just says my stuff is On My Mac instead of in iCloud.

 

 

 

 

Share this post


Link to post
13 hours ago, robwalton said:

It is a URL callback and that looks like a good library, thanks. I'll give it a go. If it worked it would bypass a lot of cruft (I have an old school fifo pipe in the mix too)---it depends, I guess, on wether ulysses is happy sending its callback's to an http server rather than a locally registered url application scheme tied by OSX to a specific application (described here).

 

On iOS at least, http URLs work fine with x-success. Theoretically, applications shouldn't care what the URL scheme is, as they just pass it off to the system to open.

Share this post


Link to post
18 hours ago, robwalton said:

I've uploaded a new version 1.0 to packal and github. Big improvements are the addition of uf find command to search inside content and the introduction of fuzzy searching (from @deanishe's workflow) to other search commands. Don't know I lived without those two @katie. Also updates Alfred-Workflow to fix hanging in Sierra and add's @dfay's code for opening or importing files--no appending yet.

 

 

It's amazing!! Works great!! Thank you so much!! :D 

Share this post


Link to post

Is there any chance you could add a keyboard modifier to uf to copy the link to a sheet instead of opening the sheet?  Ulysses lacks an easy way to create links between sheets (the one thing I miss from abandoning NVAlt....) and this would be a great way to fill that gap. 

Share this post


Link to post
On 11/04/2017 at 1:53 PM, deanishe said:

 

On iOS at least, http URLs work fine with x-success. Theoretically, applications shouldn't care what the URL scheme is, as they just pass it off to the system to open.

So I tried this out and unfortunately Ulysses won't send callbacks to html schemes. I asked Ulysses technical support and got a very quick and helpful answer. This is a deliberate choice for security reasons. They pointed me at a command line utility they'd made for calling a url scheme and getting a callback xcall. I'll go with this. Thanks for the suggestion though, it would have been tidy and self contained.

Share this post


Link to post
On 11/04/2017 at 7:25 AM, dfay said:

One quirk I've noticed with the Ulysses Alfred workflow -- even though i have "On My Mac" turned off in Ulysses prefs., and all my sheets are in iCloud, the subtitle for the workflow always shows the path as /On My Mac/correct path from here on out .  Everything else works, it just says my stuff is On My Mac instead of in iCloud.

Sond like there are two issues:

1. The workflow should only look in 'On My Mac' if preference is set.

2. I don't see the 'On My Mac' prefixes that you see. Is it possible you once enabled 'On My Mac' and had some stuff in there? The workflow will find old things on disk---this would not account for everything having 'On My Mac' in front, but could cause confusion

 

Will look at #1. Regarding #2, could you check that you don't have stuff in 'On My Mac' that be causing confusion

 

P.S. Could just disable On My Mac support until I have time look at it properly. This was just a stepping stone to supporting external folders.

 

Edited by robwalton
Add P.S.

Share this post


Link to post
5 hours ago, dfay said:

Is there any chance you could add a keyboard modifier to uf to copy the link to a sheet instead of opening the sheet?  Ulysses lacks an easy way to create links between sheets (the one thing I miss from abandoning NVAlt....) and this would be a great way to fill that gap. 

This would be a very cool feature. I guess we'd just want to paste it at the cursor location of the currently open sheet? I loose track of keyboard modifiers: would this warrant a new ul (for link) command?

Share this post


Link to post
47 minutes ago, robwalton said:

Sond like there are two issues:

1. The workflow should only look in 'On My Mac' if preference is set.

2. I don't see the 'On My Mac' prefixes that you see. Is it possible you once enabled 'On My Mac' and had some stuff in there? The workflow will find old things on disk---this would not account for everything having 'On My Mac' in front, but could cause confusion

 

Will look at #1. Regarding #2, could you check that you don't have stuff in 'On My Mac' that be causing confusion

 

P.S. Could just disable On My Mac support until I have time look at it properly. This was just a stepping stone to supporting external folders.

 

 

Let me see.  I have On My Mac turned off in Ulysses > Preferences > Library .  I just tried turning it on & it's definitely empty (there's just the empty Inbox).  Interestingly I added a sheet to it, and the workflow doesn't find it....but it's still showing everything else as being in 'On My Mac'

 

And if I use the ug command, and search for 'On My Mac' it shows my groups as expected whereas if I search for "iCloud" (where they are) I get no results.

 

Odd, isn't it?  It's not really a problem since the workflow works perfectly otherwise.  

 

I poked around in the XML in the library a bit and it seems that the DisplayName for my root is set to "On My Mac" even though it is in fact in iCloud.  Seems like a Ulysses bug that is otherwise invisible but which the workflow may make evident by using the DisplayName?  Just a guess.

Share this post


Link to post
56 minutes ago, robwalton said:

This would be a very cool feature. I guess we'd just want to paste it at the cursor location of the currently open sheet? I loose track of keyboard modifiers: would this warrant a new ul (for link) command?

My initial thought would be Option key and copy the link to the clipboard .  But using ul instead you could then have copying to clipboard as default behavior and paste at cursor location as a modifier (or vice versa).  

Share this post


Link to post

Well I got a version of ul working last night, but it requires manually loading all the sheet titles and URLs into a list filter.  From a CSV that's generated by a workflow in Workflow for iOS.  Working as a proof of concept :wacko: .  Since there seems to be no way to get a sheet identifier from the sheet file itself, it seems we would need to use Ulysses get-root-items callback URL, and parse the whole library.

 

I began to pick apart & try to replicate what the Workflow workflow does (it came from here: https://workflow.is/workflows/84934276b51643678643e1aa326637bd , & I modified it to output title, identifier, [title](ulysses url w/identifier) to a CSV), and convert it to write JSON which can then be read by a Script Filter.

 

The xcall utility linked above was really helpful at that stage.

 

Where I ran into trouble (and went to sleep) was at the stage of parsing the JSON.  

 

Here's the code I was running (in CodeRunner -- had to manually authenticate with Ulysses first, too):

 

#!/usr/bin/python

import json
import os

tmp = os.popen('/Applications/xcall.app/Contents/MacOS/xcall -url "ulysses://x-callback-url/get-root-items?recursive=YES&access-token=your token here"').read()

ulyssesLib = json.loads(tmp)['items']
print (ulyssesLib)

When I look at that output, it looks to me like it's a (long, messy) list of dictionaries but running type(ulyssesLib) says it's unicode.

 

 

Share this post


Link to post

Ok I've got it working.  Here's the code for the script filter, which also requires that you check "Alfred filters results" and replace the access token in the code:

 

#!/usr/bin/python

import json
import os
import urllib

def my_generator(json_input):
	if isinstance(json_input, dict):
		for k, v in json_input.iteritems():
			if k == 'title':
				ti = v.replace('"', '').replace(',', '').replace(':','')
			elif k == 'identifier':
				arg = v
			elif k == 'type':
				subti = v
			else:
				for child_val in my_generator(v):
					yield child_val
		yield json.dumps({u'title': ti, u'arg': '['+ti+'](ulysses://x-callback-url/open?id='+arg+')', u'subtitle': subti})
	elif isinstance(json_input, list):
		for item in json_input:
			for item_val in my_generator(item):
				yield item_val

tmp = os.popen('/Applications/xcall.app/Contents/MacOS/xcall -url "ulysses://x-callback-url/get-root-items?recursive=YES&access-token=youraccesstokengoesHERE"').read()

ulyssesLib = json.loads(urllib.unquote(tmp).decode('utf8'))

items = json.loads(urllib.unquote(ulyssesLib['items']))[0]

print('{"items":'+str(list(my_generator(items))).replace('\\\'','').replace('\'','')+'}')

It searches the sheet/group titles for the query.

it outputs a Markdown formatted URL, so to paste it into Ulysses you need to use Edit > Paste from > Markdown a.k.a. command-option-V

 

Edited by dfay

Share this post


Link to post

Hi @dfay, Looks like you've made good progress on the ul command. I've gone off on one and pretty much finished a full ulysses python client to the 23 calls described here and put it up at github. Example use:

 

>>> from ulysses_client import ulysses
>>> ulysses.get_version()
u'2'
>>> token = ulysses.authorize()
>>> ulysses.set_access_token(token)
>>> library = ulysses.get_root_items(recursive=True)
>>> print library[0]
Group(title='iCloud', n_sheets=0, n_containers=4, identifier='4A14NiU-iGaw06m2Y2DNwA')
>>> print '\n'.join(ulysses.treeview(library[0]))
...

Will use it (or about 5% of it!) to add an append command to the workflow and also your cool ul workflow fragment fairly soon.

 

I was all fired up to replace all calls in the Ulysses workflow with this (rather than the reverse engineered file system parsing approach it currently uses). However, it would be hard or impossible to replace the uf find command at his point. This command currently uses spotlight to find the location on disk of entries who's internal content matches a query; and then uses this list of file locations to filter the richer structure built up from reverse engineering the library. The problem is that it would currently take a call to

ulysses.get_quick_look_url(id)

on every single node in the tree obtained via x-callback to get path that backs each node. (The Ulysses API has no find call.) I'll ask the Ulysses guys about this, but with this version of the Ulysses API it doesn't seem like I can do a full replacement---was hoping this would fix the muddle with 'On My Mac' items and provide access to external folders without more reverse engineering (also its always nice to throw away fiddly code!)

Edited by robwalton
typos

Share this post


Link to post

Very nice - thanks for all your work on this - my approach to a workflow like this is always to do just enough to get it working for my needs, so I am always impressed when someone covers every option so exhaustively.

 

Re: uf, my thinking is to always let the OS do the work if you can.  (again I'm lazy B))

 

I am enjoying using that ul command -- I've cleaned up dozens of dead links from notes imported from NVAlt & it's so much faster that wading through the Ulysses UI.

 

Still, I feel like a lot of this would be a lot easier if the Ulysses devs would implement a rudimentary AppleScript dictionary.  I'd really link to do a cross-linking script, i.e. create a link in sheet A to sheet B and automatically append a link in sheet B back to sheet A.  But there's no equivalent in Ulysses automation to AppleScript's current document, hence no way to automatically get the title / identifier of the active document.  I could use find or mdfind to get the most recently modified file (i.e. sheet A) immediately after pasting the link to sheet B, but that doesn't get me far b/c there's no way to get an identifier from a file as far as I can tell.

 

 

 

 

 

 

Edited by dfay

Share this post


Link to post
10 hours ago, robwalton said:

Hi @dfay, Looks like you've made good progress on the ul command. I've gone off on one and pretty much finished a full ulysses python client to the 23 calls described here and put it up at github.

 

Cool. Love the way you're using xcall.

 

Couple of observations. I haven't been through all the code (let alone run it—I don't have Ulysses), but it seems to me that this line will explode if you aren't running it from your project root due to the relative path.

 

Also, the from ulysses_client import ulysses seems a little awkward. Would it not be better to rename the top-level package ulysses and import the API functions into __init__.py (or define them there), so you can just do import ulysses?

 

Edited by deanishe

Share this post


Link to post
On 17/04/2017 at 11:18 AM, deanishe said:

Cool. Love the way you're using xcall.

Thanks. Using xcall is an easy way to handle things and Martin signed it too so no brainer to use it. I haven't profiled it to see how much overhead there is in firing it up every time---in principle leaving the url-callback receiver part running would be quicker---but it seems to be fast enough. 100ms to Ulysses and back again for most calls most of the time.

 

On 17/04/2017 at 11:18 AM, deanishe said:

Couple of observations. I haven't been through all the code (let alone run it—I don't have Ulysses), but it seems to me that this line will explode if you aren't running it from your project root due to the relative path.

Good spotting! I've fixed that.

 

On 17/04/2017 at 11:18 AM, deanishe said:

Also, the from ulysses_client import ulysses seems a little awkward. Would it not be better to rename the top-level package ulysses and import the API functions into __init__.py (or define them there), so you can just do import ulysses?

Totally agree. I was a bit shy about taking the package name ulysses which is partly why it was awkward. Have followed your suggestions.

 

I just need to do something useful with it now (like ul @dfay), to see what needs changing about the API.

 

Share this post


Link to post

Just had a few minutes to tool around with this library - it's great.

 

I was able to quickly write the basis for the cross-referencing script.  Here's a working draft.

 

import re
import ulysses

# authenticate 1st & add your token here

token = ""
ulysses.set_access_token(token)

# test - enter test sheet ID here (and make sure it has some links or nothing will happen!)

sheetID = ""

# get source sheet's title and body, and search for links

iItem = ulysses.get_item(sheetID)
iBody = ulysses.read_sheet(sheetID,text=True).text

# get the Ulysses links to cross-reference
uLinks = re.findall("ulysses://x-callback-url/open\?id\=[A-Za-z0-9_-]*", iBody)

# get the BibDesk links in case we want to do something with them later
bLinks = re.findall("x-bdsk://[A-Za-z0-9_-]*", iBody)

# get the IDs for linked sheets

idList = []
for aLink in uLinks:
	id = aLink.replace("ulysses://x-callback-url/open?id=", "")
	idList=idList+[id]

linkToI = "["+iItem.title+"](ulysses://x-callback-url/open?id="+sheetID+")"

# add a link to the source sheet to all its linked sheets

referenceHeader = "#### Cross-references\n\n"
	
for linkedSheet in idList:
	ulysses.insert(linkedSheet, referenceHeader+linkToI, format='markdown', position='end', newline=None)

(After using this for a couple of hours I am finding it really useful for annotating legal cases and commentaries!)

Edited by dfay

Share this post


Link to post

One thing I noticed, though, was that the authentication token I'd acquired earlier today no longer worked and I had to re-authenticate.  Have you been able to consistently use the same token without repeat re-authentication?

Share this post


Link to post

Hi Rob, Thanks for the very useful workflow.

 

I'm wondering is there any way to add a search that will produce the same results as appear in Ulysses "Last 7 Days" folder, but within the Alfred results window? Or even just sort the sheet results by modification date?

 

I see there is a 'open-recent' action in the callback documentation you post above, though using it is beyond me except for creating a bookmark of the type:

ulysses://x-callback-url/open-recent

Thanks!

 

Edited by patgilmour

Share this post


Link to post

Actually there's a typo in Rob's code … in ulysses_calls.py --  open_recent() should call the URL that patgilmour pasted above.  

 

def open_all():  # @ReservedAssignment
    """Open special group 'All', bringing Ulysses forward."""
    call('ulysses://x-callback-url/open-all')


def open_recent():  # @ReservedAssignment
    """Open special group 'Last 7 Days', bringing Ulysses forward."""
    call('ulysses://x-callback-url/open-all')

 

 

Here's a script which can be put in a script filter to search the Ulysses library returning newest results at the top.  I call it usd for Ulysses-sorted-(by)-date :)

 

This assumes you have Rob's library installed in your workflow.

 

#!/usr/bin/python

# get Ulysses library sorted by date & optionally search for query
# outputs the identifier of the found sheet

import ulysses
import json
import sys

query = sys.argv[1]

token = "your token here"
ulysses.set_access_token(token)

library = ulysses.get_root_items(recursive=True)


def all_sheets(g):
	if hasattr(g, 'containers'):
		for c in g.containers:
			for child in all_sheets(c):
				 yield child
	if hasattr(g, 'sheets'):
		for s in g.sheets:
			yield [s.modificationDate, json.dumps({u'title': s.title.replace('"', '').replace(',', '').replace(':',''), u'arg': s.identifier, u'subtitle': s.type.capitalize()})]

print '{"items":['			
for i in sorted(list(all_sheets(library[0])), key=lambda item: item[0], reverse=True):
	if query.lower() in json.loads(i[1])['title'].lower():
		print i[1].replace('\\\'','').replace('\'','')+","
print ']}'

connect it to an Open URL item with the following:

ulysses://x-callback-url/open?id={query}

and if you want to create a link to the found sheet instead, add a keyboard modifier and connect to the Markdown link maker in the post after this one

Edited by dfay

Share this post


Link to post

Here's a cleaner version of ul that makes use of the ulysses-python-client library.  I also changed it to output just the identified as the argument.

 

#!/usr/bin/python
# search for a sheet's title and return its identifier 

import ulysses
import json

token = "your token here"
ulysses.set_access_token(token)

library = ulysses.get_root_items(recursive=True)

def all_sheets(g):
	if hasattr(g, 'containers'):
		for c in g.containers:
			yield json.dumps({u'title': c.title.replace('"', '').replace(',', '').replace(':',''), u'arg': c.identifier, u'subtitle': c.type.capitalize()})
			for child in all_sheets(c):
				yield child
	if hasattr(g, 'sheets'):
		for s in g.sheets:
			yield json.dumps({u'title': s.title.replace('"', '').replace(',', '').replace(':',''), u'arg': s.identifier, u'subtitle': s.type.capitalize()})
			
print('{"items":'+str(list(all_sheets(library[0]))).replace('\\\'','').replace('\'','')+'}')

I now have it set up to output to this script to build the Markdown link:

 

#!/usr/bin/python
# create a Markdown link from an identifier

import sys
import ulysses

token = "your token here"
ulysses.set_access_token(token)

query = sys.argv[1]

title = ulysses.get_item(query).title

print('['+title+'](ulysses://x-callback-url/open?id='+query+')')

Now thinking of what else one can do with the identifier...

 

(Update) I've added a modifier (ctrl) to send the identifier only to the keyboard -- this can then be pasted into Ulysses group search (command-shift-F) and it will return all sheets that contain a link to the queried sheet. 

 

 

Edited by dfay

Share this post


Link to post

Just occurred to me that both of these assume there's only one root level library item i.e. library[0]

 

Presumably for this to work with On My Mac or External Folders you'd need to iterate through the root level items.

Share this post


Link to post
12 hours ago, robwalton said:

I was a bit shy about taking the package name ulysses

 

Call it Odysseus :) 

Edited by deanishe

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...