Jump to content
Florian

Speed and compiled workflows

Recommended Posts

Hey people,

 

So I like optimisation. Like I just mixed some bash and php to give my piratebay workflow a little multithreading boost (updates are regular so be sure to install with packal).

 

But now I'm thinking: aren't compiled languages supposed to be faster? Technically yes, but workflows are usually very small pieces of code, so is it worth it? Is there a "loading" cost like there is for php for example? 

Share this post


Link to post

Very, very, very few workflows will benefit from compilation. Most workflows actually _do_ very little work. If a script takes 100 milliseconds and a compiled version takes 10 milliseconds, you won't be able to tell the difference. The exception is when your workflow takes seconds to run _and_ the reason for the delay is computation, not network lag. A much more likely reason for compiling code for a workflow is that it's the only (or easiest) way to interact with something in OS/X.

Share this post


Link to post

Part of the problem with compiled code and workflows is that most of us are scripters and not as good at writing code in compiled languages. They're also less accessible.

 

I think that Phyllistein wrote an Objective-C library for Alfred, but it is a bit outdated at this point.

 

For workflows, especially script filters, my approach is always to try to write the thing in Bash if possible for the best responsiveness and then fallback to PHP or Ruby if it becomes untenable to write it in Bash. That point usually occurs when I need to use some structured data like JSON.

 

Florian, since we had this optimization conversation privately, I'll outline the results for anyone else reading.

 

The way to optimize the PHP workflow is to write as much of it in bash as possible. PHP's sluggishness in script filters comes from how Alfred uses PHP. Basically (actually), Alfred just issues the command `php -r "<workflow code>" whenever it runs something with PHP. The sluggishness comes because OS X has to load the PHP binary before interpreting the script, and, if the script hasn't been run recently, then the script must be compiled and cached into byte code. The added milliseconds that the first step takes is noticeable in script filters, especially when typing quickly, and, if the script filter asks the Internet for data, it goes slower due to the response. Further, Alfred waits for each command to be finished before sending a new one, so you have to wait for an http response for (basically) every keystroke you press.

 

The trick that works with PHP is that, starting with PHP 5.4 (available in 10.9), you can start a temporary PHP webserver from a simple command: `php -S localhost:port#`. There is very little difference in the way that the PHP server and the CLI binary work on OS X, so you can use almost the exact same scripts, but the advantage comes from PHP already running and thus reducing the load time.

 

So, basically, each time the script filter is run, the workflow checks to see if a local PHP server has been started, and, if not, it launches it as well as a kill script (more on that later), and if a minimum number of characters have been typed, then it sends the command to the PHP server and gets the response quickly.

 

The kill script is important because no one should leave a webserver running on someone's computer. So the key is to have the PHP script run by the webserver to write to some sort of text file in the cache directory each time it's used. Then the kill script checks every 60 seconds or so to see when the last time the PHP server had any activity, and, if enough time has passed in a period of inactivity, the kill script issues a kill command for the webserver and then exits itself.

 

It's also important to note that trying to launch the server on a lower port number requires "sudo," so, to be safe, use one much higher (5000+) and make sure that it is random enough not to interfere with any other service that might have an active port.

 

The last optimization is for anything that requires an http response. If you know that each keystroke would take precedence over previous ones, then it is best to just kill the curl (or other http) request so that Alfred will stop waiting for it and move on to the next one.

 

This setup works only for PHP on OS X 10.9 or 10.10. Something similar could be done with Python and Ruby, but a slightly different approach would need to happen because of the way that Python loads packages and Ruby loads gems: you need to make sure that both are loaded into the initial "webserver" scripts to make them go fast enough. However, killing the previous requests could benefit most workflows with long running processes; however, make sure that you `grep` it specifically enough that you don't accidentally kill another process, and never use `killall`.

 

For most workflows, this setup is way overkill, but some can make good use of it.

 

Generally, if you're using a script filter and each keypress takes about 400ms, then it might be worth it. Less than that (as ctwise mentioned) isn't worth it.

 

Test the setup thoroughly as this becomes complicated and fragile. Don't leave processes running indefinitely on anyone's computer.

Share this post


Link to post

Oh, and if you want a sample kill script, look into the inner-workings of the Packal updater. In order to get the GUI working, I did something similar to this, but the callback to write the text file was in an ajax call in the webpages that are the GUI. You'd need to put those sorts of things in the script itself, and you shouldn't use ajax but just a simple "file_put_contents" call.

Share this post


Link to post

I've been using Go a lot recently, and there is definitely something to be said for compiled languages and workflows.

 

Go and Objective-C/Swift run about 20x faster than scripting languages, and the programs start up a lot more quickly, too. That gives you a lot of headroom to do things that would be too slow in a scripting language (or to avoid some of the complicated workarounds we use to manage the slowness).

Share this post


Link to post

If you want really quick startups, use Rust. The executable is more than half that of a go compiled program (what I've tested so far. Just been learning it). Haskell is the the second largest size, and then Swift. Go has always had a large executable size.

 

Rule of thumb: the smaller the executable programs size, the faster the load. I wrote a script workflow that uses tinyscheme and it runs really fast due to it's real small size. PHP's executable is much larger than Ruby's, and hence much slower in running scripts. Python is about just as fast as Ruby on execution.

 

Of course, these comparisons are based on my usage of the different language. Your experience may vary.

Share this post


Link to post

PHP starts up more slowly than Ruby/Python because it's monolithic. As soon as you start importing the libraries you need to write something non-trivial, the difference shrinks or disappears.

Share this post


Link to post

It seems some here might have an idea. Anyone have a rough estimate of how long it takes, on a modern system, for a script filter to run a minimal python script and return results to Alfred? Perhaps somewhere in the hundreds of milliseconds? I'm wondering how much I should try try to optimize my workflow, e.g., reductions in execution time of for example 0.05 seconds might be perceivable but require too much effort.

Share this post


Link to post

The thing with Python is the startup time. If you’re using my library, for example, it takes ~0.05s to load Python and import the libraries.

 

0.05s is definitely perceptible.

 

As far as optimising Python workflows goes, I focus on caching and lazy imports.

 

They're always going to be noticeably slower than compiled workflows, though.

 

I mostly use Go these days. It’s somewhat more effort to write than Python, but 20x faster.

Share this post


Link to post

Thank you deanishe. I started my one workflow long ago when my programming abilities were less than ok. They're perhaps only slightly better now. Working on it here and there over time, I haven't looked at it in a while since the beginning. I recently tried to make some improvements and additions to get it close to sharing. As mentioned elsewhere, I used Peewee for ease of use at the start and trying to reduce code duplication with configuration of different db backends. That takes about .09 seconds to load, most of it seems to be the loading of two drivers. Now after having read up on SQLite, I'm using only that thought haven't removed the rest of the code. A full-text search over a fairly large database takes about 150 ms total from script start to returning workflow items. I tried to optimize it and it's maybe as close as it can get; only so much can be done it seems. Removing Peewee might gain me half that time, unsure. You encouraged me to try Go. As I was getting back to my workflow, been looking into Go in recent weeks. The main part of the workflow for which I'm concerned about speed is a script filter of a full-text search over the entire db. Like a search engine, I might refine the search terms to narrow down and explore results. Since you're reply, I've tried to get just that most used part working and got it going earlier today. So nice. ~8–50 ms.

 

And thank you for the wonderful alfred-workflow and awgo.

Share this post


Link to post

If you're still using Peewee, that's an obvious place to gain speed. Apart from the import time, there's a lot of magic going on in an ORM, and it might not be generating particularly efficient SQL queries.


At 150ms, it sounds like the database query might be taking some time. If I were you, I'd try to find out if that's the case. SQLite is the same C library in every language, so rewriting the workflow in Go/Swift/whatever won't make that any faster.


If I were you, I'd be looking at the queries Peewee is generating and benchmarking those. Perhaps you can figure out a more efficient query.

Share this post


Link to post

Indeed removing Peewee is a source for more speed. At this point, the workflow maybe in maintenance mode with fixes, new features, etc. but not larger efforts of recoding. As I started porting it to Go, at least the most used part that would benefit most from speed, I use both in combo and will slowly convert the rest to Go as I learn the language, available modules, etc.

 

As for what's taking up the ~150 ms for one script filter, I was curious how such a time compared to total round trip time of a Python script filter. As is:

 

- full-text relevancy ranked search is done on an SQLite FTS5 table, sorted by relevancy in 3 columns. DB is at ~1.2 million rows. Should be a few dozen ms max or less. Read some notes the other day about FTS query syntax that would result in the same results but one might run slightly faster. various pragma statements might make a small difference.

- loading of three DB drivers (apsw for SQLite, psycopg2 for postgresql, pymysql for sphinx) seems to take up about 2/3s of execution time. ~100 ms of the total ~150. Wasn't too familar with SQLite at the time I began so used postgresql/sphinx. I could remove the code to optionally use a different backend. should reduce load time but I'll likely leave it in there for those that want it. It was only recently in getting the workflow mostly good enough for sharing that I looked into SQLite how to configure it. Still possibly more work could be done with that. Overall seems Peewee possibly could completely load and parse modules in less than 50 ms if just using apsw for SQLite.

 

total time includes steps:

- init db, create tables if needed, create triggers if they don't exist for FTS table population, SQLite pragma statements. Unsure what is deal. All of that seems to go quick.

- reading of cached query string. so bringing up the script filter will populate text field and results with previous search. query string is saved on each execution of script filter, unsure if there's a way to save it only after and if one has pressed return or selected a filter result. as such, query string is saved on each execution of script filter. seems like this step, at least file write, could be threaded.

- image thumbnails are used if they exist for filter results. so for each result that goes into the filter (searching index ebook TOC entries to open book to section), a check if exists a thumbnail is done and then the icon type is set. Search is limited to 100 results so 100 checks are done. Maybe that doesn't take too much time. As there could be results from the same file occurring multiple times in results (different sections of same book), maybe some map could be done to set icon path for all entries to the same file. Not too strong with such tasks but will look into it; unsure if such a task before creating workflow items would be faster or slower.

 

In general, unsure how fast all that could run in optimal conditions. under 60 ms? Unsure still of additional overhead of Alfred initiating the script filter, running AppleScript before and after, and parsing and displaying script filter results. Part I mostly wanted to get faster was remade in Go; as such, perhaps will mostly be devoting effort that.

 

As it seems mostly ready for others to start using, updated: 

 

 

Share this post


Link to post
5 hours ago, h2ner said:

Unsure still of additional overhead of Alfred initiating the script filter

 

Tiny. And unavoidable…

 

AppleScript can be extremely slow when you're using tell application. Individual actions can be very fast, but connecting to the application takes a very long time (>200ms on my machine). Typically, there's not much you can do about that, though, other than reduce tell application clauses to a minimum.

 

6 hours ago, h2ner said:

loading of three DB drivers (apsw for SQLite, psycopg2 for postgresql, pymysql for sphinx) seems to take up about 2/3s of execution time.

 

You could make the imports conditional based on which database is being used:

if os.getenv('GNOSIS_DB') == 'postgresql':
    import postgres
else:
    import sqlite3

In any case, when you're trying to optimise code, always benchmark the code to find out where it's actually slow. Otherwise, you can waste a lot of time trying to speed up something that isn't slow.

 

FWIW, the workflow isn't working for me. I'm trying to search the contents of an ePub in Calibre's built-in viewer, and keep getting the error:

 

[2019-03-31 09:01:52][ERROR: input.scriptfilter] Code 1: 1308:1311: syntax error: Expected end of line but found identifier. (-2741)

 

I had a very brief look at the code, but couldn't find the problems. I did find one potential source of errors:

set cmd to cmd & "\"" & theQuery & "\""

That will break if there's a double quote in the query. Do this instead:

set cmd to cmd & (quoted form of theQuery)

Share this post


Link to post
15 hours ago, deanishe said:

Tiny. And unavoidable…

 

I was out of mind when I had phrased that. Closer to what I was originally thinking, trying to get an idea of potential lowest total execution time a script filter. I haven't looked much at the AppleScript in a while but will do. Launching a minimal python script w/o imports that returns one result, unsure what is the overhead of launching python. Unsure if Alfred needs to create a shell environment first, or if there's anything else. Perhaps all of that doesn't add much and the potential of a script filter to execute and return results, in Python, might be in the dozens of milliseconds? Helps to get an idea of let's say my script execution time could be reduced by perhaps 100 ms, that is perceivable; if it's just 20 ms or so, what percentage of total time might that be? A rough estimate would be nice to know though now I've started to port it to Go, less important. Though over time, having an idea of potential speed and trying to reach it is a nice aim.

 

15 hours ago, deanishe said:

You could make the imports conditional based on which database is being used:


if os.getenv('GNOSIS_DB') == 'postgresql':
    import postgres
else:
    import sqlite3

 

 

Recently I had tried to separate the Peewee import as such of their apsw module and Postgres extended. Problem was the main Peewee module imports all drivers if available, rather than by class instantiation, and I had some issue with it breaking a table field type that is extended in it's extended SQLite module that uses apsw, but seems was imported from main Peewee. Reverted back since it didn't make any speed difference. Now I'm more comfortable with SQL, Python, etc., I may try to remove Peewee though now with Go, who knows. Only in recent weeks had I started to seriously look into SQLite, and I'm pretty happy with it, so may remove the other code and drivers. Will wait for that, though will concentrate on Go. As as far SQLite, seems the load time of the apsw driver itself and running a query, if I remember correctly, might be around 40 ms. Timing the query within sqlite3, it rounds to 10s of ms, is 10 ms w/a query that returns one result. Maybe there are compile options of the driver. … All these details, perhaps less important now though I may try some, though future efforts will be porting to Go. Also would like to better know SQLite options, pragma statements that might affect queries that return many results, and there was also a mention in the sqlite-users mailing list of someone writing a ranking algorithm that adds a fair amount of Sphinx's SPH04. Little details I may try to look into over time.

 

Thanks for the AppleScript tip.

 

As far as the error, on a new setup of Alfred with a different macOS user, I was unable to produce it. Seems like an AppleScript error. Will keep looking into it and thinking what it might be. Of note, seemingly unrelated, a recent version of calibre (within the last ~2 years) is needed for EPUB use. I had asked, specifically for this workflow, and there was an addition to the ebook-viewer command-line parameters to specify opening a file to a TOC entry by title. Not ideal since it might mismatch but it's a start. Unaware of any other EPUB viewer that has anything similar or AppleScript support.

Share this post


Link to post
7 minutes ago, h2ner said:

Launching a minimal python script w/o imports that returns one result, unsure what is the overhead of launching python.

 

That's why I said you need to benchmark that stuff. There's little point trying to optimise a program before you've identified where it's actually slow.

 

22 minutes ago, h2ner said:

Only in recent weeks had I started to seriously look into SQLite, and I'm pretty happy with it

 

SQLite is amazing. It's quite likely the most widely-used piece of software in the world.

 

27 minutes ago, h2ner said:

As far as the error, on a new setup of Alfred with a different macOS user, I was unable to produce it.

 

Doesn't matter. I don't really use Calibre's built-in eReader, anyway.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...