Alternative file extension for Alfred-3-only workflows

[deanishe]

A significant problem in adding Alfred 3 features to my workflows is that Alfred-Workflow's update mechanism, which is based on GitHub releases and used by scores of workflows, has no way to tell if a workflow is Alfred 2 compatible or not.
 
Either there's a release with a higher version number containing an .alfredworkflow file or there isn't. That's how GitHub releases work.

Developers currently have 2 options to prevent their updated-for-Alfred-3 workflows trying to break themselves in Alfred 2:
 

1. Start a new repo
2. Mangle the release in some way that older library versions ignore it

 
No. 1 is not a popular solution for obvious reasons.
 
With option 2, you can either mess up the tag (version number) to something that isn't a valid semver, but contains one, like "a3v1.0.1", or you change the file extension of the upload to something that isn't .alfredworkflow.
 
Changing the file extension involves deliberately breaking the fewest things, so that's currently the preferred solution, and it's the one I'm going to go with. The library will rename the file after download if need be.
 
The problem then is that anyone who downloads the file directly from the releases page then won't be able to install it without first changing the extension.

That seems to be the least evil on offer, however, and it would be awesome if Alfred 3 "blessed" this behaviour by adding support from an additional, Alfred-3-only file extension.
 
Alfred 2 apparently also has issues with installing workflows it shouldn't because it doesn't support them. A v3-only file extension could head off issues in that direction much more simply that heaps of logic in Alfred 2 to figure out if it's about to overwrite a functioning workflow with a newer, but incompatible, version.
 
In the above-linked GitHub issue, Shawn goes further and says Alfred 3 should use a different extension by default (i.e. if I export a workflow from Alfred 3, it gets a different extension) to provide "a consistent visible indication to the user that a workflow is only for Alfred 3".

 

Certainly a valid point, but it not one I came here to push, as I'd be happy with an "everything works and nothing is broken" solution to the problem as laid out above.

(It would also allow parallel Alfred-2 and -3 workflows in the same GitHub release, not that this strikes me as a particularly compelling reason.)

[vitor]

One of the things that has always bothered me with Alfred 2 is how “2” is part of the name. Since Alfred has official support for features that allow us to call it (or it to call itself) via AppleScript, this lead to the logical conclusion that with Alfred 3, all those AppleScript calls were broken and needed to be manually fixed in every workflow that makes use of them.
 
I mention this to reinforce the notion that a .alfredworkflow3 extension wouldn’t even be an unexpected change. I’d go even as far as to say it is completely logical. It would also be completely invisible to the user¹ and possibly helpful to a developer (for the reasons deanishe mentions) which in turn is better for the user.
 
On another note, I seem to remember in the past a fairly simple addition to a non-stable Alfred version which made Alfred’s stable version complain and not install the workflow². This also seems like a fairly straightforward way to solve the issue except if we’re only making the aforementioned AppleScript Alfred 2 → Alfred 3 change, then there really isn’t anything new for Alfred 2 to identify. Luckily, there’s a simple Alfred 3-only field that could be made mandatory to get around this:
 

 
So I see two main solutions:

• Make Alfred 3 export .alfredworkflow3 by default (but continue to support .alfredworkflow).
• Make the Workflow Version field mandatory (like the Title field is mandatory in Keyword nodes, for instance). If necessary, release a final Alfred 2 update that will actively check for Workflow Version, and if found, prevent the user from installing.

Having the new extension, even though it’s a solution I don’t particularly like (in the same sense I don’t like how Alfred’s version is part of its name), seems like the most robust option.

---

¹ In the sense of “who cares?”, as long as Alfred registers those, having the different extension or not is the same to a user.
² From the linked github issue, it seems the warning still appears, but an overwrite is still possible under certain circumstances. Edited June 3, 2016 by Vítor

[deanishe]

One of the things that has always bothered me with Alfred 2 is how “2” is part of the name.

The Breaking of The AppleScripts is particularly annoying.

 

In most cases, you only need to change one character to get v2 and v3 support, and normally, that'd be a pretty trivial fix.

 

But AppleScript is never normal. As best I can tell, characteristic insanity makes it effectively impossible to write a single, self-contained AppleScript that supports both Alfreds 2 and 3 in practice¹.

You have to duplicate each script, rewrite in JXA or use some form of generation/indirection², all because one character needs to be different

 

Having the new extension, even though it’s a solution I don’t particularly like (in the same sense I don’t like how Alfred’s version is part of its name), seems like the most robust option.

 

 

I'd like to point out that a mandatory version field in info.plist doesn't entirely solve the problem of workflows self-updating themselves to death. Even if Alfred 2 prevents the workflow from actually committing suicide (Alfred-Workflow has always called Alfred to install the update itself), many workflows will display UPDATE AVAILABLE! messages for evermore.

 

Mangling the release in some way that means all previous versions of AW ignore it is the only solution that occurs to me that won't break anything existing. Using an extension that isn't .alfredworkflow for the release binary is the cleanest solution. It's the most "optional" with the fewest side-effects, imo. Using an invalid version tag might have knock-on effects in other software, and you can always come back later and change the release binaries if you want (or include an .alfredworkflow file for Alfred 2 and an .alfredworkflow3 file for Alfred 3 if it's a security update).

 

Hmm. Perhaps I should go and start a thread in the Workflows forum. I think there are a good few users of the library that might have a better idea.

 

Regarding the version field: Is the presence thereof in info.plist necessarily proof that a workflow is incompatible with Alfred 2? Can workflows exported from v3 not be installed in v2 regardless?

 

Like, what if I just fixed a bug in one of the scripts or libraries in an existing Alfred-2 workflow and put a version number in the box while I was there. Would that then be incompatible with Alfred 2 if I export it from v3?

¹My attempts have been thwarted because (1) you can't use a variable as the target of a tell clause, and (2) any tell clauses within the script, called or not, appear to get resolved when you try to run the script, so it will ask the user to locate the missing application if they don't have Alfreds 2 and 3 installed.

2 The cleanest solution is, I think, to use a Script box with the {query} macro for Alfred's version number in an AS script, or workflow variables in a bash heredoc containing the AS. I think it'd be only fair for Alfred to give you an environment variable containing only the major version, too.

[ipaterson]

Context is very important to this decision, there are many contexts in which the ability to distinguish Alfred 2/3 support is important.

 

Installing an incompatible workflow in Alfred 2

There may be many different ways that Alfred 2 could be updated to detect and refuse to install an Alfred 3 workflow. The easiest but least robust approach is to do nothing; use a new file extension in Alfred 3 so that new workflows are not recognized by Alfred 2. Surely some Alfred-3-only workflows have already been released with .alfredworkflow but it is still early enough for this to be a passable, if disappointing, solution.

 

Preferably, inspect the extension before installing as mentioned by Vítor. Not everyone will want to use the new file extension (e.g. third-party update manager does not support the new extension yet) and some people will just remove the 3 and try to install the workflow anyway.

 

Developers accidentally dropping support for Alfred 2

Currently, making any change and then exporting the workflow from Alfred 3 results in a workflow that is incompatible with Alfred 2. Try it with the Should I watch this movie example workflow. Exported from Alfred 2, updated to use a different keyword in Alfred 3, exported in Alfred 3, then installed back to Alfred 2 yields an incompatible workflow. That is quite unfortunate, developers will accidentally drop support for Alfred 2 by making innocuous changes in Alfred 3. 

 

There may be a few approaches to that:

1. Avoid generating an incompatible workflow in Alfred 3 unless an incompatible feature is used – I assume this is much easier said than done
2. Track whether a workflow was Alfred 2 compatible and warn before exporting – again, not sure how simple it is to determine whether a workflow was Alfred 2 compatible
3. Export with the .alfred3workflow extension as a cue to the developer that the workflow will no longer work in Alfred 3 – not the best solution but probably the simplest

 

Update managers that use GitHub releases

Deanishe's Alfred-Workflow checks the workflow's GitHub repository for updates with the option to notify the workflow user when an update is available and to download that update. This is a tremendous feature to have in a workflow and Alfred-Workflow even supports pre-release updates for workflows like mine that always have new features in active development for anyone willing to test. GitHub releases are based on master branch tags which, at least for the case of Alfred-Workflow, are semver version numbers. A higher version with a single .alfredworkflow binary means there is a release available.

 

Most importantly for me, the workflow cannot download the .alfredworkflow file to inspect whether it is compatible or not. GitHub tracks per-release download counts and these have become very important metrics to me since it is otherwise impossible to know how many people are using a workflow; downloading a workflow is not only wasteful of the end-user's bandwidth but would also inflate download counts. This leaves us with a distinction that can be modeled in the metadata for the GitHub release.

 

Options here include

1. a release tag naming convention – not preferable since straight semver is clear and simple
2. something in the release name itself – a decent indication to the user but it will get old to always include marker text in the release title
3. .alfred3workflow file extension – clear to the end-user and 
4. start a new repository specifically for Alfred 3 – for workflows that do not rely on GitHub issues, stars, release download counts, etc this may be an option.

Of those options, #1, #3, and #4 have the benefit of blocking workflows released with an older version of Alfred-Workflow from downloading an update meant for Alfred 3. Furthermore, the logic allows consecutive updates to Alfred 2 and Alfred 3 workflows which is beneficial for any workflow with a large number of Alfred 2 users and a backlog of bugs to fix.

 

I am not sure whether any other workflow libraries provide automatic updates.

 

Update managers that use static version files

Alleyoop allowed developers to publish a static JSON file with the latest version of the workflow. I am not sure whether alleyoop is still supported, but if so there would probably be a few ways to handle the version compatibility. It is more flexible than GitHub releases since the JSON could be modified as necessary to denote Alfred 3 compatibility.

 

End-users downloading workflow files from the forums or GitHub

Assuming that Alfred 2 will be updated to not install incompatible workflows, there is not a huge risk here. Upon downloading a workflow and installing I would see that it is not compatible with Alfred 2. Bummer, but after this happens a dozen times I would probably just pay for the upgrade to Alfred 3. This incompatibility would be obvious without even downloading the workflow if it has the .alfred3workflow extension.

 

Integrated workflow updates in Alfred

I stumbled across this note from Andrew in red about supporting secure workflow updates within Alfred. This may not still be the plan, but it would be great to support GitHub releases. Packal seems like another sensible place to look for updates. I suspect that this is a very challenging problem since integrating into Alfred involves considerable responsibility to protect the user and validate workflows. An integrated updater would need a way to distinguish 

 

Packal

I think that Packal has far fewer limitations; without any changes in Alfred Shawn could add a checkbox for developers to select "Alfred 3 only" on a release and show a corresponding message on the download page to warn users. Of course it might be nicer to be able to detect that automatically when the workflow is uploaded, and even nicer still to be able to offer a version of the workflow for both Alfred 2 and 3 at the same time. Since Packal keeps all submitted workflows in a GitHub repository the .alfred3workflow might be preferable here since Alfred 2 and Alfred 3 workflow binaries could both exist at once.

 

Packal updater

Again a lot of freedom since the appcast.xml could be updated to distinguish Alfred 3 compatibility. Since the updater is standalone rather than integrated into a workflow like the Alfred-Workflow updater it does not have the same problem of making sure old versions do not download Alfred 3 releases. Just publish an update to packal-updater that respects the new Alfred 3 distinction and 

 

 

I don't suspect that many people will continue to develop Alfred 2 versions of their workflow after they make changes in Alfred 3. For awhile we will see bug reports about workflows not installing in Alfred 2 simply because the developer does not realize that even minor changes in Alfred 3 can render the workflow incompatible with Alfred 2. Alfred 2 use will decline over time as users upgrade but since it is a paid upgrade for most we can assume that it will not be an abrupt transition.

 

Of all the options I prefer the alternate file extension since it seems to provide a workable solution in every case. It is not a particularly elegant solution and is not a complete solution in some of the contexts described above, but like Vítor said, it seems to be the most robust way to handle these various problems.

 

If the new file extension is supported in Alfred 3 I still recommend making it the default extension for workflows exported from Alfred 3. Like I said on GitHub, it provides a consistent visible indication to the developer, the end-user, and any software that is unable to inspect the binary contents that a workflow is only for Alfred 3.

[deanishe]

Personally, I'm loathe to drop Alfred 2 support in an existing workflow unless/until it would benefit greatly from Alfred 3's new bells and whistles (or a decently-long time has passed). Indeed, it looks like I'm going to have to reinstall v2 to continue to support them.
 
Thanks very much for clarifying the compatibility situation, Ian.
 

GitHub tracks per-release download counts

I did not know this. This is the best thing I've heard all week. I am so going to write an Übersicht widget or something for this at the weekend.
 

Integrated workflow updates in Alfred
I stumbled across this note from Andrew in red about supporting secure workflow updates within Alfred. This may not still be the plan, but it would be great to support GitHub releases.

Ten minutes ago, I would have said "use git". You have built-in support for squillions of hosts right out of the box. Now I know there are release stats, +1 for GitHub releases

 

a consistent visible indication to the developer, the end-user, and any software that is unable to inspect the binary contents that a workflow is only for Alfred 3.

Extremely well put.

[ipaterson]

If anyone is looking for another way to digest my earlier post, here are a list of actions that would need to be taken to proceed with the file extension change if no better ideas come up:

 

For workflow developers

• Be very clear about Alfred 2/3 compatibility when you share your workflows, especially if you have existing users on Alfred 2
• Until you intend to drop support for Alfred 2 users, do not modify and export your workflow from Alfred 3, not even to make minor changes. Your workflow will likely no longer work in Alfred 2 (see below).
• Once you drop support for Alfred 2, export your workflows from Alfred 3 as .alfred3workflow files

For Andrew in a patch release of Alfred 2

• Inspect all workflows before installing to ensure object-level compatibility (see below), show a message and fail to install incompatible workflows

For Andrew in the next release of Alfred 3

• Add .alfred3workflow file association
• Export workflows with .alfred3workflow extension (or alfredworkflow3 as others have suggested)
• Inspect all workflows before installing to ensure object-level compatibility to guard against incompatibility with future changes in Alfred (see below), show a message and fail to install incompatible workflows
• Make the major version of Alfred 3 accessible in an environment variable (per deanishe)

For Dean in the next release of Alfred-Workflow

• Allow updates to the .alfred3workflow file extension but only if running Alfred 3

 

I'm not familiar enough with how Packal works or what Shawn is planning to suggest anything on that front.

 

Also, based on a diff of the workflows from the earlier example of changing the keyword in the Should I watch this movie? example workflow, I tracked down the specific part of the info.plist that makes this incompatible with Alfred 2. It is not the addition or modification of config keys and values, nor the change from real to integer for positions, nor the addition of the version key. Instead, the only change that makes Alfred 2 show that the workflow is incompatible are the incremented object versions:

  <key>uid</key>

  <string>F8C5D133-7C4D-44F2-845B-FAE1568A09E7</string>

  <key>version</key>

- <integer>0</integer>

+ <integer>1</integer>

 

This completely makes sense; the configuration for the Open URL and Default Web Search actions in this workflow has been updated and Alfred 2 does not have support for these versions. The extra/changed keys in most cases will not matter because Alfred 2 does not know to look for them so they will be ignored. On the plus side, this suggests that determining whether a workflow is compatible may be exclusively a matter of looking for any objects with unsupported versions. Such a test would be independent of "Alfred 2 vs Alfred 3" since the component versioning is already built in. There are already some components on version 1 which means that during the course of Alfred 2 development there have been changes that would cause workflows in the latest Alfred 2 to be incompatible with certain older versions of Alfred 2. It is not unrealistic to believe that Alfred 3 will have similar changes before Alfred 4 is released and would therefore benefit from this type of object-level compatibility test logic. We can't deal with those at the file extension or download manager level, so Alfred needs to know not to install them.

 

Unfortunately, it also suggests that any component that has been updated in Alfred 3 will cause an incompatibility with Alfred 2 even if it is not edited directly as shown in my earlier example. I think that's pretty much every component that you can put in a workflow. Any change to the workflow causes all of its objects to be upgraded to Alfred 3's version of each respective component. This is why developers need to avoid editing workflows in Alfred 3 until they are ready to drop support for Alfred 2, and if you choose to ignore that warning be sure to test every release in Alfred 2. 

 

 

Oh and Dean, shields.io has a good widget for download counts of GitHub releases (and plenty other interesting badges), here's an example URL :

 

https://img.shields.io/github/downloads/idpaterson/alfred-wunderlist-workflow/0.6.0-beta.3/total.svg 

[deanishe]

On an almost entirely unrelated note, after pulling some download stats from GitHub, the numbers are uncannily similar to the Packal ones. Almost exactly a 50/50 split.

[Andrew]

Hey chaps, just a quick message to say I have seen this thread.

 

Just a few notes:

• ipaterson is correct that during V2's release, object versions increased for particular objects when features were added, and will continue to update during the life of v3 when changes are made to objects.
• It's absolutely correct that Alfred shouldn't be installing a workflow if it's not compatible (when installed through Alfred itself). Installing and then showing that it's incompatible isn't a good experience at all. I've added a note for the next update of Alfred 2 and Alfred 3 to prevent incompatible workflows from being installed.
• I've never really liked the loose coupling of an AppleScript Output calling a Workflow Trigger. While the need for doing this is hugely reduced in Alfred 3 by allowing object chaining, I am going to address this in a future version by adding a "Call Workflow Trigger" output object. This will not only allow other workflows to be called, but also give a very tight coupling when calling back on the current workflow regardless of naming etc. This should better future-proof workflows.
• It's unlikely that i'll be changing the workflow extension for Alfred 3, but I can add a field into the info.plist to say which Alfred version the workflow was exported from, but again, the compatibility is more down to the specific workflow object versioning as ipaterson spotted.

It's also worth mentioning that we currently haven't been pushing Alfred 3 particularly hard from a marketing perspective as there is only so much Vero and I can do at once, and as you can imagine, the post v3 release period is manic. As we start enlightening Alfred 2 users that Alfred 3 exists (as it doesn't auto update to v3, many don't even know yet), and with experience of what happened with Alfred 1 to Alfred 2, it won't be long until the vast majority of users are on Alfred 3.

 

Cheers,

Andrew

[deanishe]

I am going to address this in a future version by adding a "Call Workflow Trigger" output object. This will not only allow other workflows to be called, but also give a very tight coupling when calling back on the current workflow regardless of naming etc. This should better future-proof workflows.

Why not just improve the External Trigger UX? I mean, it's not like you have to use AppleScript to call back into Alfred. We just do it because External Triggers aren't very good.

 

It's unlikely that i'll be changing the workflow extension for Alfred 3, but I can add a field into the info.plist to say which Alfred version the workflow was exported from, but again, the compatibility is more down to the specific workflow object versioning as ipaterson spotted.

Shame. One way or the other, we have to use a different extensions in GitHub releases, simply because of the way they work. It'd reduce developer effort and head off user confusion if Alfred also recognised the extension.

Oh well.

@ipaterson: Guess it's our choice then. .alfred3workflow? .alfredworkflow3? .zip?

[ipaterson]

@ipaterson: Guess it's our choice then. .alfred3workflow? .alfredworkflow3? .zip?

 

I'm not keen on changing the extension in GitHub releases if the file won't be executable without renaming for those who download it directly. I would rather see a solution that requires developers who are concerned about the transition to do a little extra work on their releases (e.g. adding a second binary or using a different tag format would both stop invalidate the release in old versions of Alfred-Workflow). If an alternate file extension is not going to happen the decision about how to approach the problem will be largely based on balancing developer effort (both yours and workflow developers) and mitigating the discontentment of users who choose not to pay for the Alfred 3 upgrade. Such a solution would be very specific to Alfred-Workflow rather than something nicely standardized across all workflow distribution channels so we should continue this part of the discussion on GitHub.

[Andrew]

I tell you what, I'll make it so Alfred also recognises alfred3workflow the same as alfredworkflow in 3.0.2... but exporting from Alfred 3 I'd like to keep as always .alfredworkflow

 

Why not just improve the External Trigger UX? I mean, it's not like you have to use AppleScript to call back into Alfred. We just do it because External Triggers aren't very good.

 

The External Trigger will stay the same, all I'll be doing is adding a better way of calling the external triggers from within workflows using a specific "Call Workflow Trigger" object, avoiding AppleScript altogether

[deanishe]

That's plenty good enough for my purposes. Thanks very much!

[ipaterson]

Thanks, Andrew! 

[noivad]

After the upgrade breaking some of my “prized*” workflows, I’m looking forward to this change. Since I already know AppleScript, it’s easier for me to write & call an Applescript embedded in a workflow. Perhaps next update the workflow imported can parse for calls to “Alfred 3” and change it to the next rev? At least the solution is embedded in the example text, so it was a 5 second fix to go to each script and change from “tell application "Alfred 2"” to “tell application "Alfred 3".” So, ~ a minute or two to ferret out all the scripts. Thanks.

 

*okay most were trivially easy using Applescript

Edited June 8, 2016 by noivad

[deanishe]

Perhaps next update the workflow imported can parse for calls to “Alfred 3” and change it to the next rev?

 

 

I'm afraid I don't think that's realistic. A lot of workflows generate their AppleScript in scripts written in less insane languages. .scpt files are also not plain text, but compiled scripts.

[Carlos-Sz]

Regarding compatibility:

 

- Some workflows using AppleScript call Alfred 2 application to use its dictionary (I know Andrew know about it)

 

- In addition, some workflows have the Alfred 2 data folder path hard coded and they have to be changed to the Alfred 3 one

 

All my workflows have to be updated and, after that, they will only work with Alfred 3.

 

Just in time: there should be an option to export a workflow to an Universal Alfred version or only for Alfred 3. However, I´m not sure if there will be too much Alfred 2 users after all.

Edited June 9, 2016 by Carlos-Sz

[Andrew]

- In addition, some workflows have the Alfred 2 data folder path hard coded and they have to be changed to the Alfred 3 one

 

Alfred 2 and 3 both have environment variables for the data folder and cache folder, if you use these, they are cross compatible.

 

- Some workflows using AppleScript call Alfred 2 application to use its dictionary (I know Andrew know about it)

 

Once I've added a way to call a workflow trigger from within a workflow without needing AppleScript, this should future proof us against incompatibilities

[vitor]

Once I've added a way to call a workflow trigger from within a workflow without needing AppleScript

 

Now that this has been implemented in pre-release and I had a chance to try it, I can say that although the new node seems to work reliably and is very clear, I still miss the option to just call Alfred (as opposed to Alfred 3) from AppleScript. Reason being we still need to call Alfred 3 when making a call from outside an Alfred Workflow. I have a few workflows that allow the user to install macOS services and launchd jobs, and those will still need to be updated (and break compatibility) with new Alfred versions.

 

iTerm seems to accept both iTerm and iTerm2 when being called via AppleScript. Why not make Alfred accept bot Alfred and Alfred 3? If necessary, the call could even be changed from tell application "Alfred 3" to run trigger "trigger_name" in workflow "workflow_id" with argument "test" to something like tell application "Alfred" to run trigger "trigger_name" in workflow "workflow_id" with argument "test" minversion "3".

Edited August 11, 2016 by Vítor

[deanishe]

Absolutely. Developers changing the name of their applications is a PITA for anyone trying to script it.

 

The AppleScript baffles me. The API remained the same, so every single AppleScript would have continued to work perfectly on Alfred 3. But changing the application name broke every single AppleScript instead, and also made it very difficult to support both versions.

[Andrew]

Hmm, I didn't even realise that I could make Alfred's AppleScript respond to Alfred and Alfred 3 - I'll put a ticket in to look at this as this would be ideal!

 

For internal calling though, I'd definitely recommend using the new object as the performance is night and day faster than AppleScript.

 

Cheers,

Andrew