Newbie question - Workflows relying on Python 2

[CWM]

I love Alfred!

And I'm hoping that someone can help me out. 

It appears that many of my workflows are broken. Although I can evoke them, when I attempt to trigger them, Alfred defaults to the standard Search Google, Search Amazon, and Search wikipedia pages.

For example, when I enter "CC" to evoke the "Case Converter 2"  workflow and transform the word "test" into another case, I just get the "Search Google" (etc) pages. 

I've attached a screen shot. 

It appears that some setting has changed. 

Can someone think of a solution?

Thanks!
 

[lycopodiopsida]

It seems, that macOS 12.3 (the last update) finally removed python2 from the system. The decision to do it in minor release is questionable, but it is what it is. A surprising number of older workflows is still written in python2 and authors never bothered to port them to python3.

 

You could install python2 manually, but it is not really advisable, since it received no updates/patches for 2 years and is a potential security issue. Other than that, there are not so many options: port the workflow to python3 by yourself, wait for author to do so or look for another.

[CWM]

31 minutes ago, lycopodiopsida said:

It seems, that macOS 12.3 (the last update) finally removed python2 from the system. The decision to do it in minor release is questionable, but it is what it is. A surprising number of older workflows is still written in python2 and authors never bothered to port them to python3.

 

You could install python2 manually, but it is not really advisable, since it received no updates/patches for 2 years and is a potential security issue. Other than that, there are not so many options: port the workflow to python3 by yourself, wait for author to do so or look for another.

 

Ah, ok! That explains it. Thanks so much . . . I had no idea

[bwbecker]

I just gotta add that it seems really, really lame on the part of the developers to not provide an easy way to invoke python3.  

 

The "Script Filter" and "Run Script" both have drop-downs to choose your language.  Both have "/usr/bin/python" disabled on Monterey.  It seems nearly trivial to add /usr/bin/python3 to that list.

 

[vitor]

@CWM See the Updated Third-Party Python 2 Workflows GitHub page. It explains the situation, links to instructions on regaining access to Python 2 and includes fixed versions of Case Converter and others.

 

@bwbecker It is trivial to invoke Python 3. Save your script inside the Workflow with a proper shebang (#!/usr/bin/python3) and call it as an External Script as the Language. Or use /bin/zsh as the Language and call your script with /usr/bin/python3 SCRIPT_NAME. You were never limited by the languages in the dropdown, you can easily use anything on your system. 

[CWM]

12 minutes ago, vitor said:

@CWM See the Updated Third-Party Python 2 Workflows GitHub page. It explains the situation, links to instructions on regaining access to Python 2 and includes fixed versions of Case Converter and others.

 

@bwbecker It is trivial to invoke Python 3. Save your script inside the Workflow with a proper shebang (#!/usr/bin/python3) and call it as an External Script as the Language. Or use /bin/zsh as the Language and call your script with /usr/bin/python3 SCRIPT_NAME. You were never limited by the languages in the dropdown, you can easily use anything on your system. 

Thanks so much, Vitor. Big help!

[bwbecker]

@vitor, yes, it's trivial to invoke Python3 using one of the methods you mentioned.  But that also holds true for php, ruby, python2, etc listed in that menu.  Why give them special support but not Python3?

 

In any case, thanks for documenting the options and taking the time to reply.  Appreciated.

[deanishe]

14 hours ago, lycopodiopsida said:

authors never bothered to port them to python3

 

That's because Apple never bothered to include Python 3 with macOS. But sure, blame workflow developers for using the tools Apple gave them.

[deanishe]

12 hours ago, bwbecker said:

It seems nearly trivial to add /usr/bin/python3 to that list

 

Python 3 isn't actually there on a normal system. It's part of Apple's developer tools. So you can't just add /usr/bin/python3 to the dropdown list.

[lycopodiopsida]

51 minutes ago, deanishe said:

 

That's because Apple never bothered to include Python 3 with macOS. But sure, blame workflow developers for using the tools Apple gave them.

I will not go them off the hook: writing is on the wall for Python2 since two years at least - from python developers and from apple's side. The problem is python2 being outdated and not supported, thus not trivial to install anymore with e.g. homebrew. Shipping an python3 distribution is trivial, alfred could provide an own.

[vitor]

2 hours ago, lycopodiopsida said:

Shipping an python3 distribution is trivial, alfred could provide an own

It is not trivial. Shipping a Python 3 installation means maintaining it, including in two architectures, plus every other language (why is Python special?). It means bloating the Alfred distribution, debugging a bunch of runtimes, keep up-to-date with every version change, handle deprecations, handle every request for every niche or hard to embed language…

 

Apple—literally the richest company in the world—decided that was too much of a hassle to keep doing even when they were making a subpar job at it, and you’re suggesting Alfred’s development essentially grinds to a halt replicating the scenario.

 

No one is saying the solution is to do nothing, but good solutions take time.

 

2 hours ago, lycopodiopsida said:

The problem is python2 being outdated and not supported

No, the problem is Apple making a major breaking change in a point release. No one batted an eye with the removal of PHP because one expects a degree of change with major releases. With Python, everyone got a month or so of warning and many were caught unaware. It’s hard to prioritise when the rug is pulled from under you.

Edited March 17, 2022 by vitor
Removed rogue newline

[lycopodiopsida]

37 minutes ago, vitor said:

It is not trivial. Shipping a Python 3 installation means maintaining it, including in two architectures, plus every other language (why is Python special?). It means bloating the Alfred distribution, debugging a bunch of runtimes, keep up-to-date with every version change, handle deprecations, handle every request for every niche or hard to embed language…

 

Apple—literally the richest company in the world—decided that was too

much of a hassle to keep doing even when they were making a subpar job at it, and you’re suggesting Alfred’s development essentially grinds to a halt replicating the scenario.

Apple has other implications - apple shipping a python distribution with the current read-only system partition means either a drastically increased system update cadence, or a potential security issue for all users in case of a vulnerability. The time of kitchensink-style operation systems, shipping everything - from JVM to web server - is gone, for better or worse. Getting these tools/languages installed is also pretty easy these days.

 

37 minutes ago, vitor said:

No, the problem is Apple making a major breaking change in a point release. No one batted an eye with the removal of PHP because one expects a degree of change with major releases. With Python, everyone got a month or so of warning and many were caught unaware. It’s hard to prioritise when the rug is pulled from under you.

Apples remove schedule is as erratic as ever, but to be fair python2 was marked as deprecated in Catalina in 2019. Thus, it is hardly a surprise. What weights heavier, is that there is no support from Python development team for Python2 since 2 years. The timing may be wrong (should have been with the Big Sur release), but the action is still justified.

Edited March 17, 2022 by lycopodiopsida

[vitor]

34 minutes ago, lycopodiopsida said:

Apple has other implications - apple shipping a python distribution with the current read-only system partition means either a drastically increased system update cadence, or a potential security issue for all users in case of a vulnerability.

 

Which supports my argument. If a company with limitless resources can’t do it, it’s unreasonable to expect that from those on a much smaller scale.

 

34 minutes ago, lycopodiopsida said:

Getting these tools/languages installed is also pretty easy these days.

 

Yes it is. Which is why Apple doesn’t ship them anymore and Alfred shouldn’t either.

 

34 minutes ago, lycopodiopsida said:

What weights heavier, is that there is no support from Python development team for Python2 since 2 years.

 

It weighs even heavier that they botched it multiple times and kept reneging on the deprecation. Python 2 was EOL in January 2020 and they still made another release in April. The whole thing was a mess and you couldn’t trust the team behind it. It caused a bunch of problems for Homebrew to the point we had to decide to just get on with removing it.

 

34 minutes ago, lycopodiopsida said:

Thus, it is hardly a surprise (…) the action is still justified.

 

No one is arguing it was unexpected or unreasonable that they made the removal.

 

34 minutes ago, lycopodiopsida said:

Apples remove schedule is as erratic as ever (…) The time may be wrong (should have happen with Big Sur release)

 

The argument is that the removal wasn’t done right, and it could’ve been.

 

We seem to be mostly in agreement on the facts, so it baffles me that you’re excusing a multi-trillion dollar behemoth who should know better while refusing to give a pass to an indie developer which has always done right for the users. Especially when the difference is the big company is taking something away without replacement and you’re demanding the independent one adds on all the work when the tradeoffs are immense.

 

Again, no one is saying there’s nothing to be to mitigate or improve the situation, but let’s get things into perspective and make sure our priorities are in the right place.

Edited March 17, 2022 by vitor

[lycopodiopsida]

My point is not about counting apple's money, but whether shipping such things as python with operation system is justified. I think it is fair to say that there is a general agreement in the developer's community these days, that it is not.

How Alfred solves it, is an entirely different question which is not important, since this problem is solvable - from providing instructions for homebrew to triggering installation for xcode-tools or leaving it completely up to workflow-authors. And to be honest, even if apple would remove in 2 more years, there would be be people completely surprised by it.

[vitor]

5 minutes ago, lycopodiopsida said:

How Alfred solves it, is an entirely different question which is not important

 

I now realise I misread your earlier comment. You were saying you won’t let developers “off the hook”. That has not been the focus of my argument, which explains why we seem to mostly agree but have different conclusions. Apologies for the confusion.

 

1 minute ago, lycopodiopsida said:

but whether shipping such things as python with operation system is justified. I think it is fair to say that there is a general agreement in the developer's community these days, that it is not.

 

I would say it’s at best contested. Those who provide quick tools for others and believe in lowering the barrier to learning programming generally think tools should ship with the OS. Isn’t that what most Linux distros do? Developers like @deanishe would have preferred to use a more recent version of Python but gave greater weight to the user convenience of not having to set up anything, thus used what the system provided. You seem to disagree with the choice—which is fair—but keep in mind many of those Workflows were made before Python was EOL, at a point where it was still reasonable to expect Apple might ship Python 3.

 

For what it’s worth, I’m opting to port my Workflows from Ruby. I’m doing so to preserve the user convenience of no setup when it gets the boot from macOS, so in a way I guess I’m agreeing with you regarding updating Workflows to the changing world. But I have one final thought to share, which is that all those Workflow developers gave away their work and time for free without asking for anything in return. Do they have an obligation to update their Workflows? I wouldn’t say that. Instead, I think that as a community we should empower them to do so by helping them when they ask and lowering their burden.

 

Do you have the availability and expertise to help Workflow developers port their Workflow to Python 3? If so, please do, there are others. If not, you may still be able to figure out what is preventing specific ports and help their creators overcome those hurdles.

 

[lycopodiopsida]

I can see the value in a widely used language such as python being delivered with the operation system, but... besides it being a very convenient attack vector most users are not aware about, I have to say that it is also not very useful as power user/developer. Even in an ideal world, where apple would take full responsibility and updates python in time there is still version-pinning, question of environments and so on. I think the general wisdom was "never use the macOS-provided python version for any serious task" so that is was more of a inconvenience since you have then to reconfigure aliases/links by hand.

 

Of course, part of the problem is than Python has handled the transition from 2 -> 3 in the worst possible way, cause a big rupture out of a pretty minor change. These days it would be done with some kind of "-lang_feature=python2" tag, but we all learned from it.  I don't know what is the position of Apple on Ruby, afaik it was used a lot in AppStore and old itunes app... but even if it will be excluded from the default shipment in future, it is pretty stable in a good sense. On the other side, I do not think that Python will make the same mistake again.

 

Workflow developers own us nothing, and I have fixed some incompatible workflows in the past. But on the other side, python2 cannot be provided forever just because some workflows are abandoned, especially since in that case the blame is more on python itself.

I have not touched python in years, and most code I wrote was in Python3, but as far as I remember the changes from 2 -> 3 were pretty trivial in most cases, there were even some automatic transition tools. If not, we should find people in the community who help port essential workflows to python3. VSCode/Pycharm CE are free as a beer, after all. I am in the middle of moving my household, thus a bit of time-constrained, but for non-urgent workflows I could offer my help.

Edited March 17, 2022 by lycopodiopsida

[vitor]

4 minutes ago, lycopodiopsida said:

I think the general wisdom was "never use the macOS-provided python version for any serious task" so that is was more of a inconvenience since you have then to reconfigure aliases/links by hand.

 

Again, your view of the general wisdom doesn’t match mine. I expect none of us has definitive numbers on which group is larger, so it’s not worth pursuing further.

 

14 minutes ago, lycopodiopsida said:

Of course, part of the problem is than Python has handled the transition from 2 -> 3 in the worst possible way

 

Now that I would say is the general wisdom, and I’d wager the Python team might agree. Guido has even publicly said there may never be a Python 4.

 

16 minutes ago, lycopodiopsida said:

I don't know what is the position on Apple on Ruby, afaik it was used a lot in AppStore and old itunes app... but even if it will be excluded from the default shipment in future

 

It’s not about the language itself, but the external scripting language runtimes. In the initial announcement, Apple mentioned Python, Ruby, and Perl. Start the REPL for either and you’ll see macOS deprecation messages.

 

20 minutes ago, lycopodiopsida said:

But on the other side, python2 cannot be provided forever just because some workflows are abandoned, especially since in that case the blame is more on python itself.

 

No objections.

 

20 minutes ago, lycopodiopsida said:

VSCode/Pycharm CE are free as a beer, after all.

 

But they have multiple full-time employees of for-profit corporations working on them. That’s a far cry from a single person working on a thing in their free time amidst other obligations.

 

23 minutes ago, lycopodiopsida said:

but for non-urgent workflows I could offer my help.

 

Pinging @xilopaint. Someone else willing to help with Python 2 to Python 3 transitions of Workflows.

[deanishe]

1 hour ago, lycopodiopsida said:

I think it is fair to say that there is a general agreement in the developer's community these days, that it is not.

 

The only people who think that are the ones who think developer convenience trumps user convenience. I do not. I think asking normal users to unnecessarily install developer tools is very poor practice.

 

People should of course feel free to use whichever languages they want (it’s their time, not mine), but please do not minimise the huge inconvenience of asking regular, non-technical users to install developer tools.

 

Go read any of the threads for Node-based workflows that aren't aimed at developers and see all the fun it leads to.

 

4 minutes ago, lycopodiopsida said:

big rupture out of a pretty minor change

 

A pretty minor change? No non-trivial Python 2 program runs unchanged under Python 3. The one, massive, break-the-world change in Python 3 was how text processing works, which is most of what workflows do.

 

37 minutes ago, lycopodiopsida said:

Python has handled the transition from 2 -> 3 in the worst possible way

 

Yes, yes they did. But there was still nothing minor about the change.

[lycopodiopsida]

15 minutes ago, deanishe said:

A pretty minor change? No non-trivial Python 2 program runs unchanged under Python 3. The one, massive, break-the-world change in Python 3 was how text processing works, which is most of what workflows do.

I would not claim that the changes were huge, just because Python2 programs do not run unchanged under Python3. If anything, it only proves that it was handled in a very bad way. Looking at Swift and Rust a lot is possible with feature switches and language editions, and Java is an example of a language of a comparable age, which handles the changes nicely, although it has removed some deprecated API's and had some module-related breaking changes with Java 9 which already caused enough problems. But even for huge changes, like project Valhalla, they are absolutely committed to backward compatibility. Python, on the other side... well, we know how it went.

Edited March 17, 2022 by lycopodiopsida

[lycopodiopsida]

17 minutes ago, vitor said:

But they have multiple full-time employees of for-profit corporations working on them. That’s a far cry from a single person working on a thing in their free time amidst other obligations.

It was meant more in the sense "everyone can help out, the coding tools are free"

[xilopaint]

6 hours ago, vitor said:

Pinging @xilopaint. Someone else willing to help with Python 2 to Python 3 transitions of Workflows.

 

Well, I’m starting with my own Alfred PDF Tools workflow. As the work gets stable enough to be released, which is not trivial since I adopted a new library for PDF manipulation, I will start to make transitions of workflows that I used to use. A lot of Python workflows take advantage from the non-Py3 compatible Alfred-Workflow library, which is why so many workflows still use Python 2 despite of the language constrains.

Edited March 17, 2022 by xilopaint

[lycopodiopsida]

3 minutes ago, xilopaint said:

 

Well, I’m starting by my own Alfred PDF Tools workflow. After the work gets stable enough to be released, which is not trivial since I adopted a new library for PDF manipulation, I will start to make transitions of workflows that I used to use. A lot of Python workflows take advantage from the non-Py3 compatible Alfred-Workflow library, which is why so many workflows still use Python 2 despite of the language constrains.

 

I see... it is indeed a problem also for the "Search Apple Music workflow" which is broken now - it would be trivial to fix it, but alfred module is imported. Is there any chance it could be ported to python3? I assume that it would ease the porting efforts for the rest of the workflows significantly.

[vitor]

25 minutes ago, xilopaint said:

A lot of Python workflows take advantage from the non-Py3 compatible Alfred-Workflow library

 

15 minutes ago, lycopodiopsida said:

Is there any chance it could be ported to python3?

 

See deanishe/alfred-workflow#97. Near the bottom there are people who started on forks for Python 3 compatibility. @deanishe is the only one who can say for sure, but for now it seems unlikely the library will be updated officially.

[xilopaint]

3 hours ago, lycopodiopsida said:

 

I see... it is indeed a problem also for the "Search Apple Music workflow" which is broken now - it would be trivial to fix it, but alfred module is imported. Is there any chance it could be ported to python3? I assume that it would ease the porting efforts for the rest of the workflows significantly.

 

No contribution to this transition would be better than a solid ported version of the Alfred-Workflow library. Not only a dirty version that "works" but something with good, clean and maintainable code. It would speed up a lot the porting of the Python workflows. But I would still wait for @deanisheto say about his plans on the future of the library before starting (or not) a community version of the library.

 

Btw, I have my own altered version of the Alfred-Workflow library. Not perfect but still cleaner in some aspects than the forked version some devs are using.

Edited March 17, 2022 by xilopaint

[vitor]

@jeffsui is porting EmojiTaco to Python3. @Acidham, whose majority of (all?) Workflows are in Python 3, may also be interested in the conversation or have some insights.