Alfred uses Python 3.7 instead of 3.8.2

[Acidham]

When I add shebang #!/usr/bin/env python3 to my python WFs Alfred is always picking up Python 3.7.3 but I am expecting version 3.8.2.

 

When I run /usr/bin/env python3 -V on iTerm or Terminal I am getting version 3.8.2

 

I checked the PATH in Alfred WF python script and I am getting: PATH:  /usr/bin:/bin:/usr/sbin:/sbin

In iTerm or Terminal, I am getting: /Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/usr/local/go/bin:/Users/jjung/Library/Python/2.7/bin:/usr/local/opt/fzf/bin

 

I am using zsh and therefore I checked which SHELL the Alfred WF python script is using with os.environ['SHELL']: /bin/zsh

 

With where python3 I am getting:

where python3
/Library/Frameworks/Python.framework/Versions/3.8/bin/python3
/usr/local/bin/python3
/usr/bin/python3

 

 

Conclusion: Alfred is using zsh shell which I am also using but why it PATH in Python script is different form PATH in shell?

 

I am not able to get Python 3.8.2 used in my scripts. What I am missing here?

Edited April 5, 2020 by Acidham

[vitor]

Using the same shell doesn’t mean you’re using the same environment.

You’re bound to have some configuration file (likely .zshrc) which is setting the desired python version as the default when used inside a shell. This might not be an outright addition to PATH, but an external tool like a python version manager.

A quick and dirty way to fix this would be to source the shell configuration before calling the script.

[Acidham]

Ok got it, but this means that Alfred uses own env? Is there a way to configure Alfred's zsh env? 
 

strange, I never experienced this issue but since I decided to ignore Python Version < python 3.8 this seems to be a bummer 😕

[deanishe]

2 minutes ago, Acidham said:

Ok got it, but this means that Alfred uses own env? Is there a way to configure Alfred's zsh env?

 

Not really. Alfred uses the same env as everything else. Your shell uses its own env.

 

If you want to use a non-standard Python interpreter, use its absolute path. Messing around with your global macOS environment is a bad idea because you'll end up publicly releasing a workflow that only runs on your system.

[Acidham]

3 minutes ago, deanishe said:

Messing around with your global macOS environment is a bad idea because you'll end up publicly releasing a workflow that only runs on your system.

 

This is what I want to avoid. Therefore I thought that good practice is to use/usr/bin/env python3 shebang. I thought when I  would use absolute path to python 3.8 interpreter I am releasing a WF that just works on my system 😕

[deanishe]

Just now, Acidham said:

Therefore I thought that good practice is to use/usr/bin/env python3 shebang.

 

There's good and bad to be said about it. Using /usr/bin/env python3 in my personal scripts means they'll continue to work on Catalina without my needing to install a Homebrew Python. OTOH, it also means they might die in flames if I'm in a virtual env.

 

If you're distributing the software, it's also generally not a bad idea to lock it down to a specific interpreter, so you don't accidentally develop using a different version to all your users, and so the script isn't overly affected by some user's exotic environment, where /usr/bin/env python resolves to some stripped-down Miniconda (or indeed Python 3).

[Bhishan]

You are in better luck than me. My Alfred uses Python 2. For each workflow I write separate python script and run in Alfred with `/Users/poudel/miniconda3/bin/python myscript.py` rather than typing python script right on the Alfred objects.

[deanishe]

8 hours ago, Bhishan said:

rather than typing python script right on the Alfred objects

 

Yes, this is exactly how we are running the scripts, too.

[Acidham]

On 4/7/2020 at 4:03 AM, Bhishan said:

rather than typing python script right on the Alfred objects.

Confused now. Is this approach not binding your workflow to your machine only?

 

@Andrew not sure if that makes sense but an Alfred enhancement to allow users to configure env would solve this problem or am I wrong ? 

[vitor]

8 hours ago, Acidham said:

@Andrew not sure if that makes sense but an Alfred enhancement to allow users to configure env would solve this problem or am I wrong ? 

 

That would make it worse. When you customise an environment, you’re making it less like other people’s. Alfred’s environment needs to be as bare as possible so that there are as few surprises as possible. That’s presumably why Alfred’s environment doesn’t even include /usr/local/bin in its PATH: that location is too unpredictable. Alfred should and does only rely of what it can trust the system to provide.

[deanishe]

9 hours ago, Acidham said:

Confused now. Is this approach not binding your workflow to your machine only?

 

No. Not when you use /usr/bin/python script.py "$1" in the Script box.

 

A lot of developers have done it that way for years because it's better than pasting loads of code into the Script box. Now there's the External Script option, though.

 

9 hours ago, Acidham said:

allow users to configure env would solve this problem or am I wrong

 

You can already do that by setting Language = /bin/bash and treating the Script box like a shell:

PATH=/usr/local/bin:$PATH python3.8 myscript.py

There are also ways to change the global environment Alfred uses if you really want to. But that can cause the problems Vítor mentioned, so Alfred actually takes steps to prevent you from doing that (it won't let you alter PATH in workflow variables) rather than making it easier.

Edited April 8, 2020 by deanishe

[Acidham]

@deanishe thx, the latter is how I am doing it right now. I assume most other user can change PATH if necessary.

 

Thank you @deanishe and @vitor for quick and fast help on that...much appreciated!

[deanishe]

On 4/8/2020 at 8:42 AM, Acidham said:

Confused now. Is this approach not binding your workflow to your machine only?

 

Uh, I just realised you were replying to @Bhishan, not me. So my answer made no sense…

 

Yes Bhishan's script will only run on his system and nobody else's. Using an "exotic" Python that only exists on your system is one thing, but saving your scripts in that Python's "bin" directory is a very bad idea (unless it's just symlinked there). It's a great way to lose a lot of your code when re-installing or updating a virtual env.

[Acidham]

@deanishe All  good

 

In the meantime I played around on how to run python3 script but in the end i need to build my wf for 3.7.3 since this is the default for macOS. One of the biggest pain with python is that setup mess. pyenv helps to work around but I am still not happy...But this is another topic 😉

[deanishe]

2 hours ago, Acidham said:

i need to build my wf for 3.7.3 since this is the default for macOS

 

This is the right choice, imo. What's new in 3.8 that makes you so keen to use it?

 

2 hours ago, Acidham said:

pyenv helps to work around but I am still not happy

 

I am not at all a fan of pyenv. It breaks everything.

[Acidham]

21 hours ago, deanishe said:

What's new in 3.8 that makes you so keen to use it?

 

1. I am seeing that some of my workflows are getting a performance boost.
2. I started to play around with type hints and I am seeing a big value when it comes to maintainability of my code
3. The whole unicode mess that I recently struggled with German umlauts seems to be easier to handle with 3.8 (with 2.7 it drove me nuts, really!)
4. Assuming Apple keeps 3.7.3 for the next 20 years I would like to be flexible instead of struggling on an old version and fight with things like this unicode sh***

[deanishe]

1 hour ago, Acidham said:

I started to play around with type hints and I am seeing a big value when it comes to maintainability of my code

 

You should try a statically-typed compiled language (Go, Swift etc.). They catch a lot of bugs and run stupidly faster.

 

1 hour ago, Acidham said:

• The whole unicode mess that I recently struggled with German umlauts seems to be easier to handle with 3.8 (with 2.7 it drove me nuts, really!)
• Assuming Apple keeps 3.7.3 for the next 20 years I would like to be flexible instead of struggling on an old version and fight with things like this unicode sh***

 

I think you're confusing 3.7 and 2.7. Your Unicode issues are with Python 2. Python 3.7 works exactly the same as 3.8. Earlier versions of Py3 have a really stupid Unicode issue, but 3.7 finally fixed it.

 

Unicode in Python 2 is a bit fiddly, but not complicated. You just have to remember to encode and decode at IO boundaries. Python 3 has some much worse Unicode issues. They're harder to run into, but also much harder to fix.

Edited April 17, 2020 by deanishe

[Acidham]

Quote

you should try a statically-typed compiled language (Go, Swift etc.)

 

I started with GO but I am not there to really use it. So far I am totally happy with Python and I can use type hints without the IDE is constantly interrupting me like GO.

 

I am ok with Python 3.7 but as mentioned I am looking for something more scalable like the usecase "Apple decides to stay with 3.7.3 for the next 100 years". Therefore my ask to build env config into Alfred which allows to customize environment if required.

[vitor]

1 hour ago, Acidham said:

like the usecase "Apple decides to stay with 3.7.3 for the next 100 years".

 

That hasn’t happened with Ruby. With each macOS release, Apple has upgraded it, going from 1 to 2 without a hitch. The reason Apple stayed with Python 2.7 for so long is due to the Python 2 to Python 3 kerfuffle. And it’s a mess that still continues: Python 2 has been officially declared EOL in January, and yet they’ll still have another release in April. It’s a huge headache. I’ve never seen anyone saying “this transition is going quite well”, no matter how much you love Python.

Tangentially, it’s not that Apple didn’t want to upgrade to Bash 4, it’s that it’s not feasible due to licensing (GPL).

So I wouldn’t say that criticising Apple is fair in this instance. They’ve been updating when they can, following the most conservative approach when in doubt.

Since Apple is deprecating the bundled language runtimes, we’d be lucky if they stick with Python 3.7.3 for one year, let alone one hundred.

[deanishe]

2 hours ago, vitor said:

I’ve never seen anyone saying “this transition is going quite well”, no matter how much you love Python

 

Yeah, because not many people actually transitioned. A lot of the people using Py3 are people who never used Py2. Many devs either stuck with Py2 or started using a different language because Py3 is awful for the things they used Py2 for.

 

It took them 10 years—till 3.7—to fix the ridiculous behaviour that stopped me from ever using Python 3. By that time I'd long lost all confidence in the Python core team, and switched to Go when they announced Py2 would be EOL'ed.

[Acidham]

Even this whole python setup and version is mess, Python is the only language that allows me to write quick and "dirty" code for achieving common task on my computer or Alfred. I like to have a the ability to control of how a language or IDE keeps me honest instead only the system applies the truth and nothing else. Go, for example, sometimes drives me nuts because I would like to achieve something quick and then the IDE always give me a slap in the face 😕 . And when I try to achieve the same I am horrible fast. But in the end it counts how big my investment is in something vs what value I am getting out of it and therefore Python is a good match. 

 

Anyhow I don't want to start debate on principles. I believe you guys do a great Job when it comes to your workflows... in the end the value and outcome counts and here 😉

 

Quote

Software is magic until you look into the code

 

[deanishe]

9 hours ago, Acidham said:

I would like to achieve something quick and then the IDE always give me a slap in the face 😕

 

Yeah, a higher-level language like Python is way better for simple scripts. But unless your IDE is screwed up, it will only complain about code that is factually incorrect. It's the same thing as type hints in Python, except done properly.

[Aviesh]

To reiterate, So what would be the currently best method of creating a portable workflow ?

 

Looks like /usr/bin/env python3 is not the way to go since default paths are not including /usr/bin/local which brew populates with python3 which appears to be the most common way to get python3 into OSX.

 

Furthermore, changing OXS's default $PATH looks like a bad idea both for portability and for OSXs apps predictability.

 

Executing the python script after setting PATH inside a wrapper shell script is the quickest workaround, am I right in my understanding ? ( Although it does not fully solve the issue since a user may need to change the PATH in case it's setup is not using brew )

 

 

 

 

Edited August 29, 2020 by Aviesh

[deanishe]

1 hour ago, Aviesh said:

To reiterate, So what would be the currently best method of creating a portable workflow ?

 

Use a language that is included with macOS.

 

Unless spending loads of time helping non-technical users install developer tools sounds like fun to you, of course.

Edited August 29, 2020 by deanishe