Please allow users to specify PATH used for workflows

[illegalhex]

Hi there,

 

it would be extremely helpful to allow users to specify the PATH being set for workflows.

1) It seems that workflows do not use the PATH environmental variables of the parent Alfred process group. If I configure the Alfred daemon via launchctl the Alfred proces wil have the path of ‘/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/opt/local/sbin’ as I intended, however any process spawned in workflows have a very different path of ‘/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin‘ which isn’t correct for my machine. 

2) Workflows now use zsh with the ‘—no-rcs’ flag, including the 1Password workflow which is how I found this problem, so zsh won’t read ~/.zshenv. That is normally how I ensure that /opt/local/bin and /opt/local/sbin are in the path since I want them to be set even for non-login and non-interactive shells, for example say an Alfred workflows . But with the ‘—no-rcs’ flag being set only /etc/zshrc will get read resulting in PATH being wrong for my machine. 

3) I can’t find where the ‘/opt/homebrew/bin:/usr/local/bin’ string is stored, I can tell that there is some string formatting somewhere when Alfred spawns zsh since if I change ‘/etc/zshenv’ to set PATH of ‘$PATH:/opt/local/bin:/opt/local/sbin‘ the shell has PATH of ‘/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/opt/local/sbin‘ so it would seem that Alfred is using ‘/opt/homebrew/bin:/usr/local/bin:$PATH’ but I can’t find where the string of ‘/opt/homebrew/bin:/usr/local/bin‘ is coming from. It doesn’t appear to be any user exposed setting nor can I find that in any plist file. Is it statically set in the binary? I really don’t want to do decompile the binary to find out. 
 

4) it seems odd that the path for homebrew is being set but not macports if there is no user configurable way of setting that value. To assume that no user would have binaries in ‘/opt/local/bin’ and ‘/opt/local/sbin’, to not inherent values from the parent process group, give no way to set PATH by the user, and to launch zsh in a mode that only reads ‘/etc/zshenv’ is frustrating from a user prospective. 
 

 

it would seem that unless Alfred exposes a way to change how PATH is set users are left with either using /etc/zshenv to ensure that PATH is correct which feels not great as editing system files on macOS comes with some issues, or ensure that binaries exist in the PATH being set. I could see something like ‘sudo mkdir -p -m 755 /opt/homebrew/bin’ and then symlink subdirectories for bin and sbin to /opt/local/bin and sbin but that feels worse than using /etc/zshenv. 
 

But unless Alfred has some way, even if it is something like edit a plist file, to set PATH users are going to need to go out of their way to accommodate Alfred’s quirks. 

[vitor]

Welcome @illegalhex,

 

Understanding the Scripting Environment explains everything.

[KingMob]

"Understanding the Scripting Environment" explains what Alfred's doing, but doesn't enable solving the problem, not for me, anyway.

 

In my case, Alfred and nix are incompatible (and it sounds like macports is broken, too). Nix stores multiple versions of programs, and adjusts the PATH to point to whatever version is needed. By ignoring the PATH, Alfred can't find executables, and by making its PATH immutable, users can't adjust it to fix their problems.

 

There's a related problem that's even worse. I can't fix other people's Workflows because Alfred hard-codes the executable locations. E.g., a Workflow that selected `/usr/bin/php` as its interpreter is broken if I install php with Nix, because `/usr/bin/php` doesn't exist. And since the path to php is absolute, the PATH envvar is irrelevant, even if Alfred could pick up one of my nix paths.

 

Hard-coding paths is a time-honored recipe for problems.

[vitor]

Welcome,

 

They aren’t incompatible, the solutions are at the end of that article as is the reason why this choice was made. Workflows aren’t broken by it, they would be if the reverse were true. If Alfred read your environment, workflow sharing wouldn’t work predictably between machines because you’d need to replicate the other person’s environment. By setting it to a standard, you can be more reasonably confident it will work.

You can change the PATH, it is not fixed. Before Alfred 5, it was common for developers to extend it (method 2) if the workflow had a dependency on some Homebrew package. That works the same today. If you find a workflow with /usr/bin/php, you can change it. Edit the workflow itself and/or submit a PR. Furthermore, for tools managed by macOS, developers should use the absolute path like /usr/bin/python3 if that works for their workflow because it ensures a more predictable experience that the developer can debug. Almost nothing in computing is all bad or all good; it’s always a tradeoff. More than a decade of Alfred development and evolution with the community informs these decisions.

In your own personal workflows you should do whatever you prefer. Workflows which are shared with the wider community should adhere to a standard that will work for most people. Homebrew is the de facto package manager for macOS. It’s what most people know and use and even the people who use an alternative package manager know of it thus have the chops to work around it. Furthermore it is self-contained so there’s not really an issue to having it alongside another package manager for use with Alfred. It’s also the package manager that gets the most attention from Apple. I could go on, but what matters is that it was a pondered decision, and Homebrew came out by far as the best choice for this case while still not stopping you from using something else if you so choose.

 

That was a bit long but I hope it helped to clarify. Have a great week!

[KingMob]

I tried method 2 first, and setting the PATH didn’t seem to have any effect, so I figured maybe the post was out-of-date. It’s why I commented in the first place. 
 

If you think it still works, I’ll try it again when I get home and post an update. 
 

In a more general sense, I understand your arguments, and I know why you would prioritize the most common setup. I’m not arguing that you support nix directly, only that there be working escape hatches.

[KingMob]

So, it turns out method 2 (setting the PATH) partially works. I can set PATH to something, but Alfred can't prepend to the existing PATH. E.g., if I set PATH to `/some/directory/bin:$PATH` Alfred treats "$PATH" there as a string literal, which I did not expect.

 

Regardless, focusing on the PATH is kind of a side issue; being able to set PATH doesn't fix the issue of Alfred hard-coding the available interpreter paths in the dropdowns.

 

I tried to work around it in one workflow by changing the interpreter to `/bin/bash`, and then using a shebang in the script field, but that doesn't appear to work, either. Neither `#!/usr/bin/env php` nor `#!/absolute/path/to/my/php` worked. I just got errors about PHP code being invalid bash syntax. I'm guessing that whatever Alfred does with the script field, it doesn't build an actual script file with it as is.

 

At this point, I'm looking at writing custom scripts to alter half a dozen workflows, which is not ideal, especially since I really think Alfred could better support this use case.

 

Why not a settings page that allows people to override where their interpreter for a language is? That could be compatible with Alfred's existing behavior, and is a very common way to support alternative setups.

[vitor]

29 minutes ago, KingMob said:

if I set PATH to `/some/directory/bin:$PATH` Alfred treats "$PATH" there as a string literal, which I did not expect.

 

$PATH isn’t a special value, it is interpreted by the shell. In any other language the syntax would be different. Reading it as a string literal is the only correct behaviour for that case. The PATH Alfred uses is listed in the linked page, so if you want to append to it you just need to add those. OR, as it is more common, you add it in the Run Script itself:

 

Quote

PATH="/some/path:/another/path:${PATH}"

some-command arguments

 

32 minutes ago, KingMob said:

I tried to work around it in one workflow by changing the interpreter to `/bin/bash`, and then using a shebang in the script field, but that doesn't appear to work

 

Well, yes, that wouldn’t work anywhere. A shebang is a clue for which interpreter to use when one isn’t specified. But you already specified one, /bin/bash, so the shebang is ignored. This is not an Alfred quirk, it would work the same in a terminal. If you write PHP code but tell Bash to execute it, it’s normal that it produces an error. You’re doing the equivalent to /bin/bash myscript.php but shebangs are only interpreted in cases like /path/to/myscript.php.

See Calling non-standard runtimes from Alfred for the proper way to do it. Again, this is how it would work in a terminal too. Alfred isn’t doing anything weird. In fact it is adhering to the letter to how this is expected to work. This is the Unix model that has been in place for decades and is used by all well-behaved software.

 

41 minutes ago, KingMob said:

Why not a settings page that allows people to override where their interpreter for a language is?

 

Because, as explained in the first link and the previous post, that’s how you get code which is broken and hard to diagnose, when everyone has different setups. Emphasis added:

 

Quote

Making your Workflow independent of external settings benefits you both as a Workflow developer and user: by ensuring a common starting point, changing this environment must be a conscious decision with clear consequences.

 

[KingMob]

I think you have a lot of internalized knowledge and/or documentation about how Alfred works that is opaque to outsiders, that I don't think you quite appreciate.

 

A lot of the links you shared had plenty of data, but didn't contain what I was looking for, and hadn't guided me to a solution. They might be clear to you after a decade, but I found them less helpful. Hence throwing stuff at the wall to try to decipher how Alfred operates. Sadly, that "Calling non-standard runtimes" link was actually pretty useful in revealing how Alfred uses the "Script" field; I wish you had shared that earlier.

 

Let me ask one question. Is the answer to my problem:

1. Alfred can't handle this
2. I'm going to have to alter the entry point of each affected workflow
   1. ...by extracting the contents of the "Script" fields into self-contained shell script files (e.g. "workflow.sh") and invoking them as "External Script"
   2. ...by choosing a shell from the dropdowns (e.g., /bin/bash), extracting the lang-specific parts into a separate file (e.g., workflow.php), and then invoking that in the "Script" body (e.g., "php workflow.php")
      1. ...optionally setting the PATH at the start of the "Script" body if necessary (e.g. "PATH=/path/to/php php workflow.php")
3. Something else?

 

[Andrew]

On 1/31/2024 at 3:51 PM, KingMob said:

There's a related problem that's even worse. I can't fix other people's Workflows because Alfred hard-codes the executable locations. E.g., a Workflow that selected `/usr/bin/php` as its interpreter is broken if I install php with Nix, because `/usr/bin/php` doesn't exist. And since the path to php is absolute, the PATH envvar is irrelevant, even if Alfred could pick up one of my nix paths.

 

Hard-coding paths is a time-honored recipe for problems.

 

Just to clarify, Alfred doesn't hard code paths. What you're selecting in the popup is the language you want to use, and Alfred will go off and find it in one of the following locations:
 

/usr/bin/
/opt/homebrew/bin/
/usr/local/bin/
/opt/local/bin/

 

For example, I have php installed via Homebrew, so my dropdown looks like this:

 

 

If one of these default languages doesn't suit you, then ultimately you should be using External Script. For advanced users, I always recommend using External Script regardless (with the script saved in the workflow's folder), as this allows you to more easily use version control on your scripts, limitless control over desired script language, and script environment.

 

You mention in your first reply that you're seeing /usr/bin/php - do you see this, or do you see /usr/bin/php [not installed]? If it's the latter, then Alfred hasn't found php in one of the paths above, otherwise you may have a broken version of php in /usr/bin/php.

 

Where do you have php installed?

 

On 1/31/2024 at 3:51 PM, KingMob said:

In my case, Alfred and nix are incompatible (and it sounds like macports is broken, too). Nix stores multiple versions of programs, and adjusts the PATH to point to whatever version is needed.

 

In this case, you should be using External Script as your selected language.

 

Thanks,

Andrew

 

 

[KingMob]

It says /usr/bin/php [not installed], which seems suggestive that it's hard-coded to look there. I can only speak for myself, but if it had said php, or php [not found], I wouldn't have assumed that.

 

I'll try External Script if I decide to proceed.

 

Thank you for your time.

[illegalhex]

Sorry for opening this and not following up, and I do appreciate the replies. 

 

Firstly, I do see binaries located in /opt/local/bin, see 00_workflow_creation.png, which is great but the problem that I'm dealing with is specifically consuming previously created workflow. Either workflows that I've created myself; and presumably I can control but let's put a pin in that; but importantly workflows someone else has created. The problem I have is that /opt/local/bin is only present for selecting which language to use, but that location isn't present when executing a workflow. 

Initially I thought that /opt/local/bin/python and /opt/local/bin/php are listed because I manually changed PATH via launchctl, see 01_launchctl, so that both the Alfred and the Alfred Preferences binary, have PATH set as `PATH=/opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin` since I've modified launchd to change the default path, see  02_alfred_processes for details but the important thing is that PATH is indeed set to what I've set via launchctl. I did that under the naïve assumption that the preferences and main application were using PATH from their running process group, but it's quite clear that can't be completely right since executing workflow do not have /opt/local/bin even when the preferences application can find executables in /opt/local/bin. At this point I took a different tack, I removed launchd from the equation and launch both Alfred and Alfred Preferences directly as my user so that I could easily control PATH and set it back to launchd's default of `PATH=/usr/bin:/bin:/usr/sbin:/sbin` and got the same result. Even though /opt/local/bin is no longer in PATH for that process.

 

At this point I was reasonably confident that Alfred isn't using the value of PATH set via launchd and having spawned processes use values inherent by the the process group. This largely leaves two options, that Alfred is hard coding PATH or Alfred is executing somesort of shell as my user to get PATH. The first option I discounted because of the statment of "Alfred doesn't hard code paths" so I focused on the second option. I have in my ~/.zshenv the single of `export PATH="$PATH:/opt/local/bin:/opt/local/sbin"` to ensure that macports binaries are always there, even when being ran as a non-interactive shell. To that end I've created a workflow that is `printenv` by both `/bin/zsh` and `/bin/zsh --no-rcs` and the only difference is exactly what I expected `/bin/zsh` has `/opt/local/bin:/opt/local/sbin` at the end of the path and `/bin/zsh --no-rcs` does not, see 03_workflow and 04_results. I then went to confirm that my ~/.zshenv file is indeed controlling that value and changed it to be `/opt/local/bin:/opt/local/sbin:$PATH` ran the workflow again with no change to `/bin/zsh --no-rcs` as I would expect, and `/bin/zsh` has a path of `/opt/local/bin:/opt/local/sbin:/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin` which does confirm that I can controlling PATH at the time of execution of `printenv` and via `~/.zshenv` but its quite clear that by the time zsh evaluates `~/.zshenv` PATH is already set to `/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin` which doesn't quite make sense. 

So another test to confirm my suspicion. I update `~/.zshenv` and the workflow as below. The idea is that since /etc/zshenv isn't setting anything the next file to be loaded is `~/.zshenv`—see `STARTUP/SHUTDOWN FILES` section under `man zsh(1)` if you aren't sure about the order that zsh loads files in—I should see PATH set as `/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin` then I export PATH with `/opt/local/bin:/opt/local/sbin` appended, then when the script is ran I should see that folder that doesn't exists, `/opt/homebrew/bin`, will be in PATH and zsh won't be able to find it. And then just to be sure, `~/.zshenv` was updated to have the single line of `echo "When reading /etc/zshenv the PATH is currently $PATH"`

```
echo "When reading ~/zshenv (sic) the PATH is currently $PATH"
export PATH="$PATH:/opt/local/bin:/opt/local/sbin"
echo "        I've updated PATH in ~/zshenv  (sic) to be $PATH"
```
```
for folder in $(echo "$PATH" | tr ':' '\n'); do [ ! -d $folder ]; echo "check for $folder returned $?"; done
echo "PATH is set to: $PATH"
```  

Indeed we end up with `When reading /etc/zshenv the PATH is currently /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
When reading ~/zshenv (sic) the PATH is currently /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
        I've updated PATH in ~/zshenv  (sic) to be /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/opt/local/sbin
` and being shown that the folder `/opt/homebrew/bin` since that folder doesn't exists, along with PATH being set to what ~/.zshenv set PATH to. See 05_path_is_strange. 

```
check for /opt/homebrew/bin returned 0
check for /usr/local/bin returned 1
check for /usr/bin returned 1
check for /bin returned 1
check for /usr/sbin returned 1
check for /sbin returned 1
check for /opt/local/bin returned 1
check for /opt/local/sbin returned 1
PATH is set to: /opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/local/bin:/opt/local/sbin
```

Now after all that, I have to conclude that Alfred _must_ be setting PATH as hard coded value. The folder `/opt/homebrew/bin` does not exists on this machine and yet it is included in PATH as soon as `/etc/zshenv` is read. That folder isn't part of PATH of the running process so it can't come from there. Alfred couldn't read from disk to emulate from a list since that folder doesn't exist. The only plausible explanation is that somewhere Alfred has a hard coded value of `/opt/homebrew/bin` and is setting that value as part of PATH when dispatching processes as part of a workflow. Where, and how I have no idea.

Speculation time, I can't say for certain, but I have a strong suspicion that Alfred has strings for `/opt/homebrew/bin`, `/usr/local/bin`, and `/opt/local/bin` hardcoded, somewhere. And while all three are used for searching for a scripting environment, only the first two are being used to set PATH during process dispatch from workflows. This would match my observed behaviour and the quite obvious amount of effort put in by the Alfred team about making sure that users have a stable path. I'm sure there is some history there, and honestly understandable. However, if that is the case, there is weird behaviour on where `/opt/local/bin` can be used for a scripting language but can't be used as location for any other binary which is very odd behaviour and strikes me as unintentional behaviour.

Happy to go into more details if you need them, I realize this is an info dump. I don't think I need to tell you that managing PATH is hard, you've obvious done a lot of work to make sure that most users don't need to think about this, but I'm pretty sure I've found a bug that has slipped by since this a non standard use case. 

[vitor]

You’re conflating the language dropdown (not hardcoded) with the PATH variable (which is hardcoded but can be overridden; see method 2). They are different things. In addition, ~/zshenv isn’t a standard file to be read by Zsh, ~/.zshenv is (note the leading dot). But that will only be read when the language is set to /bin/zsh, not when it’s /bin/zsh --no-rcs or any other non-Zsh language. launchctl doesn’t figure into it.