Alfred 2.0.3 doesn't respect the user locale in bash scripts, unexpected encoding

[mklement0]

Bash scripts launched by Alfred in the context of workflows don't respect the OS X user's locale and instead default to the generic "C" locale.

 

Alfred should use the same locale that is used when a user starts an interactive shell in Terminal.app

 

(For instance, running locale in Terminal on my US-English system returns:

 

LANG="en_US.UTF-8"

LC_COLLATE="en_US.UTF-8"

LC_CTYPE="en_US.UTF-8"

LC_MESSAGES="en_US.UTF-8"

LC_MONETARY="en_US.UTF-8"

LC_NUMERIC="en_US.UTF-8"

LC_TIME="en_US.UTF-8"

LC_ALL=

)

 

To verify the problem, use the following test workflow:

• download and install https://dl.dropboxusercontent.com/u/10047483/localetest.alfredworkflow
• type localetest into Alfred, which will display the effective locale in large type

 

To work around the problem, use the following at the beginning of your script:

 

export LANG="$(defaults read -g AppleLocale).UTF-8" 

 

Possibly related [update: NOT related - see below]: non-ANSI characters are currently encoded in unexpected ways:

 

In normal UTF-8 encoding, "ü" should be passed in (via {query}) as the following multi-byte sequence:

• expected: 0xc3 0xbc

• instead, Alfred currently passes: 0x75  0xcc  0x88   - 3(!) bytes

Edited April 25, 2013 by mklement0

[_mk_]

+1 for respecting the user locale in Bash scripts.

 

The issue with the non-ANSI chars was already discussed here and here. The unexpected encoding is due to the decomposition done by NSTask. There is already a workaround for this in the topics.

[Andrew]

In normal UTF-8 encoding, "ü" should be passed in (via {query}) as the following multi-byte sequence:

• expected: 0xc3 0xbc

• instead, Alfred currently passes: 0x75  0xcc  0x88   - 3(!) bytes

 

Alfred uses NSTask, not Terminal, for running scripts and the UTF-8 normalisation is happening outside of Alfred's hands. If you look at _mk_'s links, I have created a workaround for this issue with a small command line app to renormalise UTF-8.

 

I've moved this to 'investigating', but there isn't much Alfred can do for now without overhauling some low level build in Cocoa framework.

[mklement0]

Thanks, _mk_ and Andrew.

 

(As an aside: I should have searched the forums for "encoding"; however, I did search for utf (no quotes; rejected as too short), then "utf", "utf8", "utf-8" - none of which produced hits, even though they should.)

 

I've updated my original post to (a) point out that the encoding issue is unrelated to the locale issue and ( b ) to include a workaround for the locale issue.

To reiterate, add the following at the beginning of your script:

 

 

export LANG="$(defaults read -g AppleLocale).UTF-8"  

Edited April 25, 2013 by mklement0

[mklement0]

As it turns out, when you invoke a shell script via do shell script from an NSAppleScript action, UTF-8 encoding works as expected (no decomposition).

 

Based on that, I came up with a workaround that ensures:

• that the user's locale is in effect in the bash script invoked
• that UTF-8 encoding works as expected - without the need for an external utility.
• that the entire query string is passed correctly to the script invoked as a single parameter - AppleScript takes care of escaping with quoted form of.

The basic approach is to have a shell script invoked with a carefully crafted command line from a helper NSAppleScript action rather than directly.

 

Because do shell script defaults to / as the working directory (rather than the workflow's folder), I had to determine the workflow's own folder beforehand, which turned out to be non-trivial.

Aside: Does Alfred provide a way to for a workflow to query its own folder path?

 

The workaround is demonstrated in the following workflow:

• https://dl.dropboxusercontent.com/u/10047483/encoding%20and%20locale%20in%20scripts%20-%20workaround%20demo.alfredworkflow
• Invoke it as follows, to see how an accented character is encoded and what locale is in effect: encodingandlocaledemo ü

Given the amount of code involved, I'm not sure if the workaround is worth it, but perhaps there's something useful in there.

 

Here's the code stored in the NSAppleScript action:

on alfred_script(q)
  # -------- BEGIN: MUST BE CUSTOMIZED
  # * Create a bash script in this workflow's folder,
  #   which will be invoked below.
  #   IMPORTANT: use a UNIQUE NAME, such as created with `uuidgen`, e.g.:
  #     "D0ACD509-E05A-4B5C-B45A-B2A7DECE0938.sh"
  # * Fill in this unique name on the next line.
  set shScriptName to "D0ACD509-E05A-4B5C-B45A-B2A7DECE0938.sh"
  # -------- END: MUST BE CUSTOMIZED

  # The workaround command string that ensures
  # - use of the current OS X user's locale
  # - normal (non-decomposed) UTF-8 encoding of non-ANSI characters.
  set workaroundCmd to "export LANG='" & user locale of (system info) & ".UTF-8'"  

  # Determine the parent folder of where Alfred's preferences are stored.
set prefsFolderParent to do shell script "defaults read ~/Library/Preferences/com.runningwithcrayons.Alfred-Preferences.plist syncfolder | sed -E 's:^~:'$HOME':'"

  # Determine THIS workflow's folder, via globbing and its unique script name.
  # Note: We canNOT use a workflow UID, because it changes on every export
  # and import.
  set thisFolder to do shell script "dirname \"$(echo " & quoted form of (prefsFolderParent & "/Alfred.alfredpreferences/workflows/user.workflow.") & "*/" & quoted form of shScriptName & ")\""

  # Synthesize the cd command to change to this workflow's folder.
  set cdCmd to "cd " & quoted form of thisFolder

  # Construct the whole command line, passing q as a single,
  # properly escaped parameter.
  set cmdLine to workaroundCmd & "; " & cdCmd & "; /bin/bash ./" & shScriptName & " " & quoted form of q

  # Invoke the entire command line and pass out its stdout output.
  do shell script cmdLine

end alfred_script

Edited April 25, 2014 by mklement0

[_mk_]

I think the AS solution is a bit overkill just to get the user locale. Doing the re-normalisation of the query with Andrews tool and getting the locale with

 

export LANG="$(defaults read -g AppleLocale).UTF-8" 

seems the be the easier solution for me. That way you can also avoid the logic to find the workflow directory.

[mklement0]

Makes sense, _mk_ - I was seduced by the lure of the workaround solving three issues at once, but it turned out to be quite a bit more complex than I thought.

Edited August 13, 2014 by mklement0