Problems and Fixes for Applescript UI Scripting

[rice.shawn]

A troubleshooting conversation in the "Mavericks Merge All Finder Windows" workflow thread recently pointed out that errors can easily arise with UI Scripting because the variable names change. For instance, the command in that workflow was

 

click_menu('Finder','Window','Merge All Windows')

where click_menu was a custom function to make UI scripting easier. But, some users were having problems because their operating system wasn't in English. Kopischke pointed out the fix: 

 

His solution for German was

 

click_menu("Finder", "Fenster", "Alle Fenster zusammenführen")

While I haven't seen this solution before, I'm pretty damn sure that it's been the source of problems for other workflows.

 

So, I have some proposals as to how to fix it. The first is a stop-gap measure, and the second is a larger project.

 

The stopgap measure would just be to drop in a few lines of code with a text file:

 

set thelang_ to user locale of (get system info)
if thelang_ is not "en_US" then
  tell application "TextEdit"
    activate
    open (((path to desktop) as text) & "non-en-instructions.txt")
  end tell
end if

Obviously, you'd want to store the "non-en-instructions.txt" file in the Workflow itself, so you'd probably want to play around with the path variable. Also, it would probably be good to set it to not en_US or en_UK (is it UK?). The contents of the text file could be as simple as:

To make this workflow function in non-English, open the workflow folder, edit the "workflow.scpt" file and change the variables "Window" and "Merge All Windows" to match your operating system language. And then take out the lines of code that read:

 

set thelang_ to user locale of (get system info)
if thelang_ is not "en_US" then
  tell application "TextEdit"
    activate
    open (((path to desktop) as text) & "non-en-instructions.txt")
  end tell
end if

 

 

That's it.

 

The more robust option would be to develop a translation library for UI scripting. One might already exist, but I haven't looked. Then, in the library, we'd have a file for each language that would have the translations from English (or from whatever language we want). So, German would have the entries

Window: "Fenster"
Merge All Windows: "Alle Fenster zusammenführen"

along with many more. We could then set the variables with a quick function call after getting the user's language.

 

I think that this could be a great project and one that would need to be undertaken collaboratively, but right now I don't have the time to tackle it. If anyone wants to start on it, then please do. When my schedule opens up, I'll start on it if no one else has already.

 

Anyway, thoughts?

 

--Shawn

[ipaterson]

Normally, AppleScript makes this localization lookup quite easy. Since most apps just use a strings file for localization, AppleScript includes a built-in keyword to localize strings. Take a look at appl10n from my Wunderlist workflow for a fairly generic example. The localized string keyword reads strings files that most applications have embedded for handling localization. 

 

Some apps will use a direct word-for-word translation from English ("Merge All Windows" = "Alle Fenster zusammenführen"), but that is not always the case. As shown here, in the case of Wunderlist the translation keys were formatted "smart_list_week" rather than "Week". In order to find the proper translation keys, find the .app file in Finder, right click and "Show Package Contents" then find the proper strings file in Contents/Resources/en.lproj (usually just Localizable.strings). Of course you can also get there through the terminal and grep for the English string.

 

Sometimes the strings file is in a binary format that may not display well in a text editor. Just use plutil -p /path/to/Localizable.strings to view the file contents. FYI, Finder.app is in /System/Library/CoreServices

 

Unfortunately, the particular example given is not localized in a strings file. Instead, the entire nib file that defines the layout of that menu is duplicated in each locale (common practice when more about the layout is changing than just the text). Furthermore, the nib file is compiled so you can't necessarily use ibtool to extract the localizable strings from it. I used some tricks to reinflate the MenuBar.nib file in German and English, then used ibtool --export-strings-file to generate a strings file for each. Both map "178.title" to the merge all windows text, but I wouldn't consider this a stable or usable thing. It's pretty much not possible to derive efficiently and once you get it, it won't work with the localized string keyword in Applescript.

 

One unsavory option is to use indexes rather than names for the menu items. However, since each localization has its own copy of the menu nib you can assume now that there are differences based on the locale and this approach is also sensitive to changes in the menu due to OS updates or other unforeseen things. Let's avoid that.

 

The best option in this particular case, also something I've used successfully in Alfred workflows, is building your own strings files for the workflow. Remember, this is only in this case where the actual text is not available in the application's strings files. Let's assume you have automated the process above and found all of the translations for this particular menu item. In the workflow directory, create a matching .lproj folder for each localization supported by Finder and add a Localizable.strings file to it. In that file, map the translations in whatever format makes sense to you for all the strings that the workflow needs (simple is fine, like "Merge All Windows" = "Alle Fenster zusammenführen";). Collecting these strings is the hard part.

 

An implementation of loading those strings from the workflow can be found here. Now instead of asking an application for a localized string, you are grabbing a localized string from a bundle. The bundle is just the folder in which your workflow resides where the *.lproj/Localizable.strings files can be found. If you use qWorkflow, getWorkflowFolder() will get that for you. localized string of someString in bundle pathToWorkflow allows you to load any of the translations specified in your workflow's strings files. Simply add any strings there which cannot be localized by consulting the application directly and you've created a fairly efficient solution to the problem.

[deanishe]

I got bitten by a similar mistake (which Kopischke also found—and fixed).

I was building Spotlight queries such as kind:Movies. In this case, there's usually a universal alternative of the com.apple.whatever form.

Also, isn't it possible to "click" menu items by index instead of name? Obviously not a universal solution, but a whole lot simpler than the alternative in cases where it's possible.

[smarg19]

While not, strictly speaking, on topic, I've been working on a collection of helper functions for generating Applescript user-interaction elements. I've posted about it here.

 

I think that adding stuff to ease UI scripting would really help to take the library to a new level of utility. When I get some free time, I will research more on UI scripting using index as well as localization. If anyone else has been working on this problem, you can fork my library or pull request your additions. The GitHub repo is here.

 

I would love to reach a place where the library makes using the various elements of Applescripts UI access as easy as possible. 

 

P.S. For those workflow writers who generate a decent number of AS user-interactions (i.e. display dialog, choose from list, choose file, etc), these helper functions provide a terse, consistent way of ensuring your dialog/pop-up is always front-and-center, regardless of the user's current environment, and also offer graceful failure for user-canceling, while preserving error reporting for other issues. If you'd like to see them in action, I'm using the functions in my ZotQuery workflow. Specifically in the configuration scripts (there are examples of pure AS scripts, and piping AS from Python). 

Edited March 3, 2014 by smarg19

[dfay]

Here are a couple other useful resources:

 

     applescript - How to know the name of UI elements using Accessibility inspector (or any other tool) - Ask Different

 

     Finding Control and Menu Items for use in AppleScript User Interface Scripting - Mac OS X Hints