[Solved] applescript click event not working in Alfred

[rounak]

set dialNumber to "{query}"
try
tell application "Safari"
activate
tell window 1
try
set current tab to tab "AirDroid Web | Manage your phone on web"
on error e
try
set current tab to tab "AirDroid"
on error e
set the clipboard to e as text
end try
end try
end tell

delay 2
say "here"
tell application "System Events"
tell process "Safari"
click at {1014, 106}
delay 1
say "here2"
click at {752, 397}
keystroke dialNumber
delay 1
click at {796, 635}
end tell
end tell
end tell
end try

I have set this osascript as my phone action.

I can hear "here1" and "here2" but no clicking is done. The same script run fine when run from Script Editor.app.

Edited August 31, 2016 by rounak

[rounak]

After trying for few more times, I got the warning of Aflred 3 trying to use Accessibility Features. Its working now.

Edited August 31, 2016 by rounak

[MEs]

Hi,

 

What do the numbers in click at {796, 635} mean ?

I tried to use keystroke for some windows as well, even with Accessibility features enabled, it never worked. So I was wondering if I missed something and just found your post. Maybe this could help.

 

Regards

Edited September 3, 2016 by MEs

[rounak]

They are co-ordinates on the screen where I want to to perform the click event. You can see those co-ordinates near your mouse cursor after pressing the relevant keyboard shortcut as listed in 

/System/Library/PreferencePanes/Keyboard.prefPane

>>Shortcuts

>>Screen Shots

[deanishe]

You can see those co-ordinates near your mouse cursor after pressing the relevant keyboard shortcut as listed in 

/System/Library/PreferencePanes/Keyboard.prefPane

>>Shortcuts

>>Screen Shots

 

 

That's a very handy trick. Would never have occurred to me to do that.

[MEs]

Hi rounak,

 

Thank you for your explanation.

Have you tried this with a second monitor plugged into your mac by any chance ? If yes, how do you tell which monitor to work on ?

 

Regards

[deanishe]

EDIT: Disregard; this is incorrect.

tell process "Safari"
    click at {100,100}
end tell

 You're telling an application to click on its own window. The coordinates refer to that application's active window, not the whole screen. Edited September 4, 2016 by deanishe

[MEs]

Maybe I am missing something here. How can you get those coordinates using the Cmd+Shift+4 shortcut ? Or maybe by selecting the window itself and "measuring" the place where we need it to click ?

 

EDIT : I did a quick test. The only way I made it work was by using the coordinate relative to the entire screen.

Edited September 4, 2016 by MEs

[deanishe]

You're right. My bad. I was doing this the other day, but forgot I'd included the window bounds in the calculation.

 

So, you either need to make sure the window is in a known position (e.g. fullscreen, where the CMD+SHIFT+4 trick works) or you calculate where to click based on the window's current position, like the above-linked script.

[rounak]

Hi rounak,

 

Thank you for your explanation.

Have you tried this with a second monitor plugged into your mac by any chance ? If yes, how do you tell which monitor to work on ?

 

Regards

 

No, never used a second monitor. 

Any details you can share about what you are trying to do?

(Stating the known: UI scripting should be the last resort). 

[deanishe]

(Stating the known: UI scripting should be the last resort).

Yeah. It's slow and unreliable. Especially simulating mouse clicks (accessing the window's widgets is slightly better, but also very fragile).

If you're trying to automate a webpage, using Safari's do JavaScript to run your own JS on the page (much like a GreaseMonkey userscript) is a more robust solution. That way, you can access elements using CSS selectors, which is much more reliable.

 

In fact, rounak just posted an example of this.

Edited September 5, 2016 by deanishe

[MEs]

Thank you for the examples. Will see how it fits. I know UI scripting is not a great solution, just looking at all the options since I am still discovering Alfred