Jump to content
Sign in to follow this  

Auto-detect clickable screen areas?

Recommended Posts

I'm using AutoIt on an app that I'm running on an Android emulator; my script basically clicks a spot, sleeps, clicks another, etc, etc.
Instead of waiting the same fixed number of miliseconds every time and clicking the same predefined position, I'd like for the script to detect when & where the next clickable object pops up on on the emulator window, and either trigger it or pick another response depending on its coordinates.

As a side note, I've been rewriting my code and using PixelChecksum(), PixelSearch(), and PixelGetColor() to this end, and it's getting the job done now, but I'd be much happier with something that automatically detects buttons like a screen reader would, if that makes sense.

Edited by Elyonne
Pixel functions work now & also thread was recategorized

new to AutoIt and online forums, so I apologize in advance for being annoying.

Share this post

Link to post
Share on other sites

Another method (depending on how advanced you want to do it) is to investigate (with Inspect.exe from Windows SDK) if your objects are recognizable for UI Automation code. If so, you can probably use an UI Automation structure change event handler to monitor structure changes (objects created/deleted) in your window. See these examples.

The advantage of this method is that all the hard work is done by the code in uiautomationcore.dll. Your AutoIt script only have to listen for the events.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • By Fenzik
      my question is based on topic about Nvda Screen reader development atThis Link
      Autoit GUI is totaly nice for blind users, because it's controls are mostly standart.
      But in cooperation with Screen Readers this guis are much slower than other guis.
      Do you have any idea about reason of this behaviour?
      Thanks a lot for any answer.
  • Create New...