How to only click a button if some text is somewhere else on the screen

[FredS]

I'm sure i'm not the first with a question like this, but after searching and reading a lot, i still have not found the answer, (i did find a lot of answers, but the question is probably somewhat different than what i am trying to do.
Should the exact question already exist, so sorry. Could you please forward me in that case?

I have been using AutoIT in the past and recently i joined a group of volunteers who are doing a lot of good work and i am trying to do my part as well.
This work involves taking pictures and putting them into a database and then describe the content of these pictures.
The last part can be made easier if you first select all pictures that are about the same (i did) and then giving the description, which is the same for this group of pictures.
So when i started going through the page where i had to give the description, i immediately thought of AutoIT again and what a help it is!
Instead of manually doing the same thing over and over again, it just clicks where it is supposed to do and i made it stop just before the Submit-button, just to be sure that it did the right thing before really submitting it (i don't want to automatically 'put rubbish in the database').
I think the steps i made are correct and i am thinking about making a counter (if i have 50 photos, run the steps 50 times), but i am looking to automate the step that i am doing manually at this time.
If filling out the form is ready, i give a 'beep'-command and at that moment i look at the screen to see if i see 2 things on the screen: Napkins and Avoca (the number is not important, if it goes wrong, it goes wrong by not selecting the dropdown-value, so instead of 'Avoca', it might have choosen another value. If the right values were selected in dropdown-boxes during filling the form then i can press 'submit'.

Is there a way to do this step without AutoIT as well? I did look into 'winexist', but i think that is not the right way in this case? I also tried a little with au3info.exe, but well, maybe i haven't tried enough yet, but maybe you can point me in the right direction, because i want to get moving with this work (but also don't want to sit next to the computer, waiting for hundreds of beeps to manually check if something is correct before moving on.

I hope you understand my question but i am not sure what information you need to get me going, so i'll give it a try:

See the printscreen for a part of what i am working with. AutoIT does all the clicking and results in 2 values that have been choosen (for all pictures this will be the same value, for instance "Napkins: 1" and "Avoca: 1" in this example.
If i see that these 2 are there (and not for instance "Napkins: 1" and "Ashal: 1" (in that case it would have choosen the wrong value and i would submit the wrong data),
then i can press the submit-button and all repeats with the next picture.
So how can i check if value 1 is on the screen (the script can stop if it is not there, i will manually correct it, submit this one and let autoIT take the next pictures.
(Then after checking for value 1, i also let it check for value 2 (or stop if it is not there) and if so, i can make it press the submit and repeat all steps.

Well, not quite sure what info i have to look at (i'm unfortunately not a real developer, but this is what i have been looking at, and trying to use with commands, but whatever i try, autoIT says it did not find "Avoca" somewhere on the screen).
I hope you can help (and once againg, sorry if this has been handled before.

 

[Nine]

By using IE.udf you can use _IETagNameGetCollection for li tags and read the innerText of the object.  Something similar as this :

#include <IE.au3>
#include <MsgBoxConstants.au3>

Local $oIE = _IECreate("you url here")
Local $oInputs = _IETagNameGetCollection($oIE, "li")
Local $sTxt = ""
For $oInput In $oInputs
    $sTxt &= $oInput.innerText & @CRLF
Next
MsgBox($MB_SYSTEMMODAL, "li tags", $sTxt)

_IEQuit($oIE)

You will need to add error checking and inspect the content of the $sTxt to determine the course of actions.

[FredS]

So far i've only use 'record and playback' so it looks a but overwhelming but i will sure give it a try to get this to work,
thanks for taking your time to help me, really happy with it!

[Danp2]

FWIW, some of your concerns about "something going wrong" could be reduced or eliminated if you stopped using Send / Mouseclick and switched to using one of the UDFs designed for interacting with a browser.

[FredS]

Hi Danp2, thanks for your tip,
I think that sure would be a better option, and makes it much saver.
I only just started using autoIT and i've tried some different options with record and playback (for me the easiest way of getting started), but you are right, it would be better to use some intelligence instead of just clicking. Especially to be foolproof when and extra option is inserted in the dropdown.
I will look into that as well. It's just that i wanted to get starting and it is already working fine, except for the manual-check step.
I hope to get the script going so i can start, and meanwhile learn more about these UDFs.

[paw]

I imagine this is some sort of AI training? There are multiple ways to do this. It would be good to know the website. Does it have an API?

 

[FredS]

Hi Paw, thanks for your reply!
Not really an AI training, although this will probably be part of it as well.
For now we (people around the world) are litterpicking. (if you see trash on the street, we clean it), but apart from that, we also take a picture of it and we upload these pictures.

After that, we also tag the items, to give it some extra data (in future, this will be done by AI), so that other people can use the data (for instance to talk with producers of wrapping material, or municipalities to change the places where they clean et cetera.

For now i am picking litter and uploading the pictures and i am doing that by uploading a batch of the same kind of pictures (for instance all pictures with a cigarettebox on it). So i have to do a lot of the same actions:
click one kind of scrolldown-menu and choose 'smoking', then another dropdown and select 'cigarettebox', then another scrollmenu and select brand and another, select marlboro.

Unfortunately i do not have access to the db itself, otherwise i would probably just write a query to insert all data directly so for now it is just a matter of using the frontend and clicking a lot :-)

I did not really find a way yet to do it the way it is supposed to (the tips given above), but i will not give up yet 😉
At this moment i have made a script that keeps running 'for ever', it does all the right clicking, it then beeps (to get my attention 😉 and pauses with a popup with the question what to do, stop, retry or continue. If i press continue, it submits and starts to load the next one. If some wrong tag was chosen, i will remove it manually and click 'retry'.
So far i am happy with it and i now started to upload and tag my pictures, but i hope to find some way to skip the manual check and replace it with a check that looks if the two right ones were chosen.

 

(As mentioned above by Danp2, i can probably eliminate the possibility that the wrong ones are chosen if i write a better script but at this moment i got the problem that if i run the script that i wrote (which has the "

Local $oIE = _IECreate("you url here")

" in it, it opens a new window of the site and i have to do a lot of other clicks to get to the screen where i normally already am at the moment when i start the script that i am using now. (i hope that makes sense).

The site that i am using is openlittermap.org, but the page that i am using to give all the data can (i guess) only be reached after making an account and after uploading a picture, that's why i gave a screenprint of the developertools-information.

[Nine]

You can _IEAttach instead of _IECreate.  check help file for it.

[FredS]

Cool, and thanks for your help!
i've also started to look at some tutorials which are really helpful, but they start with the basics (ofcourse), and i think what i am trying to do is a little bit further than just the basics.
So trying to learn, and meanwhile using your help/information, really glad with it!