ControlClick doesn't work, but Mouseclick does?

[peakbagger66]

I am trying to automate a button click in a Silverlight control. Of course it is not visible to AutoIT as something I can manipulate, so I am trying to use ControlClick on the IE control, which is basically the whole window where the page appears. .

This is a partial view of the IE Page. The arrow is where I am trying to send the click event to. Basically that whole graph and button is silverlight.

Using the Info tool, I get the coords and IE window control size here:

 

So, my code looks like this:

local $hWnd = _IEPropertyGet($oIE, "hwnd")
Local $hCtrl = ControlGetHandle($hWnd, "", "[ClassNN:Internet Explorer_Server1]")
WinActivate("[Class:IEFrame]")
Local $aPos = ControlGetPos($hWnd,"",$hCtrl);Grab the X and Y of IE Control
ControlClick($hWnd, "", $hCtrl, "left", 1, $aPos[2]-40, $aPos[3]-258); Left click on the area where the button appears in the silverlight control

Nothing happens. I have tried changing the MouseCoordMode to all the available modes, 0, 1, and 2 - no happiness each and every time.

Out of desperation, I also tried (using Primary and Left)

ControlClick($hWnd, "", $hCtrl, "left", 1, $aPos[0]+$aPos[2]-40, $aPos[1]+$aPos[3]-258)

MouseClick, however, works in this form:

Mouseclick("primary",$aPos[0]+$aPos[2]-40,$aPos[1]+$aPos[3]-258,1)

I can't figure this out. I need ControlClick to work because this needs to run in Session #0 on a server.

I'm sure it must be something stupid I am doing. Can anyone help?

Thanks!

 

[jdelaney]

The X and Y are relative to the 0,0 of the control....try this instead:

ControlClick($hWnd, "", $hCtrl, "left", 1,$aPos[2]-40,$aPos[3]-258)

It's a good idea to activate the window prior (you activate some window, but might not be exactly the window you need...use the handle instead):

WinActivate($hwnd)

Edited February 7, 2014 by jdelaney

[peakbagger66]

 

The X and Y are relative to the 0,0 of the control....try this instead:

ControlClick($hWnd, "", $hCtrl, "left", 1,$aPos[2]-40,$aPos[3]-258)

It's a good idea to activate the window prior (you activate some window, but might not be exactly the window you need...use the handle instead):

WinActivate($hwnd)

 

Thanks jdelaney, I have that in the first code example, that is what I thought it should be. Good idea about the WinActivate, I just tried it but the results are the same - no click.

This is driving me crazy. I use the exact same technique for the  "Do you want to open or save the file ...." MS security download dialog in IE and it works perfectly. 

[BrewManNH]

Is there anything related to that control that can be read from the pages source code? Have you tried manipulating it with the _IE* functions?

[peakbagger66]

Is there anything related to that control that can be read from the pages source code? Have you tried manipulating it with the _IE* functions?

 

Sort of...the silverlight is contained in a DIV of class "chartcontainer" and it describes width and height. 

<DIV class=chartcontainer style="HEIGHT: 275px"><OBJECT width=950 height=275 id=RealTimeUsage..../DIV>

How can I use IE.AU3 to click on the relative coordinates of say, 940,137?  (10 pixels from end)

[AutID]

If there is a button you can click it. If not try sumbitting it. If you know the html of the next page try loading directly the html.

[peakbagger66]

More messing about. I adjusted the Y coord of the control click so it clicked on an element outside of the Silverlight object - the question mark.  Guess what? It worked! It must have something to do with controlclicking WITHIN a silver light object that does not work.

[peakbagger66]

If there is a button you can click it. If not try sumbitting it. If you know the html of the next page try loading directly the html.

Thanks AutID, no buttons that can be seen by _IE. No URL to call, the button I need to click is part of a silverlight object. Sort of like trying to click on a button in a Flash object, I guess...

Edited February 7, 2014 by peakbagger66

[BrewManNH]

Can you explain why you can't use mouseclick on this? I understand you're running it on a server, but I still don't get why that affects what you need to do.

[peakbagger66]

Can you explain why you can't use mouseclick on this? I understand you're running it on a server, but I still don't get why that affects what you need to do.

 

Well, basically the script will run in Session #0 of the server, which you get to via Interactive Services Detection. it's just like trying to use the command when the worksation is locked. Or am I wrong?  Send definitely didn't work for me, so I think the same rules apply for Session #0.

I believe they talk about it here regarding locked workstations and  certain commands.

http://www.autoitscript.com/wiki/FAQ#Why_doesn.27t_my_script_work_on_a_locked_workstation.3F

I guess the workaround is to use Mouseclick and get the script running on some crappy XP box, or maybe even a virtual machine that is never locked.

[AutID]

Then you can't click it with the ie udf. The only thing you can do is mouseclick or controlclick. If that doesn't work then there is nothing you can do.
You can try Send() or ControlSend() if there is a shortcut for that submit.
Try _SendMessage for a control, it might work better than controlsend

[jdelaney]

My code is not that same as yours.

Re-review it.

The clicks are relative to the control, so since you are first moving to the X of the start of the control, relative to your window, after you add in the width of the control, you effectively click outside of the control.

Edit: are you running this application on session 0 as well as the script?  The script must run on the same user session that the application is running on.

Edited February 7, 2014 by jdelaney

[peakbagger66]

My code is not that same as yours.

Re-review it.

The clicks are relative to the control, so since you are first moving to the X of the start of the control, relative to your window, after you add in the width of the control, you effectively click outside of the control.

Edit: are you running this application on session 0 as well as the script?  The script must run on the same user session that the application is running on.

 

Hi Jdelaney, yes I looked it over a few times before responding that it was the same. If you look at line 5 of my first code snippet, it is the same as your suggestion.

Yes, the app is running in session #0 as well as the script.

I've given up, there's just no control clicking in the silverlight object! We'll get a VM or workstation and just have it run with Mouseclick and no screen locking.

Even looked at UIAutomation but it's just one big black box...

[jdelaney]

ah, I was looking at the desperation attempts

[peakbagger66]

ah, I was looking at the desperation attempts

 

Hahah, desperate I was!

[AutID]

I hadn't read you entire 1st post. I will just tell you something. If your mouseclick function clicks then the controlclick function clicks as well. Just your coords are wrong. ControlClick has some wierd things than can be explained.
But we are just shooting in the dark. I am not seeing an url of the page so noone here can test. We only guess. And we will not be guessing for long more

[BrewManNH]

What does that Silverlight object show you when you click the button, and can you get to that information without clicking the button but going to it directly? What are you doing with the information that button shows to you that has any use when the computer is locked, and can it be done differently?