How to click GoogleTranslate "listen" button?

[Exit]

The click on the source 'listen' button at Google translate does not fire speech.

Any ideas how to fix it ?

;~ #AutoIt3Wrapper_Run_Debug_Mode=1

; speech recognition

#include <string.au3>
#include <IE.au3>

_Language("Dies ist ein deutscher Text")
;~ _Language("Questo è un testo in lingua spagnola")
;~ _Language("Questo è un testo in italiano")
;~ _Language("Ceci est un texte en français")
;~ _Language("?")

Func _Language($text)
    $oIE = _IECreate("https://translate.google.com/?hl=en&tab=wT#auto/en/" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
    $oObejct = _IEGetObjById($oIE, "gt-src-listen")
    $rc = _IEAction($oObejct, "click")
    MsgBox(262144, "Debug " & @ScriptName, "Language: " & $aLanguage[0] & @LF & @LF & "Error: " & @error & " Extended: " & @extended & @LF & "RC: " & $rc, 0)
    _IEQuit($oIE)
EndFunc   ;==>_Language

[Danyfirex]

¿why not something like this using inetget+soundplay?

http://translate.google.com/translate_tts?tl=de&q=Dies%20ist%20ein%20deutscher%20Text

Saludos

 

[Exit]

¿why not something like this using inetget+soundplay?

 

http://translate.google.com/translate_tts?tl=de&q=Dies%20ist%20ein%20deutscher%20Text

 

Saludos

 

Interesting approach, but ....   tl=auto / sl=auto  is not supported.

But see my new thread with a solution >here. 

 

 

@All

My original question is not yet answered: How to click the listen symbol ?

Edited November 13, 2014 by Exit

[SmOke_N]

I meant to post this yesterday.  For some unknown reason, what I wanted to do with parentwindow.eval was not working on this PC and IE9, and I just don't have the patience to debug why.

expandcollapse popup

#include <IE.au3>

_IEGoogleTransClickListen("Dies ist ein deutscher Text")
Switch @error
    Case 1
        MsgBox(16 + 262144, "Error", "Invalid create/attach object")
    Case 2
        MsgBox(16 + 262144, "Error", "Click object failed")
EndSwitch


Func _IEGoogleTransClickListen($s_text, $s_win = "Google Translate", _
        $s_url = "https://translate.google.com/?hl=en&tab=wT#auto/en/")

    Local $o_obj = 0, $f_attach = False
    If WinExists($s_win) Then
        $o_obj = _IEAttach($s_win)
        $f_attach = True
    EndIf
    If Not IsObj($o_obj) Then
        $o_obj = _IECreate($s_url & $s_text)
        $f_attach = False
    EndIf
    If Not IsObj($o_obj) Then
        Return SetError(1, 0, 0)
    EndIf

    If Not _IEJS_GoogleTransClickListen($o_obj, $f_attach) Then
        Return SetError(2, @error, 0)
    EndIf

    Return 1
EndFunc

Func _IEJS_GoogleTransClickListen(ByRef $o_obj, $f_funcexists = True, $s_id = "gt-res-listen")

    If Not IsObj($o_obj) Then
        Return SetError(1, 0, 0)
    EndIf

    Local $s_query = ""

    ; if we're only attaching or running this function many times on the same page
    ;  we don't need to keep adding the function itself
    ;  should only be ran on the first load, monitor however you wish
    ;  I had a eval method that looked for the function, but for some reason
    ;  on this machine, document.parentwindow.eval is not working
    If Not $f_funcexists Then
        $s_query &= "function IEAu3TriggerMouseEvent (node, eventType) {" & @CRLF
        $s_query &= "var clickEvent = document.createEvent('MouseEvents'); "  & @CRLF
        $s_query &= "clickEvent.initEvent (eventType, true, true); " & @CRLF
        $s_query &= "node.dispatchEvent (clickEvent);}" & @CRLF
        $s_query &= "var IEAu3srcListenButton  = 0;" & @CRLF
    EndIf
    $s_query &= "IEAu3srcListenButton = document.getElementById('" & $s_id & "');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mouseover');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mousedown');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mouseup');"

    Execute("$o_obj.document.parentwindow.execScript($s_query)")

    Return 1
EndFunc

[Exit]

That's a tricky circumvention using a JavaScript instead of a  _IEAction(... , "click")
I adapted it to my script and it works like a charm. Many thanks to SmOke_N

#include <string.au3>
#include <IE.au3>

_Speak("This is an English text")
_Speak("Dies ist ein deutscher Text")
_Speak("Se trata de un texto español")
_Speak("Questo è un testo italiano")
_Speak("Ceci est un texte en français")
_Speak("Die Vorführung ist beendet.")

Func _Speak($text)
    $oIE = _IECreate("https://translate.google.com/?hl=en&tab=wT#auto/en/" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
;~     $oObejct = _IEGetObjById($oIE, "gt-src-listen")
;~     $rc = _IEAction($oObejct, "click")
    $rc = _MouseClick($oIE, "gt-src-listen")
    MsgBox(262144, Default, "Press ENTER to hear translation....", 4)
    $rc = _MouseClick($oIE, "gt-res-listen")
    MsgBox(262144, Default, "Press ENTER to exit....", 3)
    _IEQuit($oIE)
EndFunc   ;==>_Speak

Func _MouseClick(ByRef $o_obj, $s_id)
    $s_JS = "function IEAu3TriggerMouseEvent(node, eventType) {;" & _
            "var clickEvent = document.createEvent('MouseEvents'); " & _
            "clickEvent.initEvent (eventType, true, true); " & _
            "node.dispatchEvent (clickEvent);}" & _
            "Button = document.getElementById('" & $s_id & "');" & _
            "IEAu3TriggerMouseEvent (Button, 'mousedown');" & _
            "IEAu3TriggerMouseEvent (Button, 'mouseup');"
    Execute("$o_obj.document.parentwindow.execScript($s_JS)")
EndFunc   ;==>_MouseClick

But nevertheless I would like to know, why it is not working with _IEAction(... , "click")
 
Any  <IE.au3>  UDF gurus online ?

Edit: Changed funtionname from Do to IEAu3TriggerMouseEvent

Edited November 15, 2014 by Exit

[SmOke_N]

It was written with a specific event (that I've provided).

Note the "Mouseover", "Mousedown", "Mouseup", it was written to accept that specific event, not the onclick method.

Run it through your IE debugger deobfuscated to track down the function that processes it if you really need to understand all the details.

Edit:

No idea why you changed the function name in the javascript.  You've named it something really non-unique.  You should have left it the way I had it to avoid issues in the future with actual javascript code they've written.

Edited November 14, 2014 by SmOke_N

[Exit]

Run it through your IE debugger deobfuscated to track down the function that processes it if you really need to understand all the details.

Can you please explain this to a noob?

 

No idea why you changed the function name in the javascript.  You've named it something really non-unique.  You should have left it the way I had it to avoid issues in the future with actual javascript code they've written.

Good point. Changed it back in above post.

---

But my main question is not yet answered:

I would like to know, why it is not working with _IEAction(... , "click")

[SmOke_N]

You'll need to read up on F12 option for Internet Explorer.

There's really way too much to just explain.

Debugbar is generally all that is needed, but sometimes you just find those stubborn websites you need to "inject" javascript into to achieve what you're trying to do.  That's when it helps to be able to use things like fiddler and IE's F12 options.

I've done a TON of website automation, and only have run into maybe 4 or 5 sites I've had to use those last two options, so if you feel you that you just have to understand, you have your search engine query's there .

[junkew]

With iuiautomation in examples section I was able in chrome to do this (but thats a different UDF)

_UIA_action($oUIElement,"click")

IE.AU3 in the end just calls $oObject.Click() and doesn't look special.

 

Looking at the html you can see the span img is within the div and as such my assumption is the img gets the click and not your div tag but anyhow your mouspointer is not at the right place so probably nothing happens

 

Microsoft click documentation states

If the user clicks the left mouse button, the onclick event for an object occurs only if the mouse pointer is over the object and an onmousedown and an onmouseup event occur in that order. For example, if the user clicks the mouse on the object but moves the mouse pointer away from the object before releasing, no onclick event occurs.

 

maybe this kind of code is an alternative (not sure if calculation of position will work for all browsers or versions)

Func _Language($text)
    $oIE = _IECreate("[url=https://translate.google.com/?hl=en&tab=wT#auto/en/]https://translate.google.com/?hl=en&tab=wT#auto/en/[/url]" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
    $oObejct = _IEGetObjById($oIE, "gt-src-listen")
 
$x1=$oIE.document.parentWindow.screenLeft
$Y1=$oIE.document.parentWindow.screenTop
 
$oObject1=$oIE.document.body
$oObject2= $oObejct 
 
$oRectangle1=$oObject1.getBoundingClientRect()
consolewrite("left1:" & $oRectangle1.left & @CRLF & "top1:" & $oRectangle1.top & @CRLF)
 
$oRectangle2=$oObject2.getBoundingClientRect()
consolewrite("left2:" & $oRectangle2.left & @CRLF & "top2:" & $oRectangle2.top)
 
mousemove($x1+ $oRectangle1.left + $oRectangle2.left + ($oRectangle2.height/2) , $y1 + $oRectangle1.top + $oRectangle2.top + ($oRectangle2.width/2) ,0)
  mouseclick("left")
sleep(5000)
 
;~  $oObject2.Click()
;~  sleep(200)
 
;~     $rc = _IEAction($oObejct, "click")
;~     MsgBox(262144, "Debug " & @ScriptName, "Language: " & $aLanguage[0] & @LF & @LF & "Error: " & @error & " Extended: " & @extended & @LF & "RC: " & $rc, 0)
 
_IEQuit($oIE)
EndFunc   ;==>_Language

I do not think it are stubborn websites. Its just that the element itself does not have an click event handler and as such calling it makes no sense. 

The click is catched at a higher level and most likely the js script then determines over which element the mousecursor is and based on that element makes decisions what to do.

Edited November 16, 2014 by junkew

[SmOke_N]

@junkew

By stubborn, I meant that the author(s) generally and or purposely wrote them unconditionally, thus making them harder to automate.

IMO, using an actual MouseClick procedure is really not ideal in any automation routine.  If the window ever loses focus, you could find yourself in trouble.  A controlclick method with x and y position may be more ideal.

[AutID]

If the coords can be returned then either MouseClick or ControlClick are not a problem. However I wouldn't go that way. If the window looses focus then a mouseclick would eventually fail but a controlclick could do the job.
If the window is minimized then non of the above would work. It would require a window restore, click and the minimize again which is a bad solution.

OP you have your solution on post 4 and your answer on your question on post 9.
Figure it out