Sign in to follow this  
Followers 0
Exit

How to click GoogleTranslate "listen" button?

11 posts in this topic

The click on the source 'listen' button at Google translate does not fire speech.

Any ideas how to fix it ?

;~ #AutoIt3Wrapper_Run_Debug_Mode=1

; speech recognition

#include <string.au3>
#include <IE.au3>

_Language("Dies ist ein deutscher Text")
;~ _Language("Questo è un testo in lingua spagnola")
;~ _Language("Questo è un testo in italiano")
;~ _Language("Ceci est un texte en français")
;~ _Language("?")

Func _Language($text)
    $oIE = _IECreate("https://translate.google.com/?hl=en&tab=wT#auto/en/" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
    $oObejct = _IEGetObjById($oIE, "gt-src-listen")
    $rc = _IEAction($oObejct, "click")
    MsgBox(262144, "Debug " & @ScriptName, "Language: " & $aLanguage[0] & @LF & @LF & "Error: " & @error & " Extended: " & @extended & @LF & "RC: " & $rc, 0)
    _IEQuit($oIE)
EndFunc   ;==>_Language

My UDFs, Abbrevs and Snippets
If you like my post, just click the like button :) 

Share this post


Link to post
Share on other sites



Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

¿why not something like this using inetget+soundplay?

 

http://translate.google.com/translate_tts?tl=de&q=Dies%20ist%20ein%20deutscher%20Text

 

Saludos

 

Interesting approach, but ....   tl=auto / sl=auto  is not supported.

But see my new thread with a solution >here

 

 

@All

My original question is not yet answered: How to click the listen symbol ?

Edited by Exit

My UDFs, Abbrevs and Snippets
If you like my post, just click the like button :) 

Share this post


Link to post
Share on other sites

I meant to post this yesterday.  For some unknown reason, what I wanted to do with parentwindow.eval was not working on this PC and IE9, and I just don't have the patience to debug why.

#include <IE.au3>

_IEGoogleTransClickListen("Dies ist ein deutscher Text")
Switch @error
    Case 1
        MsgBox(16 + 262144, "Error", "Invalid create/attach object")
    Case 2
        MsgBox(16 + 262144, "Error", "Click object failed")
EndSwitch


Func _IEGoogleTransClickListen($s_text, $s_win = "Google Translate", _
        $s_url = "https://translate.google.com/?hl=en&tab=wT#auto/en/")

    Local $o_obj = 0, $f_attach = False
    If WinExists($s_win) Then
        $o_obj = _IEAttach($s_win)
        $f_attach = True
    EndIf
    If Not IsObj($o_obj) Then
        $o_obj = _IECreate($s_url & $s_text)
        $f_attach = False
    EndIf
    If Not IsObj($o_obj) Then
        Return SetError(1, 0, 0)
    EndIf

    If Not _IEJS_GoogleTransClickListen($o_obj, $f_attach) Then
        Return SetError(2, @error, 0)
    EndIf

    Return 1
EndFunc

Func _IEJS_GoogleTransClickListen(ByRef $o_obj, $f_funcexists = True, $s_id = "gt-res-listen")

    If Not IsObj($o_obj) Then
        Return SetError(1, 0, 0)
    EndIf

    Local $s_query = ""

    ; if we're only attaching or running this function many times on the same page
    ;  we don't need to keep adding the function itself
    ;  should only be ran on the first load, monitor however you wish
    ;  I had a eval method that looked for the function, but for some reason
    ;  on this machine, document.parentwindow.eval is not working
    If Not $f_funcexists Then
        $s_query &= "function IEAu3TriggerMouseEvent (node, eventType) {" & @CRLF
        $s_query &= "var clickEvent = document.createEvent('MouseEvents'); "  & @CRLF
        $s_query &= "clickEvent.initEvent (eventType, true, true); " & @CRLF
        $s_query &= "node.dispatchEvent (clickEvent);}" & @CRLF
        $s_query &= "var IEAu3srcListenButton  = 0;" & @CRLF
    EndIf
    $s_query &= "IEAu3srcListenButton = document.getElementById('" & $s_id & "');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mouseover');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mousedown');" & @CRLF
    $s_query &= "IEAu3TriggerMouseEvent (IEAu3srcListenButton, 'mouseup');"

    Execute("$o_obj.document.parentwindow.execScript($s_query)")

    Return 1
EndFunc

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

That's a tricky circumvention using a JavaScript instead of a  _IEAction(... , "click")
I adapted it to my script and it works like a charm. Many thanks to SmOke_N

#include <string.au3>
#include <IE.au3>

_Speak("This is an English text")
_Speak("Dies ist ein deutscher Text")
_Speak("Se trata de un texto español")
_Speak("Questo è un testo italiano")
_Speak("Ceci est un texte en français")
_Speak("Die Vorführung ist beendet.")

Func _Speak($text)
    $oIE = _IECreate("https://translate.google.com/?hl=en&tab=wT#auto/en/" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
;~     $oObejct = _IEGetObjById($oIE, "gt-src-listen")
;~     $rc = _IEAction($oObejct, "click")
    $rc = _MouseClick($oIE, "gt-src-listen")
    MsgBox(262144, Default, "Press ENTER to hear translation....", 4)
    $rc = _MouseClick($oIE, "gt-res-listen")
    MsgBox(262144, Default, "Press ENTER to exit....", 3)
    _IEQuit($oIE)
EndFunc   ;==>_Speak

Func _MouseClick(ByRef $o_obj, $s_id)
    $s_JS = "function IEAu3TriggerMouseEvent(node, eventType) {;" & _
            "var clickEvent = document.createEvent('MouseEvents'); " & _
            "clickEvent.initEvent (eventType, true, true); " & _
            "node.dispatchEvent (clickEvent);}" & _
            "Button = document.getElementById('" & $s_id & "');" & _
            "IEAu3TriggerMouseEvent (Button, 'mousedown');" & _
            "IEAu3TriggerMouseEvent (Button, 'mouseup');"
    Execute("$o_obj.document.parentwindow.execScript($s_JS)")
EndFunc   ;==>_MouseClick

But nevertheless I would like to know, why it is not working with _IEAction(... , "click")
 
Any  <IE.au3>  UDF gurus online ?

Edit: Changed funtionname from Do to IEAu3TriggerMouseEvent

Edited by Exit

My UDFs, Abbrevs and Snippets
If you like my post, just click the like button :) 

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

It was written with a specific event (that I've provided).

Note the "Mouseover", "Mousedown", "Mouseup", it was written to accept that specific event, not the onclick method.

Run it through your IE debugger deobfuscated to track down the function that processes it if you really need to understand all the details.

Edit:

No idea why you changed the function name in the javascript.  You've named it something really non-unique.  You should have left it the way I had it to avoid issues in the future with actual javascript code they've written.

Edited by SmOke_N

Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

Run it through your IE debugger deobfuscated to track down the function that processes it if you really need to understand all the details.

Can you please explain this to a noob?

 

No idea why you changed the function name in the javascript.  You've named it something really non-unique.  You should have left it the way I had it to avoid issues in the future with actual javascript code they've written.

Good point. Changed it back in above post.


But my main question is not yet answered:

I would like to know, why it is not working with _IEAction(... , "click")


My UDFs, Abbrevs and Snippets
If you like my post, just click the like button :) 

Share this post


Link to post
Share on other sites

You'll need to read up on F12 option for Internet Explorer.

There's really way too much to just explain.

Debugbar is generally all that is needed, but sometimes you just find those stubborn websites you need to "inject" javascript into to achieve what you're trying to do.  That's when it helps to be able to use things like fiddler and IE's F12 options.

I've done a TON of website automation, and only have run into maybe 4 or 5 sites I've had to use those last two options, so if you feel you that you just have to understand, you have your search engine query's there ;) .


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

With iuiautomation in examples section I was able in chrome to do this (but thats a different UDF)

_UIA_action($oUIElement,"click")
IE.AU3 in the end just calls $oObject.Click() and doesn't look special.

 

Looking at the html you can see the span img is within the div and as such my assumption is the img gets the click and not your div tag but anyhow your mouspointer is not at the right place so probably nothing happens

 

Microsoft click documentation states

If the user clicks the left mouse button, the onclick event for an object occurs only if the mouse pointer is over the object and an onmousedown and an onmouseup event occur in that order. For example, if the user clicks the mouse on the object but moves the mouse pointer away from the object before releasing, no onclick event occurs.

 

maybe this kind of code is an alternative (not sure if calculation of position will work for all browsers or versions)

Func _Language($text)
    $oIE = _IECreate("[url=https://translate.google.com/?hl=en&tab=wT#auto/en/]https://translate.google.com/?hl=en&tab=wT#auto/en/[/url]" & $text, 0, 1)
    $sHtml = _IEBodyReadHTML($oIE)
    $aLanguage = _StringBetween($sHtml, 'value="auto">', " - detected")
    $oObejct = _IEGetObjById($oIE, "gt-src-listen")
 
$x1=$oIE.document.parentWindow.screenLeft
$Y1=$oIE.document.parentWindow.screenTop
 
$oObject1=$oIE.document.body
$oObject2= $oObejct 
 
$oRectangle1=$oObject1.getBoundingClientRect()
consolewrite("left1:" & $oRectangle1.left & @CRLF & "top1:" & $oRectangle1.top & @CRLF)
 
$oRectangle2=$oObject2.getBoundingClientRect()
consolewrite("left2:" & $oRectangle2.left & @CRLF & "top2:" & $oRectangle2.top)
 
mousemove($x1+ $oRectangle1.left + $oRectangle2.left + ($oRectangle2.height/2) , $y1 + $oRectangle1.top + $oRectangle2.top + ($oRectangle2.width/2) ,0)
  mouseclick("left")
sleep(5000)
 
;~  $oObject2.Click()
;~  sleep(200)
 
;~     $rc = _IEAction($oObejct, "click")
;~     MsgBox(262144, "Debug " & @ScriptName, "Language: " & $aLanguage[0] & @LF & @LF & "Error: " & @error & " Extended: " & @extended & @LF & "RC: " & $rc, 0)
 
_IEQuit($oIE)
EndFunc   ;==>_Language
I do not think it are stubborn websites. Its just that the element itself does not have an click event handler and as such calling it makes no sense. 

The click is catched at a higher level and most likely the js script then determines over which element the mousecursor is and based on that element makes decisions what to do.

Edited by junkew

Share this post


Link to post
Share on other sites

@junkew

By stubborn, I meant that the author(s) generally and or purposely wrote them unconditionally, thus making them harder to automate.

IMO, using an actual MouseClick procedure is really not ideal in any automation routine.  If the window ever loses focus, you could find yourself in trouble.  A controlclick method with x and y position may be more ideal.


Common sense plays a role in the basics of understanding AutoIt... If you're lacking in that, do us all a favor, and step away from the computer.

Share this post


Link to post
Share on other sites

If the coords can be returned then either MouseClick or ControlClick are not a problem. However I wouldn't go that way. If the window looses focus then a mouseclick would eventually fail but a controlclick could do the job.
If the window is minimized then non of the above would work. It would require a window restore, click and the minimize again which is a bad solution.

OP you have your solution on post 4 and your answer on your question on post 9.
Figure it out ;)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0