Jump to content
Sign in to follow this  
Seminko

How to grab javascript generated data?

Recommended Posts

Is there a way to grab non-hardcoded but rather javascript generated data from a webpage?

Tried a get request as well as _IEBodyReadHTML but both seem to grab the code without the javascript generated data.

$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
$oHTTP.Open("GET", "link", False)
$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
Global $DataArray[10][5]

If $oStatusCode <> 200 Then
    Exit
    MsgBox(1, "Error", "Status Code <> 200")
EndIf

FileWrite(@ScriptDir & "\output.txt", $oReceived)

; ////////

#include <IE.au3>

Local $FullLink = "link"
Local $oIE = _IECreate($FullLink, 0, 0)
_IELoadWait($oIE)
Local $sText = _IEBodyReadHTML($oIE)
FileWrite(@ScriptDir & "\output.txt", $sText)

 

Share this post


Link to post
Share on other sites

So as it turns out _IEBodyReadHTML does what I'm looking for, but unfortunately the site seems to have trouble with Internet Explorer - it runs smoothly in Chrome, but in IE it totally bugs out.

I tried the WebDriver UDF but haven't been able to find the option to create a hidden window just like _IECreate does.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Zaoka
      I'm trying to create script that would login on my SAP WEBI "web page" and replace some uploaded excel files. But i'm having problem on even the log in.
      These are user fill elements with Firefox Inspector
      <input type="text" id="_id0:logon:CMS" name="_id0:logon:CMS"> <input type="text" id="_id0:logon:USERNAME" name="_id0:logon:USERNAME"> <input type="password" id="_id0:logon:PASSWORD" name="_id0:logon:PASSWORD"> LogOnButton
      <input type="submit" id="_id0:logon:logonButton" value="Log On" class="logonButtonNoHover logon_button_no_hover" onmouseover="this.className = 'logonButtonHover logon_button_hover';" onmouseout="this.className = 'logonButtonNoHover logon_button_no_hover';">  
       
      My example, I tried _IEGetObjByName and _IEGetObjById but the script only open page but will not even fill out the form
        #include <IE.au3> Call("signIn") Func signIn() Global $oIE = _IECreate("http://192.xxx.xxx.xx:xxx/BOE/BI") Sleep (5000) Local $server = _IEGetObjByName($oIE, "_id0:logon:CMS") Local $username = _IEGetObjByName($oIE, "_id0:logon:USERNAME") Local $password = _IEGetObjByName($oIE, "_id0:logon:PASSWORD") Local $button = _IEGetObjById($oIE, "_id0:logon:logonButton") _IEFormElementSetValue($server, "TestServer:1000") _IEFormElementSetValue($username, "MyUserName") _IEFormElementSetValue($password, "MyPass") ; THIS PART ABOUT BUTTON CLICK I DONT UNDERSTAND AT ALL $sSelector = "body > div:nth-of-type(2) > div > div:nth-of-type(3) > div > table > tbody > tr > td > table > tbody > tr > td > table > tbody > tr:nth-of-type(1) > td > div > div:nth-of-type(1) > form > div:nth-of-type(5) > button" $signInUC = $oIE.document.QuerySelector($sSelector) _IEAction($signInUC, "click") EndFunc ;==>signIn  
      I would be very grateful for any help
    • By amosrinat
      Hi,
      i have an AutoIT script that sends emails with two embedded images.
      When users get the email they don't see the images. How should I fix my code?
      Thank you for your support,
      Amos
      Func fSendMail() $TemplateName = StringRegExpReplace($Template, "^.*\\|\..*$", "") $SmtpServer = "smtp.**.com" $FromName = "do-not-reply@*********" $FromAddress = "do-not-reply@******" $ToAddress = $MG1_Mgr & ";"& $Owner $Subject = "ACTION REQUIRED - " & $TemplateName $Body = $sBody5 $IPPort = 25 $ssl = 0 Global $oMyRet[2] Global $oMyError = ObjEvent("AutoIt.Error", "MyErrFunc") $rc = fINetSmtpMailCom($SmtpServer, $FromName, $FromAddress, $ToAddress, $Subject, $Body, $IPPort, $ssl) If @error Then MyErrFunc() EndIf EndFunc;fSendMail Func fINetSmtpMailCom($s_SmtpServer, $s_FromName, $s_FromAddress, $s_ToAddress, $s_Subject = "", $as_Body = "", $IPPort = 25, $ssl = 0) Local $objEmail = ObjCreate("CDO.Message") $objEmail.From = '"' & $s_FromName & '" <' & $s_FromAddress & '>' $objEmail.To = $s_ToAddress Local $i_Error = 0 Local $i_Error_desciption = "" $objEmail.Subject = $s_Subject If StringInStr($as_Body, "<") And StringInStr($as_Body, ">") Then $objEmail.HTMLBody = $as_Body Else $objEmail.Textbody = $as_Body & @CRLF EndIf $objEmail.Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/sendusing") = 2 $objEmail.Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpserver") = $s_SmtpServer If Number($IPPort) = 0 Then $IPPort = 25 $objEmail.Configuration.Fields.Item("http://schemas.microsoft.com/cdo/configuration/smtpserverport") = $IPPort $objEmail.Configuration.Fields.Update $objEmail.Fields.Update ;$objEmail.Send **************** If @error Then MyErrFunc() EndIf $objEmail = "" EndFunc;fINetSmtpMailCom  
    • By Colduction
      Hi guys, i'm using Telegram UDF by @LinkOut from github with latest update, but this UDF has not any parse_mode ability in SendDocument and other send file's functions to make texts bold, italic or underline and i can't send Emojis via these functions too. i've tried to change HTML section of multipart/form-data but i did not get correct results.

      For example, i can't get correct results by sending a document with this URL Encoded caption: %F0%9F%93%84%20*Test*%20%F0%9F%93%84

      I will be happy to help me in this section. Thanks!
    • By DannyJ
      I use _ClipPutHTML UDF function 
      My problem is that I am not able to write characters with accets.
      When I paste this code to an Mail program the accent characters will be Chinese characters or '???' characters.
      Here is a snippet of my code:
      #include <_ClipPutHTML.au3> $sHTMLStr='<html><head>'&@CRLF & " <title>Page Title</title>"&@CRLF & _ ' <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'&@CRLF & _ "</head>"&@CRLF & "<body>"&@CRLF & "<h1>Headline Text</h1>"&@CRLF & _ "<p>" & "ófiéááéllááéáéá:" & Chr(225) & BinaryToString("á",4) &@CRLF & _ '<a href="http://www.autoitscript.com/forum/index.php?showtopic=96556">_ClipPutHTML() functions</a>.'&@CRLF& _ " The regular modifiders, such as <strong>bold</strong>, <i>italics</i>, and <u>underlines</u> work as usual,"&@CRLF& _ " just like all other HTML formatting.</p>"&@CRLF & "<p>&nbsp;</p>"&@CRLF & _ "<p><strong>Here's an example list:</strong></p>"&@CRLF & "<ul>"&@CRLF & _ " <li>List <i>itemü</i> #1.</li>"&@CRLF & _ " <li>List <i>itemá</i> #2.</li>"&@CRLF & _ ' <li>List <i>itemé</i> #3 with a <a href="http://www.google.com">Hyperlink</a></li>'&@CRLF & _ "</ul>"&@CRLF & "</body>"&@CRLF & "</html>" $sPlainTextStr="Headline Text"&@CRLF&@CRLF& _ "ófigyeljáéáéá" & Chr(225) & "_ClipPutHTML() functions."& _ "The regular modifiders, such as bold, italics, and underlines work as usual, just like all other HTML formatting."&@CRLF&@CRLF& _ "Here's an example list:"&@CRLF& _ " * List itemü #1."&@CRLF& _ " * List itemá #2."&@CRLF& _ " * List itemé #3 with a Hyperlink"&@CRLF ;I have tired this way, but it does not work. ;$UTF8HTML = BinaryToString($sHTMLStr,4) ;ConsoleWrite($UTF8HTML) ;$sUTF8String=BinaryToString($sPlainTextStr,4) ConsoleWrite($sUTF8String) _ClipPutHTML($UTF8HTML,$sUTF8String) ; Special Unicode text call ;_ClipPutHyperlink("http://www.google.co.jp/",ChrW(0x30B0)& ChrW(0x30FC)& ChrW(0x30B0)& ChrW(0x30EB)& " (Japanese Google)") ; Regular text ;_ClipPutHyperlink("http://www.google.com","itt")  
    • By mdwerne
      Hello,
      Before Broadcom took over Symantec, I was able to use the following code as the base to scrape daily definition downloads from the web. Since the pages were moved over to the Broadcom web servers, I get a page full of what I believe may be JavaScript instead of the fully rendered page that lies behind it. Does anyone have any suggestions as to how I can read the full rendered webpage using AutoIt?
      #include <IE.au3> $oIE_DEFS = _IECreate("https://www.broadcom.com/support/security-center/definitions/download/detail?gid=sep14", 0, 0, 1, 1) $sString_DEFS = _IEBodyReadHTML($oIE_DEFS) MsgBox(0, "https://www.broadcom.com/support/security-center/definitions/download/detail?gid=sep14", $sString_DEFS) The above code will show the JavaScript, but if you go to the URL in a browser that has JavaScript enabled, you will see the fully rendered page that I would like to access. I hope my question makes sense, and I would appreciate any suggestions to get this working again.
      All the best,
      -Mike
×
×
  • Create New...