Jump to content
Jahar

InetGet was working previously, but not extracting full html

Recommended Posts

I have an AutoIT script It monitors 2 websites for content that applys to me and the services that I provide. One site is : www.Freelancer.com The other: www.PeoplePerHour.com Both sites publish new jobs on their site hourly or so. My AutoIT app, will view those sites and present new jobs to me in a grid that pops up on my screen. Lately, the app has stopped showing me any jobs from PeoplePerHour.

 

For freelancer.com,  Inetget is giving full html but for peopleperhour, now its not coming.

Func _CheckPPH()
    Local Static $hTimer = 0
    Local Static $hDownload = 0
    Local $aTitlesandUrls = 0
    Local Static $sTempFile = ""
    If $hTimer = 0 Then $hTimer = TimerInit()
    If $hDownload = 0 Then
        $sTempFile = _WinAPI_GetTempFileName(@TempDir)
        ConsoleWrite("Checking PPH..." & @CRLF)
        ConsoleWrite(">Downloading..." & @CRLF)
;~         $hDownload = InetGet("http://www.peopleperhour.com/freelance-jobs", $sTempFile, $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)
        $hDownload = InetGet("http://www.peopleperhour.com/freelance-jobs", $sTempFile, $INET_FORCERELOAD)
;~         Return 0
    EndIf
;~     Sleep(30)
;~     Local $isCompleted = InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)
;~     Local $isError = InetGetInfo($hDownload, $INET_DOWNLOADERROR)
;~     Sleep(30)
;~     If TimerDiff($hTimer) > 3000 And $isError Then
;~         ConsoleWrite("!PPH Fail" & @CRLF)
;~         InetClose($hDownload)
;~         $hDownload = 0
;~         Return 0
;~     EndIf
;~     Sleep(30)
    Local $Show = 0
;~     If TimerDiff($hTimer) > 3000 And $isCompleted Then
    If $hDownload > 0 Then
        ConsoleWrite("+Downloaded..." & @CRLF)
        Local $sPPHHtml = FileRead($sTempFile)
        $aTitlesandUrls = _StringBetween($sPPHHtml, '"title">' & @LF, 'time>')
;~         _ArrayDisplay($aTitlesandUrls)
        Local $aPPH[0][4]
        Local $sTitle = ""
        Local $sUrl = ""
        Local $sID = ""
        Local $sDate = ""
        Local $iRet=0
        Sleep(30)
        For $i = 0 To UBound($aTitlesandUrls) - 1
            $sTitle = _StringBetween($aTitlesandUrls[$i], '<a title="', '" class')
            $sUrl = _StringBetween($aTitlesandUrls[$i], 'href="', '">')
            $sDate = _GetDate($aTitlesandUrls[$i])
            If IsArray($sTitle) And IsArray($sUrl) Then
                $sID = _GetID($sUrl[0])
;~                 _ArrayAdd($aPPH, $sDate & "|" & $sTitle[0] & "|" & $sUrl[0] & "|" & $sID)
                $iRet = _BuildPopupsPPH($sID, $sDate, "PPH: " & $sTitle[0], $sUrl[0])
                If $iRet Then $Show+=1
            EndIf
        Next

        Sleep(30)
;~         If $Show > 0 Then ShowLatestJobs()
;~         _ArrayDisplay($aPPH)
        FileDelete($sTempFile)
        InetClose($hDownload)
        $hDownload = 0
        $hTimer = 0
        Return $Show
    EndIf
    Sleep(30)
EndFunc   ;==>_CheckPPH

Share this post


Link to post
Share on other sites
Posted (edited)

Is this topic related to your previous topic?  If so, why did you start another topic?  Also, why didn't you answer my question in the previous topic?  Is it because you knew that harvesting data from the sites that you referred to above is prohibited by their terms of use which would also mean that helping you to do so here would be prohibited?

 

Edited by TheXman
Typo

Share this post


Link to post
Share on other sites

@TheXman This is new script and new issue. For previous one, you have asked me to go thru scripts given as examples. That's different.

Share this post


Link to post
Share on other sites
9 minutes ago, Jahar said:

For previous one, you have asked me to go thru scripts given as examples.

No, I asked you why were asking for help to access a non-existent domain.

 

 

Share this post


Link to post
Share on other sites
Posted (edited)

@Jahar As stated above (and in the other thread) both sites you specify have verbiage in their TOS that states scraping or crawling of their site pages is not permitted. Case closed, please do not open another thread on this topic.

Edited by JLogan3o13

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By learner123
      Hi All,
       
      I am new to this AUTO IT and I have created a script that will open an app,enter pin and copy the code generated to clipboard. My java code call this autoIT script and use the copied generated code from clipboard.
      This works fine when server  window is on focus. My server is an windows server. 
      But when I minimize or disconnect the server, the script opens the app.exe but doesn't copy any value to clipboard.  
      Can anyone help me on this 😐
       
      Run("C:\Program Files (x86)\RSA SecurID Software Token\SecurID.exe")
      Local $hWnd=WinWait("abc - RSA SecurID Token") ; waits until the window is the active window
      $hWin = WinGetHandle("abc - RSA SecurID Token");
      ControlSend($hWnd,"","","1111") ; simulates pressing the Home key
      ControlSend($hWnd,"","","{ENTER}");
      ControlSend($hWnd,"","","^c");
      Sleep(1000) ;
      ControlSend($hWnd,"","","^c");
       
    • By learner123
      Hi All,
      So I have created a small autoIT script to enter pin into a RSA token(app which generate new code every 30 second), and copy the generated code.
      I have a java application which requires this code so every time my java-code requires this RSA code, it runs the autoIT script and the copied generated code is then used in my java application. 
      I have deployed this code on a windows server and it works fine when I am logged in and the window is on focus, But as soon as I schedule task and disconnect the server (not logged out only disconnect), or even minimize the server window, the autoIT scripts fails and its not able to copy the value.
       
      Please find below the code for AUTOIT.
       
      WinActivate("rsa - RSA SecurID Token") ; activates the window that has old in the tilte bar
      WinWaitActive("rsa - RSA SecurID Token") ; waits until the window is the active window
      Send("1111") ; simulates pressing the Home key, enters password to get the code
      Send("{ENTER}") ; simulates pressing the Enter key
      Sleep(1000) ;
      Send("^c") ; simulates pressing the CTRL+c keys (copy)
       
      Also I saw some post regarding that WINACTIVE only works when window is active. But my below AUTO IT script to handle windows pop up  works perfectly fine when the server is disconnected. 
       
      Opt("WinTitleMatchMode", 1)
      WinWait("https://url","","10")
      WinWaitActive("https://url","","10")
      Sleep(2000)
      Send("userid")
      Sleep(1000)
      Send("{TAB}")
      Sleep(1000)
      Send("passwrd")
      Send("{TAB}")
      Sleep(500)
      Send("{ENTER}")
       
       
    • By JuanFelipe
      Hello gyus, I try to make a code to login in a web page in my job, buy i can’t understand why my form is blocked, this is my code: 
      #include <ButtonConstants.au3> #include <EditConstants.au3> #include <GUIConstantsEx.au3> #include <StaticConstants.au3> #include <StringConstants.au3> #include <WindowsConstants.au3> #include <WinAPIFiles.au3> #include <FileConstants.au3> #include <File.au3> #include <Array.au3> #include <IE.au3> #include <Excel.au3> #include <GuiEdit.au3> #include <GuiStatusBar.au3> #include <DateTimeConstants.au3> #include <Date.au3> #Region ### START Koda GUI section ### Form=C:\noentry\koda_1.7.3.0\Forms\spoa.kxf $Form1 = GUICreate("Eva y Vehículos", 1200, 1000, 10, 10) $prueba = GUICtrlCreateButton("Prueba",10,990,50,20) ;VARIABLES ========================================================================================== $psi = "https://psi.policia.gov.co/PSI/Login.aspx?ReturnUrl=%2fPSI#no-back-button" $chequeVehiculos = "https://psi.policia.gov.co/PSI/frm_lista_chk.aspx" $evaluacionEva = "https://psi.policia.gov.co/PSI/eva_frmver.aspx" $oIE = ObjCreate("Shell.Explorer.2") $GUIActiveX = GUICtrlCreateObj ($oIE, 10, 10, 1180, 980) ;GUICtrlSetState(-1, $GUI_DISABLE) GUISetState(@SW_SHOW) $oIE.navigate($psi) ;_IELoadWait($oIE) _InicioSesion() #EndRegion ### END Koda GUI section ### While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit Case $prueba _InicioSesion() EndSwitch WEnd Func _InicioSesion() Local $username = _IEGetObjByName($oIE, "txtUsuario") Local $pass = _IEGetObjByName($oIE, "txtClave") Local $logina = _IEGetObjByName($oIE, "btnIngresar") $username.value="example" $pass.value="examplepass" _IEAction($logina, 'click') _IELoadWait($oIE) MsgBox(16,"","") EndFunc  
    • By shelly
      Here is the below code for handling pop-up when window is  inactive ..but I don't know how to change sleep and when i run this script it runs sometimes and sometimes it stops .
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{SPACE}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{DOWN}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{ENTER}")
      --- these 3 lines never worked while TAB lines works sometimes but not in accurate way
      I am new too AutoIt .. help me out why this script behaves in strange way
      ControlFocus("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog", "", "Internet Explorer_Server1","1")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog", "", "Internet Explorer_Server1","the request is send")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{SPACE}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{DOWN}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      Sleep(3000)
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{TAB}")
      ControlSend("Policy Decisions -- Webpage Dialog","","Internet Explorer_Server1","{ENTER}")
       
×
×
  • Create New...