Jump to content

inetget() Catch the content of the 404 response returned by the webserver?

rudi

By rudi,
in AutoIt General Help and Support

 Share Followers 1

Recommended Posts

rudi

rudi 50

Posted
Posted (edited)

Hello,

is there a way to use inetget() to catch the content of an 404 error page returned by the web server?

  

$URL="https://www.autoitscript.com/ThisPathDoesntExist"

$content=InetGet($url,"c:\temp\xxx.html",1+2)
ConsoleWrite('@@ Debug(' & @ScriptLineNumber & ') : $content = "' & $content &  """" & @CRLF & "@Extended: """ & @extended & """" & @CRLF & '>Error code: ' & @error & @CRLF) ;### Debug Console

  

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\temp\löschmich\xxx.au3" /UserParams    
+>15:27:05 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0   Keyboard:00000407  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0407)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\admin.AD\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\admin.AD\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\temp\löschmich\xxx.au3
+>15:27:05 AU3Check ended.rc:0
>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\temp\löschmich\xxx.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
@@ Debug(6) : $content = "0"
@Extended: "0"
>Error code: 13
+>15:27:05 AutoIt3.exe ended.rc:0
+>15:27:05 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 0.9361

 

The browser (I use Chrome) is displaying this 404 page: (That's what I'd like to catch) 

Not Found
The requested URL /ThisPathDoesntExist was not found on this server.

html code (Browser ctrl+u): 

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /ThisPathDoesntExist was not found on this server.</p>
</body></html>

 

Wireshark response 404 packet: 

Hypertext Transfer Protocol
    HTTP/1.1 404 Not Found\r\n
    Server: nginx\r\n
    Date: Wed, 06 Apr 2022 13:34:26 GMT\r\n
    Content-Type: text/html; charset=iso-8859-1\r\n
    Content-Length: 217\r\n
    Connection: keep-alive\r\n
    Vary: Accept-Encoding\r\n
    \r\n
    [HTTP response 1/1]
    [Time since request: 0.056074000 seconds]
    [Request in frame: 1476]
    [Request URI: http://www.autoitscript.com/ThisPathDoesntExist]
    File Data: 217 bytes


Line-based text data: text/html (7 lines)
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /ThisPathDoesntExist was not found on this server.</p>
</body></html>

any suggestions appreciated,

<edit> also tried _inetgetsource() and inetread() </edit>

Rudi

Edited by rudi

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites
rudi

rudi 50

Posted
  • Author
Posted

Hello @Danp2

you are right, I searched and found that WinHTTP UDF by @trancexx and @ProgAndy , didn't know it before.

 

Thanks for pointing out that UDF!

 

This code is doing excatly what I need:

  

; modified from the CHM help file for WinHTTP 1.6.4.1, topic _WinHttpReadData
#include-once
#include "C:\temp\WinHTTP\1.6.4.1\WinHttp.au3"
#include "C:\temp\WinHTTP\1.6.4.1\WinHttpConstants.au3"


$sDomain = "www.autoitscript.com"
$sPage = "/ThisPathDoesntExist"

; Data to send
; $sAdditionalData = "name=" & $sUserName & "&email=" & $sEmail

; Initialize and get session handle
$hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0")

; Get connection handle
$hConnect = _WinHttpConnect($hOpen, $sDomain)

; Make a request
$hRequest = _WinHttpOpenRequest($hConnect, "GET", $sPage)

; Send it. Specify additional data to send too. This is required by the Google API:
_WinHttpSendRequest($hRequest, "Content-Type: application/x-www-form-urlencoded")

; Wait for the response
_WinHttpReceiveResponse($hRequest)

; See what is returned
Dim $sReturned
If _WinHttpQueryDataAvailable($hRequest) Then ; if there is data
    Do
        $sReturned &= _WinHttpReadData($hRequest)
    Until @error
EndIf

; Close handles
_WinHttpCloseHandle($hRequest)
_WinHttpCloseHandle($hConnect)
_WinHttpCloseHandle($hOpen)

; See what is returned
MsgBox(4096, "Returned", $sReturned)
ConsoleWrite($sReturned & @CRLF)

  

>"C:\Program Files (x86)\AutoIt3\SciTE\..\AutoIt3.exe" "C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.au3" /run /prod /ErrorStdOut /in "C:\temp\löschmich\xxx.au3" /UserParams    
+>17:15:42 Starting AutoIt3Wrapper v.18.708.1148.0 SciTE v.4.1.0.0   Keyboard:00000407  OS:WIN_10/  CPU:X64 OS:X64  Environment(Language:0407)  CodePage:0  utf8.auto.check:4
+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\admin.AD\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\admin.AD\AppData\Local\AutoIt v3\SciTE 
>Running AU3Check (3.3.14.5)  from:C:\Program Files (x86)\AutoIt3  input:C:\temp\löschmich\xxx.au3
+>17:15:42 AU3Check ended.rc:0
>Running:(3.3.14.5):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\temp\löschmich\xxx.au3"    
--> Press Ctrl+Alt+Break to Restart or Ctrl+Break to Stop
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /ThisPathDoesntExist was not found on this server.</p>
</body></html>

+>17:15:55 AutoIt3.exe ended.rc:0
+>17:15:55 AutoIt3Wrapper Finished.
>Exit code: 0    Time: 13.99

 

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites
rudi

rudi 50

Posted
  • Author
Posted (edited)

@Danp2

<cite> can you explain more what your script is doing?</cite>

There was a major release upgrade for our PPS

 

For many articles in the PPS there are CAD drawings. One possible property of articles is the name of the associated drawing

for the old PPS system there was a java servlet that did return the drawing name for a given article number:

 

Example: $URL='http://10.27.10.81/Fehlteil/ident2zeichnr.jsp?identnr="123456"'

It always [200] returned content: either holding the name of the according drawing, or, if the article existed but no drawing, or even the article didn't exist at all a "empty page" (http 200)

any @error returned pointed out a malfunction of the java servlet URL (what showed up time by time, quite rarely)

 

This java servlet had to be rewritten due to the release upgrade mentioned above, and this was done using different java servlet tools.

Now the behavior changed:

  • Article exists, property "drawing" is populated:
    http [200] <drawing name>
  • Article exists, property "drawing" not populated:
    http [404] "ERROR: ZEICHNR NOT FOUND"
  • Article doesn't exist at all:
    http [404] "ERROR: IDENTNR NOT FOUND"

My script for the old java servlet used inetread(), also tried inetget(), both return just an error but not the content of the html [404] page.

 

Thanks for your help, Rudi.

Edited by rudi

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Followers 1
Go to topic listing

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • Jahar
      InetGet was working previously, but not extracting full html
      By Jahar
      I have an AutoIT script It monitors 2 websites for content that applys to me and the services that I provide. One site is : www.Freelancer.com The other: www.PeoplePerHour.com Both sites publish new jobs on their site hourly or so. My AutoIT app, will view those sites and present new jobs to me in a grid that pops up on my screen. Lately, the app has stopped showing me any jobs from PeoplePerHour.
       
      For freelancer.com,  Inetget is giving full html but for peopleperhour, now its not coming.
      Func _CheckPPH()
          Local Static $hTimer = 0
          Local Static $hDownload = 0
          Local $aTitlesandUrls = 0
          Local Static $sTempFile = ""
          If $hTimer = 0 Then $hTimer = TimerInit()
          If $hDownload = 0 Then
              $sTempFile = _WinAPI_GetTempFileName(@TempDir)
              ConsoleWrite("Checking PPH..." & @CRLF)
              ConsoleWrite(">Downloading..." & @CRLF)
      ;~         $hDownload = InetGet("http://www.peopleperhour.com/freelance-jobs", $sTempFile, $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)
              $hDownload = InetGet("http://www.peopleperhour.com/freelance-jobs", $sTempFile, $INET_FORCERELOAD)
      ;~         Return 0
          EndIf
      ;~     Sleep(30)
      ;~     Local $isCompleted = InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)
      ;~     Local $isError = InetGetInfo($hDownload, $INET_DOWNLOADERROR)
      ;~     Sleep(30)
      ;~     If TimerDiff($hTimer) > 3000 And $isError Then
      ;~         ConsoleWrite("!PPH Fail" & @CRLF)
      ;~         InetClose($hDownload)
      ;~         $hDownload = 0
      ;~         Return 0
      ;~     EndIf
      ;~     Sleep(30)
          Local $Show = 0
      ;~     If TimerDiff($hTimer) > 3000 And $isCompleted Then
          If $hDownload > 0 Then
              ConsoleWrite("+Downloaded..." & @CRLF)
              Local $sPPHHtml = FileRead($sTempFile)
              $aTitlesandUrls = _StringBetween($sPPHHtml, '"title">' & @LF, 'time>')
      ;~         _ArrayDisplay($aTitlesandUrls)
              Local $aPPH[0][4]
              Local $sTitle = ""
              Local $sUrl = ""
              Local $sID = ""
              Local $sDate = ""
              Local $iRet=0
              Sleep(30)
              For $i = 0 To UBound($aTitlesandUrls) - 1
                  $sTitle = _StringBetween($aTitlesandUrls[$i], '<a title="', '" class')
                  $sUrl = _StringBetween($aTitlesandUrls[$i], 'href="', '">')
                  $sDate = _GetDate($aTitlesandUrls[$i])
                  If IsArray($sTitle) And IsArray($sUrl) Then
                      $sID = _GetID($sUrl[0])
      ;~                 _ArrayAdd($aPPH, $sDate & "|" & $sTitle[0] & "|" & $sUrl[0] & "|" & $sID)
                      $iRet = _BuildPopupsPPH($sID, $sDate, "PPH: " & $sTitle[0], $sUrl[0])
                      If $iRet Then $Show+=1
                  EndIf
              Next
              Sleep(30)
      ;~         If $Show > 0 Then ShowLatestJobs()
      ;~         _ArrayDisplay($aPPH)
              FileDelete($sTempFile)
              InetClose($hDownload)
              $hDownload = 0
              $hTimer = 0
              Return $Show
          EndIf
          Sleep(30)
      EndFunc   ;==>_CheckPPH
    • Tersion
      Why InetGet() allow only two simultaneously background downloads?
      By Tersion
      Why I can't get more than two simultaneously background downloads?
      Simultaneously background downloads with InetGet() works! It was just server limitation from which I downloaded files.
    • leb
      InetGet error 13
      By leb
      Hello there,
      I am using the function InetGet example in the help files on a website and getting the error 13. First, I searched the help files and the forum for a list of errors to consult to no avail. I suspect this is a 400's server reply based error but it would be nice to get more info about it.
      Please help, thanks.
      L
    • Dante_t
      Downloading files from https sites using inetget
      By Dante_t
      Hi Guys, I need help. I have searched the forum before posting and i couldn't find anything. The code below works fine when downloading files from "http" sites, but when trying to download from "https" sites, no files are downloaded. I tried different sites and I experience the same problem everywhere. Is there something I'm missing or doing wrong? Please note that I'm not a programmer and I'm new to this. I'm just using logic wherever i can to get things done. your help will be highly appreciated.
       
      #include <InetConstants.au3>
      #include <MsgBoxConstants.au3>
      #include <WinAPIFiles.au3>
      ; Download a file in the background.
      ; Wait for the download to complete.

      Example()
      Func Example()
          ; Save the downloaded file to the temporary folder.
          Local $sFilePath = "d:\"
          ; Download the file in the background with the selected option of 'force a reload from the remote site.'
          Local $hDownload = InetGet("https://en.wikipedia.org/wiki/HTTPS#/media/File:Internet2.jpg", $sFilePath& "Internet2.jpg", $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)
          ; Wait for the download to complete by monitoring when the 2nd index value of InetGetInfo returns True.
          Do
              Sleep(250)
          Until InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)
          ; Retrieve the number of total bytes received and the filesize.
          Local $iBytesSize = InetGetInfo($hDownload, $INET_DOWNLOADREAD)
          Local $iFileSize = FileGetSize($sFilePath&"Internet2.jpg")
          ; Close the handle returned by InetGet.
          InetClose($hDownload)
          ; Display details about the total number of bytes read and the filesize.
          MsgBox($MB_SYSTEMMODAL, "", "The total download size: " & $iBytesSize & @CRLF & _
                  "The total filesize: " & $iFileSize)
          ; Delete the file.
          ;FileDelete($sFilePath)
      EndFunc   ;==>Example
       
    • swatsapkraz
      Download and read file help
      By swatsapkraz
      First script here. Thanks for taking the time.
      I want to download a file from my dropbox or other cloud file host and I want autoit to read the file and proceed.
      Here are the references I've gone through, it's just I'm not familiar yet with autoit so I'm looking for advice:
      https://www.autoitscript.com/autoit3/docs/functions/InetGet.htm
      https://www.autoitscript.com/autoit3/docs/functions/FileRead.htm
       
      How would I start out downloading a text file from dropbox and if in the file there is a 1 then it will proceed with the rest of the script if there is a 0 or if the file cannot be downloaded I want it to just end.
       
      Thank you for taking the time to read this and I apologize in advance if this seems very trivial for some but this is my first script and I'm hoping this is the correct place to ask this question.
×
×
  • Create New...