Jump to content
Sign in to follow this  
uncommon

Many INETGET TCP Request

Recommended Posts

uncommon

Right now the fastest way I can mine someone's database is by making hundreds of individual executable that all do there on INETGET TCP Request, obviously this take up a lot of processing and RAM resources. Anyone know of a way I can make more requests for pages faster\more efficiently?

My scripts that I run look something like this...

;A setprate script makes a txt file with part of a URL to go to
#include <File.au3>
#include <Array.au3>
$htmlstore = @DesktopCommonDir & "\HTMLstore\"
$FileList = _FileListToArray($htmlstore)
For $count = 2 To $FileList[0] + 1
If FileExists($htmlstore & $count & ".txt") = 1 Then
FileMove($htmlstore & $count & ".txt", $htmlstore & @AutoItPID & @ComputerName & ".txt")
$file = FileOpen($htmlstore & @AutoItPID & @ComputerName & ".txt")
$PID = FileRead($file)
FileClose($file)
$hDownload = InetGet("http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/" & $PID, $htmlstore & $PID & ".html", 1)
InetClose($hDownload) ; Close the handle to release resources.
FileDelete($htmlstore & @AutoItPID & @ComputerName & ".txt")
Exit
EndIf
Next

If you need more info let me know.

Any recommendations would be appreciated


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites
somdcomputerguy

..by making hundreds of individual executable..

I think what you mean is you have to run that script many times in order to get done what you're trying to get done. If that's so, try commenting out (or removing) the Exit line from the If loop.

- Bruce /*somdcomputerguy */  If you change the way you look at things, the things you look at change.

Share this post


Link to post
Share on other sites
uncommon

I think what you mean is you have to run that script many times in order to get done what you're trying to get done. If that's so, try commenting out (or removing) the Exit line from the If loop.

Alright here is the real problem, every I fire an Inetget and run it in the background and then run a different Inetget, it cancels the first and uses the second one. So I need to run many (100s) of Inetgets at once. How do I do this?

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites
BrewManNH

Look at the 4th parameter for InetGet, should be fairly simple to figure out how to do it in the background while you run the next one. Also, it's explained fairly well in the help file that this is how to do multiple downloads at the same time.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites
Chance

I made a dll that will use winhttp in threads to make multiple background requests.

I couldn't get it to work with autoit and the winhttp library or wininet either.

the multithreaded dll worked nicely, maybe you can try that.

  • Like 1

Share this post


Link to post
Share on other sites
uncommon

I made a dll that will use winhttp in threads to make multiple background requests.

I couldn't get it to work with autoit and the winhttp library or wininet either.

the multithreaded dll worked nicely, maybe you can try that.

I will look into this thanks FlutterShy I was thinking about multithreading.

And thank you BrewManNH I will do more testing on Inetget maybe I need to take a better look at remarks on the Inetget help file.


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • ur
      By ur
      I have an aws cloud machine where I kept all the build process to create some set of zip files and uploading them to OneDrive.
      Once it is completed, I have another local machine in our environment which is a acting as a local share server also.
      Is there anyway to notify this machine(without public IP) to start downloading these files.
      I have written an AutoIT exe to download these files and copy them to respective locations.But don't know how to notify this from the remote machine.
       
      Any suggestion??
       
    • Nareshm
      By Nareshm
      Its Possible To auto Login with autoit script ?
    • Pintu
      By Pintu
      Good evening guys,
      i'm having a problem, not about the code (i'm only thinking about it at the moment) but about the way i can do it.
      I have a webpage (photo N.1) it has some elements in it.
      I need only the table (photo N.2), looking at the code and with _IEFunctions i can easy find the table but how i can i "copy" it?
      As i said it's really easy to found with a script but what should i do then? Copy the source? And how can i display it to the user?
      In my head i'd like to display it inside a GUI, is that possible?
       
      Thanks in advance
       
      Edit: Posted 2 times same photo. Now should be OK

       

    • leb
      By leb
      Hello there,
      I am using the function InetGet example in the help files on a website and getting the error 13. First, I searched the help files and the forum for a list of errors to consult to no avail. I suspect this is a 400's server reply based error but it would be nice to get more info about it.
      Please help, thanks.
      L
    • Dante_t
      By Dante_t
      Hi Guys, I need help. I have searched the forum before posting and i couldn't find anything. The code below works fine when downloading files from "http" sites, but when trying to download from "https" sites, no files are downloaded. I tried different sites and I experience the same problem everywhere. Is there something I'm missing or doing wrong? Please note that I'm not a programmer and I'm new to this. I'm just using logic wherever i can to get things done. your help will be highly appreciated.
       
      #include <InetConstants.au3>
      #include <MsgBoxConstants.au3>
      #include <WinAPIFiles.au3>
      ; Download a file in the background.
      ; Wait for the download to complete.

      Example()
      Func Example()
          ; Save the downloaded file to the temporary folder.
          Local $sFilePath = "d:\"
          ; Download the file in the background with the selected option of 'force a reload from the remote site.'
          Local $hDownload = InetGet("https://en.wikipedia.org/wiki/HTTPS#/media/File:Internet2.jpg", $sFilePath& "Internet2.jpg", $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)
          ; Wait for the download to complete by monitoring when the 2nd index value of InetGetInfo returns True.
          Do
              Sleep(250)
          Until InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)
          ; Retrieve the number of total bytes received and the filesize.
          Local $iBytesSize = InetGetInfo($hDownload, $INET_DOWNLOADREAD)
          Local $iFileSize = FileGetSize($sFilePath&"Internet2.jpg")
          ; Close the handle returned by InetGet.
          InetClose($hDownload)
          ; Display details about the total number of bytes read and the filesize.
          MsgBox($MB_SYSTEMMODAL, "", "The total download size: " & $iBytesSize & @CRLF & _
                  "The total filesize: " & $iFileSize)
          ; Delete the file.
          ;FileDelete($sFilePath)
      EndFunc   ;==>Example
       
×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.