Sign in to follow this  
Followers 0
uncommon

Many INETGET TCP Request

8 posts in this topic

Right now the fastest way I can mine someone's database is by making hundreds of individual executable that all do there on INETGET TCP Request, obviously this take up a lot of processing and RAM resources. Anyone know of a way I can make more requests for pages faster\more efficiently?

My scripts that I run look something like this...

;A setprate script makes a txt file with part of a URL to go to
#include <File.au3>
#include <Array.au3>
$htmlstore = @DesktopCommonDir & "\HTMLstore\"
$FileList = _FileListToArray($htmlstore)
For $count = 2 To $FileList[0] + 1
If FileExists($htmlstore & $count & ".txt") = 1 Then
FileMove($htmlstore & $count & ".txt", $htmlstore & @AutoItPID & @ComputerName & ".txt")
$file = FileOpen($htmlstore & @AutoItPID & @ComputerName & ".txt")
$PID = FileRead($file)
FileClose($file)
$hDownload = InetGet("http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/" & $PID, $htmlstore & $PID & ".html", 1)
InetClose($hDownload) ; Close the handle to release resources.
FileDelete($htmlstore & @AutoItPID & @ComputerName & ".txt")
Exit
EndIf
Next

If you need more info let me know.

Any recommendations would be appreciated


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites



..by making hundreds of individual executable..

I think what you mean is you have to run that script many times in order to get done what you're trying to get done. If that's so, try commenting out (or removing) the Exit line from the If loop.

- Bruce /*somdcomputerguy */  If you change the way you look at things, the things you look at change.

Share this post


Link to post
Share on other sites

I think what you mean is you have to run that script many times in order to get done what you're trying to get done. If that's so, try commenting out (or removing) the Exit line from the If loop.

Alright here is the real problem, every I fire an Inetget and run it in the background and then run a different Inetget, it cancels the first and uses the second one. So I need to run many (100s) of Inetgets at once. How do I do this?

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites

Look at the 4th parameter for InetGet, should be fairly simple to figure out how to do it in the background while you run the next one. Also, it's explained fairly well in the help file that this is how to do multiple downloads at the same time.


If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Share this post


Link to post
Share on other sites

I made a dll that will use winhttp in threads to make multiple background requests.

I couldn't get it to work with autoit and the winhttp library or wininet either.

the multithreaded dll worked nicely, maybe you can try that.

1 person likes this

Share this post


Link to post
Share on other sites

I made a dll that will use winhttp in threads to make multiple background requests.

I couldn't get it to work with autoit and the winhttp library or wininet either.

the multithreaded dll worked nicely, maybe you can try that.

I will look into this thanks FlutterShy I was thinking about multithreading.

And thank you BrewManNH I will do more testing on Inetget maybe I need to take a better look at remarks on the Inetget help file.


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • Dante_t
      By Dante_t
      Hi Guys, I need help. I have searched the forum before posting and i couldn't find anything. The code below works fine when downloading files from "http" sites, but when trying to download from "https" sites, no files are downloaded. I tried different sites and I experience the same problem everywhere. Is there something I'm missing or doing wrong? Please note that I'm not a programmer and I'm new to this. I'm just using logic wherever i can to get things done. your help will be highly appreciated.
       
      #include <InetConstants.au3>
      #include <MsgBoxConstants.au3>
      #include <WinAPIFiles.au3>
      ; Download a file in the background.
      ; Wait for the download to complete.

      Example()
      Func Example()
          ; Save the downloaded file to the temporary folder.
          Local $sFilePath = "d:\"
          ; Download the file in the background with the selected option of 'force a reload from the remote site.'
          Local $hDownload = InetGet("https://en.wikipedia.org/wiki/HTTPS#/media/File:Internet2.jpg", $sFilePath& "Internet2.jpg", $INET_FORCERELOAD, $INET_DOWNLOADBACKGROUND)
          ; Wait for the download to complete by monitoring when the 2nd index value of InetGetInfo returns True.
          Do
              Sleep(250)
          Until InetGetInfo($hDownload, $INET_DOWNLOADCOMPLETE)
          ; Retrieve the number of total bytes received and the filesize.
          Local $iBytesSize = InetGetInfo($hDownload, $INET_DOWNLOADREAD)
          Local $iFileSize = FileGetSize($sFilePath&"Internet2.jpg")
          ; Close the handle returned by InetGet.
          InetClose($hDownload)
          ; Display details about the total number of bytes read and the filesize.
          MsgBox($MB_SYSTEMMODAL, "", "The total download size: " & $iBytesSize & @CRLF & _
                  "The total filesize: " & $iFileSize)
          ; Delete the file.
          ;FileDelete($sFilePath)
      EndFunc   ;==>Example
       
    • Nareshm
      By Nareshm
      How can i add sleep command using internet speed.
      If my internet speed is 40-50Kbps then sleep for 5 seconds,
      If my internet speed is 30-40Kbps then sleep for 4 seconds,
      If my internet speed is 20-30Kbps then sleep for 3 seconds.

      *Sleep command used bitween my costom script, Example,
      RunWait("rasdial /disconnect", "", @SW_HIDE) ; Stop connect from start
      Sleep(5000)
      RunWait("rasdial Internet", "", @SW_HIDE) ; Connect again
       
    • Nareshm
      By Nareshm
      Please Somebody Help me i am trying to create new script but, i don't know how to create it.

      I am Creating script like this :
      ; My script here [First Script]
      Check the internet connection, if internet connected then
      ; My Script here [Second Script]
      If internet not connected, wait for connect, after connected run my second script
      If Control id/windows/title matched do nothing and end/exit autoit script
      If Control id/windows/title not matched repeat my first and second script again until control id/windows/tittle not matched

      match with :
      >>>> Window <<<<
      Title:    PS :: Version - 2.8.0.0  - [PS Bill]
      Class:    WindowsForm734a
      Position:    -8, -8
      Size:    1382, 744
      Style:    0x17CD0000
      ExStyle:    0x00050100
      Handle:    0x0000000000130572
      >>>> Control <<<<
      Class:    WindowsForm734a
      Instance:    15
      ClassnameNN:    WindowsForm34a15
      Name:    BillPrint
      Advanced (Class):    [NAME:BillPrint]
      ID:    7875
      Text:    
      Position:    39, 310
      Size:    1221, 306
      ControlClick Coords:    745, 20
      Style:    0x56010000
      ExStyle:    0x00000000
      Handle:    0x00000000000C0468
       
    • Nareshm
      By Nareshm
      How can i stop/pause my internet connection for only 7 seconds, after 7 seconds automaticaly resume my internet.
      Please give me scripte
    • swatsapkraz
      By swatsapkraz
      First script here. Thanks for taking the time.
      I want to download a file from my dropbox or other cloud file host and I want autoit to read the file and proceed.
      Here are the references I've gone through, it's just I'm not familiar yet with autoit so I'm looking for advice:
      https://www.autoitscript.com/autoit3/docs/functions/InetGet.htm
      https://www.autoitscript.com/autoit3/docs/functions/FileRead.htm
       
      How would I start out downloading a text file from dropbox and if in the file there is a 1 then it will proceed with the rest of the script if there is a 0 or if the file cannot be downloaded I want it to just end.
       
      Thank you for taking the time to read this and I apologize in advance if this seems very trivial for some but this is my first script and I'm hoping this is the correct place to ask this question.