Jump to content

This might not be an autoit question but...


 Share

Recommended Posts

Looking to run some scripts through this site. Source code a bit too big to post but I am looking for a way to search though this database in a systematic fashion but I don't know how to accomplish this.

http://www.ocpafl.org/Searches/ParcelSearch.aspx

I know how to put fields in the search box and search any address then rip the source code off the page. That's easy.

What I don't get is to make sure I don't miss any address when I am searching through the database.

Ideas comments?

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Link to comment
Share on other sites

I feel kind of dumb for not think thinking of that myself. Thanks

After testing it a few more times I found out you can search for properties via the PID like this http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/272001000000008

So now its only a matter of time.... a Long time

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Link to comment
Share on other sites

uncommon,

You should be able to automate coing through the pages and grabbing the content into array and then into delimited file - hint:

$oNext = _IEGetObjByName($oIE, "pager1", $z)

_IEAction($oNext, "click")

_IELoadWait($oIE)

Sleep(2000)

Local $oTable = _IETableGetCollection($oIE, 0)

Local $aTableData = _IETableWriteToArray($oTable, 1)

_FileWriteFromArray($file, $aTableData, 1, 0)
Link to comment
Share on other sites

uncommon,

You should be able to automate coing through the pages and grabbing the content into array and then into delimited file - hint:

$oNext = _IEGetObjByName($oIE, "pager1", $z)

_IEAction($oNext, "click")

_IELoadWait($oIE)

Sleep(2000)

Local $oTable = _IETableGetCollection($oIE, 0)

Local $aTableData = _IETableWriteToArray($oTable, 1)

_FileWriteFromArray($file, $aTableData, 1, 0)

Thanks was using the IE to get webpages but the Inetget out to file works so much for eficaintly, especailly if you are doing multiple request on 200,000 some pages. My current plan is to only work on getting all the pages into a folder, then use a seprat scrpt aftwards to pharse the HTML for the needed content into SQL or some kind of database.

Hmm I might use IE to naviget the local files once downloaded to make the scraping a but easier, thanks.

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Link to comment
Share on other sites

Right now the fastest way I can mine someone's database is by making hundreds of individual executables that all do there on INETGET TCP Request, obviously this take up a lot of processing and RAM resources. Anyone know of a way I can make more requests for pages faster\more efficiently?

My scripts that I run look something like this...

;A setprate script makes a txt file with part of a URL to go to
#include <File.au3>
#include <Array.au3>
$htmlstore = @DesktopCommonDir & "\HTMLstore\"
$FileList = _FileListToArray($htmlstore)
For $count = 2 To $FileList[0] + 1
If FileExists($htmlstore & $count & ".txt") = 1 Then
FileMove($htmlstore & $count & ".txt", $htmlstore & @AutoItPID & @ComputerName & ".txt")
$file = FileOpen($htmlstore & @AutoItPID & @ComputerName & ".txt")
$PID = FileRead($file)
FileClose($file)
$hDownload = InetGet("http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/" & $PID, $htmlstore & $PID & ".html", 1)
InetClose($hDownload) ; Close the handle to release resources.
FileDelete($htmlstore & @AutoItPID & @ComputerName & ".txt")
Exit
EndIf
Next

If you need more info let me know.

Any recommendations would be appreciated

No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...