Jump to content
Sign in to follow this  
uncommon

This might not be an autoit question but...

Recommended Posts

uncommon

Looking to run some scripts through this site. Source code a bit too big to post but I am looking for a way to search though this database in a systematic fashion but I don't know how to accomplish this.

http://www.ocpafl.org/Searches/ParcelSearch.aspx

I know how to put fields in the search box and search any address then rip the source code off the page. That's easy.

What I don't get is to make sure I don't miss any address when I am searching through the database.

Ideas comments?


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites
Jury

Use 'Property Use Search' and go through each 'Specify Property Type' listed selecting 'All' and then grab the items as you go through the pages.

Share this post


Link to post
Share on other sites
uncommon

I feel kind of dumb for not think thinking of that myself. Thanks

After testing it a few more times I found out you can search for properties via the PID like this http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/272001000000008

So now its only a matter of time.... a Long time


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites
Jury

uncommon,

You should be able to automate coing through the pages and grabbing the content into array and then into delimited file - hint:

$oNext = _IEGetObjByName($oIE, "pager1", $z)

_IEAction($oNext, "click")

_IELoadWait($oIE)

Sleep(2000)

Local $oTable = _IETableGetCollection($oIE, 0)

Local $aTableData = _IETableWriteToArray($oTable, 1)

_FileWriteFromArray($file, $aTableData, 1, 0)

Share this post


Link to post
Share on other sites
uncommon

uncommon,

You should be able to automate coing through the pages and grabbing the content into array and then into delimited file - hint:

$oNext = _IEGetObjByName($oIE, "pager1", $z)

_IEAction($oNext, "click")

_IELoadWait($oIE)

Sleep(2000)

Local $oTable = _IETableGetCollection($oIE, 0)

Local $aTableData = _IETableWriteToArray($oTable, 1)

_FileWriteFromArray($file, $aTableData, 1, 0)

Thanks was using the IE to get webpages but the Inetget out to file works so much for eficaintly, especailly if you are doing multiple request on 200,000 some pages. My current plan is to only work on getting all the pages into a folder, then use a seprat scrpt aftwards to pharse the HTML for the needed content into SQL or some kind of database.

Hmm I might use IE to naviget the local files once downloaded to make the scraping a but easier, thanks.


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites
uncommon

Right now the fastest way I can mine someone's database is by making hundreds of individual executables that all do there on INETGET TCP Request, obviously this take up a lot of processing and RAM resources. Anyone know of a way I can make more requests for pages faster\more efficiently?

My scripts that I run look something like this...

;A setprate script makes a txt file with part of a URL to go to
#include <File.au3>
#include <Array.au3>
$htmlstore = @DesktopCommonDir & "\HTMLstore\"
$FileList = _FileListToArray($htmlstore)
For $count = 2 To $FileList[0] + 1
If FileExists($htmlstore & $count & ".txt") = 1 Then
FileMove($htmlstore & $count & ".txt", $htmlstore & @AutoItPID & @ComputerName & ".txt")
$file = FileOpen($htmlstore & @AutoItPID & @ComputerName & ".txt")
$PID = FileRead($file)
FileClose($file)
$hDownload = InetGet("http://www.ocpafl.org/Searches/ParcelSearch.aspx/PID/" & $PID, $htmlstore & $PID & ".html", 1)
InetClose($hDownload) ; Close the handle to release resources.
FileDelete($htmlstore & @AutoItPID & @ComputerName & ".txt")
Exit
EndIf
Next

If you need more info let me know.

Any recommendations would be appreciated


No problem can withstand the assault of sustained thinking.Voltaire

_Array2HTMLTable()_IEClassNameGetCollection()_IEquerySelectorAll()

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • XaelloNegative
      By XaelloNegative
      Hi,
      So, I am trying to automate and simplify the retrieval of data from our company website (made by an outside company). I've had experience with IE manipulation via autoIt however, this one is a bit tricky for me.
      I have a table that has 81 cells which are buttons. What im trying to get is the information in "data-content" tag. 
      <a id="bookedVehicleLinkButton" class="btn btn-sm btn-block btn-xsm btn-success" data-toggle="popover" data-placement="bottom" data-original-title="Trip Details" data-container="body" data-content="Plate No: UGQ-857<br />Model: TOYOTA GRANDIA GL" href="javascript:__doPostBack('ctl00$ctl00$masterContentPlaceholder$reservationContentPlaceholder$bookingDataList$ctl00$vehicleDataList$ctl28$bookedVehicleLinkButton','')">IXARA (LITO SULIT)</a> Tried using the following:
      $oTagsCell = _IETagNameGetCollection($oIE, "td") For $item in $oTagsCell $test = _IEPropertyGet($item, "innertext") ClipPut($test) Next $oTagsTable = _IETagNameGetCollection($oIE, "table") For $item in $oTagsTable $test = _IEPropertyGet($item, "innertext") ClipPut($test) Next $oTagsLink = _IETagNameGetCollection($oIE, "a") For $item in $oTagsLink $test = _IEPropertyGet($item, "innertext") ClipPut($test) Next I don't receive any errors from my aforementioned attempts however no luck in getting those "data-contents". Attached is a screenshot, I do the looking of elements in chrome but the codes are for IE. And if you might ask, the cells (or buttons) have the same ID.
       
      Thank you guys for any inputs.
       
    • XinYoung
      By XinYoung
      Quick question: How can I click this "Select All" button in IE? There's no ID, and the href is "#"  
      Am I looking at the wrong html code?
      Am I looking
      Thank you all in advance!
    • Jemboy
      By Jemboy
      Hi,
      At work we have some proprietary website, users have to login to.
      I have "made" an autoit executable to start IE, go the website, login, so the user do not have input their credentials every time.
      By NDA I am not allowed disclosed the URL of the website nor the login credentials
      So I made a fake website and an autoitscript to illustrate my question.
      #include <ie.au3> $oIE = _IECreate ("about:blank", 0, 1, 1, 1) $HWND = _IEPropertyGet($oIE, "hwnd") WinActivate ($HWND,"") WinSetState($HWND, "", @SW_MAXIMIZE) _IENavigate ($oIE, "http://demo.rkilinc.nl",1) The above start my demo website. The actual website has some links in the footer that I do not want most people click on.
      I contacted the developers of the website and they are thinking of making an option to configure what links to show in the footer, but they said it's not a high priority for them.
       
      I discovered, that by click F12 and deleting the <footer> element the footer is delete from the live page view (until the page is reloaded off course)
      I want to automate the removal of the footer, without using things like send().
      I tried getting the footer with _IEGetObjById and deleting it with _IEAction, but that didn't work.
      Does any one has an idea how I could delete the footer directly from view with an autoit script?
      TIA, Jem.
       
       

×