Help request for scraping table form IE window

[William007]

Hi all,

Impressed with site and available tools

Could anyone assist me with this:

I try to scrape a table from uk.moneybee.net, a neat site that uses Fuzzy logic (like Seti@home) to predict stock price for next two weeks on limited nr of stock / indices. Now doing some research on how well price is predicted. 1e I want to write a table to a file, stripped from HTML code. Later I backtest the prediction with the actual stock price .. At first glance it shows positive results

For testing this script you need an account at moneybee (sorry for that) and login on the site. You dont need to install the screensaver or Neuralnet software, only if you want to participate in the project and keep using their data.

Than run the script..

Problems will show up: file remains empty, or get erros related to Array dimensions. tried Excel UDF, no results

Search this site extensively, but have not found the clue yet

Any idea how to get it to work? In return I share the outcome of the predictions in a week or 2-3.

Thanks in advance

William

; *******************************************************

; Before running: login at uk.moneybee.net

; Than run, go to Moneybee and scrape table, write to file

; *******************************************************

;

#include <IE.au3>

#include <Array.au3>

#include <file.au3>

#include <ExcelCOM.au3> >>>> Tried that, Array subscript problems !!!!!!!!!!!!!!!!!!!!!

$oIE = _IECreate()

_IENavigate($oIE,"http://uk.moneybee.net/mitglieder/prognosen_detail.asp?t=1")

; Fully body text shows to much text, only need table.

$sText = _IEBodyReadText ($oIE)

;MsgBox(0, "Body Text", $sText)

;_FileWriteLog("Text.log",$sText)

; Write Array to file

; NB no go with pictures in line with text !!!!!!!!!!!!!!!!!!!!!

; So search table 4 in page, thsi contains data I need

$oTable = _IETableGetCollection ($oIE,4)

$aTableData = _IETableWriteToArray ($oTable)

_ArrayDisplay($aTableData, "_ArrayDisplay() Test")

; Seem to need this, dont know why..!!!!!!!!!!!!!!!!!!!!!!!!!!

ReDim $aTableData[uBound($aTableData)+1]

; write array to file

_FileWriteFromArray("Text2.log", $aTableData, 1)

; File remains empty

Edited July 12, 2007 by William007

[PsaltyDS]

$oTable = _IETableGetCollection ($oIE,4)

$aTableData = _IETableWriteToArray ($oTable)

_ArrayDisplay($aTableData, "_ArrayDisplay() Test")

; Seem to need this, dont know why..!!!!!!!!!!!!!!!!!!!!!!!!!!

ReDim $aTableData[uBound($aTableData)+1]

; write array to file

_FileWriteFromArray("Text2.log", $aTableData, 1)

; File remains empty

That ReDim is abusrd. What is in the array at the _ArrayDisplay() before the ReDim? Is it perhaps a 2D array? Are you blowing out the data by ReDim'ing it to a 1D array... Hmmm...?

If you are getting errors displaying a 2D array with _ArrayDisplay(), it means you are NOT running the current version of AutoIt. Download at least version 3.2.4.9 before you go on...

P.S. You may have to code a little loop to turn your 2D array into a string or a 1D array for writing to the file. I don't think _FileWriteFromArray() accepts 2D arrays.

Edited July 12, 2007 by PsaltyDS

[William007]

That ReDim is abusrd. What is in the array at the _ArrayDisplay() before the ReDim? Is it perhaps a 2D array? Are you blowing out the data by ReDim'ing it to a 1D array... Hmmm...?

If you are getting errors displaying a 2D array with _ArrayDisplay(), it means you are NOT running the current version of AutoIt. Download at least version 3.2.4.9 before you go on...

P.S. You may have to code a little loop to turn your 2D array into a string or a 1D array for writing to the file. I don't think _FileWriteFromArray() accepts 2D arrays.

Hi PsaltyDS

Thanks so far.

Running Autoit 3.2.4.9, upgraded it right over an older version

It is an 2D array, 2nd Column is empty. only Row 5 is the text I need. Could I simply delete the 2nd column and other rows ?

William

[PsaltyDS]

Hi PsaltyDS

Thanks so far.

Running Autoit 3.2.4.9, upgraded it right over an older version

It is an 2D array, 2nd Column is empty. only Row 5 is the text I need. Could I simply delete the 2nd column and other rows ?

William

Then just FileWriteLine("Text2.log", $aTableData[5][0]). No need for all the fiddling if you know exactly where it is already. Perhaps you could put a test on it to be sure:

If (Ubound($aTableData, 0) = 2) and (Ubound($aTableData) >= 5) and ($aTableData[5][0] <> "") Then
     FileWriteLine("Text2.log", $aTableData[5][0])
Else
     MsgBox(16, "Error", "Invalid data from $aTableData.")
EndIf

Edited July 12, 2007 by PsaltyDS

[William007]

Then just FileWriteLine("Text2.log", $aTableData[5][0]). No need for all the fiddling if you know exactly where it is already. Perhaps you could put a test on it to be sure:

If (Ubound($aTableData, 0) = 2) and (Ubound($aTableData) >= 5) and ($aTableData[5][0] <> "") Then
     FileWriteLine("Text2.log", $aTableData[5][0])
Else
     MsgBox(16, "Error", "Invalid data from $aTableData.")
EndIf

YEP

Thats it, stupid of me not to try that option

Now it comes down to cleaning the text, like get the TABs and spaces right for importing it into Excel

Il keep u posted

Cheers, William

Edited July 12, 2007 by William007