Sign in to follow this  
Followers 0
ssubirias3

2 dimensional array, _INet, _IE help please

28 posts in this topic

#1 ·  Posted (edited)

Pardon this long post, hopefully it will sufficiently explain my goal and the questions around my situation. Also note as easy and simple as this stuff is for many of you, I'm rather new and hopefully not too dense with respect to au3.

Objective:

I want to pull multiple reports from marketwatch.com. Using AU3Recorder I have the initial script running using no arrays and using mouseclick commands to gather and cut and paste the data I want. Since there are multiple reports avaiable on each of the three markets (NYSE, AMEX, NASDAQ) I'm interested, my aim is to have the correct code loop until all reports are run. Despite _IETableWriteToArray and/or _IETableGetCollection being the "right tool" for the job, I think I would rather use _INetGetSource since _IETable functions seems to run slower having to wait for the browser and page to be "ready" before pullling the data.

It would be great if there was an _INetWriteSourceTableToArray, but I don't think such a beast exists at this time. For IE.au3 fans, perhaps you can explain why _IETable is as fast as _INetGetSource but seems slower? Then again maybe using _INetGetSource and _IETable isn't even the correct direction I should be heading. This is what I have so far in theory. Note that I'm not a programmer and am still trying to get my mind around arrays and au3 in general.

URL1 = http://www.marketwatch.com/tools/marketsum...e=15&view=0

URL2 = http://www.marketwatch.com/tools/marketsum...e=15&view=1

URL3 = http://www.marketwatch.com/tools/marketsum...e=15&view=6

URL4= http://www.marketwatch.com/tools/marketsum...e=15&view=7

I'm sure the following code is flawed, however I'm including it to show I am making an attempt on my own.

#include <INet.au3>
#include <array.au3>
#include <String.au3>

Dim $b = 0, $sStart, $sEnd
Dim $url = "http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange="


While $b <= 6 
    If $b = 0 Then Dim $site = "13&view=0", $Array1_i[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 1 Then Dim $site = "13&view=1", $Array1_ii[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 2 Then Dim $site = "13&view=6", $Array1_iii[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 3 Then Dim $site = "13&view=7", $Array1_iv[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 4 Then Dim $site = "14&view=0", $Array2_i[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 5 Then Dim $site = "14&view=1", $Array2_ii[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 4 Then Dim $site = "14&view=6", $Array2_iii[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    If $b = 5 Then Dim $site = "14&view=7", $Array2_iv[7][25] = [["Symbol"], ["Company"], ["Last"], ["Chng."], ["% Chng."], ["Volume"], ["$ Traded"]]
    ;;;;  this would continue until all the reports I wanted have been pulled
    
    _StockScan()
    $b = $b + 1
WEnd

Func _StockScan()
    $html = _INetGetSource($url & $site)
    $Array1 = _StringBetween($html, $sStart, $sEnd)
    _ArrayToClip($Array1)
EndFunc

What I'm trying to do is create an array that would contain 8 col and 26 rows that could be dumped into a txt file or whatever and the output look like the below output. Each report contains 25 stocks +1 for the header and my thinking is I need one array for each report, hence the $array1_i - iv.

Posted Image

....

The other thing that I'm not sure about is what my $sStart and $sEnd strings should read. Here's the html source code containing the information I'm after. And for this reason maybe _INetGetSource isn't the right tool for the job?? Just wish _IETable was as fast as _INetGet.

CODE
<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=QFAB" class="lk01">QFAB</a> </td><td align=left nowrap>  Quaker Fabric Corp</td><td align=right>  <NOBR>0.148</NOBR></td><td class="t18" align=right>  +<NOBR>0.043</NOBR></td><td class="t18" align=right nowrap>  +40.95%</td><td align=right>  1,596,605</td><td align=right>  236,298</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=WRLS" class="lk01">WRLS</a> </td><td align=left nowrap>  Telular Corp</td><td align=right>  <NOBR>5.95</NOBR></td><td class="t18" align=right>  +<NOBR>1.2399</NOBR></td><td class="t18" align=right nowrap>  +26.32%</td><td align=right>  773,261</td><td align=right>  4,600,903</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=TWTI" class="lk01">TWTI</a> </td><td align=left nowrap>  Third Wave Technologies Inc</td><td align=right>  <NOBR>7.39</NOBR></td><td class="t18" align=right>  +<NOBR>1.49</NOBR></td><td class="t18" align=right nowrap>  +25.25%</td><td align=right>  2,926,154</td><td align=right>  21,624,278</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=GGBM" class="lk01">GGBM</a> </td><td align=left nowrap>  Gigabeam Corp</td><td align=right>  <NOBR>5.9301</NOBR></td><td class="t18" align=right>  +<NOBR>1.0801</NOBR></td><td class="t18" align=right nowrap>  +22.27%</td><td align=right>  4,652,468</td><td align=right>  27,589,600</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=EEEI" class="lk01">EEEI</a> </td><td align=left nowrap>  Electro Energy Inc</td><td align=right>  <NOBR>1.24</NOBR></td><td class="t18" align=right>  +<NOBR>0.16</NOBR></td><td class="t18" align=right nowrap>  +14.81%</td><td align=right>  249,937</td><td align=right>  309,922</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=RONC" class="lk01">RONC</a> </td><td align=left nowrap>  Ronson Corporation</td><td align=right>  <NOBR>2.10</NOBR></td><td class="t18" align=right>  +<NOBR>0.27</NOBR></td><td class="t18" align=right nowrap>  +14.75%</td><td align=right>  48,706</td><td align=right>  102,283</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=UCTT" class="lk01">UCTT</a> </td><td align=left nowrap>  Ultra Clean Hldgs Inc</td><td align=right>  <NOBR>15.3099</NOBR></td><td class="t18" align=right>  +<NOBR>1.8699</NOBR></td><td class="t18" align=right nowrap>  +13.91%</td><td align=right>  2,437,601</td><td align=right>  37,319,428</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=PRLS" class="lk01">PRLS</a> </td><td align=left nowrap>  Peerless Systems Corporation</td><td align=right>  <NOBR>2.45</NOBR></td><td class="t18" align=right>  +<NOBR>0.29</NOBR></td><td class="t18" align=right nowrap>  +13.43%</td><td align=right>  165,252</td><td align=right>  404,867</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=MVIS" class="lk01">MVIS</a> </td><td align=left nowrap>  Microvision Inc</td><td align=right>  <NOBR>5.45</NOBR></td><td class="t18" align=right>  +<NOBR>0.60</NOBR></td><td class="t18" align=right nowrap>  +12.37%</td><td align=right>  12,125,634</td><td align=right>  66,084,705</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=BLUE" class="lk01">BLUE</a> </td><td align=left nowrap>  Blue Hldgs Inc</td><td align=right>  <NOBR>1.40</NOBR></td><td class="t18" align=right>  +<NOBR>0.14</NOBR></td><td class="t18" align=right nowrap>  +11.11%</td><td align=right>  400,167</td><td align=right>  560,234</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=CAMT" class="lk01">CAMT</a> </td><td align=left nowrap>  Camtek Ltd</td><td align=right>  <NOBR>3.75</NOBR></td><td class="t18" align=right>  +<NOBR>0.37</NOBR></td><td class="t18" align=right nowrap>  +10.95%</td><td align=right>  194,401</td><td align=right>  729,004</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=JDAS" class="lk01">JDAS</a> </td><td align=left nowrap>  JDA Software Group Inc</td><td align=right>  <NOBR>22.59</NOBR></td><td class="t18" align=right>  +<NOBR>2.17</NOBR></td><td class="t18" align=right nowrap>  +10.63%</td><td align=right>  2,313,573</td><td align=right>  52,263,614</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=PGLA" class="lk01">PGLA</a> </td><td align=left nowrap>  Progen Pharmaceuticals Limit</td><td align=right>  <NOBR>3.887</NOBR></td><td class="t18" align=right>  +<NOBR>0.367</NOBR></td><td class="t18" align=right nowrap>  +10.43%</td><td align=right>  70,169</td><td align=right>  272,747</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=APSA" class="lk01">APSA</a> </td><td align=left nowrap>  Alto Palermo S A</td><td align=right>  <NOBR>20.94</NOBR></td><td class="t18" align=right>  +<NOBR>1.9299</NOBR></td><td class="t18" align=right nowrap>  +10.15%</td><td align=right>  500</td><td align=right>  10,470</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=CJBK" class="lk01">CJBK</a> </td><td align=left nowrap>  Central Jersey Bancorp Nj</td><td align=right>  <NOBR>8.55</NOBR></td><td class="t18" align=right>  +<NOBR>0.7699</NOBR></td><td class="t18" align=right nowrap>  +9.90%</td><td align=right>  8,150</td><td align=right>  69,683</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=ASRV" class="lk01">ASRV</a> </td><td align=left nowrap>  Ameriserv Finl Inc</td><td align=right>  <NOBR>4.09</NOBR></td><td class="t18" align=right>  +<NOBR>0.34</NOBR></td><td class="t18" align=right nowrap>  +9.07%</td><td align=right>  20,408</td><td align=right>  83,469</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=HCTL" class="lk01">HCTL</a> </td><td align=left nowrap>  Healthcare Technologies Ltd</td><td align=right>  <NOBR>1.10</NOBR></td><td class="t18" align=right>  +<NOBR>0.09</NOBR></td><td class="t18" align=right nowrap>  +8.91%</td><td align=right>  12,600</td><td align=right>  13,860</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=BOMK" class="lk01">BOMK</a> </td><td align=left nowrap>  Bank Of Mckenney Va</td><td align=right>  <NOBR>9.86</NOBR></td><td class="t18" align=right>  +<NOBR>0.77</NOBR></td><td class="t18" align=right nowrap>  +8.47%</td><td align=right>  1,201</td><td align=right>  11,842</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=REVU" class="lk01">REVU</a> </td><td align=left nowrap>  Princeton Review Inc</td><td align=right>  <NOBR>5.25</NOBR></td><td class="t18" align=right>  +<NOBR>0.40</NOBR></td><td class="t18" align=right nowrap>  +8.25%</td><td align=right>  152,429</td><td align=right>  800,252</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=SYUT" class="lk01">SYUT</a> </td><td align=left nowrap>  Synutra Intl Inc</td><td align=right>  <NOBR>21.59</NOBR></td><td class="t18" align=right>  +<NOBR>1.59</NOBR></td><td class="t18" align=right nowrap>  +7.95%</td><td align=right>  14,446</td><td align=right>  311,889</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=CRNT" class="lk01">CRNT</a> </td><td align=left nowrap>  Ceragon Networks Ltd</td><td align=right>  <NOBR>14.7901</NOBR></td><td class="t18" align=right>  +<NOBR>1.0701</NOBR></td><td class="t18" align=right nowrap>  +7.80%</td><td align=right>  1,890,772</td><td align=right>  27,964,707</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=DOCX" class="lk01">DOCX</a> </td><td align=left nowrap>  Document Sciences Corporation</td><td align=right>  <NOBR>6.45</NOBR></td><td class="t18" align=right>  +<NOBR>0.45</NOBR></td><td class="t18" align=right nowrap>  +7.50%</td><td align=right>  3,840</td><td align=right>  24,768</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=FSTW" class="lk01">FSTW</a> </td><td align=left nowrap>  Firstwave Technologies Inc</td><td align=right>  <NOBR>2.09</NOBR></td><td class="t18" align=right>  +<NOBR>0.14</NOBR></td><td class="t18" align=right nowrap>  +7.18%</td><td align=right>  5,900</td><td align=right>  12,331</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=ABPI" class="lk01">ABPI</a> </td><td align=left nowrap>  Accentia Biopharmaceuticals</td><td align=right>  <NOBR>2.27</NOBR></td><td class="t18" align=right>  +<NOBR>0.15</NOBR></td><td class="t18" align=right nowrap>  +7.08%</td><td align=right>  378,125</td><td align=right>  858,344</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

<tr><td height=20 align=left nowrap><a href="/tools/quotes/quotes.asp?symb=CHKP" class="lk01">CHKP</a> </td><td align=left nowrap>  Check Point Software Technologies Ltd.</td><td align=right>  <NOBR>25.36</NOBR></td><td class="t18" align=right>  +<NOBR>1.66</NOBR></td><td class="t18" align=right nowrap>  +7.00%</td><td align=right>  8,632,709</td><td align=right>  218,925,500</td></tr><tr><td colspan="7"><div class="LineSeparator"></div></td></tr>

Many thanks in advance to those who reply and help me learn and land on a solution!

Edited by ssubirias3

Share this post


Link to post
Share on other sites



Before I forget I wanted to say thanks to chris95219 for helping me get a better understanding of arrays yesterday. Seems like the job might be easier accomplished using _IEGet instead of _INetGetSource due to all that parsing of info in the table.

I did find some example code giving the example of pulling data from amazon and writing it to the console. But I'm not able to make the connection as to what to replace in the amazon script so that the marketwatch data will work. Anyone up for helping me accomplish my goal using the _IEGet functions??

Here's the amazon code I'm talking about. Sorry don't remember the original post or poster.

#include <IE.au3>
$sURL = "http://www.amazon.com/s/ref=nb_ss_b/102-5993488-0177751?url=search-alias%3Dstripbooks&field-keywords=mystery&Go.x=0&Go.y=0&Go=Go"
$tryAttach = True
$oIE = _IECreate($sURL, $tryAttach)

$oTables = _IETableGetCollection($oIE)
For $oTable in $oTables
    If String($oTable.className) = "n2" Then
        $oTDs = _IETagnameGetCollection($oTable, "td")
        For $oTD in $oTDs
            If String($oTD.className) = "searchitem" Then
                $oSpans = _IETagNameGetCollection($oTD, "span")
                For $oSpan in $oSpans
                    Switch String($oSpan.className)
                        Case "srTitle"
                            ConsoleWrite("Title:      " & _IEPropertyGet($oSpan, "innertext") & @CR)
                        Case "listprice"
                            ConsoleWrite("ListPrice:  " & _IEPropertyGet($oSpan, "innertext") & @CR)
                        Case "saleprice"
                            ConsoleWrite("SalePrice:  " & _IEPropertyGet($oSpan, "innertext") & @CR)
                        Case "sr_price"
                            ConsoleWrite("SRPrice:    " & _IEPropertyGet($oSpan, "innertext") & @CR)
                        Case "otherprice"
                            ConsoleWrite("OtherPrice: " & _IEPropertyGet($oSpan, "innertext") & @CR)
                    EndSwitch
                Next
                ConsoleWrite("--------------------------------------------" & @CR)
            EndIf
        Next
    EndIf   
Next

I'll admit most of the problem is that I don't know much about this stuff. So any assistance making sense of this would be appreciated. Dale, Big Daddy, anyone??

Thanks in advance!

Share this post


Link to post
Share on other sites

This seems fast enough to me...

#include <IE.au3>
#include <GUIConstants.au3>
#Include <GuiListView.au3>

Opt("GuiOnEventMode", True)

$GUI = GUICreate("Table Data", 620, 370)
GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")
$ListView1 = GUICtrlCreateListView("Symbol|Company|Last|Chng.|% Chng.|Volume|$ Traded", 10, 10, 600, 350)
GUISetState(@SW_SHOW)

_EnumTable()

While 1
    Sleep(10)
WEnd

Func _EnumTable()
    _GUICtrlListViewDeleteAllItems($ListView1)

    _IEErrorHandlerRegister()
    _IEToggleImages(False)
    $sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange=15&view=0"
    $oIE = _IECreate($sURL, 1, 0, 0)

    $oTable = 0
    While Not IsObj($oTable)
        $oTable = _IETableGetCollection($oIE, 1)
        Sleep(100)
    WEnd
    _IEToggleImages(True)
    While $oTable.readyState <> "complete"
        Sleep(100)
    WEnd
    _IEAction($oIE, "stop")

    $aResults = _IETableWriteToArray($oTable)
    _IEQuit($oIE)

    For $i = 4 To UBound($aResults, 2) - 2
        If $aResults[0][$i] = "0" Then ContinueLoop
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]

        $lv_item = GUICtrlCreateListViewItem($sSymbol & "|" & $sCompany & "|" & _
                $sLast & "|" & $sChng & "|" & $sPerChng & "|" & $sVolume & "|" & $sTraded, $ListView1)
    Next
    For $i = 0 To 6
        _GUICtrlListViewSetColumnWidth($ListView1, $i, $LVSCW_AUTOSIZE)
    Next
EndFunc   ;==>_EnumTable

;===============================================================================
;
; Function Name:   _IEToggleImages
; Description::    Toggles images on/off for IE.
; Parameter(s):    $f_State - Specifies whether images are turned on (Ture) or off (False).
; Author(s):       Bob Anthony (big_daddy)
;
;===============================================================================
;
Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImages

Func _Exit()
    Exit
EndFunc   ;==>_Exit

Share this post


Link to post
Share on other sites

This seems fast enough to me...

A sincere "thank you" Big_Daddy!! I'm hoping you can lend some insight on the following points.

1 - The script executes without Au3Check errors popping up, however the console reports the following:

"--> IE.au3 Warning from function _IEAttach, $_IEStatus_NoMatch" and

"--> IE.au3 Warning from function _IETableGetCollection, $_IEStatus_NoMatch" followed by

"--> COM Error Encountered in terri.au3"

"----> $IEComErrorScriptline = 2012"

"----> $IEComErrorNumberHex = 80020009"

"----> $IEComErrorNumber = -2147352567"

"----> $IEComErrorWinDescription = Unspecified error"

"----> $IEComErrorDescription = "

"----> $IEComErrorSource = "

"----> $IEComErrorHelpFile = "

"----> $IEComErrorHelpContext = 0"

"----> $IEComErrorLastDllError = 0"

*Note: the "--> IE.au3 Warning from function _IEAttach, $_IEStatus_NoMatch" occurs on any script using $oIE = _IECreate

2 - After the console gets pounded by the above errors, there is a rapid dump of "--> IE.au3 Warning from function _IETableGetCollection, $_IEStatus_NoMatch" until the Table Data window is finally populated with the data exactly as requested! Is the rapid dump of this error normal?

3 - When I ran the script multiple times it seems it chokes resulting in the Table Data window loosing the header (Symbol, Company, etc) information. I then close the script and the console writes the following as the last entry:

">Exit code: 0 Time: 125.736"

*Note: closing and reopening SciTE did allow the script to run with the same problems menitioned in points 1 & 2 above. However the last entry in the console was:

">Exit code: 0 Time: 30.877"

It does seem the script can only be run 3-4 times in a row before point 3 can be recreated (the locking and vanishing of the Table Data header).

Its because of this slower performance that I experience that had me asking about _INet instead of the _IE.au3 functions. I'm using autoit v3.2.4.9 with SciTE v1.74 on an XP Pro 2.8GHz CPU with 512MB or Ram. I'm not expecting you to troubleshoot my computer, just providing information that might be helpful.

Again thank you VERY much for getting the script to do what I asked, my concern is if I try to loop this to collect the 12 reports, is that going to be a problem?

.

Share this post


Link to post
Share on other sites

2 - After the console gets pounded by the above errors, there is a rapid dump of "--> IE.au3 Warning from function _IETableGetCollection, $_IEStatus_NoMatch" until the Table Data window is finally populated with the data exactly as requested! Is the rapid dump of this error normal?

The way big_daddy coded his script, it loops collecting the table object until it is actually an object. Until it loads and forms the object, IE.au3 will generate the "no match" errors you are seeing. I believe the COM errors come from the object being valid, but not yet at it's ready state, also just the way big_daddy wrote it.

IE Dev ToolbarMSDN: InternetExplorer ObjectMSDN: HTML/DHTML Reference Guide[quote]It is surprising what a man can do when he has to, and how little most men will do when they don't have to. - Walter Linn[/quote]--------------------[font="Franklin Gothic Medium"]Post a reproducer with less than 100 lines of code.[/font]

Share this post


Link to post
Share on other sites

The way big_daddy coded his script, it loops collecting the table object until it is actually an object. Until it loads and forms the object, IE.au3 will generate the "no match" errors you are seeing. I believe the COM errors come from the object being valid, but not yet at it's ready state, also just the way big_daddy wrote it.

Thanks Mike! So those errors/warnings are nothing I should be concerned about. I wasn't sure if the script was running slower on my system than Big_Daddy's due to those dumps. I'll work on this more over the coming weeks. As I finish stages of this I'll ask if anyone has suggestions on streamlining the code. I'm always open to different ways of getting the job done. If nothing else gives me plenty to study in understanding this stuff.

btw, has anyone else experienced the locking or vanishing header I was talking about in point 3?

.... 3 - When I ran the script multiple times it seems it chokes resulting in the Table Data window loosing the header (Symbol, Company, etc) information. I then close the script and the console writes the following as the last entry:

">Exit code: 0 Time: 125.736"

*Note: closing and reopening SciTE did allow the script to run with the same problems menitioned in points 1 & 2 above. However the last entry in the console was:

">Exit code: 0 Time: 30.877"

It does seem the script can only be run 3-4 times in a row before point 3 can be recreated (the locking and vanishing of the Table Data header). .....

Interestingly enough I haven't been able to recreate the problem, could have been a problem with the website?? Regardless here are some visual aids on what I tried to describe in my previous post. First image is the script running successfully and the second image is the vanishing header I was able to recreate prior to this posting. Go figure!

First Image::

Posted Image

Second Image::

Posted Image

...

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

This should fit your needs better, it also eliminates the multiple errors...

#include <IE.au3>
#include <GUIConstants.au3>
#Include <GuiListView.au3>

Opt("GuiOnEventMode", True)

_IEErrorHandlerRegister()
_IEToggleImages(False)
$oIE = _IECreate("about:blank", 1, 0, 1)
_IEToggleImages(True)

$GUI = GUICreate("Table Data", 620, 370)
GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")
$ListView1 = GUICtrlCreateListView("Symbol|Company|Last|Chng.|% Chng.|Volume|$ Traded", 10, 10, 600, 350)
GUISetState(@SW_SHOW)

_EnumTable("http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange=15&view=0")
Sleep(5000)
_EnumTable("http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange=15&view=1")
Sleep(5000)
_EnumTable("http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange=15&view=6")
Sleep(5000)
_EnumTable("http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange=15&view=7")

While 1
    Sleep(10)
WEnd

Func _EnumTable($s_URL)
    _GUICtrlListViewDeleteAllItems($ListView1)

    _IENavigate($oIE, $s_URL, 0)
    _IEErrorNotify(0)
    $oTable = 0
    While Not IsObj($oTable)
        $oTable = _IETableGetCollection($oIE, 1)
        Sleep(100)
    WEnd
    _IEErrorNotify(1)
    While $oTable.readyState <> "complete"
        Sleep(100)
    WEnd
    _IEAction($oIE, "stop")

    $aResults = _IETableWriteToArray($oTable)

    For $i = 4 To UBound($aResults, 2) - 2
        If $aResults[0][$i] = "0" Then ContinueLoop
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]

        $lv_item = GUICtrlCreateListViewItem($sSymbol & "|" & $sCompany & "|" & _
                $sLast & "|" & $sChng & "|" & $sPerChng & "|" & $sVolume & "|" & $sTraded, $ListView1)
    Next
    For $i = 0 To 6
        _GUICtrlListViewSetColumnWidth($ListView1, $i, $LVSCW_AUTOSIZE)
    Next
    _IENavigate($oIE, "about:blank")
EndFunc   ;==>_EnumTable

;===============================================================================
;
; Function Name:   _IEToggleImages
; Description::    Toggles images on/off for IE.
; Parameter(s):    $f_State - Specifies whether images are turned on (Ture) or off (False).
; Author(s):       Bob Anthony (big_daddy)
;
;===============================================================================
;
Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImages

Func _Exit()
    _IEQuit($oIE)
    Exit
EndFunc   ;==>_Exit
Edited by big_daddy

Share this post


Link to post
Share on other sites

This should fit your needs better, it also eliminates the multiple errors...

Big_Daddy, you're right on the money as usual! Still getting the first warning "--> IE.au3 Warning from function _IEAttach, $_IEStatus_NoMatch", but I ain't complaining by any means.

Since I'll be using this to run 12 to 20 different reports, I was thinking I could use a loop that would set the following:

$url = "http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange="
$site = "15&view=0"

_EnumTable($url & $site)
Sleep(5000)oÝ÷ ØZ+­z÷«­æ¬¢t1¡ë'ßÛpj¹è­ê뢺'¶¬×½êÈßÙiÉ-¢Ø^½©â²§uêÌʲȬiû§rب«jf­z·z-ºÇ½ªâi¹^!öºw-åGéi¹^q©î±æ¯j¸nWj{(è^ºw^®ËZÜ!jÒ7ö÷¢w°®à#f¶§)à¡û¬x,¢g­)à)jëh×6Dim $url = "http://www.marketwatch.com/tools/marketsummary/screener.asp?exchange="

While $b <= 11 
    If $b = 0 Then Dim $site = "13&view=0"
    If $b = 1 Then Dim $site = "13&view=1"
    If $b = 2 Then Dim $site = "13&view=6"
    If $b = 3 Then Dim $site = "13&view=7"
    If $b = 4 Then Dim $site = "14&view=0"
    If $b = 5 Then Dim $site = "14&view=1"
    If $b = 4 Then Dim $site = "14&view=6"
    If $b = 5 Then Dim $site = "14&view=7"
    ;;;;  this would continue until all the reports I wanted have been pulled
    If $b = 11 Then Dim $site = "15&view=7"
    
   _EnumTable($url & $site)
   Sleep(5000)
    $b = $b + 1
WEnd

While 1
    Sleep(10)
WEnd

Func _EnumTable($s_URL)
EndFunc

Or am I going about this completely wrong? All assistance and insight is greatly appreciated.

- ss3

...

Share this post


Link to post
Share on other sites

#9 ·  Posted (edited)

This was the best way I could come up with...

#include <IE.au3>
#include <GUIConstants.au3>
#Include <GuiListView.au3>

Opt("GuiOnEventMode", True)

$sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?"
; Register IE.au3 default error handler
_IEErrorHandlerRegister()
; Toggle IE images off to improve load times
_IEToggleImages(False)
; Create an invisible browser object
$oIE = _IECreate("about:blank", 0, 0, 1)
; Toggle images back on for browser windows created from here on
_IEToggleImages(True)

$GUI = GUICreate("Table Data", 620, 370)
GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")
$ListView1 = GUICtrlCreateListView("Symbol|Company|Last|Chng.|% Chng.|Volume|$ Traded", 10, 10, 600, 350)
GUISetState(@SW_SHOW)

$i = 1
$iExch = 13
; While $iExch is less than 16 continue
While $iExch < 16
    ; Check the value of $i and set $iView accordingly
    Switch $i
        Case 1
            $iView = 0
        Case 2
            $iView = 1
        Case 3
            $iView = 6
        Case 4
            $iView = 7
    EndSwitch
    ; Write the full URL to the console just to verify
    ConsoleWrite($sURL & "exchange=" & $iExch & "&view=" & $iView & @CR)
    ; Call _EnumTable() with our generated URL
    _EnumTable($sURL & "exchange=" & $iExch & "&view=" & $iView)
    ; Start over every fourth loop, adding 1 to $iExch, and reseting $i back to 1
    ; Otherwise add 1 to $i
    If Mod($i, 4) == 0 Then
        $iExch += 1
        $i = 1
    Else
        $i += 1
    EndIf
    ; Give time to view our table data in the GUI
    Sleep(5000)
WEnd

While 1
    ; Hang around and wait for the GUI to be closed
    Sleep(10)
WEnd

Func _EnumTable($s_URL)
    Local $oTable, $aResults, $lv_item
    
    ; Remove any previous data from the listview
    _GUICtrlListViewDeleteAllItems($ListView1)
    ; Navigate to the URL without waiting for the page to load
    _IENavigate($oIE, $s_URL, 0)
    ; Turn error notification off so it doesn't fill the console with useless information
    _IEErrorNotify(0)
    ; Wait until $oTable is an object variable
    While Not IsObj($oTable)
        ; Try to get a reference to the second table on the page
        $oTable = _IETableGetCollection($oIE, 1)
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    ; Turn error notification back on now that $oTable is actually an object
    _IEErrorNotify(1)
    ; Wait for our table to finish loading, otherwise we would get partial data
    While $oTable.readyState <> "complete"
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    ; Stop the page from loading now that we have all the data we need
    _IEAction($oIE, "stop")
    ; Write the table to an array
    $aResults = _IETableWriteToArray($oTable)
    ; If you examine the source our data starts in the fifth <tr> or row
    ; We start $i at 4 because IE objects are on a 0-base index
    ; We stop at the number of total rows minus 2 to trim "Add symbols to My Portfolio" at the bottom
    For $i = 4 To UBound($aResults, 2) - 2
        ; There are seperator lines between each row of data we need, this bypasses those
        If $aResults[0][$i] = "0" Then ContinueLoop
        ; Store each column of data in a variable
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]
        ; Create our listview item
        $lv_item = GUICtrlCreateListViewItem($sSymbol & "|" & $sCompany & "|" & _
                $sLast & "|" & $sChng & "|" & $sPerChng & "|" & $sVolume & "|" & $sTraded, $ListView1)
    Next
    ; Just cosmetics
    For $i = 0 To 6
        _GUICtrlListViewSetColumnWidth($ListView1, $i, $LVSCW_AUTOSIZE)
    Next
    ; This is a very important piece of code
    ; Without this we will attach to the previous table giving us the wrong data
    _IENavigate($oIE, "about:blank")
EndFunc   ;==>_EnumTable

;===============================================================================
;
; Function Name:   _IEToggleImages
; Description::    Toggles images on/off for IE.
; Parameter(s):    $f_State - Specifies whether images are turned on (Ture) or off (False).
; Author(s):       Bob Anthony (big_daddy)
;
;===============================================================================
;
Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImages

Func _Exit()
    _IEQuit($oIE)
    Exit
EndFunc   ;==>_Exit
Edited by big_daddy

Share this post


Link to post
Share on other sites

This was the best way I could come up with...

Whoa! Slick, very slick! No errors and completely perfect!

Thank you, thank you, and of course a special thankyouverymuch big_daddy! You weren't kidding when you said this was "the best way". I think I understand what the code is doing line by line, however there are still some questions I have. Hopefully in the name of education, others won't mind helping connect the dots for me. After all its better to teach a man to fish and feed him for a life time, right?

1 - previously I asked if it was possible to use a variable as a function's parameter as in _EnumTable($url & $site). In big_daddy's latest code he successfully uses _EnumTable($sURL & "exchange=" & $iExch & "&view=" & $iView). I'm not sure what I was missing since both appear to have the same output when written to a msgbox or the console.

2 - why is it better to use Switch...Case...EndSwitch instead of If...Then...EndIf since they both are used to "Conditionally run statements." per the help file. Is it a matter of personal preference or is there a reason that makes Switch...Case...EndSwitch a better coding practice in this case or in general?

3 - and for the million dollar question. How does Func _EnumTable($s_URL) know what information to assign to $aResults. I know that ($s_URL) is the destinatination website (marketwatch.com...) but there are several tables on that page. Excel sees the data I'm interested in as table3, but I don't see where or how Func_EnumTable($s_URL) knows to pull the right table. Naturally the reason for my questions is I'd like use this script and apply it to other websites that will have different formating and table locations.

Again thank you so much big_daddy, you're truly the man!

--ss3

...

Share this post


Link to post
Share on other sites

Whoa! Slick, very slick! No errors and completely perfect!

Your welcome!

1 - previously I asked if it was possible to use a variable as a function's parameter as in _EnumTable($url & $site). In big_daddy's latest code he successfully uses _EnumTable($sURL & "exchange=" & $iExch & "&view=" & $iView). I'm not sure what I was missing since both appear to have the same output when written to a msgbox or the console.

Your logic looks correct, so I'm guessing it was an error somewhere else in the code.

2 - why is it better to use Switch...Case...EndSwitch instead of If...Then...EndIf since they both are used to "Conditionally run statements." per the help file. Is it a matter of personal preference or is there a reason that makes Switch...Case...EndSwitch a better coding practice in this case or in general?

In this particular situation it's just easier to read, but there are times when a switch statement can greatly reduce the size of your code.

3 - and for the million dollar question. How does Func _EnumTable($s_URL) know what information to assign to $aResults. I know that ($s_URL) is the destinatination website (marketwatch.com...) but there are several tables on that page. Excel sees the data I'm interested in as table3, but I don't see where or how Func_EnumTable($s_URL) knows to pull the right table. Naturally the reason for my questions is I'd like use this script and apply it to other websites that will have different formating and table locations.

Hopefully I've answered this question with the comments I added to my previous post. If they do not let me know and I'll try to explain it better.

Share this post


Link to post
Share on other sites

Your welcome! ....

Hopefully I've answered this question with the comments I added to my previous post. If they do not let me know and I'll try to explain it better.

YES!!! And thank you, the comments in the code answer the million dollar question, allowing me to tinker with the code and get partial results from other websites. Despite the mixed results I'm confident I'll get what I need after spending more time studying the code next to the help file. I do have more questions (big shock, right?) perhaps someone can help solidify my new understanding.

1 - I'm not sure why images are turned back on so early in the script. If I'm reading this correctly, the images are turned off then a hidden browswer object loading a blank page (about:blank) is kicked off then the images are turned back on. A blank page has no images to load and therefore I'm confused by the benefit of speeding up the load time of a blank page.

$sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?"
; Register IE.au3 default error handler
_IEErrorHandlerRegister()
; Toggle IE images off to improve load times
_IEToggleImages(False)
; Create an invisible browser object
$oIE = _IECreate("about:blank", 0, 0, 1)
; Toggle images back on for browser windows created from here on
_IEToggleImages(True)oÝ÷ Ûf{^ÅÇ«¦Û),Ú«²Ø¨Éh'^­«h®Æ§vØ^_ºw-Ö(¬

And that covers this round of questions. I do appreciate the hand holding on this. Unfortunately I like to understand the how and why something works as crazy as that is.

Thanks again for all the help. --ss3

...

Share this post


Link to post
Share on other sites

1 - I'm not sure why images are turned back on so early in the script. If I'm reading this correctly, the images are turned off then a hidden browswer object loading a blank page (about:blank) is kicked off then the images are turned back on. A blank page has no images to load and therefore I'm confused by the benefit of speeding up the load time of a blank page.

Images can be turned on or off per browser window. Since images are off when the browser object is created they will remain off for that window until it is closed. We turn them back on in case the user opens a browser window, otherwise they would not have images either.

2a - Next excerpt sparks 2 questions; logical operators and the mod function. Why is == used in some cases and how is it different that =. Also what is += any why is it used in this situation. Prior t this script I don't think I've come across += in the help file or other topics I've read. Then again maybe its always been there but never noticed it.

The help file can explain the operators better than I can, search for operators and it's the first result. The += operator makes $i += 1 the same as $i = $i + 1.

2b - When I searched the help file to read about Mod its example didn't make much sense to me. Maybe I'm alone on this, but there are other commands like Ubound where the example code merely confuses the situation for a person who doesn't have a background in programming. But I digress, about Mod in this code. My understanding is this. $i = 1 before the switch section and when the script hits the If Mod(1,4) == 0 then it'll add 1 to the current $iExch value and set $i =1 otherwise it will leave $iExch's value unchaged and only increase $i by 1 feeding triggering the correct switch condition. I guess what I'm not understanding is how does the 4 in Mod(1,4) == 0 play into this and since there's not a $i = 0 anywhere in the script when would == 0 be true. Again my apologies in advance for not getting this elementary concept.

The Mod function performs a modulus operation, which is division, and returns the remainder. Initially I was going about it a different way and this would have been the most logical solution. However I changed the surrounding code but didn't bother changing this as it still achieved the goal. It could just as easily be written If $i = 4 Then...

3 - If I wanted to the exchange name to column 7 and date to column 8 of the array, what command would I use? _ArrayInsert( $aResults,[7][$i],"Nasdaq") reports a syntax error and the help file doesn't give any hints on inserting into a 2d array. You'll see that I tried to add the $sExch and $sDate to $aResults with no avail. Ultimately I would like the $iExch value to set the $sExch. For example when $iExch = 13 then $sExch = "NYSE".

I'll have to come back to this part when I'm not so tired. I don't seem to be able to comprehend what exactly it is your want right at the moment.

Share this post


Link to post
Share on other sites

..... The += operator makes $i += 1 the same as $i = $i + 1. ....

The Mod function performs a modulus operation, which is division, and returns the remainder. ..... It could just as easily be written If $i = 4 Then...

I'll have to come back to this part when I'm not so tired. I don't seem to be able to comprehend what exactly it is your want right at the moment.

Thanks that's exactly what I was looking for that the help file couldn't have given me on the $i += 1 and the Mod questions!

About my third question not making sense, I thought I'd give you a visual of what I'm talking about. Picture's worth a thousand words, right?

Posted Image

The data outlined in green is what is pulled by the script from the website table. The data outlined in red is what I'd like the script to populate into the array before moving onto the next switch. What I was trying to describe was this. Since the marketwatch urls differ only in $iExch and $iView I wanted a new variable ($sExch) to be set based on the value of $iExch. For example when

$iExch = 13 Then $sExch = "NYSE"
$iExch = 14 Then $sExch = "Amex"
$iExch = 15 Then $sExch = "Nasdaq"oÝ÷ Ù.ßÙe^¶­&­µé©¶Êi×b
7êÁ1r§wMú°6­zÚ-í7é¤^²ém±ªëk(§²'^¶¢³^ÆÙh¢ÚWjëh×6    For $i = 4 To UBound($aResults, 2) - 2
        ; There are seperator lines between each row of data we need, this bypasses those
        If $aResults[0][$i] = "0" Then ContinueLoop
        ; Store each column of data in a variable
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]
        $sExch = _ArrayInsert( $aResults,[7][$i],"Nasdaq")
        $sDate = _ArrayInsert( $aResults,[8][$i],@MON &"/"&@MDAY&"/"&@YEAR)
        ; Create our listview item
        $lv_item = GUICtrlCreateListViewItem($sSymbol & "|" & $sCompany & "|" & _
                $sLast & "|" & $sChng & "|" & $sPerChng & "|" & $sVolume & "|" & $sTraded, $ListView1)
    Next

Hopefully this makes more sense than my last post. Enjoy the rest of your weekend!

--ss3

...

Share this post


Link to post
Share on other sites

The picture does clear it up nicely. There's no need to actually insert the values into the array.

#include <IE.au3>
#include <Date.au3>
#include <GUIConstants.au3>
#Include <GuiListView.au3>

Opt("GuiOnEventMode", True)

$sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?"
; Register IE.au3 default error handler
_IEErrorHandlerRegister()
; Toggle IE images off to improve load times
_IEToggleImages(False)
; Create an invisible browser object
$oIE = _IECreate("about:blank", 0, 0, 1)
; Toggle images back on for browser windows created from here on
_IEToggleImages(True)

$GUI = GUICreate("Table Data", 620, 370)
GUISetOnEvent($GUI_EVENT_CLOSE, "_Exit")
$ListView1 = GUICtrlCreateListView("Symbol|Company|Last|Chng.|% Chng.|Volume|$ Traded|Exchange|Date", 10, 10, 600, 350)
GUISetState(@SW_SHOW)

$i = 1
$iExch = 13
; While $iExch is less than 16 continue
While $iExch < 16
    ; Check the value of $i and set $iView accordingly
    Switch $i
        Case 1
            $iView = 0
        Case 2
            $iView = 1
        Case 3
            $iView = 6
        Case 4
            $iView = 7
    EndSwitch
    ; Write the full URL to the console just to verify
    ConsoleWrite($sURL & "exchange=" & $iExch & "&view=" & $iView & @CR)
    ; Call _EnumTable() with our generated URL
    _EnumTable($sURL & "exchange=" & $iExch & "&view=" & $iView)
    ; Start over every fourth loop, adding 1 to $iExch, and reseting $i back to 1
    ; Otherwise add 1 to $i
    If $i = 4 Then
        $iExch += 1
        $i = 1
    Else
        $i += 1
    EndIf
    ; Give time to view our table data in the GUI
    Sleep(5000)
WEnd

While 1
    ; Hang around and wait for the GUI to be closed
    Sleep(10)
WEnd

Func _EnumTable($s_URL)
    Local $oTable, $aResults, $lv_item

    ; Remove any previous data from the listview
    _GUICtrlListViewDeleteAllItems($ListView1)
    ; Navigate to the URL without waiting for the page to load
    _IENavigate($oIE, $s_URL, 0)
    ; Turn error notification off so it doesn't fill the console with useless information
    _IEErrorNotify(0)
    ; Wait until $oTable is an object variable
    While Not IsObj($oTable)
        ; Try to get a reference to the second table on the page
        $oTable = _IETableGetCollection($oIE, 1)
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    ; Turn error notification back on now that $oTable is actually an object
    _IEErrorNotify(1)
    ; Wait for our table to finish loading, otherwise we would get partial data
    While $oTable.readyState <> "complete"
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    ; Stop the page from loading now that we have all the data we need
    _IEAction($oIE, "stop")
    ; Write the table to an array
    $aResults = _IETableWriteToArray($oTable)
    ; If you examine the source our data starts in the fifth <tr> or row
    ; We start $i at 4 because IE objects are on a 0-base index
    ; We stop at the number of total rows minus 2 to trim "Add symbols to My Portfolio" at the bottom
    For $i = 4 To UBound($aResults, 2) - 2
        ; There are seperator lines between each row of data we need, this bypasses those
        If $aResults[0][$i] = "0" Then ContinueLoop
        ; Store each column of data in a variable
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]

        Switch $iExch
            Case 13
                $sExch = "NYSE"
            Case 14
                $sExch = "Amex"
            Case 15
                $sExch = "Nasdaq"
        EndSwitch

        $sDate = _NowDate()

        ; Create our listview item
        $lv_item = GUICtrlCreateListViewItem($sSymbol & "|" & $sCompany & "|" & _
                $sLast & "|" & $sChng & "|" & $sPerChng & "|" & $sVolume & "|" & $sTraded & "|" & $sExch & "|" & $sDate, $ListView1)
    Next
    ; Just cosmetics
    For $i = 0 To 8
        _GUICtrlListViewSetColumnWidth($ListView1, $i, $LVSCW_AUTOSIZE)
    Next
    ; This is a very important piece of code
    ; Without this we will attach to the previous table giving us the wrong data
    _IENavigate($oIE, "about:blank")
EndFunc   ;==>_EnumTable

;===============================================================================
;
; Function Name:   _IEToggleImages
; Description::    Toggles images on/off for IE.
; Parameter(s):    $f_State - Specifies whether images are turned on (Ture) or off (False).
; Author(s):       Bob Anthony (big_daddy)
;
;===============================================================================
;
Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImages

Func _Exit()
    _IEQuit($oIE)
    Exit
EndFunc   ;==>_Exit

Share this post


Link to post
Share on other sites

The picture does clear it up nicely. There's no need to actually insert the values into the array.

Thank you big_daddy! I can tell I'm going to really like these arrays and switches. Thanks for the schooling!

...

Share this post


Link to post
Share on other sites

What I'm trying to do is create an array that would contain 8 col and 26 rows that could be dumped into a txt file or whatever and the output look like the below output. Each report contains 25 stocks +1 for the header and my thinking is I need one array for each report, hence the $array1_i - iv.

I'm totally thankful for the time and energy big_daddy put into this script and helping me understand the how and why behind the magic. Over the past few days I've been digging into the code trying to see how i can move away from the GUICreate("Table Data", 620, 370) box. Nothing wrong with the gui, especially considering I have some new found knowledge and application around this command. However originally I was looking for a way to capture the data into an array that could then dump into a txt file or use _ArrayToClip($aResults) with.

After discovering that _ArrayToClip($aResults) didn't capture what I expected, I added _ArrayDisplay($aResults, "Resutls Array Contents") to the code. Then I understood why _ArrayToClip($aResults) wasn't giving me what I was looking for. So back to the drawing board and asking questions in the forum. My original though was to change:

If $aResults[0][$i] = "0" Then ContinueLoop       
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]oÝ÷ ÚÚºÚ"µÍY ÌÍØTÝ[ÖÉÌÍÚWVÌHH  ][ÝÌ  ][ÝÈ[ÛÛ[YSÛÜ  ÌÍÜÔÞ[XÛH ÌÍØTÝ[ÖÉÌÍÚWVÌB   ÌÍÜÐÛÛ[HH ÌÍØTÝ[ÖÉÌÍÚWVÌWB  ÌÍÜÓÝH ÌÍØTÝ[ÖÉÌÍÚWVÌB   ÌÍÜÐÚÈH   ÌÍØTÝ[ÖÉÌÍÚWVÌ×B ÌÍÜÔÚÈH   ÌÍØTÝ[ÖÉÌÍÚWVÍB   ÌÍÜÕÛ[YHH  ÌÍØTÝ[ÖÉÌÍÚWVÍWB  ÌÍÜÕYYH ÌÍØTÝ[ÖÉÌÍÚWVÍ

But that didn't have the result I was after. So then I realized that its $aResults = _IETableWriteToArray($oTable) that is creating the array. I'm thinking I need to write the data into a second array instead of the gui, but I'm not having much luck. Does anyone have any suggestion? What I'd like at the end of this is to have the array in the format shown below so I can use _ArrayToClip($aResults) to capture the data into variables that will ultimately dump into either excel, txt, or any other application I would want to use down the road.

Posted Image

Thanks again for any assistance.

--ss3

...

Share this post


Link to post
Share on other sites

Modified to remove all GUI action. Saves data entirely to a file:

#include <IE.au3>
#include <File.au3>

; File to save data in:
$sFile = @ScriptDir & "\MarketWatchData.txt"

; Register IE.au3 default error handler
_IEErrorHandlerRegister()

; Toggle IE images off to improve load times
_IEToggleImages(False)

; Create an invisible browser object
$oIE = _IECreate("about:blank", 0, 0, 1)

; Toggle images back on for browser windows created from here on
_IEToggleImages(True)

; Array of exchanges ([13] thru [15])
Global $avExch[16] = ["", "", "", "", "", "", "", "", "", "", "", "", "", "NYSE", "Amex", "Nasdaq"]

; Array of view selections ([0] thru [3])
Global $avViews[4] = [0, 1, 6, 7]

; While $iExch is less than 16 continue
For $iExch = 13 To 15
    For $iView = 0 To 3
        ; Call _EnumTable() with our generated URL
        _EnumTable($iExch, $iView)
    Next
Next

; Done, exit
_IEQuit($oIE)
Exit


;===============================================================================
; Function Name:   _EnumTable($E, $V)
; Description::    Enumerates MarketWatch data from web site
; Parameter(s):    $E - Specifies one of the exchanges from the $avExch array
;                   $V - Specifies one of the view indexes from the $avViews array
; Author(s):       Bob Anthony (big_daddy)
;                   Modified by PsaltyDS to remove GUI functions
;===============================================================================
Func _EnumTable($E, $V)
    Local $oTable, $aResults
    Local $sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?"
    Local $sCurrURL = $sURL & "exchange=" & $E & "&view=" & $avViews[$V]

    ; Keep user apprised on progress
    ConsoleWrite("Debug: $sCurrURL = " & $sCurrURL & @LF)
    TrayTip("MarketWatchData", "Getting data from: " & $avExch[$E] & @LF & "View " & $V + 1 & " of 4", 10)
    
    ; Navigate to the URL without waiting for the page to load
    _IENavigate($oIE, $sCurrURL, 0)
    
    ; Turn error notification off so it doesn't fill the console with useless information
    _IEErrorNotify(0)
    
    ; Wait until $oTable is an object variable
    While Not IsObj($oTable)
        ; Try to get a reference to the second table on the page
        $oTable = _IETableGetCollection($oIE, 1)
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    
    ; Turn error notification back on now that $oTable is actually an object
    _IEErrorNotify(1)
    
    ; Wait for our table to finish loading, otherwise we would get partial data
    While $oTable.readyState <> "complete"
        ; Keep the CPU usage down
        Sleep(100)
    WEnd
    
    ; Stop the page from loading now that we have all the data we need
    _IEAction($oIE, "stop")
    
    ; Write the table to an array
    $aResults = _IETableWriteToArray($oTable)
    
    ; If you examine the source our data starts in the fifth <tr> or row
    ; We start $i at 4 because IE objects are on a 0-base index
    ; We stop at the number of total rows minus 2 to trim "Add symbols to My Portfolio" at the bottom
    For $i = 4 To UBound($aResults, 2) - 2
        ; There are seperator lines between each row of data we need, this bypasses those
        If $aResults[0][$i] = "0" Then ContinueLoop
        
        ; Create our file entry
        $sLine = $aResults[0][$i] & "|" & $aResults[1][$i] & "|" & $aResults[2][$i] & "|" & $aResults[3][$i] & "|" & _
                $aResults[4][$i] & "|" & $aResults[5][$i] & "|" & $aResults[6][$i] & "|" & $avExch[$E]
        _FileWriteLog($sFile, $sLine)
    Next
    
    ; This is a very important piece of code
    ; Without this we will attach to the previous table giving us the wrong data
    _IENavigate($oIE, "about:blank")
EndFunc   ;==>_EnumTable

;===============================================================================
; Function Name:   _IEToggleImages
; Description::    Toggles images on/off for IE.
; Parameter(s):    $f_State - Specifies whether images are turned on (Ture) or off (False).
; Author(s):       Bob Anthony (big_daddy)
;===============================================================================
Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImages

Cheers!

:)


Valuater's AutoIt 1-2-3, Class... Is now in Session!For those who want somebody to write the script for them: RentACoder"Any technology distinguishable from magic is insufficiently advanced." -- Geek's corollary to Clarke's law

Share this post


Link to post
Share on other sites

Modified to remove all GUI action. Saves data entirely to a file:

Cheers!

:)

Thank you PsaltyDS! Its reallly great to see the different approaches people take to coding. I really liked the tooltip; very cool!! And _FileWriteLog() command will come in handy!

I've been playing with the code to write $aResults into a second array before writing to data to the log file. Purpose of the $avNew is to place each stock and its information in a row verses column. With the code changes found below, I've got the general idea working. I need some help getting $avNew to populate correctly.

Full Code

#include <IE.au3>
#include <File.au3>
#include <Array.au3>
#include <Date.au3>

Dim $aArray, $avNew[30][10]

; File to save data in:
$sFile = @ScriptDir & "\MarketWatchData.txt"

_IEErrorHandlerRegister()
_IEToggleImages(False)
$oIE = _IECreate("about:blank", 0, 0, 1)
_IEToggleImages(True)

; Array of exchanges ([13] thru [15])
Global $avExch[16] = ["", "", "", "", "", "", "", "", "", "", "", "", "", "NYSE", "Amex", "Nasdaq"]

; Array of view selections ([0] thru [3])
Global $avViews[4] = [0, 1, 6, 7]

For $iExch = 13 To 15
    For $iView = 0 To 3
        ; Call _EnumTable() with our generated URL
        _EnumTable($iExch, $iView)
    Next
Next

_IEQuit($oIE)
Exit


;===============================================================================
; Function Name:  _EnumTable($E, $V)
; Description::     Enumerates MarketWatch data from web site
; Parameter(s):   $E - Specifies one of the exchanges from the $avExch array
;                        $V - Specifies one of the view indexes from the $avViews array
; Author(s):        Bob Anthony (big_daddy)
;                        Modified by PsaltyDS to remove GUI functions
;===============================================================================
Func _EnumTable($E, $V)
    Local $oTable, $aResults
    Local $sURL = "http://www.marketwatch.com/tools/marketsummary/screener.asp?"
    Local $sCurrURL = $sURL & "exchange=" & $E & "&view=" & $avViews[$V]

    ConsoleWrite("Debug: $sCurrURL = " & $sCurrURL & @LF)
    TrayTip("MarketWatchData", "Getting data from: " & $avExch[$E] & @LF & "View " & $V + 1 & " of 4", 10)
    
    _IENavigate($oIE, $sCurrURL, 0)
    _IEErrorNotify(0)
    While Not IsObj($oTable)
        $oTable = _IETableGetCollection($oIE, 1)
        Sleep(100)
    WEnd
    
    _IEErrorNotify(1)
    
    While $oTable.readyState <> "complete"
        Sleep(100)
    WEnd
    
    _IEAction($oIE, "stop")
    
    ; Write the table to an array
    $aResults = _IETableWriteToArray($oTable)
    _ArrayDisplay($aResults, "_ArrayDisplay() Test 1")    ;; <== added to see the contents of the $aResults array

        ;;;;;;;;;;;;;;;;;;;;
        ;;;;
        ;;;;  Following code added back in from big_daddy's code
        ;;;;
        ;;;;;;;;;;;;;;;;;;;;

    For $i = 4 To UBound($aResults, 2) - 2
        If $aResults[0][$i] = "0" Then ContinueLoop
        ; Store each column of data in a variable
        $sSymbol = $aResults[0][$i]
        $sCompany = $aResults[1][$i]
        $sLast = $aResults[2][$i]
        $sChng = $aResults[3][$i]
        $sPerChng = $aResults[4][$i]
        $sVolume = $aResults[5][$i]
        $sTraded = $aResults[6][$i]

        Switch $iExch
            Case 13
                $sExch = "NYSE"
            Case 14
                $sExch = "Amex"
            Case 15
                $sExch = "Nasdaq"
        EndSwitch

        $sDate = _NowDate()

        ;;;;;;;;;;;;;;;;;;;;
        ;;;;
        ;;;;  Following code added in by ss3 attempting to create 2nd array in  
        ;;;;  the format of big_daddy's Table Data gui
        ;;;;
        ;;;;;;;;;;;;;;;;;;;;

            $avNew[$i][0] = $sSymbol
            $avNew[$i][1] = $sCompany
            $avNew[$i][2] = $sLast
            $avNew[$i][3] = $sChng
            $avNew[$i][4] = $sPerChng
            $avNew[$i][5] = $sVolume
            $avNew[$i][6] = $sTraded
            $avNew[$i][7] = $sExch
            $avNew[$i][8] = $sDate
            _ArrayDisplay($avNew, "_ArrayDisplay() Test")   ;; <== added to see the contents of the $avNew array

        ;;;;;;;;;;;;;;;;;;;;
        
                      ; Create our file entry
                      $sLine = $aResults[0][$i] & "|" & $aResults[1][$i] & "|" & $aResults[2][$i] & "|" & $aResults[3][$i] & "|" & _
                      $aResults[4][$i] & "|" & $aResults[5][$i] & "|" & $aResults[6][$i] & "|" & $avExch[$E]
                      _FileWriteLog($sFile, $sLine)

    Next
    
    _IENavigate($oIE, "about:blank")
EndFunc   ;==>_EnumTable

Func _IEToggleImages($f_State = True)
    If $f_State Then
        $s_Status = "yes"
    Else
        $s_Status = "no"
    EndIf
    RegWrite("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main", _
            "Display Inline Images", "REG_SZ", $s_Status)
EndFunc   ;==>_IEToggleImagesoÝ÷ ØéÜ{Mújó^Â+)¢¥j×rëyËeÈ(ºWeG­ë®*m¶azwn7é«Í{h¶*[¡ªÝ²aj×r®G­­êh®×Ø^·²È çm¢Ø^rëyËoj¸nWyú+{
â¶)ඥ¢âå,xZuÜ·öÄ°×!wRÂ+Êyú+Ó~¢V'°jwtߨÅÈHßÙ­)äx-ç(uëh±ëtߦ¯5ì7öÇ(ק¶Ëhjö«¦åzh Ùh¢IbëaÆÄÅÇ«¦×ë¢`¢÷¿m«­¢+Øìììììììììììììììììììì(ìììì(ìììì½±±½Ý¥¹½¥¸äÍÌÌÑѵÁÑ¥¹Ñ¼ÉÑɹÉÉ䥸(ììììÑ¡½ÉµÐ½¥}äÌäíÌQ±ÑÕ¤(ìììì(ìììììììììììììììììììì((ÀÌØíÙ9ÝlÀÌØí¥ulÁtôÀÌØíÍMåµ½°(ÀÌØíÙ9ÝlÀÌØí¥ulÅtôÀÌØíÍ
½µÁ¹ä(ÀÌØíÙ9ÝlÀÌØí¥ulÉtôÀÌØíÍ1ÍÐ(ÀÌØíÙ9ÝlÀÌØí¥ulÍtôÀÌØíÍ
¡¹(ÀÌØíÙ9ÝlÀÌØí¥ulÑtôÀÌØíÍAÉ
¡¹(ÀÌØíÙ9ÝlÀÌØí¥ulÕtôÀÌØíÍY½±Õµ(ÀÌØíÙ9ÝlÀÌØí¥ulÙtôÀÌØíÍQÉ(ÀÌØíÙ9ÝlÀÌØí¥ulÝtôÀÌØíÍá (ÀÌØíÙ9ÝlÀÌØí¥ulátôÀÌØíÍÑ(}ÉÉå¥ÍÁ±ä ÀÌØíÙ9Ü°ÅÕ½Ðí}ÉÉå¥ÍÁ±ä ¤QÍÐÅÕ½Ðì¤ìì±ÐìôôѼÍÑ¡½¹Ñ¹Ñ̽ѡÀÌØíÙ9ÜÉÉä(ÀÌØíÉÉå}±¥Àô}ÉÉåQ½
±¥À ÀÌØíÙ9ܤ(%ÀÌØí¥á ôÄÌ9ÀÌØí¥Y¥ÜôÀQ¡¸ÀÌØí¹åÍ|}ÕÀôÀÌØíÉÉå}±¥À(%ÀÌØí¥á ôÄÌ9ÀÌØí¥Y¥ÜôÄQ¡¸ÀÌØí¹åÍ|}¸ôÀÌØíÉÉå}±¥À(%ÀÌØí¥á ôÄÐ9ÀÌØí¥Y¥ÜôÀQ¡¸ÀÌØíµá|}ÕÀôÀÌØíÉÉå}±¥À(%ÀÌØí¥á ôÄÐ9ÀÌØí¥Y¥ÜôÄQ¡¸ÀÌØíµá|}¸ôÀÌØíÉÉå}±¥À(%ÀÌØí¥á ôÄÔ9ÀÌØí¥Y¥ÜôÀQ¡¸ÀÌØí¹Í|}ÕÀôÀÌØíÉÉå}±¥À(%ÀÌØí¥á ôÄÔ9ÀÌØí¥Y¥ÜôÄQ¡¸ÀÌØí¹Í|}ÕÀôÀÌØíÉÉå}±¥À((ìììììììììììììììììììì((ì
ÉѽÕÈ¥±¹ÑÉä(ÀÌØíÍ1¥¹ôÀÌØíIÍÕ±ÑÍlÁulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíIÍÕ±ÑÍlÅulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíIÍÕ±ÑÍlÉulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíIÍÕ±ÑÍlÍulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀì|(ÀÌØíIÍÕ±ÑÍlÑulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíIÍÕ±ÑÍlÕulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíIÍÕ±ÑÍlÙulÀÌØí¥tµÀìÅÕ½ÐíðÅÕ½ÐìµÀìÀÌØíÙá¡lÀÌØít(}¥±]É¥Ñ1½ ÀÌØíÍ¥±°ÀÌØíÍ1¥¹¤

I hope I've explained this sufficiently. Let me know if I need to break out the visual aids :P.

Many thanks!

--ss3

...

Share this post


Link to post
Share on other sites

*** Bump ***

Still looking for some help, please.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0