KimberlyJillPereira

Extract table from website until the end

16 posts in this topic

#1 ·  Posted

I could only extract the first 20 from table into Microsoft Excel by using Array Extract but I want to extract until the end what I mean is until the second page. How to do it? Please revert. Thanks.:):)

image1.jpg

image2.jpg

image3.png

Share this post


Link to post
Share on other sites



#2 ·  Posted

since there is total of 33 I want to extract from row 0 to row 32. How to do it? please help. Thanks.

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

@water   @MaoMao  please help

Edited by KimberlyJillPereira

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

#include <Array.au3>
#include <IE.au3>
#include <String.au3>

Local $oIE
Local $oDivs
Local $aPageCount, $iPageCount
Local $aTable, $oTables
Local $aResults[0][7]

$oIE = _IECreate("http://www.timeview2.net/alarms", 1)
$oDivs = _IETagNameGetCollection($oIE, "div")
For $oDiv In $oDivs
    If $oDiv.ClassName = "alarms index" Then
        $aPageCount = _StringBetween($oDiv.InnerText, "Page 1 of ", ", showing ")
        If @error Then Exit
        $iPageCount = Number($aPageCount[0])
    EndIf
Next

For $i = 1 To $iPageCount
    $oIE = _IECreate("http://www.timeview2.net/alarms/index/group:All%20Groups/page:" & $i, 1)
    $oTables = _IETagNameGetCollection($oIE, "table")
    For $oTable In $oTables
        If $oTable.ClassName = "pageindex" Then
            $nTable = $oTable.NextElementSibling
            $aTable = _IETableWriteToArray($nTable, True)
            If UBound($aTable) - 1 > 0 And UBound($aTable, 2) = 7 Then
                If $i = 1 Then
                    _ArrayConcatenate($aResults, $aTable, 0)
                Else
                    _ArrayConcatenate($aResults, $aTable, 1)
                EndIf
                ExitLoop
            EndIf
        EndIf
    Next
Next
_ArrayDisplay($aResults)
;~ Save to CSV
Local $hFileOpen = FileOpen(@ScriptDir & "\Results.csv", 2)
For $i = 0 To UBound($aResults) - 1
    FileWrite($hFileOpen, '"' & _
    StringStripWS($aResults[$i][0], 7) & '","' & _
    StringStripWS($aResults[$i][1], 7) & '","' & _
    StringStripWS($aResults[$i][2], 7) & '","' & _
    StringStripWS($aResults[$i][3], 7) & '","' & _
    StringStripWS($aResults[$i][4], 7) & '","' & _
    StringStripWS($aResults[$i][5], 7) & '"' & @CRLF)
Next
FileClose($hFileOpen)

 

Edited by Subz
Added Table Header + Removed unwanted spaces from table results
1 person likes this

Share this post


Link to post
Share on other sites

#5 ·  Posted

Thanks it works but then I want to type in username and password automatically then click submit button and then click on Alarms button to extract all the data but then it is already logged in and then extracts all the data. I copied my code to it and it only automatically types in username and password but then it doesnt click submit button. I used _IEFormSubmit($oDivs) and it doesnt work and it shows warning from function _IEAttach, $_IESTATUS_NoMatch even clicking on alarms button also doesnt work. I used this code _IELinkClickByText($oIE, "Alarms") and it also shows warning Warning from function _IELinkClickByText, $_IESTATUS_NoMatch by the way I used these codes in a seperate program and it worked but when I use my codes together with yours it doesnt work. Please help me. Thanks.

Share this post


Link to post
Share on other sites

#6 ·  Posted

Can you post your code without username and password and I'll take a look although probably later this evening.

1 person likes this

Share this post


Link to post
Share on other sites

#8 ·  Posted

Try this, just replace values for $sUserName and $sPassWord.

#include <Array.au3>
#include <IE.au3>
#include <String.au3>

Local $oIE
Local $oDivs
Local $aPageCount, $iPageCount
Local $aTable, $oTables
Local $aResults[0][7]
Local $oUserName, $oPassWord
Local $sUserName = "user"
Local $sPassWord = "demo"

$oIE = _IECreate("http://www.timeview2.net/", 1)
$oForms = _IETagNameGetCollection($oIE, "form")
For $oForm In $oForms
    If $oForm.id = "UserLoginForm" Then
        $oUserName = _IEFormElementGetObjByName($oForm, "data[User][username]")
        _IEFormElementSetValue($oUserName, $sUserName)
        $oPassWord = _IEFormElementGetObjByName($oForm, "data[User][password]")
        _IEFormElementSetValue($oPassWord, $sPassWord)
        _IEFormSubmit($oForm)
    EndIf
Next

$oIE = _IECreate("http://www.timeview2.net/alarms", 1)
$oDivs = _IETagNameGetCollection($oIE, "div")
For $oDiv In $oDivs
    If $oDiv.ClassName = "alarms index" Then
        $aPageCount = _StringBetween($oDiv.InnerText, "Page 1 of ", ", showing ")
        If @error Then Exit
        $iPageCount = Number($aPageCount[0])
    EndIf
Next

For $i = 1 To $iPageCount
    $oIE = _IECreate("http://www.timeview2.net/alarms/index/group:All%20Groups/page:" & $i, 1)
    $oTables = _IETagNameGetCollection($oIE, "table")
    For $oTable In $oTables
        If $oTable.ClassName = "pageindex" Then
            $nTable = $oTable.NextElementSibling
            $aTable = _IETableWriteToArray($nTable, True)
            If UBound($aTable) - 1 > 0 And UBound($aTable, 2) = 7 Then
                If $i = 1 Then
                    _ArrayConcatenate($aResults, $aTable, 0)
                Else
                    _ArrayConcatenate($aResults, $aTable, 1)
                EndIf
                ExitLoop
            EndIf
        EndIf
    Next
Next
_ArrayDisplay($aResults)
;~ Save to CSV
Local $hFileOpen = FileOpen(@ScriptDir & "\Results.csv", 2)
For $i = 0 To UBound($aResults) - 1
    FileWrite($hFileOpen, '"' & _
    StringStripWS($aResults[$i][0], 7) & '","' & _
    StringStripWS($aResults[$i][1], 7) & '","' & _
    StringStripWS($aResults[$i][2], 7) & '","' & _
    StringStripWS($aResults[$i][3], 7) & '","' & _
    StringStripWS($aResults[$i][4], 7) & '","' & _
    StringStripWS($aResults[$i][5], 7) & '"' & @CRLF)
Next
FileClose($hFileOpen)

 

1 person likes this

Share this post


Link to post
Share on other sites

#9 ·  Posted

Thanks it works :):)but the problem is that there is too many popup window when its on Alarms page then it shows full table extraction. How to avoid too many popup windows? thanks.

Share this post


Link to post
Share on other sites

#10 ·  Posted

#include <Array.au3>
#include <IE.au3>
#include <String.au3>

Local $oIE
Local $oDivs
Local $aPageCount, $iPageCount
Local $aTable, $oTables
Local $aResults[0][7]
Local $oUserName, $oPassWord
Local $sUserName = "user"
Local $sPassWord = "demo"

$oIE = _IECreate("http://www.timeview2.net/", 1)
$oForms = _IETagNameGetCollection($oIE, "form")
For $oForm In $oForms
    If $oForm.id = "UserLoginForm" Then
        $oUserName = _IEFormElementGetObjByName($oForm, "data[User][username]")
        _IEFormElementSetValue($oUserName, $sUserName)
        $oPassWord = _IEFormElementGetObjByName($oForm, "data[User][password]")
        _IEFormElementSetValue($oPassWord, $sPassWord)
        _IEFormSubmit($oForm)
    EndIf
Next

_IENavigate($oIE, "http://www.timeview2.net/alarms")
$oDivs = _IETagNameGetCollection($oIE, "div")
For $oDiv In $oDivs
    If $oDiv.ClassName = "alarms index" Then
        $aPageCount = _StringBetween($oDiv.InnerText, "Page 1 of ", ", showing ")
        If @error Then Exit
        $iPageCount = Number($aPageCount[0])
    EndIf
Next

For $i = 1 To $iPageCount
    _IENavigate($oIE, "http://www.timeview2.net/alarms/index/group:All%20Groups/page:" & $i, 1)
    $oTables = _IETagNameGetCollection($oIE, "table")
    For $oTable In $oTables
        If $oTable.ClassName = "pageindex" Then
            $nTable = $oTable.NextElementSibling
            $aTable = _IETableWriteToArray($nTable, True)
            If UBound($aTable) - 1 > 0 And UBound($aTable, 2) = 7 Then
                If $i = 1 Then
                    _ArrayConcatenate($aResults, $aTable, 0)
                Else
                    _ArrayConcatenate($aResults, $aTable, 1)
                EndIf
                ExitLoop
            EndIf
        EndIf
    Next
Next
_ArrayDisplay($aResults)
;~ Save to CSV
Local $hFileOpen = FileOpen(@ScriptDir & "\Results.csv", 2)
For $i = 0 To UBound($aResults) - 1
    FileWrite($hFileOpen, '"' & _
    StringStripWS($aResults[$i][0], 7) & '","' & _
    StringStripWS($aResults[$i][1], 7) & '","' & _
    StringStripWS($aResults[$i][2], 7) & '","' & _
    StringStripWS($aResults[$i][3], 7) & '","' & _
    StringStripWS($aResults[$i][4], 7) & '","' & _
    StringStripWS($aResults[$i][5], 7) & '"' & @CRLF)
Next
FileClose($hFileOpen)

 

1 person likes this

Share this post


Link to post
Share on other sites

#11 ·  Posted

No more popups. It worked. Thank you so much!:):):)

Share this post


Link to post
Share on other sites

#12 ·  Posted

@Subz , Sorry to disturb you again I just only want to extract from column 0 to 5 that means excluding the column 'Actions'. Please help me. Thanks.:)

Share this post


Link to post
Share on other sites

#13 ·  Posted

If you look at the results.csv file it already excludes the Actions column, the _ArrayDisplay shows the full table but I only wrote column 0 to 5 in the csv, see code after :~ Save to CSV

Share this post


Link to post
Share on other sites

#14 ·  Posted

It doesnt have column but it has rows. I dont want 'View'. how to remove rows too for the Column 'Action' Please help. Thanks.:)

jjjj.png

Share this post


Link to post
Share on other sites

#15 ·  Posted

You can actually remove _ArrayDisplay line it doesn't really mean anything, I just wanted to show that it had grabbed the table, the results that are saved to csv are only Col 0 to Col 5.  If you want to keep _ArrayDisplay then just use _ArrayColDelete($aResults, 6) before _ArrayDisplay

1 person likes this

Share this post


Link to post
Share on other sites

#16 ·  Posted

Thank you so much!!! It worked.:):D

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • Kiran_L
      By Kiran_L
      Hi guys,
       
      I am trying to read a pdf file with unstructured data. I dontot know how to handle pdf activities in AutoIt,
      Can you help me with any UDF to open the PDF and read the doc.
       
      Thanks for your time.
       
    • Chimp
      By Chimp
      This is for extraction of data from HTML tables to an array.
      It uses an raw html source file as input, and does not relies on any browser.
      You can get the source of the html using commands like InetGet(), InetRead(), _INetGetSource(), _IEDocReadHTML() for example, or load an html file from disc as well.
      It also takes care of the data position in the table due to rowspan and colspan trying to keep the same layout in the generated array.
      It has the option to fill the cells in the array corresponding with the "span" zones all with the same value of the first "span" cell of the corresponding area.
      ; save this as _HtmlTable2Array.au3 #include-once #include <array.au3> ; ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableGetList ; Description ...: Finds and enumerates all the html tables contained in an html listing (even if nested). ; if the optional parameter $i_index is passed, then only that table is returned ; Syntax ........: _HtmlTableGetList($sHtml[, $i_index = -1]) ; Parameters ....: $sHtml - A string value containing an html page listing ; $i_index - [optional] An integer value indicating the number of the table to be returned (1 based) ; with the default value of -1 an array with all found tables is returned ; Return values .: Success; Returns an 1D 1 based array containing all or single html table found in the html. ; element [0] (and @extended as well) contains the number of tables found (or 0 if no tables are returned) ; if an error occurs then an ampty string is returned and the following @error code is setted ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _HtmlTableGetList($sHtml, $i_index = -1) Local $aTables = _ParseTags($sHtml, "<table", "</table>") If @error Then Return SetError(@error, 0, "") ElseIf $i_index = -1 Then Return SetError(0, $aTables[0], $aTables) Else If $i_index > 0 And $i_index <= $aTables[0] Then Local $aTemp[2] = [1, $aTables[$i_index]] Return SetError(0, 1, $aTemp) Else Return SetError(4, 0, "") ; bad index EndIf EndIf EndFunc ;==>_HtmlTableGetList ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableWriteToArray ; Description ...: It writes values from an html table to a 2D array. It tries to take care of the rowspan and colspan formats ; Syntax ........: _HtmlTableWriteToArray($sHtmlTable[, $bFillSpan = False[, $iFilter = 0]]) ; Parameters ....: $sHtmlTable - A string value containing the html code of the table to be parsed ; $bFillSpan - [optional] Default is False. If span areas have to be filled by repeating the data ; contained in the first cell of the span area ; $iFilter - [optional] Default is 0 (no filters) data extracted from cells is returned unchanged. ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return values .: Success: 2D array containing data from the html table ; Faillure: An empty strimg and sets @error as following: ; @error: 1 - no table content is present in the passed HTML ; 2 - error while parsing rows and/or columns, (opening and closing tags are not balanced) ; 3 - error while parsing rows and/or columns, (open/close mismatch error) ; =============================================================================================================================== Func _HtmlTableWriteToArray($sHtmlTable, $bFillSpan = False, $iFilter = 0) $sHtmlTable = StringReplace(StringReplace($sHtmlTable, "<th", "<td"), "</th>", "</td>") ; th becomes td ; rows of the wanted table Local $iError, $aTempEmptyRow[2] = [1, ""] Local $aRows = _ParseTags($sHtmlTable, "<tr", "</tr>") ; $aRows[0] = nr. of rows If @error Then Return SetError(@error, 0, "") Local $aCols[$aRows[0] + 1], $aTemp For $i = 1 To $aRows[0] $aTemp = _ParseTags($aRows[$i], "<td", "</td>") $iError = @error If $iError = 1 Then ; check if it's an empty row $aTemp = $aTempEmptyRow ; Empty Row Else If $iError Then Return SetError($iError, 0, "") EndIf If $aCols[0] < $aTemp[0] Then $aCols[0] = $aTemp[0] ; $aTemp[0] = max nr. of columns in table $aCols[$i] = $aTemp Next Local $aResult[$aRows[0]][$aCols[0]], $iStart, $iEnd, $aRowspan, $aColspan, $iSpanY, $iSpanX, $iSpanRow, $iSpanCol, $iMarkerCode, $sCellContent Local $aMirror = $aResult For $i = 1 To $aRows[0] ; scan all rows in this table $aTemp = $aCols[$i] ; <td ..> xx </td> ..... For $ii = 1 To $aTemp[0] ; scan all cells in this row $iSpanY = 0 $iSpanX = 0 $iY = $i - 1 ; zero base index for vertical ref $iX = $ii - 1 ; zero based indexes for horizontal ref ; following RegExp kindly provided by SadBunny in this post: ; http://www.autoitscript.com/forum/topic/167174-how-to-get-a-number-located-after-a-name-from-within-a-string/?p=1222781 $aRowspan = StringRegExp($aTemp[$ii], "(?i)rowspan\s*=\s*[""']?\s*(\d+)", 1) ; check presence of rowspan If IsArray($aRowspan) Then $iSpanY = $aRowspan[0] - 1 If $iSpanY + $iY > $aRows[0] Then $iSpanY -= $iSpanY + $iY - $aRows[0] + 1 EndIf EndIf ; $aColspan = StringRegExp($aTemp[$ii], "(?i)colspan\s*=\s*[""']?\s*(\d+)", 1) ; check presence of colspan If IsArray($aColspan) Then $iSpanX = $aColspan[0] - 1 ; $iMarkerCode += 1 ; code to mark this span area or single cell If $iSpanY Or $iSpanX Then $iX1 = $iX For $iSpY = 0 To $iSpanY For $iSpX = 0 To $iSpanX $iSpanRow = $iY + $iSpY If $iSpanRow > UBound($aMirror, 1) - 1 Then $iSpanRow = UBound($aMirror, 1) - 1 EndIf $iSpanCol = $iX1 + $iSpX If $iSpanCol > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][UBound($aResult, 2) + 1] ReDim $aMirror[$aRows[0]][UBound($aMirror, 2) + 1] EndIf ; While $aMirror[$iSpanRow][$iX1 + $iSpX] ; search first free column $iX1 += 1 ; $iSpanCol += 1 If $iX1 + $iSpX > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][UBound($aResult, 2) + 1] ReDim $aMirror[$aRows[0]][UBound($aMirror, 2) + 1] EndIf WEnd Next Next EndIf ; $iX1 = $iX ; following RegExp kindly provided by mikell in this post: ; http://www.autoitscript.com/forum/topic/167309-how-to-remove-from-a-string-all-between-and-pairs/?p=1224207 $sCellContent = StringRegExpReplace($aTemp[$ii], '<[^>]+>', "") If $iFilter Then $sCellContent = _HTML_Filter($sCellContent, $iFilter) For $iSpX = 0 To $iSpanX For $iSpY = 0 To $iSpanY $iSpanRow = $iY + $iSpY If $iSpanRow > UBound($aMirror, 1) - 1 Then $iSpanRow = UBound($aMirror, 1) - 1 EndIf While $aMirror[$iSpanRow][$iX1 + $iSpX] $iX1 += 1 If $iX1 + $iSpX > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][$iX1 + $iSpX + 1] ReDim $aMirror[$aRows[0]][$iX1 + $iSpX + 1] EndIf WEnd $aMirror[$iSpanRow][$iX1 + $iSpX] = $iMarkerCode ; 1 If $bFillSpan Then $aResult[$iSpanRow][$iX1 + $iSpX] = $sCellContent Next $aResult[$iY][$iX1] = $sCellContent Next Next Next ; _ArrayDisplay($aMirror, "Debug") Return SetError(0, $aResult[0][0], $aResult) EndFunc ;==>_HtmlTableWriteToArray ; ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableGetWriteToArray ; Description ...: extract the html code of the required table from the html listing and copy the data of the table to a 2D array ; Syntax ........: _HtmlTableGetWriteToArray($sHtml[, $iWantedTable = 1[, $bFillSpan = False[, $iFilter = 0]]]) ; Parameters ....: $sHtml - A string value containing the html listing ; $iWantedTable - [optional] An integer value. The nr. of the table to be parsed (default is first table) ; $bFillSpan - [optional] Default is False. If all span areas have to be filled by repeating the data ; contained in the first cell of the span area ; $iFilter - [optional] Default is 0 (no filters) data extracted from cells is returned unchanged. ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return values .: success: 2D array containing data from the wanted html table. ; faillure: An empty string and sets @error as following: ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _HtmlTableGetWriteToArray($sHtml, $iWantedTable = 1, $bFillSpan = False, $iFilter = 0) Local $aSingleTable = _HtmlTableGetList($sHtml, $iWantedTable) If @error Then Return SetError(@error, 0, "") Local $aTableData = _HtmlTableWriteToArray($aSingleTable[1], $bFillSpan, $iFilter) If @error Then Return SetError(@error, 0, "") Return SetError(0, $aTableData[0][0], $aTableData) EndFunc ;==>_HtmlTableGetWriteToArray ; #FUNCTION# ==================================================================================================================== ; Name ..........: _ParseTags ; Description ...: searches and extract all portions of html code within opening and closing tags inclusive. ; Returns an array containing a collection of <tag ...... </tag> lines. one in each element (even if are nested) ; Syntax ........: _ParseTags($sHtml, $sOpening, $sClosing) ; Parameters ....: $sHtml - A string value containing the html listing ; $sOpening - A string value indicating the opening tag ; $sClosing - A string value indicating the closing tag ; Return values .: success: an 1D 1 based array containing all the portions of html code representing the element ; element [0] af the array (and @extended as well) contains the counter of found elements ; faillure: An empty string and sets @error as following: ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _ParseTags($sHtml, $sOpening, $sClosing) ; example: $sOpening = '<table', $sClosing = '</table>' ; it finds how many of such tags are on the HTML page StringReplace($sHtml, $sOpening, $sOpening) ; in @xtended nr. of occurences Local $iNrOfThisTag = @extended ; I assume that opening <tag and closing </tag> tags are balanced (as should be) ; (so NO check is made to see if they are actually balanced) If $iNrOfThisTag Then ; if there is at least one of this tag ; $aThisTagsPositions array will contain the positions of the ; starting <tag and ending </tag> tags within the HTML Local $aThisTagsPositions[$iNrOfThisTag * 2 + 1][3] ; 1 based (make room for all open and close tags) ; 2) find in the HTML the positions of the $sOpening <tag and $sClosing </tag> tags For $i = 1 To $iNrOfThisTag $aThisTagsPositions[$i][0] = StringInStr($sHtml, $sOpening, 0, $i) ; start position of $i occurrence of <tag opening tag $aThisTagsPositions[$i][1] = $sOpening ; it marks which kind of tag is this $aThisTagsPositions[$i][2] = $i ; nr of this tag $aThisTagsPositions[$iNrOfThisTag + $i][0] = StringInStr($sHtml, $sClosing, 0, $i) + StringLen($sClosing) - 1 ; end position of $i^ occurrence of </tag> closing tag $aThisTagsPositions[$iNrOfThisTag + $i][1] = $sClosing ; it marks which kind of tag is this Next _ArraySort($aThisTagsPositions, 0, 1) ; now all opening and closing tags are in the same sequence as them appears in the HTML Local $aStack[UBound($aThisTagsPositions)][2] Local $aTags[Ceiling(UBound($aThisTagsPositions) / 2)] ; will contains the collection of <tag ..... </tag> from the html For $i = 1 To UBound($aThisTagsPositions) - 1 If $aThisTagsPositions[$i][1] = $sOpening Then ; opening <tag $aStack[0][0] += 1 ; nr of tags in html $aStack[$aStack[0][0]][0] = $sOpening $aStack[$aStack[0][0]][1] = $i ElseIf $aThisTagsPositions[$i][1] = $sClosing Then ; a closing </tag> was found If Not $aStack[0][0] Or Not ($aStack[$aStack[0][0]][0] = $sOpening And $aThisTagsPositions[$i][1] = $sClosing) Then Return SetError(3, 0, "") ; Open/Close mismatch error Else ; pair detected (the reciprocal tag) ; now get coordinates of the 2 tags ; 1) extract this tag <tag ..... </tag> from the html to the array $aTags[$aThisTagsPositions[$aStack[$aStack[0][0]][1]][2]] = StringMid($sHtml, $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0], 1 + $aThisTagsPositions[$i][0] - $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0]) ; 2) remove that tag <tag ..... </tag> from the html $sHtml = StringLeft($sHtml, $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0] - 1) & StringMid($sHtml, $aThisTagsPositions[$i][0] + 1) ; 3) adjust the references to the new positions of remaining tags For $ii = $i To UBound($aThisTagsPositions) - 1 $aThisTagsPositions[$ii][0] -= StringLen($aTags[$aThisTagsPositions[$aStack[$aStack[0][0]][1]][2]]) Next $aStack[0][0] -= 1 ; nr of tags still in html EndIf EndIf Next If Not $aStack[0][0] Then ; all tags where parsed correctly $aTags[0] = $iNrOfThisTag Return SetError(0, $iNrOfThisTag, $aTags) ; OK Else Return SetError(2, 0, "") ; opening and closing tags are not balanced EndIf Else Return SetError(1, 0, "") ; there are no of such tags on this HTML page EndIf EndFunc ;==>_ParseTags ; #============================================================================= ; Name ..........: _HTML_Filter ; Description ...: Filter for strings ; AutoIt Version : V3.3.0.0 ; Syntax ........: _HTML_Filter(ByRef $sString[, $iMode = 0]) ; Parameter(s): .: $sString - String to filter ; $iMode - Optional: (Default = 0) : removes nothing ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return Value ..: Success - Filterd String ; Failure - Input String ; Author(s) .....: Thorsten Willert, Stephen Podhajecki {gehossafats at netmdc. com} _ConvertEntities ; Date ..........: Wed Jan 27 20:49:59 CET 2010 ; modified ......: by Chimp Removed a double "&nbsp;" entities declaration, ; replace it with char(160) instead of chr(32), ; declaration of the $aEntities array as Static instead of just Local ; ============================================================================== Func _HTML_Filter(ByRef $sString, $iMode = 0) If $iMode = 0 Then Return $sString ;16 simple HTML tag / entities converter If $iMode >= 16 And $iMode < 32 Then Static Local $aEntities[95][2] = [["&quot;", 34],["&amp;", 38],["&lt;", 60],["&gt;", 62],["&nbsp;", 160] _ ,["&iexcl;", 161],["&cent;", 162],["&pound;", 163],["&curren;", 164],["&yen;", 165],["&brvbar;", 166] _ ,["&sect;", 167],["&uml;", 168],["&copy;", 169],["&ordf;", 170],["&not;", 172],["&shy;", 173] _ ,["&reg;", 174],["&macr;", 175],["&deg;", 176],["&plusmn;", 177],["&sup2;", 178],["&sup3;", 179] _ ,["&acute;", 180],["&micro;", 181],["&para;", 182],["&middot;", 183],["&cedil;", 184],["&sup1;", 185] _ ,["&ordm;", 186],["&raquo;", 187],["&frac14;", 188],["&frac12;", 189],["&frac34;", 190],["&iquest;", 191] _ ,["&Agrave;", 192],["&Aacute;", 193],["&Atilde;", 195],["&Auml;", 196],["&Aring;", 197],["&AElig;", 198] _ ,["&Ccedil;", 199],["&Egrave;", 200],["&Eacute;", 201],["&Ecirc;", 202],["&Igrave;", 204],["&Iacute;", 205] _ ,["&Icirc;", 206],["&Iuml;", 207],["&ETH;", 208],["&Ntilde;", 209],["&Ograve;", 210],["&Oacute;", 211] _ ,["&Ocirc;", 212],["&Otilde;", 213],["&Ouml;", 214],["&times;", 215],["&Oslash;", 216],["&Ugrave;", 217] _ ,["&Uacute;", 218],["&Ucirc;", 219],["&Uuml;", 220],["&Yacute;", 221],["&THORN;", 222],["&szlig;", 223] _ ,["&agrave;", 224],["&aacute;", 225],["&acirc;", 226],["&atilde;", 227],["&auml;", 228],["&aring;", 229] _ ,["&aelig;", 230],["&ccedil;", 231],["&egrave;", 232],["&eacute;", 233],["&ecirc;", 234],["&euml;", 235] _ ,["&igrave;", 236],["&iacute;", 237],["&icirc;", 238],["&iuml;", 239],["&eth;", 240],["&ntilde;", 241] _ ,["&ograve;", 242],["&oacute;", 243],["&ocirc;", 244],["&otilde;", 245],["&ouml;", 246],["&divide;", 247] _ ,["&oslash;", 248],["&ugrave;", 249],["&uacute;", 250],["&ucirc;", 251],["&uuml;", 252],["&thorn;", 254]] $sString = StringRegExpReplace($sString, '(?i)<p.*?>', @CRLF & @CRLF) $sString = StringRegExpReplace($sString, '(?i)<br>', @CRLF) Local $iE = UBound($aEntities) - 1 For $x = 0 To $iE $sString = StringReplace($sString, $aEntities[$x][0], Chr($aEntities[$x][1]), 0, 2) Next For $x = 32 To 255 $sString = StringReplace($sString, "&#" & $x & ";", Chr($x)) Next $iMode -= 16 EndIf ;8 Tag filter If $iMode >= 8 And $iMode < 16 Then ;$sString = StringRegExpReplace($sString, '<script.*?>.*?</script>', "") $sString = StringRegExpReplace($sString, "<[^>]*>", "") $iMode -= 8 EndIf ; 4 remove all double cr, lf If $iMode >= 4 And $iMode < 8 Then $sString = StringRegExpReplace($sString, "([ \t]*[\n\r]+[ \t]*)", @CRLF) $sString = StringRegExpReplace($sString, "[\n\r]+", @CRLF) $iMode -= 4 EndIf ; 2 remove all double withespaces If $iMode = 2 Or $iMode = 3 Then $sString = StringRegExpReplace($sString, "[[:blank:]]+", " ") $sString = StringRegExpReplace($sString, "\n[[:blank:]]+", @CRLF) $sString = StringRegExpReplace($sString, "[[:blank:]]+\n", "") $iMode -= 2 EndIf ; 1 remove all non ASCII (remove all chars with ascii code > 127) If $iMode = 1 Then $sString = StringRegExpReplace($sString, "[^\x00-\x7F]", " ") EndIf Return $sString EndFunc ;==>_HTML_Filter This simple demo allow to test those functions, showing what it can extract from the html tables in a web page of your choice or loading the html file from the disc.
      ; #include <_HtmlTable2Array.au3> ; <--- udf already included (hard coded) at bottom of this demo #include <GUIConstantsEx.au3> #include <EditConstants.au3> #include <WindowsConstants.au3> #include <File.au3> ; needed for _FileWriteFromArray() #include <array.au3> #include <IE.au3> Local $oIE1 = _IECreateEmbedded(), $oIE2 = _IECreateEmbedded(), $iFilter = 0 Local $sHtml_File, $iIndex, $aTable, $aMyArray, $sFilePath GUICreate("Html tables to array demo", 1000, 450, (@DesktopWidth - 1000) / 2, (@DesktopHeight - 450) / 2 _ , $WS_OVERLAPPEDWINDOW + $WS_CLIPSIBLINGS + $WS_CLIPCHILDREN) GUICtrlCreateObj($oIE1, 010, 10, 480, 360) ; left browser GUICtrlCreateTab(500, 10, 480, 360) GUICtrlCreateTabItem("view table") GUICtrlCreateObj($oIE2, 502, 33, 474, 335) ; right browser GUICtrlCreateTabItem("view html") Local $idLabel_HtmlTable = GUICtrlCreateInput("", 502, 33, 474, 335, $ES_MULTILINE + $ES_AUTOVSCROLL) GUICtrlSetFont(-1, 10, 0, 0, "Courier new") GUICtrlCreateTabItem("") Local $idInputUrl = GUICtrlCreateInput("", 10, 380, 440, 20) Local $idButton_Go = GUICtrlCreateButton("Go", 455, 380, 25, 20) Local $idButton_Load = GUICtrlCreateButton("Load html from disk", 10, 410, 480, 30) Local $idButton_Prev = GUICtrlCreateButton("Prev <-", 510, 375, 50, 30) Local $idLabel_NunTable = GUICtrlCreateLabel("00 / 00", 570, 375, 40, 30) GUICtrlSetFont(-1, 9, 700) Local $idButton_Next = GUICtrlCreateButton("Next ->", 620, 375, 50, 30) GUICtrlCreateGroup("Fill Span", 680, 370, 80, 40) Local $iFillSpan = GUICtrlCreateCheckbox("", 715, 388, 15, 15) GUICtrlCreateGroup("", -99, -99, 1, 1) ;close group Local $idButton_Array0 = GUICtrlCreateButton("Preview array", 770, 375, 100, 30) Local $idButton_Array1 = GUICtrlCreateButton("Write array to file", 880, 375, 100, 30) ; options for filtering GUICtrlCreateGroup("Filters", 510, 410, 470, 35) Local $iFilter01 = GUICtrlCreateCheckbox("non ascii", 520, 425, 85, 15) Local $iFilter02 = GUICtrlCreateCheckbox("double spaces", 610, 425, 85, 15) Local $iFilter04 = GUICtrlCreateCheckbox("double @LF", 700, 425, 85, 15) Local $iFilter08 = GUICtrlCreateCheckbox("html-tags", 790, 425, 85, 15) Local $iFilter16 = GUICtrlCreateCheckbox("tags to entities", 880, 425, 85, 15) GUICtrlCreateGroup("", -99, -99, 1, 1) ;close group GUISetState(@SW_SHOW) ;Show GUI ; _IEDocWriteHTML($oIE2, "<HTML></HTML>") GUICtrlSetData($idInputUrl, "http://www.danshort.com/HTMLentities/") ; GUICtrlSetData($idInputUrl, "http://www.mojotoad.com/sisk/projects/HTML-TableExtract/tables.html") ; example page ControlClick("", "", $idButton_Go) ; _IEAction($oIE1, "stop") Do; Waiting for user to close the window $iMsg = GUIGetMsg() Select Case $iMsg = $idButton_Go _IENavigate($oIE1, GUICtrlRead($idInputUrl)) ; _IEAction($oIE1, "stop") $aTables = _HtmlTableGetList(_IEBodyReadHTML($oIE1)) If Not @error Then ; _ArrayDisplay($aTables, "Tables contained in this html") $iIndex = 1 _IEBodyWriteHTML($oIE2, "<html>" & $aTables[$iIndex] & "</html>") ControlClick("", "", $idButton_Prev) _IEAction($oIE2, "stop") Else MsgBox(0, 0, "@error " & @error) EndIf Case $iMsg = $idButton_Load ConsoleWrite("$idButton_Load" & @CRLF) $sHtml_File = FileOpenDialog("Choose an html file", @ScriptDir & "\", "html page (*.htm;*.html)") If Not @error Then GUICtrlSetData($idInputUrl, $sHtml_File) ControlClick("", "", $idButton_Go) EndIf Case $iMsg = $idButton_Next If IsArray($aTables) Then $iIndex += $iIndex < $aTables[0] GUICtrlSetData($idLabel_NunTable, "Table" & @CRLF & $iIndex & " / " & $aTables[0]) GUICtrlSetData($idLabel_HtmlTable, $aTables[$iIndex]) _IEBodyWriteHTML($oIE2, "<html>" & $aTables[$iIndex] & "</html>") _IEAction($oIE2, "stop") EndIf Case $iMsg = $idButton_Prev If IsArray($aTables) Then $iIndex -= $iIndex > 1 GUICtrlSetData($idLabel_NunTable, "Table" & @CRLF & $iIndex & " / " & $aTables[0]) GUICtrlSetData($idLabel_HtmlTable, $aTables[$iIndex]) _IEBodyWriteHTML($oIE2, "<html>" & $aTables[$iIndex] & "</html>") _IEAction($oIE2, "stop") EndIf Case $iMsg = $idButton_Array0 ; Preview Array If IsArray($aTables) Then $iFilter = 1 * _IsChecked($iFilter01) + 2 * _IsChecked($iFilter02) + 4 * _IsChecked($iFilter04) + 8 * _IsChecked($iFilter08) + 16 * _IsChecked($iFilter16) $aMyArray = _HtmlTableWriteToArray($aTables[$iIndex], _IsChecked($iFillSpan), $iFilter) If Not @error Then _ArrayDisplay($aMyArray) EndIf Case $iMsg = $idButton_Array1 ; Saves the array in a csv file of your choice If IsArray($aTables) Then $iFilter = 1 * _IsChecked($iFilter01) + 2 * _IsChecked($iFilter02) + 4 * _IsChecked($iFilter04) + 8 * _IsChecked($iFilter08) + 16 * _IsChecked($iFilter16) $aMyArray = _HtmlTableWriteToArray($aTables[$iIndex], _IsChecked($iFillSpan), $iFilter) If Not @error Then $sFilePath = FileSaveDialog("Choose a file to save to", @ScriptDir, "(*.csv)") If $sFilePath <> "" Then If Not _FileWriteFromArray($sFilePath, $aMyArray, 0, Default, ",") Then MsgBox(0, "Error on file write", "Error code is " & @error & @CRLF & @CRLF & "@error meaning:" & @CRLF & _ "1 - Error opening specified file" & @CRLF & _ "2 - $aArray is not an array" & @CRLF & _ "3 - Error writing to file" & @CRLF & _ "4 - $aArray is not a 1D or 2D array" & @CRLF & _ "5 - Start index is greater than the $iUbound parameter") EndIf EndIf EndIf EndIf EndSelect Until $iMsg = $GUI_EVENT_CLOSE GUIDelete() ; returns 1 if CheckBox is checked Func _IsChecked($idControlID) ; $GUI_CHECKED = 1 Return GUICtrlRead($idControlID) = $GUI_CHECKED EndFunc ;==>_IsChecked ; ------------------------------------------------------------------------ ; Following code should be included by the #include <_HtmlTable2Array.au3> ; hard coded here for easy load an run to try the example ; ------------------------------------------------------------------------ #include-once #include <array.au3> ; ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableGetList ; Description ...: Finds and enumerates all the html tables contained in an html listing (even if nested). ; if the optional parameter $i_index is passed, then only that table is returned ; Syntax ........: _HtmlTableGetList($sHtml[, $i_index = -1]) ; Parameters ....: $sHtml - A string value containing an html page listing ; $i_index - [optional] An integer value indicating the number of the table to be returned (1 based) ; with the default value of -1 an array with all found tables is returned ; Return values .: Success; Returns an 1D 1 based array containing all or single html table found in the html. ; element [0] (and @extended as well) contains the number of tables found (or 0 if no tables are returned) ; if an error occurs then an ampty string is returned and the following @error code is setted ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _HtmlTableGetList($sHtml, $i_index = -1) Local $aTables = _ParseTags($sHtml, "<table", "</table>") If @error Then Return SetError(@error, 0, "") ElseIf $i_index = -1 Then Return SetError(0, $aTables[0], $aTables) Else If $i_index > 0 And $i_index <= $aTables[0] Then Local $aTemp[2] = [1, $aTables[$i_index]] Return SetError(0, 1, $aTemp) Else Return SetError(4, 0, "") ; bad index EndIf EndIf EndFunc ;==>_HtmlTableGetList ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableWriteToArray ; Description ...: It writes values from an html table to a 2D array. It tries to take care of the rowspan and colspan formats ; Syntax ........: _HtmlTableWriteToArray($sHtmlTable[, $bFillSpan = False[, $iFilter = 0]]) ; Parameters ....: $sHtmlTable - A string value containing the html code of the table to be parsed ; $bFillSpan - [optional] Default is False. If span areas have to be filled by repeating the data ; contained in the first cell of the span area ; $iFilter - [optional] Default is 0 (no filters) data extracted from cells is returned unchanged. ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return values .: Success: 2D array containing data from the html table ; Faillure: An empty strimg and sets @error as following: ; @error: 1 - no table content is present in the passed HTML ; 2 - error while parsing rows and/or columns, (opening and closing tags are not balanced) ; 3 - error while parsing rows and/or columns, (open/close mismatch error) ; =============================================================================================================================== Func _HtmlTableWriteToArray($sHtmlTable, $bFillSpan = False, $iFilter = 0) $sHtmlTable = StringReplace(StringReplace($sHtmlTable, "<th", "<td"), "</th>", "</td>") ; th becomes td ; rows of the wanted table Local $iError, $aTempEmptyRow[2] = [1, ""] Local $aRows = _ParseTags($sHtmlTable, "<tr", "</tr>") ; $aRows[0] = nr. of rows If @error Then Return SetError(@error, 0, "") Local $aCols[$aRows[0] + 1], $aTemp For $i = 1 To $aRows[0] $aTemp = _ParseTags($aRows[$i], "<td", "</td>") $iError = @error If $iError = 1 Then ; check if it's an empty row $aTemp = $aTempEmptyRow ; Empty Row Else If $iError Then Return SetError($iError, 0, "") EndIf If $aCols[0] < $aTemp[0] Then $aCols[0] = $aTemp[0] ; $aTemp[0] = max nr. of columns in table $aCols[$i] = $aTemp Next Local $aResult[$aRows[0]][$aCols[0]], $iStart, $iEnd, $aRowspan, $aColspan, $iSpanY, $iSpanX, $iSpanRow, $iSpanCol, $iMarkerCode, $sCellContent Local $aMirror = $aResult For $i = 1 To $aRows[0] ; scan all rows in this table $aTemp = $aCols[$i] ; <td ..> xx </td> ..... For $ii = 1 To $aTemp[0] ; scan all cells in this row $iSpanY = 0 $iSpanX = 0 $iY = $i - 1 ; zero base index for vertical ref $iX = $ii - 1 ; zero based indexes for horizontal ref ; following RegExp kindly provided by SadBunny in this post: ; http://www.autoitscript.com/forum/topic/167174-how-to-get-a-number-located-after-a-name-from-within-a-string/?p=1222781 $aRowspan = StringRegExp($aTemp[$ii], "(?i)rowspan\s*=\s*[""']?\s*(\d+)", 1) ; check presence of rowspan If IsArray($aRowspan) Then $iSpanY = $aRowspan[0] - 1 If $iSpanY + $iY > $aRows[0] Then $iSpanY -= $iSpanY + $iY - $aRows[0] + 1 EndIf EndIf ; $aColspan = StringRegExp($aTemp[$ii], "(?i)colspan\s*=\s*[""']?\s*(\d+)", 1) ; check presence of colspan If IsArray($aColspan) Then $iSpanX = $aColspan[0] - 1 ; $iMarkerCode += 1 ; code to mark this span area or single cell If $iSpanY Or $iSpanX Then $iX1 = $iX For $iSpY = 0 To $iSpanY For $iSpX = 0 To $iSpanX $iSpanRow = $iY + $iSpY If $iSpanRow > UBound($aMirror, 1) - 1 Then $iSpanRow = UBound($aMirror, 1) - 1 EndIf $iSpanCol = $iX1 + $iSpX If $iSpanCol > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][UBound($aResult, 2) + 1] ReDim $aMirror[$aRows[0]][UBound($aMirror, 2) + 1] EndIf ; While $aMirror[$iSpanRow][$iX1 + $iSpX] ; search first free column $iX1 += 1 ; $iSpanCol += 1 If $iX1 + $iSpX > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][UBound($aResult, 2) + 1] ReDim $aMirror[$aRows[0]][UBound($aMirror, 2) + 1] EndIf WEnd Next Next EndIf ; $iX1 = $iX ; following RegExp kindly provided by mikell in this post: ; http://www.autoitscript.com/forum/topic/167309-how-to-remove-from-a-string-all-between-and-pairs/?p=1224207 $sCellContent = StringRegExpReplace($aTemp[$ii], '<[^>]+>', "") If $iFilter Then $sCellContent = _HTML_Filter($sCellContent, $iFilter) For $iSpX = 0 To $iSpanX For $iSpY = 0 To $iSpanY $iSpanRow = $iY + $iSpY If $iSpanRow > UBound($aMirror, 1) - 1 Then $iSpanRow = UBound($aMirror, 1) - 1 EndIf While $aMirror[$iSpanRow][$iX1 + $iSpX] $iX1 += 1 If $iX1 + $iSpX > UBound($aMirror, 2) - 1 Then ReDim $aResult[$aRows[0]][$iX1 + $iSpX + 1] ReDim $aMirror[$aRows[0]][$iX1 + $iSpX + 1] EndIf WEnd $aMirror[$iSpanRow][$iX1 + $iSpX] = $iMarkerCode ; 1 If $bFillSpan Then $aResult[$iSpanRow][$iX1 + $iSpX] = $sCellContent Next $aResult[$iY][$iX1] = $sCellContent Next Next Next ; _ArrayDisplay($aMirror, "Debug") Return SetError(0, $aResult[0][0], $aResult) EndFunc ;==>_HtmlTableWriteToArray ; ; #FUNCTION# ==================================================================================================================== ; Name ..........: _HtmlTableGetWriteToArray ; Description ...: extract the html code of the required table from the html listing and copy the data of the table to a 2D array ; Syntax ........: _HtmlTableGetWriteToArray($sHtml[, $iWantedTable = 1[, $bFillSpan = False[, $iFilter = 0]]]) ; Parameters ....: $sHtml - A string value containing the html listing ; $iWantedTable - [optional] An integer value. The nr. of the table to be parsed (default is first table) ; $bFillSpan - [optional] Default is False. If all span areas have to be filled by repeating the data ; contained in the first cell of the span area ; $iFilter - [optional] Default is 0 (no filters) data extracted from cells is returned unchanged. ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return values .: success: 2D array containing data from the wanted html table. ; faillure: An empty string and sets @error as following: ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _HtmlTableGetWriteToArray($sHtml, $iWantedTable = 1, $bFillSpan = False, $iFilter = 0) Local $aSingleTable = _HtmlTableGetList($sHtml, $iWantedTable) If @error Then Return SetError(@error, 0, "") Local $aTableData = _HtmlTableWriteToArray($aSingleTable[1], $bFillSpan, $iFilter) If @error Then Return SetError(@error, 0, "") Return SetError(0, $aTableData[0][0], $aTableData) EndFunc ;==>_HtmlTableGetWriteToArray ; #FUNCTION# ==================================================================================================================== ; Name ..........: _ParseTags ; Description ...: searches and extract all portions of html code within opening and closing tags inclusive. ; Returns an array containing a collection of <tag ...... </tag> lines. one in each element (even if are nested) ; Syntax ........: _ParseTags($sHtml, $sOpening, $sClosing) ; Parameters ....: $sHtml - A string value containing the html listing ; $sOpening - A string value indicating the opening tag ; $sClosing - A string value indicating the closing tag ; Return values .: success: an 1D 1 based array containing all the portions of html code representing the element ; element [0] af the array (and @extended as well) contains the counter of found elements ; faillure: An empty string and sets @error as following: ; @error: 1 - no tables are present in the passed HTML ; 2 - error while parsing tables, (opening and closing tags are not balanced) ; 3 - error while parsing tables, (open/close mismatch error) ; 4 - invalid table index request (requested table nr. is out of boundaries) ; =============================================================================================================================== Func _ParseTags($sHtml, $sOpening, $sClosing) ; example: $sOpening = '<table', $sClosing = '</table>' ; it finds how many of such tags are on the HTML page StringReplace($sHtml, $sOpening, $sOpening) ; in @xtended nr. of occurences Local $iNrOfThisTag = @extended ; I assume that opening <tag and closing </tag> tags are balanced (as should be) ; (so NO check is made to see if they are actually balanced) If $iNrOfThisTag Then ; if there is at least one of this tag ; $aThisTagsPositions array will contain the positions of the ; starting <tag and ending </tag> tags within the HTML Local $aThisTagsPositions[$iNrOfThisTag * 2 + 1][3] ; 1 based (make room for all open and close tags) ; 2) find in the HTML the positions of the $sOpening <tag and $sClosing </tag> tags For $i = 1 To $iNrOfThisTag $aThisTagsPositions[$i][0] = StringInStr($sHtml, $sOpening, 0, $i) ; start position of $i occurrence of <tag opening tag $aThisTagsPositions[$i][1] = $sOpening ; it marks which kind of tag is this $aThisTagsPositions[$i][2] = $i ; nr of this tag $aThisTagsPositions[$iNrOfThisTag + $i][0] = StringInStr($sHtml, $sClosing, 0, $i) + StringLen($sClosing) - 1 ; end position of $i^ occurrence of </tag> closing tag $aThisTagsPositions[$iNrOfThisTag + $i][1] = $sClosing ; it marks which kind of tag is this Next _ArraySort($aThisTagsPositions, 0, 1) ; now all opening and closing tags are in the same sequence as them appears in the HTML Local $aStack[UBound($aThisTagsPositions)][2] Local $aTags[Ceiling(UBound($aThisTagsPositions) / 2)] ; will contains the collection of <tag ..... </tag> from the html For $i = 1 To UBound($aThisTagsPositions) - 1 If $aThisTagsPositions[$i][1] = $sOpening Then ; opening <tag $aStack[0][0] += 1 ; nr of tags in html $aStack[$aStack[0][0]][0] = $sOpening $aStack[$aStack[0][0]][1] = $i ElseIf $aThisTagsPositions[$i][1] = $sClosing Then ; a closing </tag> was found If Not $aStack[0][0] Or Not ($aStack[$aStack[0][0]][0] = $sOpening And $aThisTagsPositions[$i][1] = $sClosing) Then Return SetError(3, 0, "") ; Open/Close mismatch error Else ; pair detected (the reciprocal tag) ; now get coordinates of the 2 tags ; 1) extract this tag <tag ..... </tag> from the html to the array $aTags[$aThisTagsPositions[$aStack[$aStack[0][0]][1]][2]] = StringMid($sHtml, $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0], 1 + $aThisTagsPositions[$i][0] - $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0]) ; 2) remove that tag <tag ..... </tag> from the html $sHtml = StringLeft($sHtml, $aThisTagsPositions[$aStack[$aStack[0][0]][1]][0] - 1) & StringMid($sHtml, $aThisTagsPositions[$i][0] + 1) ; 3) adjust the references to the new positions of remaining tags For $ii = $i To UBound($aThisTagsPositions) - 1 $aThisTagsPositions[$ii][0] -= StringLen($aTags[$aThisTagsPositions[$aStack[$aStack[0][0]][1]][2]]) Next $aStack[0][0] -= 1 ; nr of tags still in html EndIf EndIf Next If Not $aStack[0][0] Then ; all tags where parsed correctly $aTags[0] = $iNrOfThisTag Return SetError(0, $iNrOfThisTag, $aTags) ; OK Else Return SetError(2, 0, "") ; opening and closing tags are not balanced EndIf Else Return SetError(1, 0, "") ; there are no of such tags on this HTML page EndIf EndFunc ;==>_ParseTags ; #============================================================================= ; Name ..........: _HTML_Filter ; Description ...: Filter for strings ; AutoIt Version : V3.3.0.0 ; Syntax ........: _HTML_Filter(ByRef $sString[, $iMode = 0]) ; Parameter(s): .: $sString - String to filter ; $iMode - Optional: (Default = 0) : removes nothing ; - 0 = no filter ; - 1 = removes non ascii characters ; - 2 = removes all double whitespaces ; - 4 = removes all double linefeeds ; - 8 = removes all html-tags ; - 16 = simple html-tag / entities convertor ; Return Value ..: Success - Filterd String ; Failure - Input String ; Author(s) .....: Thorsten Willert, Stephen Podhajecki {gehossafats at netmdc. com} _ConvertEntities ; Date ..........: Wed Jan 27 20:49:59 CET 2010 ; modified ......: by Chimp Removed a double "&nbsp;" entities declaration, ; replace it with char(160) instead of chr(32), ; declaration of the $aEntities array as Static instead of just Local ; ============================================================================== Func _HTML_Filter(ByRef $sString, $iMode = 0) If $iMode = 0 Then Return $sString ;16 simple HTML tag / entities converter If $iMode >= 16 And $iMode < 32 Then Static Local $aEntities[95][2] = [["&quot;", 34],["&amp;", 38],["&lt;", 60],["&gt;", 62],["&nbsp;", 160] _ ,["&iexcl;", 161],["&cent;", 162],["&pound;", 163],["&curren;", 164],["&yen;", 165],["&brvbar;", 166] _ ,["&sect;", 167],["&uml;", 168],["&copy;", 169],["&ordf;", 170],["&not;", 172],["&shy;", 173] _ ,["&reg;", 174],["&macr;", 175],["&deg;", 176],["&plusmn;", 177],["&sup2;", 178],["&sup3;", 179] _ ,["&acute;", 180],["&micro;", 181],["&para;", 182],["&middot;", 183],["&cedil;", 184],["&sup1;", 185] _ ,["&ordm;", 186],["&raquo;", 187],["&frac14;", 188],["&frac12;", 189],["&frac34;", 190],["&iquest;", 191] _ ,["&Agrave;", 192],["&Aacute;", 193],["&Atilde;", 195],["&Auml;", 196],["&Aring;", 197],["&AElig;", 198] _ ,["&Ccedil;", 199],["&Egrave;", 200],["&Eacute;", 201],["&Ecirc;", 202],["&Igrave;", 204],["&Iacute;", 205] _ ,["&Icirc;", 206],["&Iuml;", 207],["&ETH;", 208],["&Ntilde;", 209],["&Ograve;", 210],["&Oacute;", 211] _ ,["&Ocirc;", 212],["&Otilde;", 213],["&Ouml;", 214],["&times;", 215],["&Oslash;", 216],["&Ugrave;", 217] _ ,["&Uacute;", 218],["&Ucirc;", 219],["&Uuml;", 220],["&Yacute;", 221],["&THORN;", 222],["&szlig;", 223] _ ,["&agrave;", 224],["&aacute;", 225],["&acirc;", 226],["&atilde;", 227],["&auml;", 228],["&aring;", 229] _ ,["&aelig;", 230],["&ccedil;", 231],["&egrave;", 232],["&eacute;", 233],["&ecirc;", 234],["&euml;", 235] _ ,["&igrave;", 236],["&iacute;", 237],["&icirc;", 238],["&iuml;", 239],["&eth;", 240],["&ntilde;", 241] _ ,["&ograve;", 242],["&oacute;", 243],["&ocirc;", 244],["&otilde;", 245],["&ouml;", 246],["&divide;", 247] _ ,["&oslash;", 248],["&ugrave;", 249],["&uacute;", 250],["&ucirc;", 251],["&uuml;", 252],["&thorn;", 254]] $sString = StringRegExpReplace($sString, '(?i)<p.*?>', @CRLF & @CRLF) $sString = StringRegExpReplace($sString, '(?i)<br>', @CRLF) Local $iE = UBound($aEntities) - 1 For $x = 0 To $iE $sString = StringReplace($sString, $aEntities[$x][0], Chr($aEntities[$x][1]), 0, 2) Next For $x = 32 To 255 $sString = StringReplace($sString, "&#" & $x & ";", Chr($x)) Next $iMode -= 16 EndIf ;8 Tag filter If $iMode >= 8 And $iMode < 16 Then ;$sString = StringRegExpReplace($sString, '<script.*?>.*?</script>', "") $sString = StringRegExpReplace($sString, "<[^>]*>", "") $iMode -= 8 EndIf ; 4 remove all double cr, lf If $iMode >= 4 And $iMode < 8 Then $sString = StringRegExpReplace($sString, "([ \t]*[\n\r]+[ \t]*)", @CRLF) $sString = StringRegExpReplace($sString, "[\n\r]+", @CRLF) $iMode -= 4 EndIf ; 2 remove all double withespaces If $iMode = 2 Or $iMode = 3 Then $sString = StringRegExpReplace($sString, "[[:blank:]]+", " ") $sString = StringRegExpReplace($sString, "\n[[:blank:]]+", @CRLF) $sString = StringRegExpReplace($sString, "[[:blank:]]+\n", "") $iMode -= 2 EndIf ; 1 remove all non ASCII (remove all chars with ascii code > 127) If $iMode = 1 Then $sString = StringRegExpReplace($sString, "[^\x00-\x7F]", " ") EndIf Return $sString EndFunc ;==>_HTML_Filter Any error reports or suggestions for enhancements are welcome
    • RyukShini
      By RyukShini
      #Include <file.au3> #Include <Array.au3> Local $nobrainArray $var = _FileReadToArray("example.txt", $nobrainArray) $split = StringSplit($var, ":"); split by colon? _ArrayDisplay($split) Its getting later and I am getting more and more tired so I think I should go to bed and give this another look tmr.
      but if someone could help me i'd be grateful!
       
      randomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\nrandomfirstname:randomlastname\n----------------------------------------------------------------------\n\nThe topic can be found here:\nhttps://www.websitehere.com\n\n\nYou can unsubscribe at any time here: https://www.websitehere.com/unsubscribe/Zm9ydW1zO2ZvcnVtczs0MzszOTc0MTA7Mzk3NDEwO25pa29sYXppbmRvQGdtYWlsLmNvbQ,,/\n\nIf you are not following any forums and wish to stop receiving notifications, uncheck the setting\n\"Send me news and information\" found in \'My Settings\' under \'Notification Options\'.\n',545627,'followed_forums','https://www.websitehere.com/topic/','forums','forums',43,'4745c9f0607baec3e8bc38f47d07f9bd'),(622776,49813,1457299052,1,'<a href=\'https://www.websitehere.com/!545627\'>Antepliemmo</a> posted topic <a href=\'https://www.websitehere.com\'>\n\n----------------------------------------------------------------------\n As you can see this is very messy!
      There is random first names and last names everywhere and then there is a lot of junk....
      I am extracting all the names/last names for a buddy, but I just can't seem to figure it out.

      Any help is appreciated, I'll keep working on this tomorrow again wish a fresh mindset!
       
      Regards

      Ryuk
    • 31290
      By 31290
      Hi everyone
      Hope you're doing great today.
      Well, I have a little question about extracting a certain string from a file name to make comparison after treatment.
      Let's get into the details:
      First of all, I gather and store a machine BIOS Version by running this:
      RunWait(@ComSpec & " /c " & "wmic bios get SMBIOSBIOSVERSION >> C:\Drivers\Tag.txt" & @CRLF, "", @SW_HIDE, "$STDOUT_CHILD") _FileWriteToLine("c:\Drivers\BIOS_Version.txt", 1, "", 1) $sContent = FileRead ("C:\Drivers\BIOS_Version.txt") $sContent = StringRegExpReplace($sContent, " ", "") Result is, for example, A10
      Then, I download the latest available BIOS version from the Dell related model website and the filename of the latest BIOS is stored in a $sBIOSName variable.
      Result is for example, E5440A14.exe
      What i can't figure out is how to extract the Bios name stored in the $sBIOSName knowing that each model this app would run won't have the same number of chars.
      Indeed, I have some OPL790AXX.exe / OPL7010AXX.exe / E5470AXX.exe etc...
      I imagine something like getting the whole string, left trimming it to the first A found count 2 chars on the right and store them to have the possibility to make the check between A10 and A14 (as the number of char of filenames is never the same)
      Hope this is clear  If not, don't hesitate to ask for more.
      Thanks in advance for the help
       
    • wakillon
      By wakillon
      TinyPicSharer :

      A simple and handy tool for capture window or picture by mouse grab and an uploader for 10 Pic Hosters !
      You can also Resize, Convert, Optimize, add WaterMark to your images easily.
      (optimization with jpegtran.exe, optipng.exe, gifsicle.exe command line tools)
      In plus it can extract images from Docs (pdf, doc, docx, odt, odp, ppt, pptx, pps, ppsx)
      (Doc extraction with b2xtranslator and pdfextract.exe command line tools)
      Just Drag'n drop a picture/doc on the Host Logo for load it.
      I have replaced Curl by WinHttp functions and TrIDLib.DLL by my own FileGetType function.
       

       
      Previous downloads : 1343

      Update of 8 June 2013
      source and executable are available in the Download Section
      See Tray menu for options.
      Double click on his tray icon for restore the gui.
       
      >zlib.au3, WinHttp.au3, >WinAPIEx UDF are needed. ( Thanks to Ward, trancexx, Yashield )
      Hope you 'll find it usefull !