Help with Regex (capture two columns on a html table)

[Kyan]

Hi, I'm trying to capture "volcano name" and "Last Major Eruption" columns (this is a example), how can I do it through regex?

I tried this,but no output :s

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
$aTest = StringRegExp($stbl,'(?ism)<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\v+?</caption>.+?Type of Eruption\v+?</th>(?:\v+?)?</tr>\v+?(?:<td>(\w+?)$\v+?</td>\v+?<td>.+?</td>\v+?<td>(.+?)$\v+?</td>)+',3)
_ArrayDisplay($aTest)
Exit

Don't know if regex is the best way to do this, I'm downloading the page through winhttp (ascii output), but this regex is not doing the job.

Any hint? Thank you in advance, and sorry for the bad english

[mLipok]

did you try this:
 

_IETableGetCollection ( ByRef $oObject [, $iIndex = -1] )

EDIT:

or even better this:

_IETableWriteToArray ( ByRef $oObject [, $bTranspose = False] )

Edited January 7, 2015 by mLipok

[Kyan]

 

did you try this:

 

_IETableGetCollection ( ByRef $oObject [, $iIndex = -1] )

EDIT:

or even better this:

_IETableWriteToArray ( ByRef $oObject [, $bTranspose = False] )

How do I create a object for this table? I'm getting this through winhttp (need to deal with cookies and POST requests)

[mLipok]

You can create

_IECreate(..)

and put this HTML content to created object

[Kyan]

You can create

_IECreate(..)

and put this HTML content to created object

For that IE function I can do it like this (?ism)(?:<td>(.+?)</td>)+ don't help much discriminating the different columns

[Malkey]

Here are two methods that appear to work.

expandcollapse popup

#include <IE.au3>
#include <Array.au3>

Local $sHTML = '<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
        '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
        '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$sHTML = StringReplace($sHTML, "\n", "")
;ConsoleWrite($sHTML & @LF)

; ------- Using the _IE functions ------------------------------
Local $aArray = _IE_InnerhtmlTableToArray($sHTML)
_ArrayDisplay($aArray)
_ArrayDelete($aArray, "1;3")
_ArrayTranspose($aArray)
_ArrayDisplay($aArray)

;Or

; ---------------- RE Method ---------------
Local $aArray1 = StringRegExp($sHTML, "<tr>(.+?)</tr>", 3)
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Local $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)


; From HTML source, "innerhtml", get the table into an array.
Func _IE_InnerhtmlTableToArray($sSource)
    Local $oIE = _IECreate("about:blank", 0, 0) ; Hidden
    _IEPropertySet($oIE, "innerhtml", $sSource)
    Local $oTable = _IETableGetCollection($oIE, 0)
    Local $aTableData = _IETableWriteToArray($oTable)
    _IEQuit($oIE)
    Return $aTableData
EndFunc   ;==>_IE_InnerhtmlTableToArray

[jguinch]

Here is an example of a regex, working with your code :

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'

$aTest = StringRegExp($stbl, "(?s)<tr>.*?<td[^>]*>(.*?)(?:\\|</td>).*?<td.*?<td[^>]*>(.*?)(?:\\|</td>)", 3)
If IsArray($aTest) Then 
    Local $aResult[ UBound($aTest) / 2][2]
    For $i = 0 To UBound($aTest) - 1 Step 2
        $aResult[$i / 2][0] = $aTest[$i]
        $aResult[$i / 2][1] = $aTest[$i + 1]
    Next
    _ArrayDisplay($aResult)
EndIf

Edited January 7, 2015 by jguinch

[Kyan]

@malkey, I tried your RE code and didn't output an array

#include <Array.au3>
#include <IE.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
; ---------------- RE Method ---------------
Local $aArray = _IE_InnerhtmlTableToArray($stbl)
Local $aArray1 = StringRegExp($stbl, "(?im)<tr>(.+?)</tr>", 3)
Global $aArray2
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit

@jguinch, Works 5*, damn regex expression thank you

[jdelaney]

Example with xml dom:

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'

$oXML = ObjCreate("Microsoft.XMLDOM")
$oXML.loadxml($stbl)
$oRows = $oXML.selectNodes("//tr")
Local $a[$oRows.length][1]
$i = 0
$j = 0
For $oRow In $oRows
    $oCells = $oRow.selectNodes("./td|./th")
    If UBound($a,2) < $oCells.Length Then ReDim $a[UBound($a)][$oCells.Length]
    For $oCell In $oCells
        $a[$i][$j] = $oCell.text
        $j+=1
    Next
    $i+=1
    $j=0
Next
_ArrayDisplay($a)

* note: it doesn't remove html encodings.

[Malkey]

@malkey, I tried your RE code and didn't output an array

#include <Array.au3>
#include <IE.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
; ---------------- RE Method ---------------
Local $aArray = _IE_InnerhtmlTableToArray($stbl)
Local $aArray1 = StringRegExp($stbl, "(?im)<tr>(.+?)</tr>", 3)
Global $aArray2
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit

@jguinch, Works 5*, damn regex expression thank you

 

I hope to explain the mistakes you made in this quoted script.

Line 6. - The StringFormat function does convert all the "n"'s into line feeds or vertical white spaces.

Note: With all the linefeeds, the two reg exp. patterns in lines #9 and #11 will need the DotAll option, "(?s)".

This will allow the array to be created. However, each element in the array will have an invisible linefeed character, see only when printed out when a new line appears.

In my example of post #6, StringReplace($sHTML, "n", "") eliminates the problems that "n" causes.

Line 8. - The _IE_InnerhtmlTableToArray function is not needed when using the Reg Exp method and the function was not included in this script. Therefore the variable "$aArray" is not to be used later in the script.

See further comments in this debugged example.

#include <Array.au3>
#include <IE.au3>
$stbl = '<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
        '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
        '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
;$stbl=StringFormat($stbl)
$stbl = StringReplace($stbl, "\n", "")
; ---------------- RE Method ---------------
;Local $aArray = _IE_InnerhtmlTableToArray($stbl) ; _IE_InnerhtmlTableToArray function not included
Local $aArray1 = StringRegExp($stbl, "(?i)<tr>(.+?)</tr>", 3) ; With StringFormat use "(?is)<tr>(.+?)</tr>"
;Global $aArray2 ; Can declare here and then use Dim on line #14
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3) ; With StringFormat use "(?s)<t.>(.+?)<"
    ;If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)] ; Use $aArray1 not $aArray.
    If $i = 0 Then Global $aArray2[UBound($aArray1)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit

 

[Kyan]

On your first comment (#6) I saw you using $aArray and I became intrigued what you were doing :s

Yeah, nicely done, capturing first every line, then the contents of each column thank you very much