Sign in to follow this  
Followers 0
Kyan

Help with Regex (capture two columns on a html table)

11 posts in this topic

Hi, I'm trying to capture "volcano name" and "Last Major Eruption" columns (this is a example), how can I do it through regex?

fderde.png

I tried this,but no output :s

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
$aTest = StringRegExp($stbl,'(?ism)<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\v+?</caption>.+?Type of Eruption\v+?</th>(?:\v+?)?</tr>\v+?(?:<td>(\w+?)$\v+?</td>\v+?<td>.+?</td>\v+?<td>(.+?)$\v+?</td>)+',3)
_ArrayDisplay($aTest)
Exit

Don't know if regex is the best way to do this, I'm downloading the page through winhttp (ascii output), but this regex is not doing the job.

Any hint? Thank you in advance, and sorry for the bad english


Heroes, there is no such thing

One day I'll discover what IE.au3 has of special for so many users using it.
C'mon there's InetRead and WinHTTP, way better
happy.png

Share this post


Link to post
Share on other sites



#2 ·  Posted (edited)

did you try this:
 

_IETableGetCollection ( ByRef $oObject [, $iIndex = -1] )

EDIT:

or even better this:

_IETableWriteToArray ( ByRef $oObject [, $bTranspose = False] )
Edited by mLipok

Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

 

did you try this:

 

_IETableGetCollection ( ByRef $oObject [, $iIndex = -1] )

EDIT:

or even better this:

_IETableWriteToArray ( ByRef $oObject [, $bTranspose = False] )

How do I create a object for this table? I'm getting this through winhttp (need to deal with cookies and POST requests)


Heroes, there is no such thing

One day I'll discover what IE.au3 has of special for so many users using it.
C'mon there's InetRead and WinHTTP, way better
happy.png

Share this post


Link to post
Share on other sites

You can create

_IECreate(..)

and put this HTML content to created object


Signature beginning:   Wondering who uses AutoIT and what it can be used for ?
* GHAPI UDF - modest begining - comunication with GitHub REST API *
ADO.au3 UDF     POP3.au3 UDF     XML.au3 UDF    How to use IE.au3  UDF with  AutoIt v3.3.14.x  for other useful stuff click the following button

Spoiler

Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind. 

My contribution (my own projects): * Debenu Quick PDF Library - UDF * Debenu PDF Viewer SDK - UDF * Acrobat Reader - ActiveX Viewer * UDF for PDFCreator v1.x.x * XZip - UDF * AppCompatFlags UDF * CrowdinAPI UDF * _WinMergeCompare2Files() * _JavaExceptionAdd() * _IsBeta() * Writing DPI Awareness App - workaround * _AutoIt_RequiredVersion() * Chilkatsoft.au3 UDF * TeamViewer.au3 UDF * JavaManagement UDF * VIES over SOAP * WinSCP UDF * GHAPI UDF - modest begining - comunication with GitHub REST API *

My contribution to others projects or UDF based on  others projects: * _sql.au3 UDF  * POP3.au3 UDF *  RTF Printer - UDF * XML.au3 - BETA * ADO.au3 UDF SMTP Mailer UDF *

Useful links: * Forum Rules * Forum etiquette *  Forum Information and FAQs * How to post code on the forum * AutoIt Online Documentation * AutoIt Online Beta Documentation * SciTE4AutoIt3 getting started * Convert text blocks to AutoIt code * Games made in Autoit * Programming related sites * Polish AutoIt Tutorial * DllCall Code Generator * 

Wiki: Expand your knowledge - AutoIt Wiki * Collection of User Defined Functions * How to use HelpFile * Best coding practices * 

IE Related:  * How to use IE.au3  UDF with  AutoIt v3.3.14.x * Why isn't Autoit able to click a Javascript Dialog? * Clicking javascript button with no ID * IE document >> save as MHT file * IETab Switcher (by LarsJ ) * HTML Entities * _IEquerySelectorAll() (by uncommon) * 

I encourage you to read: * Global Vars * Best Coding Practices * Please explain code used in Help file for several File functions * OOP-like approach in AutoIt * UDF-Spec Questions *  EXAMPLE: How To Catch ConsoleWrite() output to a file or to CMD *

"Homo sum; humani nil a me alienum puto" - Publius Terentius Afer
"Program are meant to be read by humans and only incidentally for computers and execute" - Donald Knuth, "The Art of Computer Programming"
:naughty:  :ranting:, be  :) and       \\//_.

Anticipating Errors :  "Any program that accepts data from a user must include code to validate that data before sending it to the data store. You cannot rely on the data store, ...., or even your programming language to notify you of problems. You must check every byte entered by your users, making sure that data is the correct type for its field and that required fields are not empty."

Signature last update: 2017-06-04

Share this post


Link to post
Share on other sites

You can create

_IECreate(..)

and put this HTML content to created object

For that IE function I can do it like this (?ism)(?:<td>(.+?)</td>)+ don't help much discriminating the different columns


Heroes, there is no such thing

One day I'll discover what IE.au3 has of special for so many users using it.
C'mon there's InetRead and WinHTTP, way better
happy.png

Share this post


Link to post
Share on other sites

Here are two methods that appear to work.

#include <IE.au3>
#include <Array.au3>

Local $sHTML = '<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
        '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
        '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$sHTML = StringReplace($sHTML, "\n", "")
;ConsoleWrite($sHTML & @LF)

; ------- Using the _IE functions ------------------------------
Local $aArray = _IE_InnerhtmlTableToArray($sHTML)
_ArrayDisplay($aArray)
_ArrayDelete($aArray, "1;3")
_ArrayTranspose($aArray)
_ArrayDisplay($aArray)

;Or

; ---------------- RE Method ---------------
Local $aArray1 = StringRegExp($sHTML, "<tr>(.+?)</tr>", 3)
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Local $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)


; From HTML source, "innerhtml", get the table into an array.
Func _IE_InnerhtmlTableToArray($sSource)
    Local $oIE = _IECreate("about:blank", 0, 0) ; Hidden
    _IEPropertySet($oIE, "innerhtml", $sSource)
    Local $oTable = _IETableGetCollection($oIE, 0)
    Local $aTableData = _IETableWriteToArray($oTable)
    _IEQuit($oIE)
    Return $aTableData
EndFunc   ;==>_IE_InnerhtmlTableToArray
1 person likes this

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

Here is an example of a regex, working with your code :

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'

$aTest = StringRegExp($stbl, "(?s)<tr>.*?<td[^>]*>(.*?)(?:\\|</td>).*?<td.*?<td[^>]*>(.*?)(?:\\|</td>)", 3)
If IsArray($aTest) Then 
    Local $aResult[ UBound($aTest) / 2][2]
    For $i = 0 To UBound($aTest) - 1 Step 2
        $aResult[$i / 2][0] = $aTest[$i]
        $aResult[$i / 2][1] = $aTest[$i + 1]
    Next
    _ArrayDisplay($aResult)
EndIf
Edited by jguinch
1 person likes this

Share this post


Link to post
Share on other sites

@malkey, I tried your RE code and didn't output an array

#include <Array.au3>
#include <IE.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
; ---------------- RE Method ---------------
Local $aArray = _IE_InnerhtmlTableToArray($stbl)
Local $aArray1 = StringRegExp($stbl, "(?im)<tr>(.+?)</tr>", 3)
Global $aArray2
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit

@jguinch, Works 5*, damn regex expression :D thank you


Heroes, there is no such thing

One day I'll discover what IE.au3 has of special for so many users using it.
C'mon there's InetRead and WinHTTP, way better
happy.png

Share this post


Link to post
Share on other sites

Example with xml dom:

#include <Array.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'

$oXML = ObjCreate("Microsoft.XMLDOM")
$oXML.loadxml($stbl)
$oRows = $oXML.selectNodes("//tr")
Local $a[$oRows.length][1]
$i = 0
$j = 0
For $oRow In $oRows
    $oCells = $oRow.selectNodes("./td|./th")
    If UBound($a,2) < $oCells.Length Then ReDim $a[UBound($a)][$oCells.Length]
    For $oCell In $oCells
        $a[$i][$j] = $oCell.text
        $j+=1
    Next
    $i+=1
    $j=0
Next
_ArrayDisplay($a)

* note: it doesn't remove html encodings.

1 person likes this

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

@malkey, I tried your RE code and didn't output an array

#include <Array.au3>
#include <IE.au3>
$stbl='<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
    '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
    '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
$stbl=StringFormat($stbl)
; ---------------- RE Method ---------------
Local $aArray = _IE_InnerhtmlTableToArray($stbl)
Local $aArray1 = StringRegExp($stbl, "(?im)<tr>(.+?)</tr>", 3)
Global $aArray2
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3)
    If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit
@jguinch, Works 5*, damn regex expression :D thank you

 

I hope to explain the mistakes you made in this quoted script.

Line 6. - The StringFormat function does convert all the "n"'s into line feeds or vertical white spaces.

Note: With all the linefeeds, the two reg exp. patterns in lines #9 and #11 will need the DotAll option, "(?s)".

This will allow the array to be created. However, each element in the array will have an invisible linefeed character, see only when printed out when a new line appears.

In my example of post #6, StringReplace($sHTML, "n", "") eliminates the problems that "n" causes.

Line 8. - The _IE_InnerhtmlTableToArray function is not needed when using the Reg Exp method and the function was not included in this script. Therefore the variable "$aArray" is not to be used later in the script.

See further comments in this debugged example.

#include <Array.au3>
#include <IE.au3>
$stbl = '<table border="1">\n<caption>Recent Major Volcanic Eruptions in the Pacific Northwest\n</caption>\n<tr>\n<th>Volcano Name\n</th>\n<th>Location\n</th>\n<th>Last Major Eruption\n</th>\n<th>Type of Eruption\n' & _
        '</th></tr>\n<tr>\n<td>Mt. Lassen\n</td>\n<td>California\n\n</td>\n<td>1914-17\n</td>\n<td>Explosive Eruption\n</td></tr>\n<tr>\n<td>Mt. Hood\n</td>\n<td>Oregon\n</td>\n<td>1790s\n</td>\n<td>Pyroclastic flows and Mudflows' & _
        '</td></tr>\n<tr>\n<td>Mt. St. Helens\n</td>\n<td>Washington\n</td>\n<td>1980\n</td>\n<td>Explosive Eruption\n</td></tr></table>'
;$stbl=StringFormat($stbl)
$stbl = StringReplace($stbl, "\n", "")
; ---------------- RE Method ---------------
;Local $aArray = _IE_InnerhtmlTableToArray($stbl) ; _IE_InnerhtmlTableToArray function not included
Local $aArray1 = StringRegExp($stbl, "(?i)<tr>(.+?)</tr>", 3) ; With StringFormat use "(?is)<tr>(.+?)</tr>"
;Global $aArray2 ; Can declare here and then use Dim on line #14
For $i = 0 To UBound($aArray1) - 1
    $aTemp = StringRegExp($aArray1[$i], "<t.>(.+?)<", 3) ; With StringFormat use "(?s)<t.>(.+?)<"
    ;If $i = 0 Then Dim $aArray2[UBound($aArray)][UBound($aTemp)] ; Use $aArray1 not $aArray.
    If $i = 0 Then Global $aArray2[UBound($aArray1)][UBound($aTemp)]
    For $j = 0 To UBound($aTemp) - 1
        $aArray2[$i][$j] = $aTemp[$j]
    Next
Next
_ArrayDisplay($aArray2)
_ArrayTranspose($aArray2)
_ArrayDelete($aArray2, "1;3")
_ArrayTranspose($aArray2)
_ArrayDisplay($aArray2)
Exit

 

1 person likes this

Share this post


Link to post
Share on other sites

On your first comment (#6) I saw you using $aArray and I became intrigued what you were doing :s

Yeah, nicely done, capturing first every line, then the contents of each column :D thank you very much  :thumbsup:


Heroes, there is no such thing

One day I'll discover what IE.au3 has of special for so many users using it.
C'mon there's InetRead and WinHTTP, way better
happy.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0