Jump to content
Sign in to follow this  
rm4453

[SOLVED] How to add a progress bar to _IETableWriteToArray

Recommended Posts

Hello,

 

I am currently writing a program that parses a massive table from a website, and need a way to add a progress bar while parsing.

I am currently using the function _IETableWriteToArray($oObj, True) to parse the array. I need the progress bar to update as the table is parsed, not just at the end of the parsing.

Any help at all would be very much appreciated!

 

*EDIT --> The array I am left with after parsing is $array[0-50000][16]

Edited by rm4453
[SOLVED]

Share this post


Link to post
Share on other sites

Hi @rm4453

a bad idea might be to change the IE library.... ( I suggest not to do it :thumbsdown:) , a better idea is to only clone the _IETableWriteToArray function and modify it by adding the progressbar and then use the modified function instead of the original one (in the modified function I marked with "; <-------- mod #" the lines I added):

#include <Array.au3>
#include <IE.au3>

Local $oIE = _IECreate("https://en.wikipedia.org/wiki/List_of_North_American_settlements_by_year_of_foundation")
Local $oTable = _IETableGetCollection($oIE, 1)
Local $aTableData = _IETableWriteToArray_mod($oTable, True)
_ArrayDisplay($aTableData)

_IEQuit($oIE)

; #FUNCTION# ====================================================================================================================
; Author ........: Dale Hohm
; modified ......: Chimp
; ===============================================================================================================================
Func _IETableWriteToArray_mod(ByRef $oObject, $bTranspose = False)
    If Not IsObj($oObject) Then
        __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_InvalidDataType")
        Return SetError($_IESTATUS_InvalidDataType, 1, 0)
    EndIf
    ;
    If Not __IEIsObjType($oObject, "table") Then
        __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_InvalidObjectType")
        Return SetError($_IESTATUS_InvalidObjectType, 1, 0)
    EndIf
    ;
    Local $iCols = 0, $oTds, $iCol
    Local $oTrs = $oObject.rows
    For $oTr In $oTrs
        $oTds = $oTr.cells
        $iCol = 0
        For $oTd In $oTds
            $iCol = $iCol + $oTd.colSpan
        Next
        If $iCol > $iCols Then $iCols = $iCol
    Next
    Local $iRows = $oTrs.length
    Local $aTableCells[$iCols][$iRows]
    Local $iRow = 0
    Local $iPercent ; <-------- mod 1

    ProgressOn("Progress Meter", "processing rows ", 0 & "/" & $iRows, -1, -1, 16) ; <-------- mod 2

    For $oTr In $oTrs
        $oTds = $oTr.cells
        $iCol = 0
        For $oTd In $oTds
            $aTableCells[$iCol][$iRow] = String($oTd.innerText)
            If @error Then ; Trap COM error, report and return
                __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_COMError", @error)
                Return SetError($_IESTATUS_ComError, @error, 0)
            EndIf
            $iCol = $iCol + $oTd.colSpan
        Next
        $iRow = $iRow + 1
        $iPercent = Int($iRow / $iRows * 100) ; <-------- mod 3
        ProgressSet($iPercent, $iRow & "/" & $iRows & @TAB & "(" & $iPercent & "%)") ; <-------- mod 4
    Next
    If $bTranspose Then
        Local $iD1 = UBound($aTableCells, $UBOUND_ROWS), $iD2 = UBound($aTableCells, $UBOUND_COLUMNS), $aTmp[$iD2][$iD1]
        For $i = 0 To $iD2 - 1
            For $j = 0 To $iD1 - 1
                $aTmp[$i][$j] = $aTableCells[$j][$i]
            Next
        Next
        $aTableCells = $aTmp
    EndIf
    ProgressOff() ; <-------- mod 5
    Return SetError($_IESTATUS_Success, 0, $aTableCells)
EndFunc   ;==>_IETableWriteToArray_mod

 

Edited by Chimp

small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites
3 minutes ago, Chimp said:

Hi @rm4453

a bad idea might be to change the IE library.... ( I suggest not to do it :thumbsdown:) , a better idea is to only clone the _IETableWriteToArray function and modify it by adding the progressbar and then use the modified function function instead of the original one (in the modified function I marked with "; <-------- mod #" the lines I added):

 

Hello, @Chimp

 

I added your modified function to the IE library, and renamed it to _IETableWriteToArray_ProgressBar. I am testing it right now to see how well it works out, and appreciate all the help so far!

Share this post


Link to post
Share on other sites

@Chimp

I have added one more modification, and it is working perfectly thank you!

*The modification allows you to append a message to be on the progress bar at the end of the function inputs.*

; #FUNCTION# ====================================================================================================================
; Author ........: Dale Hohm
; modified ......: Chimp
; ===============================================================================================================================
Func _IETableWriteToArray_ProgressBar(ByRef $oObject, $bTranspose = False, $message = "Processing Data!") ; <------- If copied into IE.au3 can call normally with _ProgressBar appended to have progress bar pop up, you can also include a message as the last arg if wanted!
    If Not IsObj($oObject) Then
        __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_InvalidDataType")
        Return SetError($_IESTATUS_InvalidDataType, 1, 0)
    EndIf
    ;
    If Not __IEIsObjType($oObject, "table") Then
        __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_InvalidObjectType")
        Return SetError($_IESTATUS_InvalidObjectType, 1, 0)
    EndIf
    ;
    Local $iCols = 0, $oTds, $iCol
    Local $oTrs = $oObject.rows
    For $oTr In $oTrs
        $oTds = $oTr.cells
        $iCol = 0
        For $oTd In $oTds
            $iCol = $iCol + $oTd.colSpan
        Next
        If $iCol > $iCols Then $iCols = $iCol
    Next
    Local $iRows = $oTrs.length
    Local $aTableCells[$iCols][$iRows]
    Local $iRow = 0
    Local $iPercent ; <-------- mod 1

    ProgressOn("Progress Meter", $message, 0 & "/" & $iRows, -1, -1, 16) ; <-------- mod 2

    For $oTr In $oTrs
        $oTds = $oTr.cells
        $iCol = 0
        For $oTd In $oTds
            $aTableCells[$iCol][$iRow] = String($oTd.innerText)
            If @error Then ; Trap COM error, report and return
                __IEConsoleWriteError("Error", "_IETableWriteToArray", "$_IESTATUS_COMError", @error)
                Return SetError($_IESTATUS_ComError, @error, 0)
            EndIf
            $iCol = $iCol + $oTd.colSpan
        Next
        $iRow = $iRow + 1
        $iPercent = Int($iRow / $iRows * 100) ; <-------- mod 3
        ProgressSet($iPercent, $iRow & "/" & $iRows & @TAB & "(" & $iPercent & "%)") ; <-------- mod 4
    Next
    If $bTranspose Then
        Local $iD1 = UBound($aTableCells, $UBOUND_ROWS), $iD2 = UBound($aTableCells, $UBOUND_COLUMNS), $aTmp[$iD2][$iD1]
        For $i = 0 To $iD2 - 1
            For $j = 0 To $iD1 - 1
                $aTmp[$i][$j] = $aTableCells[$j][$i]
            Next
        Next
        $aTableCells = $aTmp
    EndIf
    ProgressOff() ; <-------- mod 5
    Return SetError($_IESTATUS_Success, 0, $aTableCells)
EndFunc   ;==>_IETableWriteToArray_ProgressBar

 

Edited by rm4453
Fixed Spelling Mishap

Share this post


Link to post
Share on other sites

You are welcome :)
p.s. better to also add an ProgressOff() statement right after the line "If @error Then ; Trap COM error, report and return" so that the progress bar will be closed also in case of errors.


small minds discuss people average minds discuss events great minds discuss ideas.... and use AutoIt....

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Zaoka
      I'm trying to create script that would login on my SAP WEBI "web page" and replace some uploaded excel files. But i'm having problem on even the log in.
      These are user fill elements with Firefox Inspector
      <input type="text" id="_id0:logon:CMS" name="_id0:logon:CMS"> <input type="text" id="_id0:logon:USERNAME" name="_id0:logon:USERNAME"> <input type="password" id="_id0:logon:PASSWORD" name="_id0:logon:PASSWORD"> LogOnButton
      <input type="submit" id="_id0:logon:logonButton" value="Log On" class="logonButtonNoHover logon_button_no_hover" onmouseover="this.className = 'logonButtonHover logon_button_hover';" onmouseout="this.className = 'logonButtonNoHover logon_button_no_hover';">  
       
      My example, I tried _IEGetObjByName and _IEGetObjById but the script only open page but will not even fill out the form
        #include <IE.au3> Call("signIn") Func signIn() Global $oIE = _IECreate("http://192.xxx.xxx.xx:xxx/BOE/BI") Sleep (5000) Local $server = _IEGetObjByName($oIE, "_id0:logon:CMS") Local $username = _IEGetObjByName($oIE, "_id0:logon:USERNAME") Local $password = _IEGetObjByName($oIE, "_id0:logon:PASSWORD") Local $button = _IEGetObjById($oIE, "_id0:logon:logonButton") _IEFormElementSetValue($server, "TestServer:1000") _IEFormElementSetValue($username, "MyUserName") _IEFormElementSetValue($password, "MyPass") ; THIS PART ABOUT BUTTON CLICK I DONT UNDERSTAND AT ALL $sSelector = "body > div:nth-of-type(2) > div > div:nth-of-type(3) > div > table > tbody > tr > td > table > tbody > tr > td > table > tbody > tr:nth-of-type(1) > td > div > div:nth-of-type(1) > form > div:nth-of-type(5) > button" $signInUC = $oIE.document.QuerySelector($sSelector) _IEAction($signInUC, "click") EndFunc ;==>signIn  
      I would be very grateful for any help
    • By Colduction
      Hi dear friends!, i'm sorry for creating a new thread (a new problem), i have over than 9 lists that i want to combine them to be this (in this example, there are 3 test files):


      I've written a little code for splitting main information, but i really confused how to make results as "Output.txt", here is that code:
       
      $sRegex_1 = StringRegExp(FileRead("1.txt"), '(?s:(?<=\=\=\r\n)(.*?)(?=\r\n\=\=))', 3) $sRegex_2 = StringRegExp(FileRead("2.txt"), '(?s:(?<=\=\=\r\n)(.*?)(?=\r\n\=\=))', 3) $sRegex_3 = StringRegExp(FileRead("3.txt"), '(?s:(?<=\=\=\r\n)(.*?)(?=\r\n\=\=))', 3) For $i = 0 To UBound($sRegex_1) - 1 ConsoleWrite($sRegex_1[$i] & @CRLF) For $j = 0 To UBound($sRegex_2) - 1 ConsoleWrite($sRegex_2[$j] & @CRLF) For $k = 0 To UBound($sRegex_3) - 1 ConsoleWrite($sRegex_3[$k] & @CRLF) Next Next Next  
    • By adityaparakh
      Hello ,

      A website I am trying to login with my credentials.
      And retrieve the cookie into a text file.
      Unable to do so.
      Is it that certain,  Httponly , type - are not allowed to be fetched.

      Then further ,
      I will be checking every 5 minutes if my session is active , else re-login and re-fetch the cookie.
      For the second part , I will probably fetch some table and see if not in appropriate format do Part 1 : Fetch Cookie - again.
      Any better way , tips would be appreciated.
       
      Thanks
       
    • By nacerbaaziz
      hello evrybody
      here is an example about how to split your texts using a delimiter with the ability to select how much of delimiters shows in each colum  with $i_number
      e.g
      you have a long text and you want to split it in an array
      that evry colum have a number (n) of lines
      i made a function that do that for you
      just call it with a three params
      $s_text
      your text
      $i_number
      the number that you want to put in each col
      $s_siparator
      the siparator
      default is "|"
      here is the function with example
      i hope that it will be useful for you
       
      ****
       
      #include <Array.au3> $s_txt = "some text1some text2|some text3|some text4|some text5|some text6" $array = splitText($s_txt, 2) _ArrayDisplay($array) Func splitText($s_text, $i_number, $s_siparator = "|") Local $a_TXT = StringSplit($s_text, $s_siparator) Local $a_Return[$a_TXT[0] + 1] If ($a_TXT[0] <= $i_number) Or ($i_number <= 0) Then ReDim $a_Return[2] $a_Return[0] = 1 $a_Return[1] = $s_text Return $a_Return EndIf Local $i_Processed = 1, $i_arrayProcessed = 1 Do For $i = $i_Processed To ($i_Processed + $i_number) - 1 If ($a_TXT[0] < $i) Then ExitLoop If Not ($a_Return[$i_arrayProcessed]) Then $a_Return[$i_arrayProcessed] = $a_TXT[$i] Else $a_Return[$i_arrayProcessed] &= $s_siparator & $a_TXT[$i] EndIf $i_Processed += 1 Next $i_arrayProcessed += 1 Until ($a_TXT[0] < $i_Processed) ReDim $a_Return[$i_arrayProcessed] $a_Return[0] = $i_arrayProcessed - 1 Return $a_Return EndFunc ;==>splitText
      accept my greetings
      thanks to
      @Dan_555
      for his notes
       
    • By MesterPerfect
      good morning
      this is the first post here in the autoit forums
      i hope that you can help me in my problem
      i have a JSON encoded
      it a map of my forums
      where i want to make a treeview that have the same type of map
      e.g
      a system (as category)
      windows (as sub category)
      software (as an child item in the windows category)
      .....
      i don't know how to do that
      so, i know that i can do that using the json functions
      but i need your help about how we can do it as the type that i told you
      by the way i need to put the sub info for each item in an array that give me the ability to manage my items
      e.g
      can post thread
      can reply
      message cound ...
      you just give me a small example and i can continue.
      am sorry if this against the rules of the forum.
      but i realy searched a lot but i couldn't
      i hope some one give me the way.
      thank you very much in advance
       
      here is the link of json forum
      https://www.autoitscript.com/forum/topic/148114-a-non-strict-json-udf-jsmn/
      and here is my encoded json file
       
      { "tree_map": { "0": [ 1, 5, 6, 7 ], "1": [ 2 ], "2": [ 4 ], "5": [ 3 ], "6": [ 8 ], "8": [ 9, 10 ] }, "nodes": [ { "breadcrumbs": [], "description": "", "display_in_list": true, "display_order": 1, "node_id": 1, "node_name": null, "node_type_id": "Category", "parent_node_id": 0, "title": "Main category", "type_data": {} }, { "breadcrumbs": [ { "node_id": 1, "title": "Main category", "node_type_id": "Category" } ], "description": "", "display_in_list": true, "display_order": 1, "node_id": 2, "node_name": null, "node_type_id": "Forum", "parent_node_id": 1, "title": "Main forum", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [ { "node_id": 1, "title": "Main category", "node_type_id": "Category" }, { "node_id": 2, "title": "Main forum", "node_type_id": "Forum" } ], "description": "", "display_in_list": true, "display_order": 1, "node_id": 4, "node_name": null, "node_type_id": "Forum", "parent_node_id": 2, "title": "my forums1", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [], "description": "", "display_in_list": true, "display_order": 2, "node_id": 5, "node_name": null, "node_type_id": "Category", "parent_node_id": 0, "title": "Perfect", "type_data": {} }, { "breadcrumbs": [ { "node_id": 5, "title": "Perfect", "node_type_id": "Category" } ], "description": "", "display_in_list": true, "display_order": 2, "node_id": 3, "node_name": null, "node_type_id": "Forum", "parent_node_id": 5, "title": "ahmed", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [], "description": "", "display_in_list": true, "display_order": 3, "node_id": 6, "node_name": null, "node_type_id": "Forum", "parent_node_id": 0, "title": "autoit", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [ { "node_id": 6, "title": "autoit", "node_type_id": "Forum" } ], "description": "", "display_in_list": true, "display_order": 3, "node_id": 8, "node_name": null, "node_type_id": "Forum", "parent_node_id": 6, "title": "examples", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [ { "node_id": 6, "title": "autoit", "node_type_id": "Forum" }, { "node_id": 8, "title": "examples", "node_type_id": "Forum" } ], "description": "", "display_in_list": true, "display_order": 3, "node_id": 9, "node_name": null, "node_type_id": "Forum", "parent_node_id": 8, "title": "GUI", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [ { "node_id": 6, "title": "autoit", "node_type_id": "Forum" }, { "node_id": 8, "title": "examples", "node_type_id": "Forum" } ], "description": "", "display_in_list": true, "display_order": 31, "node_id": 10, "node_name": null, "node_type_id": "Forum", "parent_node_id": 8, "title": "windowEX", "type_data": { "allow_poll": true, "allow_posting": true, "can_create_thread": true, "can_upload_attachment": true, "discussion_count": 0, "last_post_date": 0, "last_post_id": 0, "last_post_username": "", "last_thread_id": 0, "last_thread_prefix_id": 0, "last_thread_title": "", "message_count": 0, "min_tags": 0, "require_prefix": false } }, { "breadcrumbs": [], "description": "", "display_in_list": true, "display_order": 4, "node_id": 7, "node_name": null, "node_type_id": "Category", "parent_node_id": 0, "title": "vbs", "type_data": {} } ] }  
×
×
  • Create New...