Jump to content

Getting info from a website with arrays and table read (Optimize my code)


Recommended Posts

Hi, first of all thanks to all the guys who always help people in the forums, I wouldn't be able to do anything if wasn't for your help, even if I don't ask it myself.

 

I've created this code to get some info on a monitoring network on my work. It relays on _IETableGetCollection and _IETableWriteToArray.

It works well, but take around 3:25 minutes to get the info from 28 pages (some of them are large and take longer to load, but most of them are small and fast).

My question is if you see a way to get the program to go faster...

 

I've tried to make it easy for you to understand and edited somethings with sensitive info.

(Some of the pages doesn't have the black divider with MIRA in the end, so I need to search if it is there or not.)

 

#include <IE.au3>
#include <array.au3>

Local $oIE = _IECreate("about:blank", 0, 0)
Local $paginas[28] = [89, 90, 91, 92, 93, 96, 105, 113, 119, 125, 126, 129, 131, 133, 135, 137, 139, 140, 141, 144, 145, 146, 148, 149, 150, 151, 158, 159]
Local $Datos_array[0][2]
Local $oTable
Local $tabla
Local $aux_x = 1
Local $ar = 1
Local $Numtables_datos = 0
MsgBox(0, "asd", "asd")
For $pag = 0 To UBound($paginas) - 1 Step 1
    _IENavigate($oIE, "<WEBSITE URL>" & $paginas[$pag]) ; <<< the pages to load are always the same except for the last digits.


    _ArrayAdd($Datos_array, $paginas[$pag] & "|" & "Entrante", 0, "|") ; <<<<<<<<<<<<<<<< adds the page number toarray  [0, 0] 

    ;############################################ START counts amount of tables with traffic
    $oTable = _IETableGetCollection($oIE)
    Local $iNumTables = @extended
    For $i = 3 To $iNumTables - 2 Step 1
        $oTable = _IETableGetCollection($oIE, $i)
        $nomb_tabla2 = _IETableWriteToArray($oTable) ; <<<<<<<< TABLE TO ARRAY.
        $string2 = StringStripWS($nomb_tabla2[1][0], 8)
        If $string2 <> "MIRA" Then $Numtables_datos = $Numtables_datos + 1
    Next
    $tabla_End = $iNumTables - $Numtables_datos
    ;############################################ FIN

    $tabla_Start = 4
    $tabla_trafico = 2
    For $for = 1 To $Numtables_datos Step 1

        $oTable = _IETableGetCollection($oIE, $tabla_Start - 1) ; <<<<<<<<<<< NAME OF THE TABLE; row2 = mira

        $nomb_tabla = _IETableWriteToArray($oTable) ; <<<<<<<< TABLE TO ARRAY

        ;########################################### ADDS the traffic number into the row
        $string = StringStripWS($nomb_tabla[1][0], 8)
        If $string == "MIRA" Then
            ;si o si pasa por aca 1 vez
            _ArrayAdd($Datos_array, $nomb_tabla[0][0])
            $nomb_aux = $nomb_tabla[0][0]
            $aux_x = 1
            $tabla_trafico = $tabla_trafico + 2
        Else
            ;esto deberia ser por row
            _ArrayAdd($Datos_array, $nomb_aux & " " & $aux_x)
            $aux_x = $aux_x + 1
            $tabla_trafico = $tabla_trafico + 1
        EndIf

        $oTable = _IETableGetCollection($oIE, $tabla_trafico)
        Local $aTableData = _IETableWriteToArray($oTable)

        $bps = _ArrayToString($aTableData, "|", 0, 0, @CRLF, 0, 0)
        $bps = StringRight($bps, 5)
        $bps = StringLeft($bps, 4)
        $trafico_actual = _ArrayToString($aTableData, "|", 0, 0, @CRLF, 2, 2)
        If $bps == "Gbps" Then $trafico_actual = $trafico_actual * 1000
        If $bps == "Kbps" Then $trafico_actual = $trafico_actual / 1000
        $Datos_array[$ar][1] = $trafico_actual
        $ar = $ar + 1

        If $string == "MIRA" Then

            $tabla_Start = $tabla_Start + 2
        Else
            $tabla_Start = $tabla_Start + 1
        EndIf
    Next
    $ar = $ar + 1




;~ ############# CAÍDA ############
;~          If $actual_entrante = 0 Then
;~              $xxx = 0
;~              Do

;~              MsgBox(0, "Tráfico Caído", $paginas[$i], 5)
;~                  $xxx = $xxx + 1
;~              Until $xxx = 10
;~          EndIf
;~ ############# CAÍDA ############.
    Local $Numtables_datos = 0
Next
_ArrayDisplay($Datos_array, "Array display")
_IEQuit($oIE)

Thanks!! 

array output.png

WEB SCREENSHOT2.png

monitoria.html

Edited by JackER4565
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By ahha
      I'm trying to get the number of columns in a specific row in a Word table and am stuck.   I need a push.  Program below and Word file attached.
      Thanks.
      #AutoIt3Wrapper_run_debug_mode=Y ;use this to debug in console window #include <Word.au3> $oWord = _Word_Create(True, True) ;Create Word application object, make it visible, and force a new instance of Word $oDoc = _Word_DocOpen($oWord, @ScriptDir&"\ColumnTest.docx", Default, Default, True) ;Open the Word document $iTablesCount = $oDoc.Tables.Count ;get Tables count in $oDoc Pause("$iTablesCount = '" & $iTablesCount & "'") $iRowCount = $oDoc.Tables.Item(1).Rows.Count ;Table hard coded $iColCount = $oDoc.Tables.Item(1).Columns.Count Pause("Table#1 $iRowCount = '" & $iRowCount & " $iColCount = '" & $iColCount & "'") ;trying to get the number of columns in each row ;$ColCountInRow = $oDoc.Tables.Item(1).Rows(1).Columns.Count ;this fails and read somewhere to use Cells.Count $ColCountInRow = $oDoc.Tables.Item(1).Rows(1).Cells.Count ;hard code Row 1 <<<<< ERROR here ;this is the error I get ;: ==> The requested action with this object has failed.: ;$ColCountInRow = $oDoc.Tables.Item(1).Rows(1).Cells.Count ;$ColCountInRow = $oDoc.Tables.Item(1)^ ERROR Pause("Row 1 has " & $ColCountInRow & " Columns") Exit Func Pause($text="") MsgBox(262144, "DEBUG", "Paused: " & $text) EndFunc  
      ColumnTest.docx
    • By vinnyMS
      #Include <Array.au3> #include <Constants.au3> $s = FileRead("2.txt") Local $w = StringRegExp($s, '(?is)(\b\w+\b)(?!.*\b\1\b)', 3) _ArrayColInsert($w, 1) For $i = 0 to UBound($w)-1 StringRegExpReplace($s, '(?i)\b' & $w[$i][0] & '\b', $w[$i][0]) $w[$i][1] = @extended Next _ArraySort($w, 1, 0, 0, 1) _ArrayDisplay($w) i have this script that returns 3 columns  
       
      i need to copy the  Col 0 and Col 1 as text to paste on notepad or excel
      you will have to create a "copy" button if possible
      array.au3 2.txt
    • By DannyJ
      I have a dataset like this, (a strubg)
      Username: User1 Type: Admin RegDate: 1999 Username: User2 Type: User RegDate: 2000 How to make a 2 dimensional array that I can display with _ArrayDisplay?
      This would be a perfect 2D array to represent my data:
      Username           Tpye RegDate User1              Admin 1999 User2              User 2000   If you run this Powershell this powershell command, you can get this dataset that I am talking about:
      Get-LocalUser | Select * With this code you can try it to read into a string:
      #include <GuiConstantsEx.au3> #include <WindowsConstants.au3> #include "GUIListViewEx.au3" #include <Array.au3> ; Just for display in example #RequireAdmin #Region ;**** Directives created by AutoIt3Wrapper_GUI **** #AutoIt3Wrapper_UseX64=y #EndRegion $sCommand = "powershell.exe Get-LocalUser | Select *" Local $iPid = Run($sCommand, @WorkingDir , @SW_SHOW , $STDOUT_CHILD) ProcessWaitClose($iPid) Local $sOutput = StdoutRead($iPID) ConsoleWrite($sOutput) How can I correctly split $sOutput into a 2D array (with the above mentioned layout) that I can display and I work with?
    • By kovlad
      My solution is to write nested arrays without copying.
      The problem was described hier.
       
      Function:
      #include <Array.au3> ; #FUNCTION# ==================================================================================================================== ; Name ..........: _ArrayNestedSet ; Description ...: Assigns a value to an element of a nested 1D array. ; Syntax ........: _ArrayNestedSet(ByRef $aArray, $vIndex, $vValue) ; Parameters ....: $aArray - an array of arrays. ; $vIndex - an index or 1d-array of indexes; ; a size if $vValue not defined (zero to delete). ; $vValue - a value (create, resize or delete if not defined). ; ; Return values .: on success - 1 ; @extended - nesting level of operation ; on failure - 0 ; @extended - nesting level of error ; @error = 1 - invalid array ; @error = 2 - invalid index ; Author ........: ; Modified ......: kovlad ; Remarks .......: ; Related .......: ; Link ..........: https://www.autoitscript.com/forum/topic/185638-assign-a-value-to-an-array-in-array-element/ ; https://www.autoitscript.com/trac/autoit/ticket/3515?replyto=description ; Example .......: Yes ; =============================================================================================================================== Func _ArrayNestedSet(ByRef $aArray, $vIndex, $vValue = Default) Local $extended = @extended + 1 If IsArray($vIndex) Then If UBound($vIndex, 0) <> 1 Then _ Return SetError(2, $extended) If UBound($vIndex) > 1 Then If UBound($aArray, 0) <> 1 Then _ Return SetError(1, $extended) ; keep index for this array Local $i = $vIndex[0] If $i < 0 Or UBound($aArray) <= $i Then _ Return SetError(2, $extended) ; delete index of this array _ArrayDelete($vIndex, 0) ; recursive function call Local $return = _ArrayNestedSet($aArray[$i], $vIndex, $vValue) If @error Then Return SetError(@error, @extended + 1, 0) Else Return SetExtended(@extended + 1, 1) EndIf Else $vIndex = $vIndex[0] EndIf EndIf If $vValue = Default Then If $vIndex < 0 Then _ Return SetError(2, $extended) If $vIndex = 0 Then ; delete array and free memory $aArray = 0 Return SetExtended($extended, 1) EndIf If UBound($aArray, 0) = 1 Then ; resize array keeping data ReDim $aArray[$vIndex] Return SetExtended($extended, 1) Else ; create new nested array Local $aTmp[$vIndex] $aArray = $aTmp Return SetExtended($extended, 1) EndIf Else If UBound($aArray) <= $vIndex Then _ Return SetError(2, $extended + 1) ; set value of array entry $aArray[$vIndex] = $vValue Return SetExtended($extended, 1) EndIf EndFunc  
      Examples:
      ; write value to 1st nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : write value to 1st nested array" & @CRLF) Local $aTmp1[4] = [1,2,3,4] _ArrayDisplay($aTmp1, "$aTmp1") Local $aArray[2] = [$aTmp1] ConsoleWrite( _ "_ArrayNestedSet($aArray[0], 3, 14) = " & _ArrayNestedSet($aArray[0], 3, 14) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay($aArray[0], "$aArray[0]") ; resize 1st nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : resize 1st nested array" & @CRLF) ConsoleWrite( _ "_ArrayNestedSet($aArray[0], 8) = " & _ArrayNestedSet($aArray[0], 8) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay($aArray[0], "$aArray[0]") ; write array to 1st nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : write array to 1st nested array" & @CRLF) Local $aTmp11[4] = [11,12,13,14] _ArrayDisplay($aTmp11, "$aTmp11") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], 2, $aTmp11) = " & _ArrayNestedSet($aArray[0], 2, $aTmp11) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay(($aArray[0])[2], "($aArray[0])[2]") ; write value to 2nd nested array using index array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : write value to 2nd nested array using index array" & @CRLF) Local $aIndex1[2] = [2,3] _ArrayDisplay($aIndex1, "$aIndex1") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex1, 140) = " & _ArrayNestedSet($aArray[0], $aIndex1, 140) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay(($aArray[0])[2], "($aArray[0])[2]") ; resize 2nd nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : resize 2nd nested array" & @CRLF) Local $aIndex1[2] = [2,8] _ArrayDisplay($aIndex1, "$aIndex1") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex1) = " & _ArrayNestedSet($aArray[0], $aIndex1) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay(($aArray[0])[2], "($aArray[0])[2]") ; create new 3rd nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : create new 3rd nested array" & @CRLF) Local $aIndex2[3] = [2,7,6] _ArrayDisplay($aIndex2, "$aIndex2") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex2) = " & _ArrayNestedSet($aArray[0], $aIndex2) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF & @CRLF) _ArrayDisplay((($aArray[0])[2])[7], ")($aArray[0])[2])[7]") ; delete 3rd nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : delete 3rd nested array" & @CRLF) Local $aIndex3[3] = [2,7,0] _ArrayDisplay($aIndex3, "$aIndex2") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex3) = " & _ArrayNestedSet($aArray[0], $aIndex3) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF) ConsoleWrite("IsArray((($aArray[0])[2])[7]) = " & IsArray((($aArray[0])[2])[7]) & @CRLF & @CRLF) ; write 0 in 1st nested array to delete the 2nd nested array ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : write 0 in 1st nested array to delete the 2nd nested array" & @CRLF) Local $aIndex4[1] = [2] _ArrayDisplay($aIndex4, "$aIndex4") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex4, 0) = " & _ArrayNestedSet($aArray[0], $aIndex4, 0) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF) ConsoleWrite("IsArray(($aArray[0])[2]) = " & IsArray(($aArray[0])[2]) & @CRLF & @CRLF) ; delete 1st nested array (same as '$aArray[0] = 0') ConsoleWrite("@@ Debug(" & @ScriptLineNumber & ") : delete 1st nested array (same as '$aArray[0] = 0')" & @CRLF) Local $aIndex5[1] = [0] _ArrayDisplay($aIndex5, "$aIndex5") ConsoleWrite( _ "_ArrayNestedSet($aArray[0], $aIndex5) = " & _ArrayNestedSet($aArray[0], $aIndex5) & @CRLF & _ " @error = " & @error & @CRLF & _ " @extended = " & @extended & @CRLF) ConsoleWrite("IsArray($aArray[0]) = " & IsArray($aArray[0]) & @CRLF & @CRLF)  
    • By DannyJ
      $sCommands1 = 'powershell.exe Get-ChildItem' $iPid = run($sCommands1   , @WorkingDir , @SW_SHOW , 0x2) $sOutput = ""  While 1     $sOutput &= StdoutRead($iPID)         If @error Then             ExitLoop         EndIf  WEnd ;~ msgbox(0, '' , $sOutput) ConsoleWrite("$sOutput") ConsoleWrite($sOutput) ConsoleWrite(@CRLF) $aOutput = stringsplit($sOutput ,@LF , 2) For $i=0 To  UBound($aOutput) - 1 Step 1     ConsoleWrite($aOutput[$i]) Next The script above reads the whole directory into a one dimensional array, but I need to work with the array, so I need to split the array into multiple dimensions.
      I have already read some forum answers here, and I have already tried these commands:
       
      Are there any way to use the $aOutput variable like in PowerShell:
      PowerShell:
      $a = Get-ChildItem $a.Mode I imagine this in AutoIt  $aOutput
      ConsoleWrite($aOutput[i].Mode) Or if I split this command into 2 dimension like:
      For $i To UBound($aOutput)-1 Step 1 ConsoleWrite($aOutput[$i][1]) ConsoleWrite($aOutput[$i][2]) Next  
×
×
  • Create New...