I am currently writing a program that parses a massive table from a website, and need a way to add a progress bar while parsing.
I am currently using the function _IETableWriteToArray($oObj, True) to parse the array. I need the progress bar to update as the table is parsed, not just at the end of the parsing.
Any help at all would be very much appreciated!
*EDIT --> The array I am left with after parsing is $array[0-50000]
So this may be more of a challenge of effective programming then specific to AutoIT but I want to solve this problem with AutoIT so i'm putting it here. (If someone has a better language to solve with I'm all ears)
So the task I'm trying to achieve is that I have multiple .CSV files that have: year, month, day, hour, value. I need to be able to sum up all the values that have the same date/time, then find which date and time has the maximum value.
The problem is that each file may or may not have same amount of days/hours as the rest. So I need to devise a way to handle this.
File A File B File C 2018 1 1 1:00 10 2018 1 1 2:00 10 2018 1 1 1:00 10 2018 1 1 2:00 12 2018 1 1 3:00 12 2018 1 2 1:00 12 2018 1 1 3:00 14 2018 1 1 4:00 14 2018 2 1 1:00 16 2018 2 1 1:00 16 2018 2 1 1:00 16
Answer I want to be spit out is Feb 1st 2018 at 2:00 with value of 48
So far I've got code to store all .CSV files to an array, then a loop to go through each csv, but not sure how to effectively manipulate the data. Keep in mind each file has over 7000 time entry points.
If anyone can solve this that would be pretty awesome!
#include <Array.au3> #include <File.au3> #include <MsgBoxConstants.au3> RefineData() Func RefineData() Local $i, $filenum, $file, $csvArray, $FilePath = @ScriptDir $fileList = _FileListToArrayRec($FilePath, "*.csv", 1) ;Create and array of all .csv files within folder Local $chkArray[UBound($fileList)] ;=====Loop through the .csv files within the folder====== For $filenum = 1 To UBound($fileList) - 1 Step 1 $file = $fileList[$filenum] $sFilePath = $FilePath & "\" & $file ;=====Create array based on csv file===== _FileReadToArray($sFilePath, $csvArray, $FRTA_NOCOUNT, ",") ;#### Operations here ###### next msgbox(0,"", "Date: " & $date_of_max & "Value: " & $maxVal );display solution endfunc
Associative Array Functions
I've seen a couple of UDFs for this on the forum. One of them I quite like. But it's still nearly not as good as this method, IMHO.
I don't recall if I discovered the "Scripting.Dictionary" COM object myself or if I got the original base code from somewhere online. I have recently searched the web (and here) hard for any AutoIt references to this, other than my own over the years I've been using this (in ffe, etc..), and I can find nothing, so I dunno. If anyone does, I'd love to give credit where it's due; this is some cute stuff! It could actually be all my own work! lol
At any rate, it's too useful to not have posted somewhere at autoitscript.com, so I've put together a wee demo.
For those who haven't heard of the COM "Scripting.Dictionary"..
If you've ever coded in Perl or PHP (and many other languages), you know how useful associative arrays are. Basically, rather than having to iterate through an array to discover it's values, with an associative array you simply pluck values out by their key "names".
I've added a few functions over the years, tweaked and tuned, and this now represent pretty much everything you need to easily work with associative arrays in AutoIt. En-joy!
The main selling point of this approach is its simplicity and weight. I mean, look at how much code it takes to work with associative arrays! The demo is bigger than all the functions put together! The other selling point is that we are using Windows' built-in COM object functions which are at least theoretically, fast and robust.
I've used it many times without issues, anyhow, here goes..
; Associative arrays in AutoIt? Hells yeah! ; Initialize your array ... global $oMyError = ObjEvent("AutoIt.Error", "AAError") ; Initialize a COM error handler ; first example, simple. global $simple AAInit($simple) AAAdd($simple, "John", "Baptist") AAAdd($simple, "Mary", "Lady Of The Night") AAAdd($simple, "Trump", "Silly Man-Child") AAList($simple) debug("It is said that Trump is a " & AAGetItem($simple, "Trump") & ".", @ScriptLineNumber);debug debug("") ; slightly more interesting.. $ini_path = "AA_Test.ini" ; Put this prefs section in your ini file.. ; [test] ; foo=foo value ; foo2=foo2 value ; bar=bar value ; bar2=bar2 value global $associative_array AAInit($associative_array) ; We are going to convert this 2D array into a cute associative array where we ; can access the values by simply using their respective key names.. $test_array = IniReadSection($ini_path, "test") for $z = 1 to 2 ; do it twice, to show that the items are *really* there! for $i = 1 to $test_array $key_name = $test_array[$i] debug("Adding '" & $key_name & "'..");debug ; key already exists in "$associative_array", use the pre-determined value.. if AAExists($associative_array, $key_name) then $this_value = AAGetItem($associative_array, $key_name) debug("key_name ALREADY EXISTS! : =>" & $key_name & "<=" , @ScriptLineNumber);debug else $this_value = $test_array[$i] ; store left=right value pair in AA if $this_value then AAAdd($associative_array, $key_name, $this_value) endif endif next next debug(@CRLF & "Array Count: =>" & AACount($associative_array) & "<=" , @ScriptLineNumber);debug AAList($associative_array) debug(@CRLF & "Removing 'foo'..");debug AARemove($associative_array, "foo") debug(@CRLF & "Array Count: =>" & AACount($associative_array) & "<=" , @ScriptLineNumber);debug AAList($associative_array) debug(@CRLF & "Removing 'bar'..");debug AARemove($associative_array, "bar") debug(@CRLF & "Array Count: =>" & AACount($associative_array) & "<=" , @ScriptLineNumber);debug AAList($associative_array) quit() func quit() AAWipe($associative_array) AAWipe($simple) endfunc ;; Begin AA Functions func AAInit(ByRef $dict_obj) $dict_obj = ObjCreate("Scripting.Dictionary") endfunc ; Adds a key and item pair to a Dictionary object.. func AAAdd(ByRef $dict_obj, $key, $val) $dict_obj.Add($key, $val) If @error Then return SetError(1, 1, -1) endfunc ; Removes a key and item pair from a Dictionary object.. func AARemove(ByRef $dict_obj, $key) $dict_obj.Remove($key) If @error Then return SetError(1, 1, -1) endfunc ; Returns true if a specified key exists in the associative array, false if not.. func AAExists(ByRef $dict_obj, $key) return $dict_obj.Exists($key) endfunc ; Returns a value for a specified key name in the associative array.. func AAGetItem(ByRef $dict_obj, $key) return $dict_obj.Item($key) endfunc ; Returns the total number of keys in the array.. func AACount(ByRef $dict_obj) return $dict_obj.Count endfunc ; List all the "Key" > "Item" pairs in the array.. func AAList(ByRef $dict_obj) debug("AAList: =>", @ScriptLineNumber);debug local $k = $dict_obj.Keys ; Get the keys ; local $a = $dict_obj.Items ; Get the items for $i = 0 to AACount($dict_obj) -1 ; Iterate the array debug($k[$i] & " ==> " & AAGetItem($dict_obj, $k[$i])) next endfunc ; Wipe the array, obviously. func AAWipe(ByRef $dict_obj) $dict_obj.RemoveAll() endfunc ; Oh oh! func AAError() Local $err = $oMyError.number If $err = 0 Then $err = -1 SetError($err) ; to check for after this function returns endfunc ;; End AA Functions. ; debug() (trimmed-down version) ; ; provides quick debug report in your console.. func debug($d_string, $ln=false) local $pre ; For Jump-to-Line in Notepad++ if $ln then $pre = "(" & $ln & ") " & @Tab ConsoleWrite($pre & $d_string & @CRLF) endfunc
I'm trying to make a program that can look at a folder directory, find all the CSV files, and then add the data from CSV's to an array.
The problem I seem to be running into is on the 2nd iteration (2nd file) when the script will not create an array. Could someone please help? Thanks in advance
#include <Array.au3> #include <File.au3> #include <MsgBoxConstants.au3> #include <Excel.au3> #include <MsgBoxConstants.au3> Global $MasterArray RefineData() Func RefineData() Local $i, $filenum, $file, $csvArray, $sFilePath = @ScriptDir $fileList = _FileListToArrayRec($sFilePath, "*.csv", 1) ;Create and array of all .csv files within folder ;=====Loop through the .csv files within the folder====== For $filenum = 1 To UBound($fileList) - 1 Step 1 $file = $fileList[$filenum] $sFilePath = $sFilePath & "\" & $file ;=====Create array based on csv file===== _FileReadToArray($sFilePath, $csvArray, $FRTA_NOCOUNT, ",") _ArrayDisplay($csvArray,"File: " & $filenum) If $filenum = 1 Then $MasterArray = $csvArray _ArrayDisplay($MasterArray, "Master") Else $MasterArray = _ArrayColInsert($MasterArray, UBound($MasterArray)) ;want column added at end For $i = 0 To UBound($MasterArray)-1 Step 1 $MasterArray[$i][UBound($MasterArray) - 1] = $csvArray[$i] Next _ArrayDisplay($MasterArray, "Master") EndIf Next EndFunc ;==>RefineData
How do I work with 2D arrays. I've tried this but get errors.
Local $aTest = [[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]]
;$aTest = [10,11,12] ; Error at 
Local $sTest = ""
For $i = 0 To UBound($aTest)-1
Local $aExtract = _ArrayExtract($aTest,$i,$i)
$sTest = $sTest & MyTest($aExtract)
; Error at v $aTemp
Return String($aTemp) & " - " & String($aTemp) & " - " & String($aTemp) & @CRLF