photonbuddy Posted May 30, 2007 Share Posted May 30, 2007 Hi All, I am using this bit of code in one of my programs. Currently testing it on a datafile with ~100,000 records, and it's pretty quick (~7 seconds), but my PC is fairly fast compared to what a lot of people have, so any speed improvements would be great. Each record consists of 3 fields, a filename, a boolean option, and a number. Func loaddata() Local $inline, $splits,$lc dim $aArray $dbhandle = FileOpen($dbname, 0) $aArray = StringSplit( StringStripCR(FileRead($dbhandle, FileGetSize($dbname))), @LF) FileClose($dbhandle) $count=$aArray[0]-2 Dim $mfile[$count+1][3],$favs[500],$favcount=-1 for $lc=0 to $count $splitst = StringSplit($aArray[$lc+1],"|") If $splitst[0] = 3 Then;==> Removing this test saved around .8 of a second. If $lc/2500 = Int($lc/2500) Then GUICtrlSetData($info1, "Loading Database ... " & $lc & " Files");==> Update the display every 2500 records EndIf $mfile[$lc][0] = $splitst[1] $mfile[$lc][1] = $splitst[2] If $mfile[$lc][1]="1" and $favcount<500 Then $favcount=$favcount+1 $favs[$favcount]=$mfile[$lc][0] EndIf $mfile[$lc][2] = $splitst[3] EndIf Next dim $aArray=0 EndFunc ;==>loaddata Any help much appreciated. Link to comment Share on other sites More sharing options...
randallc Posted May 30, 2007 Share Posted May 30, 2007 (edited) Hi, This is nearly twice as fast, but I note your favcount takes about 0.8 secs too? [Array2D.au3 in my sig uses vbs too] Best, randall expandcollapse popup#include<array2D.au3> Local $dbname1 = @ScriptDir & "\testdata.txt", $TotalTimerIntCheck $timerstamp1 = TimerInit() $ar_Answer1 = loaddata($dbname1) ConsoleWrite("Time " & Round(TimerDiff($timerstamp1)) & " ms" & @LF) $ar_Answer3 = $ar_Answer1 ReDim $ar_Answer3[399][UBound($ar_Answer3, 2) - 1] ;~ _ArrayDisplay("UBound($ar_Answer1)"&UBound($ar_Answer1)& ";Here are first 400; Time " & Round(TimerDiff($timerstamp1)) & " ms", 0) $timerstamp1 = TimerInit() $ar_Answer = loaddata2($dbname1) ConsoleWrite("Time " & Round(TimerDiff($timerstamp1)) & " ms" & @LF) ConsoleWrite("Time $TotalTimerIntCheck" & Round($TotalTimerIntCheck) & " ms" & @LF) $ar_Answer2 = $ar_Answer ReDim $ar_Answer2[399][UBound($ar_Answer2, 2) - 1] _ArrayDisplay($ar_Answer2,"UBound($ar_Answer)"&UBound($ar_Answer)& ";Here are first 400; Time " & Round(TimerDiff($timerstamp1)) & " ms", 0) Func loaddata2($dbname) local $aArray2D[1],$favs[500],$TotalTimerIntCheck ,$favcount = -1,$aArray = StringSplit(FileRead($dbname), @crLF,1) $aArray2D = _Array2DCreateFromArraySt ($aArray, 2);_Array2DCreateFromArraySt($ar1_Array_Strings, $i_Base = 0, $i_Transpose = 0, $s_Separator = "|") For $lc = 0 To UBound($aArray2D) - 1 If $aArray2D[$lc][1] = "1" And $favcount < 500 Then $favcount += 1 $favs[$favcount] = $aArray2D[$lc][0] EndIf Next Return $aArray2D EndFunc ;==>loaddata2 Func loaddata($dbname) Local $inline, $splits, $lc,$aArray $dbhandle = FileOpen($dbname, 0) $aArray = StringSplit(FileRead($dbname), @crLF,1) ;~ $aArray = StringSplit(StringStripCR(FileRead($dbhandle, FileGetSize($dbname))), @LF) FileClose($dbhandle) $count = $aArray[0] - 2 Dim $mfile[$count + 1][3], $favs[500], $favcount = -1 For $lc = 0 To $count $splitst = StringSplit($aArray[$lc + 1], "|") If $splitst[0] = 3 Then ;~ $timerstamp1 = TimerInit() ;~ If $lc/2500 = Int($lc/2500) Then;==> Removing this test saved around .8 of a second. ;~ GUICtrlSetData($info1, "Loading Database ... " & $lc & " Files");==> Update the display every 2500 records ;~ EndIf ;~ $TotalTimerIntCheck+=TimerDiff($timerstamp1) $mfile[$lc][0] = $splitst[1] $mfile[$lc][1] = $splitst[2] If $mfile[$lc][1] = "1" And $favcount < 500 Then $favcount += 1 $favs[$favcount] = $mfile[$lc][0] EndIf $mfile[$lc][2] = $splitst[3] EndIf Next Dim $aArray = 0 Return $mfile EndFunc ;==>loaddata ;~ local $ar_CreateFile[100001],$dbname1=@ScriptDir&"\testdata.txt" ;~ for $a= 1 to 100000 ;create database ;~ $ar_CreateFile[$a]="c:\programs\searchengine\test.au3|0|"&$a ;~ Next ;~ _FileWriteFromArray($dbname1,$ar_CreateFile)oÝ÷ Øô¦z+æÍ÷jëh×6$aArray = StringSplit(FileRead($dbname), @crLF,1) $favcount += 1 Edited May 30, 2007 by randallc ExcelCOM... AccessCom.. Word2... FileListToArrayNew...SearchMiner... Regexps...SQL...Explorer...Array2D.. _GUIListView...array problem...APITailRW Link to comment Share on other sites More sharing options...
photonbuddy Posted May 30, 2007 Author Share Posted May 30, 2007 Changing to the Array2D functions certainly does speed things up. I think that's probably getting to a point where if it's taking too long, you need to buy a faster PC. Thanks so much for being very helpful. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now