Jump to content

Search the Community

Showing results for tags '_mycsv2dcreator'.

  • Search By Tags

    Type tags separated by commas.
  • Search By Author

Content Type


Forums

  • General
    • Announcements and Site News
    • Administration
  • AutoIt v3
    • AutoIt Help and Support
    • AutoIt Technical Discussion
    • AutoIt Example Scripts
  • Scripting and Development
    • Developer General Discussion
    • Language Specific Discussion
  • IT Administration
    • Operating System Deployment
    • Windows Client
    • Windows Server
    • Office

Categories

  • AutoIt Team
    • Beta
    • MVP
  • AutoIt
    • Automation
    • Databases and web connections
    • Data compression
    • Encryption and hash
    • Games
    • GUI Additions
    • Hardware
    • Information gathering
    • Internet protocol suite
    • Maths
    • Media
    • PDF
    • Security
    • Social Media and other Website API
    • Windows
  • Scripting and Development
  • IT Administration
    • Operating System Deployment
    • Windows Client
    • Windows Server
    • Office

Categories

  • Forum FAQ
  • AutoIt

Calendars

  • Community Calendar

Find results in...

Find results that contain...


Date Created

  • Start

    End


Last Updated

  • Start

    End


Filter by number of...

Joined

  • Start

    End


Group


Member Title


Location


WWW


Interests

Found 1 result

  1. Good Morning All, I have an application that takes csv files - sorts, finds, and tries to make it's own individual csv files... Everything runs smooth until I get to the "_myCSV2DCreator" on extremely large CSV files... like 200+ MB with 100,000 + lines. I get an "out of memory" error from AutoIT when I convert the 2D array to a CSV file using this function "_myCSV2DCreator". I get an "out of memory" error when running the _ArrayInsert($aNewArray, "0", $columnheaders, "", ","). I believe it takes whatever array is in memory and doubles it's size as it writes just one row of data. Any ideas on how to get around this? I'm stumped... Func GEN_SeparateOutCSV03() SendAndLog("GEN_SeparateOutCSV03", $tempzipdir & '\' & $LogFileName01, True) SplashTextOn($ProgramTitle, 'Separating CSV files from data...', 400, 60, (@DesktopWidth / 2) - 200, 10, "", "") Sleep ($sleeptime) $array01 = $twoDarray $aUniqueHostname = _ArrayUnique ($array01, 1) ;_ArrayDisplay ($aUniqueHostname, "UniqueHostname ") For $i01 = Ubound($aUniqueHostname) - 1 to 0 Step - 1 For $j01 = Ubound($array01) - 1 to 0 Step - 1 If $array01[$j01][1] == $aUniqueHostname[$i01] and StringRegExp($array01[$j01][5], "MY_VALUE.txt") then ;MsgBox(0, "Computer and MY_VALUE.txt", $aUniqueHostname[$i01] & " : " & $array01[$j01][5]) $FileName01 = $tempzipdir & "\" & $array01[$j01][3] & "_" & $aUniqueHostname[$i01] & "_" & $array01[$j01][2] & ".csv" ; MsgBox(0, "File Name", $FileName01) Local $avResult = _ArrayFindAll($array01, $aUniqueHostname[$i01], 0, 0, 0, 0, 1) ;_ArrayDisplay($avResult, "$avResult") Local $aNewArray = "" Local $aNewArray[UBound($avResult)][UBound($array01, 2)] SplashTextOn($ProgramTitle, 'Loop - Array search for unique hostname', 400, 60, (@DesktopWidth / 2) - 200, 10, "", "") Sleep ($sleeptime) For $i = 0 To UBound($avResult) -1 ; Loop through the returned index numbers. For $j = 0 To UBound($array01, 2) -1 ; Loop through each of the columns. $aNewArray[$i][$j] = $array01 [$avResult[$i]] [$j] ; Populate the new array. Next Next SplashTextOn($ProgramTitle, 'Loop - Column header modification', 400, 60, (@DesktopWidth / 2) - 200, 10, "", "") Sleep ($sleeptime) ;_ArrayDisplay($twoDarray, "Removed 'User-defined Rules:' in column 3") ;_ArrayInsert($aNewArray, "0", $columnheaders, "", ",") _ArrayInsert($aNewArray, "0", $columnheaders, "", ",") ;_ArrayDisplay($aNewArray, "Inserted 0") ;MsgBox (0, "Out of the loop", "Out of the loop - File Write From Array") _myCSV2DCreator($tempzipdir & "\beta_" & $array01[$j01][3] & "_" & $aUniqueHostname[$i01] & "_" & $array01[$j01][2] & ".csv", $aNewArray, True) ;_FileWriteFromArray($FileName01, $aNewArray) ExitLoop EndIf Next Next SplashTextOn($ProgramTitle, 'Please wait a moment...', 400, 60, (@DesktopWidth / 2) - 200, 10, "", "") Sleep ($sleeptime) SplashOff() SendAndLog("GEN_SeparateOutCSV03 - Okay", $tempzipdir & '\' & $LogFileName01, True) MsgBox(262192, $ProgramTitle, "All files stored here:" & @CR & @CR & $tempzipdir) SendAndLog("Final Message Box - Exit Okay", $tempzipdir & '\' & $LogFileName01, True) Exit EndFunc Func _myCSV2DCreator($hFile, $avArray, $bEraseCreate = True) SplashTextOn($ProgramTitle, '2D to CSV file create', 400, 60, (@DesktopWidth / 2) - 200, 10, "", "") Sleep ($sleeptime) If $bEraseCreate Then FileClose(FileOpen($hFile, 2)) Local $sHoldString = "" For $x = 0 To UBound($avArray) - 1 For $i = 0 To UBound($avArray, 2) - 1 $sHoldString &= $avArray[$x][$i] & "," Next $sHoldString &= @CRLF Next Return FileWrite($hFile, StringTrimRight($sHoldString, 3)) EndFunc (out of memory) Is there a better method / solution that I should be using? Thanks Everyone!
×
×
  • Create New...