Jump to content

File to and from 2D array


SnArF
 Share

Recommended Posts

I always use _FileWriteFromArray2D written by Universalist, but the script is slow and almost useless if the file contains more then 5000 rows.

Thats why I've made _FileFromArray2D and co-function _FileToArray2D.

If you have a file containing 30000 rows, _FileReadToArray2D takes about 1 hour to create the 2D Array

_FileToArray2D can do it in a few seconds.

There are some extra options in _FileToArray2D, a positive endrow number stops importing at fileline number in file, a negative value will import startrow + nr off rows.

The script needs some error handling, so feel free to post it here.

#include-Once

#include <File.au3>
#include <Array.au3>

; #INDEX# =======================================================================================================================
; Title .........: FileArray2D
; AutoIt Version : 3.2.10++
; Language ......: English
; Description ...: Functions for creating en reading 2D arrays.
; Author(s) .....: Frans (SnArF) Ordelman
; ===============================================================================================================================

; #NO_DOC_FUNCTION# =============================================================================================================
; Not working/documented/implemented at this time
; ===============================================================================================================================

; #CURRENT# =====================================================================================================================
;_FileToArray2D
;_FileFromArray2D
; ===============================================================================================================================

; #INTERNAL_USE_ONLY# ===========================================================================================================
; ===============================================================================================================================

; #FUNCTION# ====================================================================================================================
; Name...........: _FileToArray2D
; Description ...: Creates a 2D array from a file.
; Syntax.........: _FileToArray2D($aFile, ByRef $aArray, $aDelim = ";", $aStartRow = 1, $aEndRow = 0)
; Parameters ....: $aFile - File to read
; $aArray - 2D Array to create
; $aDelim - [Optional] String delimiter, default is ";"
; $aStartRow - [Optional] Row to start, default is 1
; $aEndRow = [Optional] Row to end, default is 0 (all)
; Return values .: Success - Index of last added item
; Failure - -1, sets @error
; Author ........: Frans (SnArF) Ordelman
; Modified.......:
; Remarks .......: $aEndRow positive stops importing at filenr in file,
; a negative value will import startrow + nr off rows
; Related .......: __FileFromArray2D
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _FileToArray2D($aFile, ByRef $aArray, $aDelim = ";", $aStartRow = 1, $aEndRow = 0)
Local $aRead, $aRows, $aColums, $aString
$aRead = FileOpen($aFile)
If $aStartRow < 1 Then $aStartRow = 1
$aRows = _FileCountLines($aFile) + 1 - $aStartRow
If $aEndRow >0 Then $aRows = $aRows - ($aRows - $aEndRow + $aStartRow -1)
If $aEndRow <0 Then $aRows = $aRows - ($aRows + $aEndRow)

$aColums = UBound(StringSplit(FileReadLine($aRead, $aStartRow), $aDelim))-1

FileSetPos($aRead, 0, $FILE_BEGIN)
FileReadLine($aRead,-1 + $aStartRow)

Local $aArrayTemp[$aRows + 1][$aColums]
$aArrayTemp[0][0] = $aRows

For $i = 1 to $aRows
$aString = StringSplit(FileReadLine($aRead), ";")
For $k = 0 to $aColums -1
$aArrayTemp[$i][$k] = $aString[$k+1]
Next
Next

FileClose($aRead)
$aArray = $aArrayTemp
EndFunc

; #FUNCTION# ====================================================================================================================
; Name...........: _FileFromArray2D
; Description ...: Creates a file from a 2D array.
; Syntax.........: _FileFromArray2d($aFile, $aArray, $aDelim = ";", $aWriteMode = 2, $aStartRow = 0, $aEndRow = -1, $aStartCol = 0, $aEndCol = -1)
; Parameters ....: $aFile - File to create
; $aArray - 2D Array to read
; $aDelim - [Optional] String delimiter, default is ";"
; $aWriteMode - [Optional] Mode to open the file in, default is 2
; $aStartRow - [Optional] Row to start, default is 0
; $aEndRow = [Optional] Row to end, default is -1 (all)
; $aStartCol = [Optional] Colum to start, default is 0
; $aEndCol = [Optional] Column to end, default is -1 (all)
; Return values .: Success - Index of last added item
; Failure - -1, sets @error
; Author ........: Frans (SnArF) Ordelman
; Modified.......:
; Remarks .......:
; Related .......: __FileToArray2D
; Link ..........:
; Example .......: No
; ===============================================================================================================================
Func _FileFromArray2d($aFile, $aArray, $aDelim = ";", $aWriteMode = 2, $aStartRow = 0, $aEndRow = -1, $aStartCol = 0, $aEndCol = -1)
Local $aWrite, $aUboundRow, $aUboundCol, $aLine, $aDelimNr

If Not IsArray($aArray) Then
SetError(1)
Return 0
EndIf

$aWrite = FileOpen($aFile, $aWriteMode)
If $aWrite = -1 Then
SetError(2)
Return 0
EndIf

$aUboundRow = UBound($aArray, 1)-1
$aUboundCol = UBound($aArray, 2)-1
$aDelimNr = StringLen($aDelim)

If $aEndRow > $aUboundRow Or $aEndRow = -1 Then $aEndRow = $aUboundRow
If $aEndCol > $aUboundCol Or $aEndCol = -1 Then $aEndCol = $aUboundCol

For $i = $aStartRow To $aEndRow
$aLine = ""
For $k = $aStartCol To $aEndCol
$aLine &= $aArray[$i][$k] & $aDelim
Next
$aLine = StringTrimRight($aLine, $aDelimNr )
FileWriteLine($aWrite, $aLine)
Next

FileClose($aWrite)
EndFunc

_filearray2d.au3

Example 2D Array.zip

Edited by SnArF

My scripts: _ConsoleWriteLog | _FileArray2D

 

 

 

Link to comment
Share on other sites

Would you mind providing an example of usage. Thanks.

Edit: Also I'm surprised you say this is fast considering it reads the file twice for determining the number of rows.

Edited by guinness

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

Guinness, I've uploaded 2 examples, one with the old UFD (not written by me), this one is very slow, it takes about 5 min to add 15000 records to the array, if you double the records it takes more than 1 hour.

My UDF takes less than a second to add the records to the array and has extra options. (start and end row)

New UDF.zip

Old UDF.zip

My scripts: _ConsoleWriteLog | _FileArray2D

 

 

 

Link to comment
Share on other sites

I'm surprised that OldUDF example even worked for you. The $Delim variable is set to -1 if you don't explicitly set it, so it's trying to split the line using "-1", which would NEVER work for your file. That is a terribly written function which I would never use or compare anything else against.

BTW, _FileWriteFromArray will write 2D arrays to a file.

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to comment
Share on other sites

The first version is slow for many reason, ReDim, FileReadLine to name but a few.

Slightly quicker:

#include <Array.au3>
#include <Constants.au3>
#include <File.au3>

Local $a2dArray = 0
Local $hTimer = TimerInit()

_FileReadToArray2D(@ScriptDir & '\example15000rows.csv', $a2dArray, Default) ; Default delim is ;.

MsgBox($MB_SYSTEMMODAL, '', TimerDiff($hTimer))

_ArrayDisplay($a2dArray)

Func _FileReadToArray2D($sFilePath, ByRef $aArray, $sDelimiter = Default) ; By guinness.
    $aArray = 0
    Local $aSRE = StringRegExp(FileRead($sFilePath) & @LF, '([^\r\n]*)(?:\r\n|\n|\r)(?:[\r\n]$)?', 3) ; By DXRW4E.
    If @error Then
        Return SetError(1, 0, 0)
    EndIf
    Local $iUBound = UBound($aSRE, 1)
    If $sDelimiter = Default Then
        $sDelimiter = ';'
    EndIf
    Local $aSplit = StringSplit($aSRE[0], $sDelimiter)
    If @error Then
        Return SetError(2, 0, 0)
    EndIf

    Local $aArrayTemp[$iUBound + 1][$aSplit[0]] = [[0]]
    For $i = 0 To $iUBound - 1
        $aSplit = StringSplit($aSRE[$i], $sDelimiter)
        If @error Then
            ContinueLoop
        EndIf

        $aArrayTemp[0][0] += 1
        For $j = 1 To $aSplit[0]
            $aArrayTemp[$aArrayTemp[0][0]][$j - 1] = $aSplit[$j]
        Next
    Next
    $aArray = $aArrayTemp
    $aArrayTemp = 0
EndFunc   ;==>_FileReadToArray2D
Edited by guinness

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

BTW, _FileWriteFromArray will write 2D arrays to a file.

True, though I would recommend the beta version which contains a couple of bug fixes.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

True, though I would recommend the beta version which contains a couple of bug fixes.

Yeah, I know but not every one has the beta or is willing to use an unfinished product. Besides, I rewrote that function for the Trak ticket. :) Edited by BrewManNH

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to comment
Share on other sites

Self working Example: (See post #5 for function.)

#include <Array.au3>
#include <FileConstants.au3>

Local $aArray = 0, $hTimer = 0, $sFilePath = @ScriptDir & '\TestDB.txt'

_Test_File($sFilePath, ';') ; Creates an example using ';'
$hTimer = TimerInit()
_FileReadToArray2D($sFilePath, $aArray)
ConsoleWrite(TimerDiff($hTimer) & @CRLF)
_ArrayDisplay($aArray)

_Test_File($sFilePath, ',') ; Creates an example using ','
$hTimer = TimerInit()
_FileReadToArray2D($sFilePath, $aArray, ',')
ConsoleWrite(TimerDiff($hTimer) & @CRLF)
_ArrayDisplay($aArray)

FileDelete($sFilePath)

Func _Test_File($sFilePath, $sDelimiter = '|', $iRows = 1000, $iColumns = 10)
    Local $hFileOpen, $sOutput = ''
    For $i = 1 To $iRows
        For $j = 1 To $iColumns
            $sOutput &= 'Example_' & $j & $sDelimiter
        Next
        $sOutput = StringTrimRight($sOutput, 1) & @CRLF
    Next
    Local $hFileOpen = FileOpen($sFilePath, $FO_OVERWRITE)
    If $hFileOpen = -1 Then
        Return False
    EndIf
    FileWrite($hFileOpen, $sOutput)
    Return FileClose($hFileOpen) = 1
EndFunc   ;==>_Test_File

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

Yeah, I know but not every one has the beta or is willing to use an unfinished product. Besides, I rewrote that function for the Trak ticket. :)

Yes, but the version in the beta doesn't resemble that code anymore. Also your re-write contains an old bug. Edited by guinness

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

The bug fix from the original?

If I posted any code, assume that code was written using the latest release version unless stated otherwise. Also, if it doesn't work on XP I can't help with that because I don't have access to XP, and I'm not going to.
Give a programmer the correct code and he can do his work for a day. Teach a programmer to debug and he can do his work for a lifetime - by Chirag Gude
How to ask questions the smart way!

I hereby grant any person the right to use any code I post, that I am the original author of, on the autoitscript.com forums, unless I've specifically stated otherwise in the code or the thread post. If you do use my code all I ask, as a courtesy, is to make note of where you got it from.

Back up and restore Windows user files _Array.au3 - Modified array functions that include support for 2D arrays.  -  ColorChooser - An add-on for SciTE that pops up a color dialog so you can select and paste a color code into a script.  -  Customizable Splashscreen GUI w/Progress Bar - Create a custom "splash screen" GUI with a progress bar and custom label.  -  _FileGetProperty - Retrieve the properties of a file  -  SciTE Toolbar - A toolbar demo for use with the SciTE editor  -  GUIRegisterMsg demo - Demo script to show how to use the Windows messages to interact with controls and your GUI.  -   Latin Square password generator

Link to comment
Share on other sites

True, though I would recommend the beta version which contains a couple of bug fixes.

_FileFromArray2D with extra options

Func _FileFromArray2d($aFile, $aArray, $aDelim = ";", $aWriteMode = 2, $aStartRow = 0, $aEndRow = -1, $aStartCol = 0, $aEndCol = -1)
Local $aWrite, $aUboundRow, $aUboundCol, $aLine, $aDelimNr

If Not IsArray($aArray) Then
SetError(1)
Return 0
EndIf

$aWrite = FileOpen($aFile, $aWriteMode)
If $aWrite = -1 Then
SetError(2)
Return 0
EndIf

$aUboundRow = UBound($aArray, 1)-1
$aUboundCol = UBound($aArray, 2)-1
$aDelimNr = StringLen($aDelim)

If $aEndRow > $aUboundRow Or $aEndRow = -1 Then $aEndRow = $aUboundRow
If $aEndCol > $aUboundCol Or $aEndCol = -1 Then $aEndCol = $aUboundCol

For $i = $aStartRow To $aEndRow
$aLine = ""
For $k = $aStartCol To $aEndCol
$aLine &= $aArray[$i][$k] & $aDelim
Next
$aLine = StringTrimRight($aLine, $aDelimNr )
FileWriteLine($aWrite, $aLine)
Next

FileClose($aWrite)
EndFunc

My scripts: _ConsoleWriteLog | _FileArray2D

 

 

 

Link to comment
Share on other sites

Yeah, I know but not every one has the beta or is willing to use an unfinished product.

Shame really, as I know from my experience that when people use your product stable/beta, you feel appreciated and want to update it. When that appreciation dwindles so does your passion for maintaining your product.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

_FileFromArray2D with extra options

This isn't taken from the beta version. It's more beneficial to write to the file only once instead of every line.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

  • 4 weeks later...
  • 6 months later...

This script is pretty nice! However, in my case I find that using AutoIt's _FileReadToArray() is a bit faster for me.

I was inspired by your script so I wrote my own function that converts 1D arrays to 2D arrays.
'?do=embed' frameborder='0' data-embedContent>>
 

My script combined with your _FileFromArray2D() works wonders on large data (and obviously small data too). I was able to cut down the time from 18 minutes or more (using AutoIt's built-in UDFs) down to mere seconds or less (using the above combo).

Kudos!  :thumbsup: 

 

 

Link to comment
Share on other sites

  • 1 year later...

Very nice! The "Row to start" feature is especially useful. Here's also a way to manipulate the created arrays.

But as you may know, CVS files may look like field1, "field, 2", field3. That is, commas are used as delimiters...unless they're inside quotes.

So can you update your script to support multiple delimiters?

Here's my way (based on this):

$sSpecialChr = chr(34) ; a new variable that supports quotes, etc.

    ;$aColums = UBound(StringSplit(FileReadLine($aRead, $aStartRow), $aDelim,1)) - 1
    $aColums_temp = StringRegExp(FileReadLine($aRead, $aStartRow), '(?:' & $sSpecialChr & '.*?' & $sSpecialChr & ')+|(?<=' & $aDelim & '|^)[^' & $aDelim & ']*(?=' & $aDelim & '|$)', 3)
    _ArrayInsert($aColums_temp, 0, ubound($aColums_temp))

...

    For $i = 1 To $aRows
        ;$aString = StringSplit(FileReadLine($aRead), $aDelim,1)
        $aString_temp = StringRegExp(FileReadLine($aRead), '(?:' & $sSpecialChr & '.*?' & $sSpecialChr & ')+|(?<=' & $aDelim & '|^)[^' & $aDelim & ']*(?=' & $aDelim & '|$)', 3)
        _ArrayInsert($aString_temp, 0, ubound($aString_temp))

I've also added a feature that writes to a specific line in the output file (with a new variable called $start_from_line):

Func _FileFromArray2d($aFile, $aArray, $aDelim = ";", $aWriteMode = 2, $start_from_line = 1, $aStartRow = 0, $aEndRow = 0, $aStartCol = 0, $aEndCol = -1)

...

    If $start_from_line>1 then
        filesetpos($aWrite, 0, $FILE_BEGIN)
        For $i = 1 To $start_from_line-1
            FileReadLine($aWrite)
        next
    endif

 

Edited by LWC
Link to comment
Share on other sites

I am sure this stuff is already available in the current UDF library function?

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

I haven't tested speed difference, but I can tell you the internal _FileReadToArray:

  1. Doesn't have a "start at row" feature.
  2. Fails with CSV files which file lines have different numbers of fields.
  3. Doesn't support CSV files that quote their commas.
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...