Jump to content
rossnixon

My script runs too slow - any alternative?

Recommended Posts

rossnixon

My script works. It strips off the first 9 lines of each ASCII file (*.mif) in a folder.
But it is incredibly slow on large files. I previously used http://www.ecobyte.com/replacetext/ - but it won't work on Win 7 64-bit
Does anyone know of a way to speed up my code, or should I look for another program?

 

strip9.au3

Share this post


Link to post
Share on other sites
iamtheky

#Include <File.au3>
Dim $text

$FileList = _FileListToArray(@ScriptDir, "*.MIF", 1)
If @error = 4 Then
    MsgBox(0, "", "No MIF Files Found.")
    Exit
EndIf

For $i = 1 To $FileList[0]
    ProcessFile($FileList[$i])
Next
MsgBox(0, "", "Updated " & $i-1 & " MIF files.")

Func ProcessFile($filefromlist)

    $MaxCharsPerLine = 256
    $sFile = FileRead($filefromlist , $MaxCharsPerLine * 9)
    $aFile = stringsplit($sFile , @LF , 2)
    $hFile = FileOpen($filefromlist , 2)

      For $j = 0 To 8
         FileWriteLine($hFile , $aFile[$j] & @LF)
      Next

EndFunc   ;==>ProcessFile

 


,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
Zedna

Without posting your code you will get no help ...

EDIT: try this

Func ProcessFile($filefromlist)
    $text = FileRead($filefromlist)
    $i = StringInStr($text, @CRLF, 0, 9) ; position of 9th CRLF
    $text = StringMid($text, $i+2) ; skip first 9 lines, +2 to skip found 9th CRLF
      ;Write the new MIF file.
      $file = FileOpen($filefromlist,2)
      FileWrite($file, $text)
      FileClose($file)
EndFunc   ;==>ProcessFile

 

Edited by Zedna
  • Like 1

Share this post


Link to post
Share on other sites
iamtheky

zed, its attached...

ross, you can just use code tags and paste it in, that is how its usually done

Edited by boththose

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
rossnixon

Without posting your code you will get no help ...

EDIT: try this

Func ProcessFile($filefromlist)
    $text = FileRead($filefromlist)
    $i = StringInStr($text, @CRLF, 0, 9) ; position of 9th CRLF
    $text = StringMid($text, $i+2) ; skip first 9 lines, +2 to skip found 9th CRLF
      ;Write the new MIF file.
      $file = FileOpen($filefromlist,2)
      FileWrite($file, $text)
      FileClose($file)
EndFunc   ;==>ProcessFile

 

​Great! And runs fast!
I only had to change $i to $j, as I was already using $i
Thanks very much Zedna! :-)

Share this post


Link to post
Share on other sites
iamtheky

ahh i was backward, since you are already including file this would also do it

 

 

#Include <File.au3>
Dim $text

$FileList = _FileListToArray(@ScriptDir, "*.MIF", 1)
If @error = 4 Then
    MsgBox(0, "", "No MIF Files Found.")
    Exit
EndIf

For $i = 1 To $FileList[0]
    ProcessFile($FileList[$i])
Next
MsgBox(0, "", "Updated " & $i-1 & " MIF files.")

Func ProcessFile($filefromlist)

      for $i = 1 to 9
      _FileWriteToLine($filefromlist , $i , "" , 1)
      next

EndFunc   ;==>ProcessFile

 

 

Edited by boththose
back to functional

,-. .--. ________ .-. .-. ,---. ,-. .-. .-. .-.
|(| / /\ \ |\ /| |__ __||| | | || .-' | |/ / \ \_/ )/
(_) / /__\ \ |(\ / | )| | | `-' | | `-. | | / __ \ (_)
| | | __ | (_)\/ | (_) | | .-. | | .-' | | \ |__| ) (
| | | | |)| | \ / | | | | | |)| | `--. | |) \ | |
`-' |_| (_) | |\/| | `-' /( (_)/( __.' |((_)-' /(_|
'-' '-' (__) (__) (_) (__)

Share this post


Link to post
Share on other sites
jguinch

Here is a regexpreplace way, which could be faster :

Func ProcessFile($sFilename)
    Local $sContent = FileRead($sFilename)
    If @error Then Return SetError(1, 0, 0)
    
    $sContent = StringRegExpReplace($sContent, "^(?:\N*\R){1,9}", "")
    
    Local $hFileW = FileOpen($sFilename, 2)
    If $hFileW = -1 Then Return SetError(1, 0, 0)
    
    FileWrite($hFileW, $sContent)
    FileClose($hFileW)
    Return 1
EndFunc

 

Share this post


Link to post
Share on other sites
Zedna

@jquinch

StringRegExpReplace() in this case on such big files/texts will be definitely much slower than one simple StringInStr() + StringMid()

EDIT: Anyway thanks for nice example for RegExp pattern ...

Edited by Zedna

Share this post


Link to post
Share on other sites
UEZ

Try this:

Func FileStripFirstNLines($sFile, $iLines = 9)
    Local $hFile = FileOpen($sFile)
    If @error Then Return SetError(1, 0, 0)
    Local $sResult, $i
    For $i = 1 To $iLines
        $sResult &= FileReadLine($hFile, $i) & @CRLF
    Next
    FileClose($hFile)
    Return $sResult
EndFunc

For a 166 MB file it takes around 0.81 milli seconds to read first 9 lines.


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites
UEZ

Then I misunderstood op.

 

Sorry.


Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites
UEZ

Ok, here another version:

Func FileCutFirstNLines($sFile, $iLines = 9)
    Local $hFile = FileOpen($sFile)
    If @error Then Return SetError(1, 0, 0)
    Local $sResult, $i
    For $i = 1 To $iLines
        $sResult &= FileReadLine($hFile, $i) & @CRLF
    Next
    FileClose($hFile)
    Local $iLength = StringLen($sResult), $iFilesize = FileGetSize($sFile), $nBytes, $iDelta = $iFilesize - $iLength
    Local $tBuffer = DllStructCreate("byte[" & $iDelta & "]")
    $hFile = _WinAPI_CreateFile($sFile, 2, 2)
    _WinAPI_SetFilePointer($hFile, $iLength)
    _WinAPI_ReadFile($hFile, DllStructGetPtr($tBuffer), $iDelta, $nBytes)
    _WinAPI_CloseHandle($hFile)
    $hFile = FileOpen(StringTrimRight($sFile, 4) & "_new." & StringRight($sFile, 3) , 2)
    FileWrite($hFile, DllStructGetData($tBuffer, 1))
    FileClose($hFile)
    $tBuffer = 0
    Return 1
EndFunc

Takes around 1.7 seconds to create the new file (~170 MB).

 

The higher the $iLines value the slower the runtime.

Edited by UEZ

Please don't send me any personal message and ask for support! I will not reply!

Selection of finest graphical examples at Codepen.io

The own fart smells best!
Her 'sikim hıyar' diyene bir avuç tuz alıp koşma!
¯\_(ツ)_/¯  ٩(●̮̮̃•̃)۶ ٩(-̮̮̃-̃)۶ૐ

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.