New FAST line counter

[SlimShady]

I adjusted my version to the latest AutoIt to not leave carriage returns behind.

Func _NFileCountLines($sFile)
   Local $AArray
   Local $Content
   Local $FileSize
   Local $CountLines
   If Not FileExists($sFile) Then
      $CountLines = 0
   Else
      $FileSize = FileGetSize($sFile)
      $Content = FileRead($sFile, $FileSize)
      
      $AArray = StringSplit($Content, @CRLF, 1)
      $Content = 0
      $CountLines = $AArray[0]
   EndIf
   Return $CountLines
EndFunc

Edited December 30, 2010 by Jos

[tylo]

Hehe, check out my contribution - at least 10 times faster than any of the others!

JdeB, could you try it with your test file?

Ps: File must have CRLF line-ends.

expandcollapse popup

$TestFile = $CmdLine[1]

$StartTime = TimerInit()
$CountLines_3 = _NFileCountLines($TestFile) & " lines in "
$Test3_Time = Round(TimerDiff($StartTime) / 1000, 3) & " seconds"
$StartTime = TimerInit()
$CountLines_4 = _TylosFileCountLines($TestFile) & " lines in "
$Test4_Time = Round(TimerDiff($StartTime) / 1000, 3) & " seconds"
MsgBox(64, "Debug Test", "_NFileCountLines counted " & $CountLines_3 & $Test3_Time & @CRLF _
                       & "_TylosFileCountLines counted " & $CountLines_4 & $Test4_Time)
Exit
                       
Func _NFileCountLines($sFile)
  Local $AArray
  Local $Content
  Local $FileSize
  Local $CountLines
  If Not FileExists($sFile) Then
     $CountLines = 0
  Else
     $FileSize = FileGetSize($sFile)
     $Content = FileRead($sFile, $FileSize)    
     $AArray = StringSplit($Content, @CRLF, 1)
     $Content = 0
     $CountLines = $AArray[0]
  EndIf
  Return $CountLines
EndFunc


Func _TylosFileCountLines($file)
   If Not FileExists($file) Then
     SetError(1)
     Return 0
   EndIf
   Local $n = FileGetSize($file)
   Return $n - StringLen(StringStripCR(FileRead($file, $n))) + 1
EndFunc

Edited December 30, 2010 by Jos

[tylo]

Actually, my suggestion will count single @CR as lines, so it's not perfect. However, the following counts @LF's which is better (works also on UNIX files):

I tested a file with about 340000 lines (each about 9 chars long).

Jdeb's used about 1.77 seconds, whereas this used only 0.11 seconds:

Func _TylosFileCountLines($file)
  If Not FileExists($file) Then
    SetError(1)
    Return 0
  EndIf
  Local $n = FileGetSize($file)
  Return StringLen(StringAddCR(FileRead($file, $n))) - $n + 1
EndFunc

Edited December 30, 2010 by Jos

[Jos]

Hehe, check out my contribution - at least 10 times faster than any of the others!

JdeB, could you try it with your test file?

Ps: File must have CRLF line-ends.

We have a new winner !!!

Tylo's LineCount: counted 10000 lines in 0.843 seconds

_FileCountLines: counted 10000 lines in 4.034 seconds

_NFileCountLines: counted 10000 lines in 3.149 seconds

Internal record counter: counted 10000 lines in 1.024 seconds

Clever solution !

[SlimShady]

Awesome!

[Chris_1013]

Tylo, it's great if yours works really quick, can you explain what it does, cos I can't understand it!

[SlimShady]

Tylo, it's great if yours works really quick, can you explain what it does, cos I can't understand it!

<{POST_SNAPBACK}>

I commented a little. I hope you'll understand now.

Func _TylosFileCountLines($file)
   If Not FileExists($file) Then
      SetError(1)
      Return 0
   EndIf
   Local $n = FileGetSize($file)
  ;1. File is read completely
  ;2. StringAddCR adds a carriage return to every line feed. Function: for every new line, a character is added.
  ;3. StringLen counts how many characters; There's a new total.
  ;4. From the new total the original file size is subtracted.
  ;5. A number of extra line feeds is left behind and that number will be returned.
  ;I'm not sure why the number 1 is added to the total...
   Return StringLen(StringAddCR(FileRead($file, $n))) - $n + 1
EndFunc

Edited December 30, 2010 by Jos

[tylo]

Thanks Here's the new library function. I think it is correct only to add 1 if the last char is different from @LF. Meaning, a trailing @LF should not count as a new line.

; Returns number of lines (separated by @LF's) in a file.
; If file could not be opened, return 0 and @error = 1
; How: Adds a @CR before each @LF, and computes num of @CR's that was added.
; Note: It does not count a final @LF as a line.

Func _FileCountLines( $sFilePath )
   Local $N = FileGetSize($sFilePath) - 1
   If @error Or $N = -1 Then Return 0
   Return StringLen(StringAddCR(FileRead($sFilePath, $N))) - $N + 1
EndFunc

Edited December 30, 2010 by Jos

[Jon]

I'll bet that tylo used his knowledge of how autoit works interally....it's a whole different sort of cheating

[tylo]

me? no. i no no-ting

[Josbe]

Thanks    Here's the new library function. I think it is correct only to add 1 if the last char is different from @LF. Meaning, a trailing @LF should not count as a new line.

; Returns number of lines (separated by @LF's) in a file.
; If file could not be opened, return 0 and @error = 1
; How: Adds a @CR before each @LF, and computes num of @CR's that was added.
; Note: It does not count a final @LF as a line.

Func _FileCountLines( $sFilePath )
   Local $N = FileGetSize($sFilePath) - 1
   If @error Or $N = -1 Then Return 0
   Return StringLen(StringAddCR(FileRead($sFilePath, $N))) - $N + 1
EndFunc

<{POST_SNAPBACK}>

Very good.

@tylo: You left obsolete my only one(_FileCountLines) contributed to the AutoIt's Library.

I am glad that there are better alternatives.

[Guest Guidosoft]

Very good.

@tylo: You left obsolete my only one(_FileCountLines) contributed to the AutoIt's Library.

I am glad that there are better alternatives. 

<{POST_SNAPBACK}>

So how does it work?

[Jos]

So how does it work?

<{POST_SNAPBACK}>

as documented above:

; How: Adds a @CR before each @LF, and computes num of @CR's that was added.

[Guest Guidosoft]

as documented above:

<{POST_SNAPBACK}>

That some clever (beeeeeeeep).

[zcoacoaz]

i can make clever beeps to but i usually am to lazy

[Guest Guidosoft]

i can make clever beeps to but i usually am to lazy

<{POST_SNAPBACK}>

lol.

So whats new?

How are you?

How do you do?

[ALFJ]

Great code here, I know the post is old but it's still a great function

Getting the line count for me on a list of 151671 names was almost 2 seconds(without this function), but with this Function it's less than a second only 0.334

I always like to tweak my codes as much as possible, I don't like the lazy coders out there who produce programs that eat CPU & Memory! If a program does the same thing but eats less resources or takes for ever to start, guess which one I will pick

Edited December 30, 2010 by ALFJ

[GEOSoft]

I guess my measly .023 seconds for 10,000 lines or .21 seconds for 88,711 lines is out of the running here.

SRE's people, SRE's

[Jos]

@George,

just for clarity, this thread was created waaayyyyyyy before SRE was implemented in AutoIt3.

Having said that, we should probably look at the current UDF's for improvements using SRE.

Jos

[GEOSoft]

@George,

just for clarity, this thread was created waaayyyyyyy before SRE was implemented in AutoIt3.

Having said that, we should probably look at the current UDF's for improvements using SRE.

Jos

Agreed Jos. That may well become an extension for the current help file documentation group once it's all caught up to date.

By the way, The function is here and it's _File_CountLines2(). I haven't made the changes to make it work with a string input like I did with my original _File_CountLines() which is in the same UDF. It does have the advantage of being able to ignore blank lines though which is something I didn't implement on the original.

Edited December 30, 2010 by GEOSoft