Sign in to follow this  
Followers 0
kor

need some help reading text files and averaging numbers

15 posts in this topic

#1 ·  Posted (edited)

I am needing some help to get started with a new script that will save me TONS of time every month. Below I have included 2 documents. Both documents are the result of an automated scan of the particular folders, and their folder size is saved into a text documents with the name of the folder as the file name. As you can see from both examples, they are all just a little different, but their total size report at the bottom of each scan is always in the same format. Here are the main points that I need the script to preform.

Read each text file and find each final scan total amount

Round each final scan total amount to the next highest GB unless GB equals exactly 0. (18.0 equals 18, but 18.2 equals 19)

Take each final scan total amount after rounding and average the numbers

Create a popup window with a list of all the scanned documents and their final averaged number.

I'm not asking for everyone to do anything FOR me, but based on my requirements above can you guys start to point me to the commands and procedures that I will need to look at to accomplish each step. Thank you.

I'd also like to say that these text files (even the same ones) can change week by week. This first example might only have 7 folders the first week, but might have 9 folders the second week.. so I'm unsure how to code that stuff with the filereadline. I would need to specifically look for the end of the scan ... IE the ---------------- with the number below and above the second closing -------------------.

DirSize report, created at 5/3/2009 5:56:19 AM

Total + subs    Folder name  
--------------  ----------------------------------------
       18.9 GB  F:\test1  
      227.8 MB  |--folder1 ...  
      522.9 MB  |--folder2 ...  
        3.1 GB  |--folder3 ...  
      869.4 MB  |--folder4 ...  
        8.5 GB  |--folder5 ...  
      250.1 MB  |--folder6 ...  
        5.4 GB  +--folder7 ...  
--------------  ----------------------------------------
       18.9 GB  Cluster size: 4 KB  Wastage: 469.2 MB
--------------  ----------------------------------------

DirSize report, created at 5/10/2009 6:58:44 AM

Total + subs    Folder name  
--------------  ----------------------------------------
       19.5 GB  F:\test1  
      227.1 MB  |--folder1 ...  
      518.4 MB  |--folder2 ...  
        3.1 GB  |--folder3 ...  
      876.4 MB  |--folder4 ...  
        9.1 GB  |--folder5 ...  
      250.1 MB  |--folder6 ...  
        5.4 GB  +--folder7 ...  
--------------  ----------------------------------------
       19.5 GB  Cluster size: 4 KB  Wastage: 471.1 MB
--------------  ----------------------------------------

DirSize report, created at 5/3/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
    3,021.0 GB  E:\test2  
    3,020.8 GB  |--files ...  
      178.1 MB  +--log ...  
--------------  ----------------------------------------
    3,021.0 GB  Cluster size: 4 KB  Wastage: 43.0 GB
--------------  ----------------------------------------

DirSize report, created at 5/10/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
    3,036.2 GB  E:\test2  
    3,036.0 GB  |--files ...  
      196.1 MB  +--log ...  
--------------  ----------------------------------------
    3,036.2 GB  Cluster size: 4 KB  Wastage: 43.0 GB
--------------  ----------------------------------------
Edited by kor

Share this post


Link to post
Share on other sites



$FileToOpen = ''; replace it with the path of the file to open
$FileOpen = FileOpen($FileToOpen, 0)
$String = FileRead($FileOpen)
$Sizes = StringRegExp($string, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
Dim $Average, $ArrSize = UBound($Sizes)-1
;~ _ArrayDisplay($Sizes)
For $i = 0 To $ArrSize
    $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
    $Sizes[$i] = Ceiling(Number($Sizes[$i]))
    $Average += Ceiling($Sizes[$i]/$ArrSize)
    $Sizes[$i] &= ' GB'
Next
_ArrayDisplay($Sizes)
MsgBox(0, '', $Average&' GB')

it's up to you now to modify it to suits your need

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

Thank you very much, it works great but maybe you can point me to what commands I need to use to have it read multiple files. I added the text file and it works, but I want to scan all txt files in a directory and display the final numbers from all of them. I tried doing

$FileToOpen = 'C:\*.txt'

But it won't read that way.

I've also noticed that the final msgbox doesn't average the figures, it seems to only add them together.

EDIT: well, it doesn't seem to be adding together files with more than 2 scans, but it's not calculating the average correctly. Here is another sample file.

DirSize report, created at 4/5/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
      109.1 GB  D:\test3  
        2.0 GB  |--folder1 ...  
        2.5 MB  |--folder2 ...  
       28.9 MB  |--folder3 ...  
        120 KB  |--folder4 ...  
        1.2 GB  |--folder5 ...  
      988.8 MB  |--folder6 ...  
       21.2 MB  |--folder7 ...  
        920 KB  |--folder8 ...  
      240.5 MB  |--folder9 ...  
       84.3 GB  |--folder10 ...  
       26.8 MB  |--folder11 ...  
       47.0 MB  |--folder12 ...  
       73.3 MB  |--folder13 ...  
       16.0 GB  |--folder14 ...  
        2.7 GB  |--folder15 ...  
      518.6 MB  |--folder16 ...  
        444 KB  |--folder17 ...  
        428 KB  |--folder18 ...  
       61.4 MB  |--folder19 ...  
      892.6 MB  +--folder20 ...  
--------------  ----------------------------------------
      109.1 GB  Cluster size: 4 KB  Wastage: 3.9 GB
--------------  ----------------------------------------

DirSize report, created at 4/12/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
      114.1 GB  D:\test3  
        2.0 GB  |--folder1 ...  
        2.5 MB  |--folder2 ...  
       29.0 MB  |--folder3 ...  
        120 KB  |--folder4 ...  
        1.3 GB  |--folder5 ...  
      988.8 MB  |--folder6 ...  
       21.2 MB  |--folder7 ...  
        928 KB  |--folder8 ...  
      240.5 MB  |--folder9 ...  
       84.3 GB  |--folder10 ...  
       28.3 MB  |--folder11 ...  
       47.0 MB  |--folder12 ...  
        4.2 GB  |--folder13 ...  
       16.9 GB  |--folder14 ...  
        2.7 GB  |--folder15 ...  
      545.7 MB  |--folder16 ...  
        456 KB  |--folder17 ...  
        456 KB  |--folder18 ...  
       60.9 MB  |--folder19 ...  
      892.6 MB  +--folder20 ...  
--------------  ----------------------------------------
      114.1 GB  Cluster size: 4 KB  Wastage: 3.9 GB
--------------  ----------------------------------------

DirSize report, created at 4/19/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
      114.3 GB  D:\test3  
        2.0 GB  |--folder1 ...  
        2.5 MB  |--folder2 ...  
       29.0 MB  |--folder3 ...  
        120 KB  |--folder4 ...  
        1.3 GB  |--folder5 ...  
      988.8 MB  |--folder6 ...  
       21.2 MB  |--folder7 ...  
        924 KB  |--folder8 ...  
      240.5 MB  |--folder9 ...  
       84.3 GB  |--folder10 ...  
        3.2 MB  |--folder11 ...  
       47.0 MB  |--folder12 ...  
        5.0 GB  |--folder13 ...  
       16.2 GB  |--folder14 ...  
        2.7 GB  |--folder15 ...  
      543.0 MB  |--folder16 ...  
       16.5 MB  |--folder17 ...  
        500 KB  |--folder18 ...  
       57.3 MB  |--folder19 ...  
      892.6 MB  +--folder20 ...  
--------------  ----------------------------------------
      114.3 GB  Cluster size: 4 KB  Wastage: 3.9 GB
--------------  ----------------------------------------

DirSize report, created at 4/26/2009 4:00:00 AM

Total + subs    Folder name  
--------------  ----------------------------------------
      114.8 GB  D:\test3  
        2.0 GB  |--folder1 ...  
        2.5 MB  |--folder2 ...  
       29.0 MB  |--folder3 ...  
        120 KB  |--folder4 ...  
        1.3 GB  |--folder5 ...  
        1.1 GB  |--folder6 ...  
       21.2 MB  |--folder7 ...  
        924 KB  |--folder8 ...  
      240.5 MB  |--folder9 ...  
       84.3 GB  |--folder10 ...  
       25.3 MB  |--folder11 ...  
       47.0 MB  |--folder12 ...  
        5.3 GB  |--folder13 ...  
       16.2 GB  |--folder14 ...  
        2.8 GB  |--folder15 ...  
      549.5 MB  |--folder16 ...  
        512 KB  |--folder17 ...  
        532 KB  |--folder18 ...  
       57.3 MB  |--folder19 ...  
      892.6 MB  +--folder20 ...  
--------------  ----------------------------------------
      114.8 GB  Cluster size: 4 KB  Wastage: 3.9 GB
--------------  ----------------------------------------

The average is reporting as 154GB, when it should REALLY be 113.75 (or following the same rounding as before, 114GB)

Edited by kor

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

fixed the error:

#include <Array.au3>
#include <File.au3>
Dim $Folder = 'C:\', '*.txt'
Dim $FileArray = _FileListToArray($Folder)

For $i = 0 To UBound($FileArray-1)
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += Ceiling($Sizes[$i] / $ArrSize)
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    _ArrayDisplay($Sizes)
    MsgBox(0, '', $Average & ' GB')
EndFunc;==>ReadFiles

it displays for every text file in the C: folder each file scan amount and the average

Edited by oMBRa

Share this post


Link to post
Share on other sites

C:\newread.au3 (3) : ==> No variable given for "Dim", "Local", "Global", "Struct" or "Const" statement.:

Dim $Folder = 'C:\', '*.txt'

Dim $Folder = 'C:\', ^ ERROR

>Exit code: 1 Time: 0.207

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

#include <Array.au3>
#include <File.au3>
Dim $Folder = 'C:'
Dim $Files = '*.txt'
Dim $FileArray = _FileListToArray($Folder, $File)

For $i = 0 To UBound($FileArray-1)
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += Ceiling($Sizes[$i] / $ArrSize)
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    _ArrayDisplay($Sizes)
    MsgBox(0, '', $Average & ' GB')
EndFunc ;==>ReadFiles

u could fix the error by yourself reading the helpfile though

Edited by oMBRa

Share this post


Link to post
Share on other sites

#7 ·  Posted (edited)

C:\newread.au3 (8) : ==> Subscript used with non-Array variable.:

ReadFiles($FileArray[$i])

ReadFiles($FileArray^ ERROR

>Exit code: 1 Time: 0.209

EDIT: found the problem, I was trying to scan a directory with no text files in it.

Although, can you suggest a way to not have it display a msgbox for the very first array value? It is always blank.. then it begins to work correctly.

EDIT: after running the script I get this.

C:\newread.au3 (8) : ==> Array variable has incorrect number of subscripts or subscript dimension range exceeded.:

ReadFiles($FileArray[$i])

ReadFiles(^ ERROR

>Exit code: 1 Time: 6.905

It works, but when it's done it gives this error after the final msgbox closes.

Edited by kor

Share this post


Link to post
Share on other sites

Ive changed

For $i = 0 To UBound($FileArray-1)

to

For $i = 0 To UBound($FileArray)-1

And that seems to have fixed the "Array variable has incorrect number of subscripts or subscript dimension range exceeded."

But I'm not sure if by changing that I have altered the script in an adverse way. Running it still seems to work as normal.

Share this post


Link to post
Share on other sites

You don't even need the Ubound(), the count is already in the array.

Just use the [0] element like:

#include <Array.au3>
#include <File.au3>
Dim $Folder = @ScriptDir
Dim $FileArray = _FileListToArray($Folder, '*.txt')
For $i = 1 To $FileArray[0]
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
    _ArrayDisplay($Sizes)
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Average += Int(Number($Sizes[$i]) +.99)
    Next
    FileClose($FileOpen)
    $Average /= $ArrSize
    MsgBox(0, '', $Average & ' GB')
EndFunc;==>ReadFiles

Share this post


Link to post
Share on other sites

Here is the code I have settled with. Thank you.

#include <Array.au3>
#include <File.au3>
Dim $Folder = 'C:\'
Dim $Files = '*.txt'
Dim $FileArray = _FileListToArray($Folder, $Files)

;loop
For $i = 0 To $FileArray[0]
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
;   _ArrayDisplay($Sizes, $FileToOpen)
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += Ceiling($Sizes[$i] / $ArrSize)
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    _ArrayAdd($Sizes, "Avg = " & $Average & ' GB')
    _ArrayDisplay($Sizes, $FileToOpen)
EndFunc;==>ReadFiles

There is only 1 issue I am having, and that is the math is a little off on the averages. I have a test file with the following values, 3314.7, 1498.4, 5832.8, 2233.1, 1021.0.

They are all correctly rounded; 3315, 1499, 5833, 2234, 1021.... however their average is displayed as 2782... when it should be 2781. Is there any way to get the math more accurate?

Share this post


Link to post
Share on other sites

#11 ·  Posted (edited)

?

#include <Array.au3>
#include <File.au3>
Dim $Folder = 'C:\'
Dim $Files = '*.txt'
Dim $FileArray = _FileListToArray($Folder, $Files)
;loop
For $i = 0 To $FileArray[0]
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
;   _ArrayDisplay($Sizes, $FileToOpen)
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += $Sizes[$i]
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    $Average = Ceiling($Average / $ArrSize)
    _ArrayAdd($Sizes, "Avg = " & $Average & ' GB')
    _ArrayDisplay($Sizes, $FileToOpen)
EndFunc;==>ReadFiles

Edit: Changed "Int($x + .99)" to "Ceiling($x)", if that's your preference. Although, behind the scenes, it does exactly that, I'm sure.

Edit2: Pardon, replaced the code to one based off your last example. not your earliest. PS - Why the stringreplace to strip out commas? There aren't any commas in the array... Removed the + from the avg total I'd inadvertantly left in

Edited by Spiff59

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

Here is my finished code. I like this much better than the array display

#include <Array.au3>
#include <File.au3>
Dim $Folder = @ScriptDir
Dim $Files = '*.txt'
Dim $FileArray = _FileListToArray($Folder, $Files)

;loop
For $i = 0 To $FileArray[0]
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $FileName = StringTrimRight($FileToOpen, 4);trims the .txt off
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
;   _ArrayDisplay($Sizes, $FileToOpen);displays figures before rounding
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += Ceiling($Sizes[$i] / $ArrSize)
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    _ArrayAdd($Sizes, "")
    _ArrayAdd($Sizes, "Average:")
    _ArrayAdd($Sizes, $Average & ' GB')
    If _ArrayToString($Sizes, @LF, 0, 10) = "" Then;stops a blank msgbox from appearing
    Else
        MsgBox(4096, $FileName, _ArrayToString($Sizes, @LF, 0, 10))
    EndIf
  ;_ArrayDisplay($Sizes, $FileToOpen)
EndFunc;==>ReadFiles

I'm still working on the more accurate math. Spiff, suggestions with my updated code?

Edited by kor

Share this post


Link to post
Share on other sites

Yes, the funky averaging.

Add it all together, then divide once at the end.

I ran your numbers through mine and got 2781, not 2782.

Share this post


Link to post
Share on other sites

Thats weird, because I am still getting incorrect numbers for multiple test files. I am using Vista, and I have also tested the script when compiled on a server 2003 box, and the math is still wrong.

For example, here are the most blatent errors.

5, 5, 5, 5 : gives me average of 8

133, 133, 133, 133 : gives me average of 136

110, 115, 115, 115 : gives me average of 115

3, 3 : gives me average of 4

13, 13, 13, 13 : gives me average of 16

I'm not sure why these values are wrong.

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

Here is the fixed code that now correctly calculates the average withing being wrong.

#include <Array.au3>
#include <File.au3>
Dim $Folder = @ScriptDir
Dim $Files = '*.txt'
Dim $FileArray = _FileListToArray($Folder, $Files)

;loop
For $i = 0 To $FileArray[0]
    ReadFiles($FileArray[$i])
Next

Func ReadFiles($FileToOpen)
    $FileOpen = FileOpen($FileToOpen, 0)
    $FileName = StringTrimRight($FileToOpen, 4);trims the .txt off
    $String = FileRead($FileOpen)
    $Sizes = StringRegExp($String, '(?:\s*)(.+\s+)(?i)GB\s*(?:Cluster)', 3)
;   _ArrayDisplay($Sizes, $FileToOpen);displays figures before rounding
    Dim $Average, $ArrSize = UBound($Sizes)
    For $i = 0 To $ArrSize - 1
        $Sizes[$i] = StringReplace($Sizes[$i], ',', '')
        $Sizes[$i] = Ceiling(Number($Sizes[$i]))
        $Average += $Sizes[$i]
        $Sizes[$i] &= ' GB'
    Next
    FileClose($FileOpen)
    $Average = Ceiling($Average / $ArrSize)
    _ArrayAdd($Sizes, "")
    _ArrayAdd($Sizes, "Average:")
    _ArrayAdd($Sizes, $Average & ' GB')
    If _ArrayToString($Sizes, @LF, 0, 10) = "" Then;stops a blank msgbox from appearing
    Else
        MsgBox(0, $FileName, _ArrayToString($Sizes, @LF, 0, 10))
    EndIf
   ;_ArrayDisplay($Sizes, $FileToOpen)
EndFunc;==>ReadFiles
Edited by kor

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0