Jump to content
Sign in to follow this  
rudi

Searching for the *POSITION* from that on files are different.

Recommended Posts

rudi

Hi.

I have a problem with a certain Server. It happens sometimes, that large files copied to the server are not 100% copied. The resulting file on the server is "Filled with NULL" starting at the point of copy break down, which *CAN* be not noticed if certain conditions are met. (set NCP signature option = 0)

This code does find the position fine, but it's really slow: Giga link, and approxymately only 1MB/s are processed.

My approach is to binary read parts of 1MB to memory and to compare them. (it's enough to me if I just get the "trouble position" with an accuracy of +/- 1MB)

Howto improve?

#cs
    Author: Rudolf Thilo
    Suche nach der Position, *AB* wo sich 2 Dateien unterscheiden.
#ce

$Orginal="X:\download\Original-file-600Meg.zip" ; local disk. 
$Kopie="H:\Restore\corrupted\Copy-of-Original-File-600Meg.zip" ; Network Drive
$MaxCompare=FileGetSize("File1.zip")

$Block = 1024 * 1024 ; Ein Happen = 1 Megabyte
$FullBlocks = Floor($MaxCompare / $Block)
$Rest = $MaxCompare - $FullBlocks * $Block
ConsoleWrite("full size         : " & $MaxCompare & @CRLF)
ConsoleWrite("# of 1 Meg blocs  : " & $FullBlocks & @CRLF)
ConsoleWrite("Size of all blocks: " &$FullBlocks * $Block & @CRLF)
ConsoleWrite("Rest after blocks : " & $Rest & @CRLF)
ConsoleWrite("calced together   : " & $FullBlocks * $Block + $Rest & @CRLF)


$OHandle = FileOpen($Orginal, 16) ; Lesen; Binary
$KHandle = FileOpen($Kopie, 16) ; Lesen; Binary

Dim $Compared=0

While 1
    ConsoleWrite("Block Count: " & $FullBlocks & ", current block: " & $Compared & @CRLF)
    $OrgHappen = FileRead($OHandle, $Block)
    if @error then 
        ConsoleWrite("Failing to read Orginal file: @Error = " & @error & @CRLF & $Orginal & @crlf)
        Exit
    EndIf
    
    $KopHappen = FileRead($KHandle, $Block)
    if @error then 
        ConsoleWrite("Failing to read Copied file  : @Error = " & @error & @CRLF & $Kopie & @CRLF)
        Exit
    EndIf
    
    If StringCompare($OrgHappen, $KopHappen, 1) <> 0 Then
        ; Unterschied gefunden.
        $DiffAt = $Compared * $Block
        ConsoleWrite("Differnce at: " & $DiffAt & @CRLF)
        Exit
    EndIf
    $Compared += 1
    If $Compared = $FullBlocks Then
        $OrgHappen = FileRead($OHandle, $Rest)
        $KopHappen = FileRead($KHandle, $Rest)
        If StringCompare($OrgHappen, $KopHappen, 1) <> 0 Then
            ; Unterschied gefunden.
            $DiffAt = $Compared * $Block
            ConsoleWrite("Difference at: " & $DiffAt & @CRLF)
            Exit
        EndIf
        ; ende erreicht
        ConsoleWrite("No differences found." & @CRLF)
        Exit
    EndIf
WEnd

Regards, Rudi.

Edit: Typo

Edit2: syntax nonsense, by translating to English .... : $DiffAt = $Compared & $Block > $DiffAt = $Compared * $Block

Edited by rudi

Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites
rudi

Bump. :mellow:

I've used this code already successfully. Anyways, I'm interested how it could be improved to speed up.

Thanks, Rudi.

Hi.

I have a problem with a certain Server. It happens sometimes, that large files copied to the server are not 100% copied. The resulting file on the server is "Filled with NULL" starting at the point of copy break down, which *CAN* be not noticed if certain conditions are met. (set NCP signature option = 0)

This code does find the position fine, but it's really slow: Giga link, and approxymately only 1MB/s are processed.

My approach is to binary read parts of 1MB to memory and to compare them. (it's enough to me if I just get the "trouble position" with an accuracy of +/- 1MB)

Howto improve?

#cs
    Author: Rudolf Thilo
    Suche nach der Position, *AB* wo sich 2 Dateien unterscheiden.
#ce

$Orginal="X:\download\Original-file-600Meg.zip" ; local disk. 
$Kopie="H:\Restore\corrupted\Copy-of-Original-File-600Meg.zip" ; Network Drive
$MaxCompare=FileGetSize("File1.zip")

$Block = 1024 * 1024 ; Ein Happen = 1 Megabyte
$FullBlocks = Floor($MaxCompare / $Block)
$Rest = $MaxCompare - $FullBlocks * $Block
ConsoleWrite("full size         : " & $MaxCompare & @CRLF)
ConsoleWrite("# of 1 Meg blocs  : " & $FullBlocks & @CRLF)
ConsoleWrite("Size of all blocks: " &$FullBlocks * $Block & @CRLF)
ConsoleWrite("Rest after blocks : " & $Rest & @CRLF)
ConsoleWrite("calced together   : " & $FullBlocks * $Block + $Rest & @CRLF)


$OHandle = FileOpen($Orginal, 16) ; Lesen; Binary
$KHandle = FileOpen($Kopie, 16) ; Lesen; Binary

Dim $Compared=0

While 1
    ConsoleWrite("Block Count: " & $FullBlocks & ", current block: " & $Compared & @CRLF)
    $OrgHappen = FileRead($OHandle, $Block)
    if @error then 
        ConsoleWrite("Failing to read Orginal file: @Error = " & @error & @CRLF & $Orginal & @crlf)
        Exit
    EndIf
    
    $KopHappen = FileRead($KHandle, $Block)
    if @error then 
        ConsoleWrite("Failing to read Copied file  : @Error = " & @error & @CRLF & $Kopie & @CRLF)
        Exit
    EndIf
    
    If StringCompare($OrgHappen, $KopHappen, 1) <> 0 Then
        ; Unterschied gefunden.
        $DiffAt = $Compared * $Block
        ConsoleWrite("Differnce at: " & $DiffAt & @CRLF)
        Exit
    EndIf
    $Compared += 1
    If $Compared = $FullBlocks Then
        $OrgHappen = FileRead($OHandle, $Rest)
        $KopHappen = FileRead($KHandle, $Rest)
        If StringCompare($OrgHappen, $KopHappen, 1) <> 0 Then
            ; Unterschied gefunden.
            $DiffAt = $Compared * $Block
            ConsoleWrite("Difference at: " & $DiffAt & @CRLF)
            Exit
        EndIf
        ; ende erreicht
        ConsoleWrite("No differences found." & @CRLF)
        Exit
    EndIf
WEnd

Regards, Rudi.

Edit: Typo

Edit2: syntax nonsense, by translating to English .... : $DiffAt = $Compared & $Block > $DiffAt = $Compared * $Block


Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites
rover

Hi.

I have a problem with a certain Server. It happens sometimes, that large files copied to the server are not 100% copied. The resulting file on the server is "Filled with NULL" starting at the point of copy break down, which *CAN* be not noticed if certain conditions are met. (set NCP signature option = 0)

This code does find the position fine, but it's really slow: Giga link, and approxymately only 1MB/s are processed.

My approach is to binary read parts of 1MB to memory and to compare them. (it's enough to me if I just get the "trouble position" with an accuracy of +/- 1MB)

Howto improve?

removed posted code, it obviously must be rubbish if it doesn't rate comment. Edited by rover

I see fascists...

Share this post


Link to post
Share on other sites
kjcdude

I've delt with MD5 comparing and it seems to be extremely slow in autoit.

In some instances i found it faster to copy all than md5 compare the files and copy the changed ones.

I've resorted to comparing filesize, modified date, and created date.

Share this post


Link to post
Share on other sites
rover

I've delt with MD5 comparing and it seems to be extremely slow in autoit.

In some instances i found it faster to copy all than md5 compare the files and copy the changed ones.

I've resorted to comparing filesize, modified date, and created date.

Hi kjcdude

yes, very slow unless using plugins (external dll) or LazyCats run assembler code hack (very fast)

most practical would be an external file compare exe.

that's what I was thinking of instead of doing it in AutoIt.

OP seems to have file corruption problem with a server

and wants the file position (approximate +- 1mb) where errors begin.

Cheers :mellow:


I see fascists...

Share this post


Link to post
Share on other sites
rudi

Hi Rover.

removed posted code, it obviously must be rubbish if it doesn't rate comment.

Pardon? :mellow: Sorry, I absolutely miss what that shall tell me? :(

Can you say it using slang free English, pls?

Regards, Rudi.


Earth is flat, pigs can fly, and Nuclear Power is SAFE!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×