Different file size after Encrypt-Decrypt file

[valdemar1977]

Hello!
At the moment I'm having a problem with integrity of decrypted file.
First time I mean that it is my script fault and I'm try standart _Crypt_EncryptFile.au3 and _Crypt_decryptFile.au3 scripts for investigate a problem. Result are the same - different files size (and MD5/SHA1 checksum) after encrypting-decrypting. Try to use AES 128-256 and 3DES.
Original file is huge - 55.4 GB (59 579 039 744 bytes) binary file.
After encryption size is 55.4 GB (59 579 039 760 bytes)
After decryption 55.4 GB (59 579 039 748 bytes).
As you see decrypted file size is on 4 bytes larger then original, of course after that file is corrupted.
8-9 GB file successfully encrypted and decrypted. 
OS: Windows 7 Pro SP1 x64 

Any opinions are welcome,

[Jos]

This looks pretty much the same as your bug report, just some very basic info and no substance that allows us to help you.
Have you check with some Hex editor to see what the difference is? (Not sure WinMerge would be happy to do a compare of 55GB files)
Any way you can replicate this with a smaller sized file?

Jos

 

[jchd]

Comparing smaller chunks (or by dichotomy on filesize) will help locate where the differences are.
Also it would help posting the (simplified to barest form) failing script and AutoIt version used, just to be sure.

[valdemar1977]

I'm using 3.3.12.0 version of Autoit on Windows 7 Pro x64 with SP1 and all latest hotfixes.

It is easy to reproduce a problem:

1) Create empty big file

start cmd and type a command fsutil file createnew c:\test_big_file.orig 68719476736

2) Start _Crypt_EncryptFile.au3 from Autoit_HOME\Examples\Helpfile

Select c:\test_big_file.orig as input file, c:\test_big_file.encr as output.

Type any password and remember it.

Select AES256  algorithm. 

Press encrypt. Wait when encrypting completed, then close script.

3) Start _Crypt_DecryptFile.au3

Select c:\test_big_file.encr as input file, c:\test_big_file.decr as output.

Type a password from step 2.

Select AES256 algorithm.

Press decrypt. Wait when decrypting completed, then close script.

4) Compare size of c:\test_big_file.orig and c:\test_big_file.decr it must be not identical (68719476736 vs 68719476752 bytes in my case)

Also you can calclulate and check MD5 sum with any available utility, WinMD5Free for example.

Regards.

P.S. I'm try the same procedure with 32 GB file and dont have a problem.

Edited June 19, 2015 by valdemar1977

[valdemar1977]

Please any help...

[jchd]

Your last post doesn't make sense, probably you were trying to post links which didn't work out.

From my own (lengthy) experience with this round-tripping of large files, I've found zero difference in size nor in content with a 45 Gb file with RC4, then with a 68Gb file with AES256. Here's my test script:

expandcollapse popup

#include <FileConstants.au3>
#include <Crypt.au3>
#include <Math.au3>

Local $iSizeGb = 68
Local $kCipher = $CALG_AES_256
Local $sPlainFile = "big\plain" & $iSizeGb & ".bin"
Local $sCipherFile = "big\cipher" & $iSizeGb & ".bin"
Local $sRecoverFile = "big\recover" & $iSizeGb & ".bin"
Local $sKey = "There is a big big big secret hidden somewhere"
Local $hPlainFile = FileOpen($sPlainFile, $FO_OVERWRITE + $FO_CREATEPATH + $FO_BINARY)
FileWrite($hPlainFile, StringToBinary("Begin of adventure", 4))
Local $bEnd = StringToBinary("End of game", 4)
FileSetPos($hPlainFile, $iSizeGb * 1024 * 1024 * 1024 - BinaryLen($bEnd), $FILE_BEGIN)
FileWrite($hPlainFile, $bEnd)
FileClose($hPlainFile)
_Crypt_EncryptFile($sPlainFile, $sCipherFile, $sKey, $kCipher)
_Crypt_DecryptFile($sCipherFile, $sRecoverFile, $sKey, $kCipher)
FileDelete($sCipherFile)

_FileBinCompare($sPlainFile, $sRecoverFile)
FileDelete($sPlainFile)
FileDelete($sRecoverFile)

Func _FileBinCompare($sSourceFile1, $sSourceFile2)
    Local   $bTempData1, _
            $iMinSize = _Min(FileGetSize($sSourceFile1), FileGetSize($sSourceFile2)), _
            $iRead = 0, _
            $hInFile1 = FileOpen($sSourceFile1, $FO_BINARY), _
            $hInFile2 = FileOpen($sSourceFile2, $FO_BINARY), _
            $iBlock = 0, _
            $iChunk = 128 * 1024 * 1024

    Do
        $iBlock += 1
        $bTempData1 = FileRead($hInFile1, $iChunk)
        $bTempData2 = FileRead($hInFile2, $iChunk)
        If BinaryLen($bTempData1) = BinaryLen($bTempData2) Then
            $iRead += BinaryLen($bTempData1)
        ElseIf BinaryLen($bTempData1) > BinaryLen($bTempData2) Then
            $iRead += BinaryLen($bTempData2)
            ConsoleWrite("End of file2 reached before file1" & @LF)
        Else
            $iRead += BinaryLen($bTempData1)
            ConsoleWrite("End of file1 reached before file2" & @LF)
        EndIf
        ConsoleWrite("Blocks " & $iBlock & " are " & ($bTempData1 = $bTempData2 ? "equal." : "different.") & @LF)
    Until $iRead = $iMinSize
    FileClose($hInFile1)
    FileClose($hInFile2)
EndFunc   ;==>_FileBinCompare

Please note that I'm not convinced by the fsutil trick. What I'm doing here is actually creating a real plaintext with known content from offset 0 to filesize - 1. I've used the release version in 32-bit (I bet it can't be the cause of any issue). Compare the speed of creating with fsutil to that of actually writing, setpos-ing writing again then closing.

So either you can come up with a reproducible script (in full, please) resulting in a failure, or I'm going to consider the ticket as "works for me".

Edited June 20, 2015 by jchd

[valdemar1977]

Thank you jchd.

I'm try to make a 72 GB RAR5 (with BLAKE2 checksum) file with hd video inside and try _Crypt_Encrypt-_Crypt_Decrypt on it. No errors. Strange...

I will additionally investigate.