Jump to content
valdemar1977

Different file size after Encrypt-Decrypt file

Recommended Posts

valdemar1977

Hello!
At the moment I'm having a problem with integrity of decrypted file.
First time I mean that it is my script fault and I'm try standart _Crypt_EncryptFile.au3 and _Crypt_decryptFile.au3 scripts for investigate a problem. Result are the same - different files size (and MD5/SHA1 checksum) after encrypting-decrypting. Try to use AES 128-256 and 3DES.
Original file is huge - 55.4 GB (59 579 039 744 bytes) binary file.
After encryption size is 55.4 GB (59 579 039 760 bytes)
After decryption 55.4 GB (59 579 039 748 bytes).
As you see decrypted file size is on 4 bytes larger then original, of course after that file is corrupted.
8-9 GB file successfully encrypted and decrypted. 
OS: Windows 7 Pro SP1 x64 

Any opinions are welcome,

Share this post


Link to post
Share on other sites
Jos

This looks pretty much the same as your bug report, just some very basic info and no substance that allows us to help you. :)
Have you check with some Hex editor to see what the difference is? (Not sure WinMerge would be happy to do a compare of 55GB files)
Any way you can replicate this with a smaller sized file?

Jos

 


SciTE4AutoIt3 Full installer Download page   - Beta files       Read before posting     How to post scriptsource        Forum Rules
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Share this post


Link to post
Share on other sites
jchd

Comparing smaller chunks (or by dichotomy on filesize) will help locate where the differences are.
Also it would help posting the (simplified to barest form) failing script and AutoIt version used, just to be sure.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
valdemar1977

I'm using 3.3.12.0 version of Autoit on Windows 7 Pro x64 with SP1 and all latest hotfixes.

It is easy to reproduce a problem:

1) Create empty big file

start cmd and type a command fsutil file createnew c:\test_big_file.orig 68719476736

2) Start _Crypt_EncryptFile.au3 from Autoit_HOME\Examples\Helpfile

Select c:\test_big_file.orig as input file, c:\test_big_file.encr as output.

Type any password and remember it.

Select AES256  algorithm. 

Press encrypt. Wait when encrypting completed, then close script.

3) Start _Crypt_DecryptFile.au3

Select c:\test_big_file.encr as input file, c:\test_big_file.decr as output.

Type a password from step 2.

Select AES256 algorithm.

Press decrypt. Wait when decrypting completed, then close script.

4) Compare size of c:\test_big_file.orig and c:\test_big_file.decr it must be not identical (68719476736 vs 68719476752 bytes in my case)

Also you can calclulate and check MD5 sum with any available utility, WinMD5Free for example.

Regards.

P.S. I'm try the same procedure with 32 GB file and dont have a problem.

Edited by valdemar1977

Share this post


Link to post
Share on other sites
valdemar1977

Please any help...

Share this post


Link to post
Share on other sites
jchd

Your last post doesn't make sense, probably you were trying to post links which didn't work out.

From my own (lengthy) experience with this round-tripping of large files, I've found zero difference in size nor in content with a 45 Gb file with RC4, then with a 68Gb file with AES256. Here's my test script:

#include <FileConstants.au3>
#include <Crypt.au3>
#include <Math.au3>

Local $iSizeGb = 68
Local $kCipher = $CALG_AES_256
Local $sPlainFile = "big\plain" & $iSizeGb & ".bin"
Local $sCipherFile = "big\cipher" & $iSizeGb & ".bin"
Local $sRecoverFile = "big\recover" & $iSizeGb & ".bin"
Local $sKey = "There is a big big big secret hidden somewhere"
Local $hPlainFile = FileOpen($sPlainFile, $FO_OVERWRITE + $FO_CREATEPATH + $FO_BINARY)
FileWrite($hPlainFile, StringToBinary("Begin of adventure", 4))
Local $bEnd = StringToBinary("End of game", 4)
FileSetPos($hPlainFile, $iSizeGb * 1024 * 1024 * 1024 - BinaryLen($bEnd), $FILE_BEGIN)
FileWrite($hPlainFile, $bEnd)
FileClose($hPlainFile)
_Crypt_EncryptFile($sPlainFile, $sCipherFile, $sKey, $kCipher)
_Crypt_DecryptFile($sCipherFile, $sRecoverFile, $sKey, $kCipher)
FileDelete($sCipherFile)

_FileBinCompare($sPlainFile, $sRecoverFile)
FileDelete($sPlainFile)
FileDelete($sRecoverFile)

Func _FileBinCompare($sSourceFile1, $sSourceFile2)
    Local   $bTempData1, _
            $iMinSize = _Min(FileGetSize($sSourceFile1), FileGetSize($sSourceFile2)), _
            $iRead = 0, _
            $hInFile1 = FileOpen($sSourceFile1, $FO_BINARY), _
            $hInFile2 = FileOpen($sSourceFile2, $FO_BINARY), _
            $iBlock = 0, _
            $iChunk = 128 * 1024 * 1024

    Do
        $iBlock += 1
        $bTempData1 = FileRead($hInFile1, $iChunk)
        $bTempData2 = FileRead($hInFile2, $iChunk)
        If BinaryLen($bTempData1) = BinaryLen($bTempData2) Then
            $iRead += BinaryLen($bTempData1)
        ElseIf BinaryLen($bTempData1) > BinaryLen($bTempData2) Then
            $iRead += BinaryLen($bTempData2)
            ConsoleWrite("End of file2 reached before file1" & @LF)
        Else
            $iRead += BinaryLen($bTempData1)
            ConsoleWrite("End of file1 reached before file2" & @LF)
        EndIf
        ConsoleWrite("Blocks " & $iBlock & " are " & ($bTempData1 = $bTempData2 ? "equal." : "different.") & @LF)
    Until $iRead = $iMinSize
    FileClose($hInFile1)
    FileClose($hInFile2)
EndFunc   ;==>_FileBinCompare

Please note that I'm not convinced by the fsutil trick. What I'm doing here is actually creating a real plaintext with known content from offset 0 to filesize - 1. I've used the release version in 32-bit (I bet it can't be the cause of any issue). Compare the speed of creating with fsutil to that of actually writing, setpos-ing writing again then closing.

So either you can come up with a reproducible script (in full, please) resulting in a failure, or I'm going to consider the ticket as "works for me".

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
valdemar1977

Thank you jchd.

I'm try to make a 72 GB RAR5 (with BLAKE2 checksum) file with hd video inside and try _Crypt_Encrypt-_Crypt_Decrypt on it. No errors. Strange...

I will additionally investigate.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×