masvil

How to progressively store encrypted text in a file

19 posts in this topic

#1 ·  Posted (edited)

A way to progressively store encrypted text in a file, then easy decrypt it.
 

#include <Crypt.au3>

Local $Encrypted = @ScriptDir & "\Encrypted.txt"
Local $DecryptedJoined = @ScriptDir & "\DecryptedJoined.txt"
Local $password = "any"

_Crypt_Startup()

$hFileOpen = FileOpen($Encrypted, 10)
FileWrite($hFileOpen, String(_Crypt_EncryptData("One", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

$hFileOpen = FileOpen($Encrypted, 1)
FileWrite($hFileOpen, String(_Crypt_EncryptData("Two", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

$hFileOpen = FileOpen($Encrypted, 1)
FileWrite($hFileOpen, String(_Crypt_EncryptData("Three", $password, $CALG_AES_256) & " "))
FileClose($hFileOpen)

MsgBox (0,"", "Original strings have been encrypted, joined and separated by space in " & $Encrypted)

$hFileOpen = FileOpen($Encrypted, 0)
$EncryptedContent = StringTrimRight(FileRead($hFileOpen), 1) ; StringTrimRight is to cut out the last separator
FileClose($hFileOpen)

$SplittedString = StringSplit($EncryptedContent, " ")

FileDelete($DecryptedJoined)
For $i = 1 To $SplittedString[0]
    ; MsgBox(0, "Encrypted", $SplittedString[$i])
    ; MsgBox(0, "Decrypted", BinaryToString(_Crypt_DecryptData($SplittedString[$i], $password, $CALG_AES_256)))
    $hFileOpen = FileOpen($DecryptedJoined, 1)
    FileWrite($hFileOpen, BinaryToString(_Crypt_DecryptData($SplittedString[$i], $password, $CALG_AES_256)))
    FileClose($hFileOpen)
Next

MsgBox (0,"", "Encrypted Strings have been splitted in a array, one-by-one decrypted, and re-joined in " & $DecryptedJoined)

_Crypt_Shutdown()

$hFileOpen = FileOpen($DecryptedJoined, 0)
$DecryptedJoinedContent = FileRead($hFileOpen)
FileClose($hFileOpen)

MsgBox(0, "", "So the final result is: " & $DecryptedJoinedContent)

 

Edited by masvil

Share this post


Link to post
Share on other sites



The ciphertext is binary so you should Fopen the file for binary write mode.

Then the encryption/decryption passwords don't match!

Finally what are you trying to achieve by appending data? It won't work like this.


This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#3 ·  Posted (edited)

@jchd I followed your suggestions and fixed the code, as you can see. The decryption still doesn't work.

I have to progressively append data because in my project I have the data progressively available. I thought the 4th _Crypt_EncryptData optional parameter ([, $bFinal = True]) it's useful to do so.

Edited by masvil

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

EDIT: my first one was a stupid advice (and maybe this one too).

Have you tried to memorize all data in a single buffer and then write it to the file?

Edited by j0kky

Share this post


Link to post
Share on other sites
1 hour ago, j0kky said:

Have you tried to memorize all data in a single buffer and then write it to the file?

Good advice, but I need to continuously store data to avoid loss from PC power failure or something.

Share this post


Link to post
Share on other sites

When invoking CryptEncrypt() the data passed must adhere to the underlying Windows function requirements (https://msdn.microsoft.com/en-us/library/windows/desktop/aa379924(v=vs.85).aspx):

" When a block cipher is used, this data length must be a multiple of the block size unless this is the final section of data to be encrypted and the Final parameter is TRUE. "

Since you embed to encryption function call inside a FileWrite you have exactly zero chance to remark that an error was raised inside.

So your alternatives are either to use a stream cipher or to postpone encrypting stuff until you gather enough data (multiple of AES block).

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

Just an idea, what about:

- filewrite the content encrypted by _Crypt_EncryptData (with last param setted to True)

- write to an .ini file the lenght, in bytes, written

- filewrite another content encrypted by _Crypt_EncryptData (with last param setted to True)

- write to an .ini file the lenght, in bytes, written

- ......

Then you can FileRead and decrypt each block of data (because you can retrieve its lenght by the .ini file)

1 person likes this

Share this post


Link to post
Share on other sites

Why not, but then you can spare the .INI files altogether by prepending the block length before the block itself, an int32 would do.

Note that AES uses a fixed 128-bit blocksize, so dealing with multiples of blocksize shouldn't be very hard indeed!

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites
13 hours ago, jchd said:

Note that AES uses a fixed 128-bit blocksize, so dealing with multiples of blocksize shouldn't be very hard indeed!

How do you apply it to the user case? He said he has data progressively available and he must continuosly store data to avoid loss...

It means if he sets Final param to False, he should manually pad each block of data he currently has to make it multiple of 128 bit blocksize and make algorithm works... And when he has another block of data he will do the same thing and so on until he has the final block which he will crypt with Final param setted to True (so the function automatically pads his data).

It works with no error, but when you try to decrypt this data, you get something like:

DATA pad DATA pad DATA pad

Which is not exatly useful, right?

Share this post


Link to post
Share on other sites

#10 ·  Posted (edited)

@jchd and @j0kky you pointed me right, thank you. I updated my code, take a look.

I opted to simply use a space as separator while progressively storing the encrypted text, so no need to care about size of blocks (it really doesn't matter).

Then I use StringSplit (based on that space separator) to catch strings while decrypting and re-joining.

What do you think about this method?

And thank you to @Melba23 for his example on how to do some things.

Edited by masvil

Share this post


Link to post
Share on other sites

#11 ·  Posted

10 minutes ago, masvil said:

Then I use StringSplit (based on that space separator) to separate strings while decrypting.

What if an encrypted byte is 0x20 (which is ASCII white space)?

In my opinion my solution (improved by jchd's suggestion about Int32) will fit for you.

Share this post


Link to post
Share on other sites

#12 ·  Posted (edited)

9 minutes ago, j0kky said:

What if an encrypted byte is 0x20 (which is ASCII white space)?

It works as well (if I well understood your question). You can replace one of the three original strings with " " in my code to check it out.

Edited by masvil

Share this post


Link to post
Share on other sites

#13 ·  Posted (edited)

I'm not saying your code fails when you have a whitespace in plaintext, I'm saying I'm not sure it will work if an encrypted byte (a plaintext byte which has been encrypted) results equal to 0x20, which (I repeat, I'm not sure) is interpreted as a whitespace.

Edited by j0kky

Share this post


Link to post
Share on other sites

#14 ·  Posted

2 hours ago, j0kky said:

I'm not saying your code fails when you have a whitespace in plaintext, I'm saying I'm not sure it will work if an encrypted byte (a plaintext byte which has been encrypted) results equal to 0x20, which (I repeat, I'm not sure) is interpreted as a whitespace.

You mean maybe StringSplit could interprete an eventual 0x20 string in the text (clear or encrypted should be the same) as a white space (so a separator), right?

If so, I've coded an example (grabbed from the Help) to test it. It doesn't detect 0x20 as a white space separator:

Example()

Func Example()
    Local $aDays = StringSplit("Mon0x20Tues0x20Wed0x20Thur0x20Fri0x20Sat0x20Sun", " ") ; Split the string of days using the delimiter " " and the default flag value.

    For $i = 1 To $aDays[0] ; Loop through the array returned by StringSplit to display the individual values.
        MsgBox(0, "", "$aDays[" & $i & "] - " & $aDays[$i])
    Next
EndFunc   ;==>Example

 

Share this post


Link to post
Share on other sites

#15 ·  Posted

My doubt was about String() interpretation of binary data, but I noticed it translates "as is" into a string, adding the classical 0x in front of the string, so there should not be problems with white spaces.

1 person likes this

Share this post


Link to post
Share on other sites

#16 ·  Posted

51 minutes ago, masvil said:

"Mon0x20Tues0x20Wed0x20Thur0x20Fri0x20Sat0x20Sun"

These "0x20" are substrings, not characters!

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#17 ·  Posted

1 hour ago, jchd said:

These "0x20" are substrings, not characters!

So how can I test if @j0kky's suspicion is reasonable?

Share this post


Link to post
Share on other sites

#18 ·  Posted

An implementation of what I recommended in previous post, immune to character misinterpretation:

#include <Crypt.au3>
#include <FileConstants.au3>

Local $Encrypted = @ScriptDir & "\Encrypted.txt"
Local $Decrypted = @ScriptDir & "\Decrypted.txt"
Local $password = "any"

_Crypt_Startup()

Local $vBlock
Local $hCipherText = FileOpen($Encrypted, $FO_OVERWRITE + $FO_CREATEPATH + $FO_BINARY)
$vBlock = _Crypt_EncryptData("One; ", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

$hCipherText = FileOpen($Encrypted, $FO_APPEND + $FO_BINARY)
$vBlock = _Crypt_EncryptData("Two but with text longer than AES blocksize; ", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

$hCipherText = FileOpen($Encrypted, $FO_APPEND + $FO_BINARY)
$vBlock = _Crypt_EncryptData("Three.", $password, $CALG_AES_256)
FileWrite($hCipherText, BinaryLen($vBlock))
FileWrite($hCipherText, $vBlock)
FileClose($hCipherText)

MsgBox (0,"", "Original strings have been encrypted separately, with their binary length prepended, in " & $Encrypted)

Local $iFileSize = FileGetSize($Encrypted), $iBlockLength

$hCipherText = FileOpen($Encrypted, $FO_BINARY)
$hRecoverText = FileOpen($Decrypted, $FO_OVERWRITE + $FO_CREATEPATH)

While $iFileSize > 0
    $iBlockLength = Int(FileRead($hCipherText, 4))
    $iFileSize -= 4
    $EncryptedContent = FileRead($hCipherText, $iBlockLength)
    $iFileSize -= $iBlockLength
    FileWrite($hRecoverText, BinaryToString(_Crypt_DecryptData($EncryptedContent, $password, $CALG_AES_256)))
WEnd
FileClose($hRecoverText)

MsgBox (0,"", "Encrypted Strings have been decrypted and re-joined in " & $Decrypted)

Two points to consider:
  1) this works as long as plaintext strings are less than MaxInt32
  2) if you may have input strings containing Unicode characters beyond ANSI, they should be converted to UTF8 before encryption and from UTF8 after decryption.
 

1 person likes this

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Share this post


Link to post
Share on other sites

#19 ·  Posted (edited)

1 hour ago, masvil said:

So how can I test if @j0kky's suspicion is reasonable?

$t = DllStructCreate("byte element[1]")
$t.element = 0x20
ConsoleWrite(String($t.element) & @CRLF)

As I stated in post #15, it is not (instead the suspect is valid for BinaryToString function).

Anyhow jchd's code is better than yours, because it uses n bytes + 4 for each block of data of n bytes, while yours uses 2 * n bytes + 3 for each block.

Edited by j0kky
1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now