Jump to content
trancexx

Memory compression

Recommended Posts

trancexx

Appears that memory compression is in these days. So, to be in, here is another method. This one is called "Native API Compression".

Apparently it's LZ1(LZ77) compression algorithm or something very similar and it's available through one or two dll calls. Of course that files can be compressed too by this method.

Functions with small example:

#NoTrayIcon

$sString = "lzwlzwlzwlzwlzwlzwlzwlzwlzwlzwlzwlzw"
ConsoleWrite("Before compression: " & Binary($sString) & @CRLF)

$BC = _LZNTCompress($sString)
ConsoleWrite("Compressed: " & $BC & @CRLF)

$CB = _LZNTDecompress($BC)
ConsoleWrite("Decompressed: " & $CB & @CRLF)






; #FUNCTION# ;===============================================================================
;
; Name...........: _LZNTDecompress
; Description ...: Decompresses input data.
; Syntax.........: _LZNTDecompress ($bBinary)
; Parameters ....: $vInput - Binary data to decompress.
; Return values .: Success - Returns decompressed binary data.
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - Error decompressing.
; Author ........: trancexx
; Related .......: _LZNTCompress
; Link ..........; http://msdn.microsoft.com/en-us/library/bb981784.aspx
;
;==========================================================================================
Func _LZNTDecompress($bBinary)

    $bBinary = Binary($bBinary)

    Local $tInput = DllStructCreate("byte[" & BinaryLen($bBinary) & "]")
    DllStructSetData($tInput, 1, $bBinary)

    Local $tBuffer = DllStructCreate("byte[" & 16 * DllStructGetSize($tInput) & "]") ; initially oversizing buffer

    Local $a_Call = DllCall("ntdll.dll", "int", "RtlDecompressBuffer", _
            "ushort", 2, _
            "ptr", DllStructGetPtr($tBuffer), _
            "dword", DllStructGetSize($tBuffer), _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "dword*", 0)

    If @error Or $a_Call[0] Then
        Return SetError(1, 0, "") ; error decompressing
    EndIf

    Local $tOutput = DllStructCreate("byte[" & $a_Call[6] & "]", DllStructGetPtr($tBuffer))

    Return SetError(0, 0, DllStructGetData($tOutput, 1))

EndFunc   ;==>_LZNTDecompress



; #FUNCTION# ;===============================================================================
;
; Name...........: _LZNTCompress
; Description ...: Compresses input data.
; Syntax.........: _LZNTCompress ($vInput [, $iCompressionFormatAndEngine])
; Parameters ....: $vInput - Data to compress.
;                  $iCompressionFormatAndEngine - Compression format and engine type. Default is 2 (standard compression). Can be:
;                  |2 - COMPRESSION_FORMAT_LZNT1 | COMPRESSION_ENGINE_STANDARD
;                  |258 - COMPRESSION_FORMAT_LZNT1 | COMPRESSION_ENGINE_MAXIMUM
; Return values .: Success - Returns compressed binary data.
;                          - Sets @error to 0
;                  Failure - Returns empty string and sets @error:
;                  |1 - Error determining workspace buffer size.
;                  |2 - Error compressing.
; Author ........: trancexx
; Related .......: _LZNTDecompress
; Link ..........; http://msdn.microsoft.com/en-us/library/bb981783.aspx
;
;==========================================================================================
Func _LZNTCompress($vInput, $iCompressionFormatAndEngine = 2)

    If Not ($iCompressionFormatAndEngine = 258) Then 
        $iCompressionFormatAndEngine = 2
    EndIf   
    
    Local $bBinary = Binary($vInput)

    Local $tInput = DllStructCreate("byte[" & BinaryLen($bBinary) & "]")
    DllStructSetData($tInput, 1, $bBinary)

    Local $a_Call = DllCall("ntdll.dll", "int", "RtlGetCompressionWorkSpaceSize", _
            "ushort", $iCompressionFormatAndEngine, _
            "dword*", 0, _
            "dword*", 0)

    If @error Or $a_Call[0] Then
        Return SetError(1, 0, "") ; error determining workspace buffer size
    EndIf

    Local $tWorkSpace = DllStructCreate("byte[" & $a_Call[2] & "]") ; workspace is needed for compression

    Local $tBuffer = DllStructCreate("byte[" & 16 * DllStructGetSize($tInput) & "]") ; initially oversizing buffer

    Local $a_Call = DllCall("ntdll.dll", "int", "RtlCompressBuffer", _
            "ushort", $iCompressionFormatAndEngine, _
            "ptr", DllStructGetPtr($tInput), _
            "dword", DllStructGetSize($tInput), _
            "ptr", DllStructGetPtr($tBuffer), _
            "dword", DllStructGetSize($tBuffer), _
            "dword", 4096, _
            "dword*", 0, _
            "ptr", DllStructGetPtr($tWorkSpace))

    If @error Or $a_Call[0] Then
        Return SetError(2, 0, "") ; error compressing
    EndIf

    Local $tOutput = DllStructCreate("byte[" & $a_Call[7] & "]", DllStructGetPtr($tBuffer))

    Return SetError(0, 0, DllStructGetData($tOutput, 1))

EndFunc   ;==>_LZNTCompress

Available in Microsoft Windows XP and later versions of all Windows operating systems.

There is $tBuffer there if you look. I'm oversizing it plenty initially (saw this bad example in c++). Probably should be some better way to do that but documentation for this functions on MSDN is not over yet apparently :)

Two compression rates are available. Default COMPRESSION_ENGINE_STANDARD is lightening speedy. Better compression is gained by COMPRESSION_ENGINE_MAXIMUM which is slower.

  • Like 2

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
smashly

Thank You for sharing and nice work trancexx,

I just tried it on a 12KB script as standard and the output file is 6KB..

50% compression of my script... nice

Set compression as max and I get 5KB filesize.

The compression isn't as high as 3rd party programs, but since it's using the native windows to do it's thing I love the idea.

I'm off to play with it bit more :)

Cheers

Edited by smashly

Share this post


Link to post
Share on other sites
trancexx

Thank You for sharing and nice work trancexx,

I just tried it on a 12KB script as standard and the output file is 6KB..

50% compression of my script... nice

Set compression as max and I get 5KB filesize.

The compression isn't as high as 3rd party programs, but since it's using the native windows to do it's thing I love the idea.

I'm off to play with it bit more ^_^

Cheers

Ok, enough playing with it! You're gonna break something.

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
knightz

thank you

this script is very useful

Share this post


Link to post
Share on other sites
netegg

so sad, would you like to explain how to use it? Why i can't get the result as above?

Share this post


Link to post
Share on other sites
trancexx

It's very hard to explain.You just do.

Understanding internal workings might help; 150 sentences about the working.

Edited by trancexx

♡♡♡

.

eMyvnE

Share this post


Link to post
Share on other sites
AdmiralAlkex

This UDF is great. I use it for the "CompressPlaylist"-option in ShiftER, end result is around a third/sixth that of the non-compressed file.

Thank you trancexx

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×