Jump to content

DllCall to BZIP2.dll method fails when I try to use function


Recommended Posts

Hi guys.

I try to choose an algorithm to compress some string data in-memory, without file medium to store in sqlite as blob. I plan to store about 100k blobs each 80k symbols (uncompressed). First opinion was LZMA udf by Ward, but udf file doesn't exist.

Then i tried to use BZIP2 dll in DllCall, but it simply doesn't work and throws @error = 1.

 

Here is an example:

Spoiler
$hBZDll = DllOpen(@ScriptDir & "\bzip2.dll") ; obtained at http://gnuwin32.sourceforge.net/packages/bzip2.htm

$data = "binbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbinbin"

$Ret = _DB_CompressData($data)
MsgBox(0, $Ret[2], Binary($Ret[0]))

Func _DB_CompressData($data)

    Local $Size_O, $Size_C, $Size_R, $a_pBuffer, $ret[3]
    $Size_O = BinaryLen($data)
    $Size_R = Ceiling(($Size_O*1.01)+600) ; reserve some bytes accordingly to BZIP2 documentation
    
    $a_pBuffer = DllStructCreate('source byte[' & $Size_O & '];sourceLen uint;dest byte[' & $Size_R & '];destLen uint;blockSize100k int;verbosity int;workFactor int;')
    DllStructSetData($a_pBuffer, "source", $data)
    DllStructSetData($a_pBuffer, "sourceLen", $Size_O)
    DllStructSetData($a_pBuffer, "destLen", $Size_R)
    DllStructSetData($a_pBuffer, "blockSize100k", 9) ; this and below are default values in BZIP2 docs
    DllStructSetData($a_pBuffer, "verbosity", 0)
    DllStructSetData($a_pBuffer, "workFactor", 30)
    
    $dllres = DllCall($hBZDll, "int", "BZ2_bzBuffToBuffCompress", _
    "dest", DllStructGetPtr($a_pBuffer, "dest"), _
    "destLen", DllStructGetPtr($a_pBuffer, "destLen"), _ ; destLen is uint * type, so we need a pointer
    "source", DllStructGetPtr($a_pBuffer, "source"), _
    "sourceLen", DllStructGetData($a_pBuffer, "sourceLen"), _
    "blockSize100k", DllStructGetData($a_pBuffer, "blockSize100k"), _
    "verbosity", DllStructGetData($a_pBuffer, "verbosity"), _
    "workFactor", DllStructGetData($a_pBuffer, "workFactor"))
    
    MsgBox(0, "Error", "@error = " & @error & @CRLF & "@extended = " & @extended)
    
    $Size_C = DllStructGetData($a_pBuffer, "destLen")
    
    $Ret[0] = DllStructGetData($a_pBuffer, "dest")
    $Ret[1] = $Size_O
    $Ret[2] = $Size_C

    Return $ret

EndFunc   ;==>_CompressData


#cs
BZ2_bzBuffToBuffCompress function prototype on C++

int BZ2_bzBuffToBuffCompress( char*         dest,
                              unsigned int* destLen,
                              char*         source,
                              unsigned int  sourceLen,
                              int           blockSize100k,
                              int           verbosity,
                              int           workFactor );
#ce

 

 

What should I do with this error? Where am I wrong?

Thanks.

Edited by Hammerfist
Link to post
Share on other sites

And trancexx , I know about your Native API Compression =) Didn't use it due to low compression ratio - it's 34%. Btw $COMPRESSION_FORMAT_XPRESS_HUFF shows awesome 11% of compression, but I could not make it decompress the data.

_WinAPI_DecompressBuffer($a_pBuffer[0], $Size_O, $a_pBuffer[1], $Size_C, $COMPRESSION_FORMAT_XPRESS_HUFF) shows @error = 10 and @extended = c00000e8.

_WinAPI_CompressBuffer and _WinAPI_DecompressBuffer using the engine $COMPRESSION_FORMAT_XPRESS work very well, but if we add "_HUFF" it throws error.

Link to post
Share on other sites

If the use is for compression of blob in SQLite storage and if you don't have concurency timing requirement, I'd develop an SQLite extension to compress/decompress data on the fly, keeping the applicative layer clean, simple, blindly ignorant of what's being done under its feet and avoiding manipulating data several times within AutoIt (forcibly slower than within C code).

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites

Can I write proper subroutines for SQLite using AutoIt? I think no.

And my app is kinda archive. It keeps a database in compressed state and extracts one blob of data on demand, one, two or ten times per day. So i can sacrifice some processor time at the import stage to reach maximum compression.

Applicative layer remains clean enough. I'll simply change the "$data = $uncompressed_blob" to "$data = _DB_UnpackBlob($compressed_blob)".

Link to post
Share on other sites

Yes you can (create SQLite functions written in AutoIt) but it isn't worth the burden, especially in you "archive" use case.

When the expected benefit is large enough it's much better to write directly an extension function in C and load it at connection time, or auto-load it for all subsequent connections in the same run, or statically include it in a homebrew SQLite.dll build.

If ever you're interessed to add AutoIt functions as extensions to SQLite you can use this (posted on the French forum): https://autoitscript.fr/forum/viewtopic.php?f=21&t=13556&p=94618

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By IndianSage
      Hi,
      My AutoIt script is as folllows:
      ;use for calling function add2NosA in dll ;Local $vNo1 = 33 ;Local $vNo2 = 11 ;use for calling function sortNos in dll Local $vNo1 = [11,7,9] Local $vNo2 = [1,3,2] ; _ArrayDisplay($vNo1, "vNo1 display") ; _ArrayDisplay($vNo2, "vNo2 display") ;Local $hWnd = DllOpen("E:\CV-Dell-1\autoit3\myComObj1.dll") Local $hWnd = ObjCreate("myComObj1.clsMath") if (@error) Then MsgBox (0, "Error", "Error1 = " & @error) Exit EndIf ;function call method - DllCall with function name ;Local $aRes = DllCall($hWnd, "int", "addNosA", "int", $vNo1, "int", $vNo2) ;Local $aRes = DllCall($hWnd, "Ptr", "sortNos", "Array", $vNo1, "Array", $vNo2) ;function call method - $hWnd.<function name> ;Local $aRes = $hWnd.add2NosA($vNo1, $vNo2) ; this works fine with ObjCreate Local $aRes = $hWnd.sortNos($vNo1, $vNo2) if (@error) Then MsgBox (0, "Error", "Error2 = " & @error) DllClose($hWnd) Exit EndIf ;use appropriate msgbox ;MsgBox(0,"Result", "Result = " & $vNo1[0]) MsgBox(0,"Result", "Result = " & $aRes) _ArrayDisplay($vNo1, "vNo1 display") _ArrayDisplay($aRes , "aRes display") DllClose($hWnd) My VB.Net - ClassLibrary - Dll - COM obj is as follows - has 2 functions - add2NosA and sortNos:
      <ComClass(clsMath.ClassId, clsMath.InterfaceId, clsMath.EventsId)> Public Class clsMath Public Const ClassId As String = "3A42F85E-24C8-4BAA-91B5-AE56C4683C13" Public Const InterfaceId As String = "D99D7C79-2BA7-4A33-B7BC-9B7F19FDF828" Public Const EventsId As String = "CA128AC4-580C-4112-9EAD-8D1599E3F37A" Public Sub New() MyBase.New() End Sub Public Function add2NosA(ByVal no1 As Integer, ByVal no2 As Integer) As Integer Return (no1 + no2) End Function Public Sub sortNos(ByRef no1 As Array, ByRef no2 As Array) Array.Sort(no1) no2 = no1 End Sub End Class  Over all I tried various 8 options mentioned in the attached Excel file - with only 1 combination working.
      Overall could not make Array returned capture in AutoIt script. 
      Can someone help please?
      Thanks in  advance.
      Options-Tried-Matrix-Results.xlsx
    • By IndianSage
      I have create a function in FreeBasic like below:
      Extern "Windows-MS" Type tA f1 As Integer f2 As Integer End Type Public Function _switchOrder(ByVal no1 As Integer, ByVal no2 As Integer) As tA Export Dim result As Integer Dim taa As tA taa.f1 = no2 taa.f2 = no1 Return taa End Function End Extern Caller AutoIt code is:
      #include <MsgBoxConstants.au3> Global Const $sTag_ftdi_version_info="struct; int no1a; int no2a; endstruct" Local $aRet=DllCall("Math1.dll","Ptr","_switchOrder", "Int", 10, "Int", 30) ;MsgBox (0,"",@error & "-" & $aRet[0] & "-" & $aRet[1]& "-" & $aRet[2]) Local $t_ftdi_version_info=DllStructCreate($sTag_ftdi_version_info,$aRet[0]) MsgBox (0,"msg1=",@error & "---" & $aRet[0] & "-" & $aRet[1]& "-" & $aRet[2]) ;Local $retData1 = DllStructGetData($t_ftdi_version_info,"",1) Local $retData1 = DllStructGetData($t_ftdi_version_info,"no1a") MsgBox (0,"msg2=",@error & "--" & $retData1) ;Local $retData1 = DllStructGetData($t_ftdi_version_info,"",2) Local $retData1 = DllStructGetData($t_ftdi_version_info,"no2a") MsgBox (0,"msg2=",@error & "--" & $retData1) ;ConsoleWrite(DllStructGetData($t_ftdi_version_info,"",2) & "--" & @error) ;ConsoleWrite(DllStructGetData($t_ftdi_version_info,"no2a") & @CRLF) ;ConsoleWrite(DllStructGetData($tversion_str,1) & @CRLF) Getting error 2 for DllStructGetData or it give Close Application AutoIt popup message. 
      Certainly DllCall is not returning pointer to the Structure in $aRet[0] hence issue.
      Can someone help me fix this please?
      Thanks in advance.
       
       
    • By xYuri
      This simple dllcall gives me error 5, access denied,
      Func _WinAPI_VkKeyScan($__key) _WinAPI_SetLastError(0) $res = DllCall('User32.dll', 'SHORT', 'VkKeyScan', 'CHAR', $__key) _xConsole('res: '&$res) $_LastErr = _WinAPI_GetLastError() If $_LastErr <> 0 Then _xConsole('Err: {' & $_LastErr & '}> ' & _WinAPI_GetLastErrorMessage()) Return $res EndFunc Am i doing something wrong?
      Also tried VkKeyScanA and W
      Edit:
      I want to send `:` via PostMessage() WM_KEYDOWN
    • By Skysnake
      This is relevant
      From here https://stackoverflow.com/questions/3454315/is-it-possible-to-pin-a-dll-in-memory-to-prevent-unloading
      I use several UDFs on the Forum to do various things.  Those UDFs work very well.
      Effectively the UDFs are DLL wrappers, that make it possible to access DLL functions easily without the long hard slog of DLLCall() every time.
      However, I have now run into the issue that multiple UDF DLLCalls are slow. Not mind numbingly slow, but slow enough to become noticeable with a large of repeated function calls.
      So I was wondering, is it possible to "load a DLL into memory" and leave it there for the duration of my script's lifetime, avoid repeated DLL on-disk reads with a persistent in memory DLL?
      From Microsoft
      https://docs.microsoft.com/en-us/windows/desktop/dlls/about-dynamic-link-libraries
      Looks like what I want to do is: load-time dynamic linking,
      So next question, (a) how do I do this with AutoIt (b) How would this impact on standard AutoIt type DLL calls?
       
      The point is speed.  Is there a different approach?
      Or am I barking up the wrong tree?
      Skysnake
    • By supersonic
      Hi -
      Currently I'm playing around with Windows Credential Manager. I'm trying to access it with DllCall("advapi32.dll", ...) using the functions 'CredWriteW', 'CredReadW' and 'CredDeleteW'. All well. Another function I have to deal with is 'CredEnumerateW': https://docs.microsoft.com/en-us/windows/desktop/api/wincred/nf-wincred-credenumeratew/ .
      That's the test code I have so far:
      #include <Array.au3> #include <String.au3> Local $tCredentialsCount = DllStructCreate("DWORD;") Local $tPointerToArrayOfPointers = DllStructCreate("PTR;") ; Local $tPointerToArrayOfPointers = DllStructCreate(_StringRepeat("PTR;", 200)) ; ??? Local $aResult = DllCall("advapi32.dll", "BOOL", "CredEnumerateW", _ "WSTR", Null, "DWORD", 1, "DWORD", DllStructGetPtr($tCredentialsCount), "PTR", DllStructGetPtr($tPointerToArrayOfPointers)) If (Not @error) Then Local $iCredentialsCount = DllStructGetData($tCredentialsCount, 1) _ArrayDisplay($aResult, $iCredentialsCount) Local $hPointerToArrayOfPointers = DllStructGetData($tPointerToArrayOfPointers, 1) MsgBox(0, "$hPointerToArrayOfPointers", $hPointerToArrayOfPointers) ; Fails... For $i = 1 To 10 ; $iCredentialsCount MsgBox(0, $i & "___" & (($i * 2) - 1), DllStructGetData($tPointerToArrayOfPointers, ($i * 2) - 1)) Next $tCredentialsCount = 0 $tPointerToArrayOfPointers = 0 DllCall("advapi32.dll", "NONE", "CredFreeW", "PTR", $hPointerToArrayOfPointers) EndIf The DllCall seems to function properly - I get a valid count of credentials (on my computer ~ 133) and a pointer "to array of pointers".
      What is meant by "array of pointers"?
      Microsoft says: Pointer to an array of pointers to credentials. The returned credential is a single allocated block. Any pointers contained within the buffer are pointers to locations within this single allocated block.
      How to access these pointers... Contained within the buffer???
      Any information you can provide me would be greatly appreciated.
×
×
  • Create New...