Jump to content

Get the HTML response from cURL / StdOutRead with a DLL (libcurl)

Recommended Posts

I'm using to upload a file to a server. After the upload the server returns a URL to that file. My program needs to give that URL to the user.

libcURL is working fine and the upload is successful, and the HTML response needed is output to scite's console. I need the information put into the console to be stored in a variable, though, and I can't figure it out.

I think I need to use stdout or stdread to do this. I have used these before when using a run() command but this library is using a dll (and is much cleaner than my previous version which used run commands).

How do I record the output buffer of a DLL? (or whatever stdout is called).

This attachment is what I'm testing it with, just open "playground.au3" and run it. It should spit a bunch of information to the console. The top listing of variables is all from curl_getdata(), the part with brackets and arrows is from my web server. I need the second part to be put into a variable.

PS: The cURL process ID is stored as $__hcurl_process

seangriffin's manages to accomplish this, it looks like the key is to use "DllCallbackRegister". I'm trying to plug this in to smartee's UDF but the two UDF's look like completely different languages. I don't think I will be able to figure this out on my own, but I'm going to keep trying.

EDIT: You need to move the DLL's to the same folder as playground.au3, and edit "libcURL.au3" and change the #include to quotes instead of brackets. I packed it wrong.

Edited by Rad
Link to post
Share on other sites

Um, i don't see any attachment ;)

Here's a little demo showing how to save to a variable

#include <libcURL.au3>

Global $bMyData = Binary("")


Func _urlget($sURL)

    Local $curl_handle = _curl_easy_init()

    If ($curl_handle) Then

        Local $tCURLSTRUCT_URL = DllStructCreate("char[" & StringLen($sURL) + 1 & "]")
        DllStructSetData($tCURLSTRUCT_URL, 1, $sURL)
        _curl_easy_setopt($curl_handle, $CURLOPT_URL, DllStructGetPtr($tCURLSTRUCT_URL))

        _curl_easy_setopt($curl_handle, $CURLOPT_VERBOSE, 0)

        _curl_easy_setopt($curl_handle, $CURLOPT_FOLLOWLOCATION, 1)

        _curl_easy_setopt($curl_handle, $CURLOPT_NOPROGRESS, 1)

        Local $handle = DllCallbackRegister("_my_vwrite", "uint:cdecl", "ptr;uint;uint;ptr")
        _curl_easy_setopt($curl_handle, $CURLOPT_WRITEFUNCTION, DllCallbackGetPtr($handle))


EndFunc   ;==>_urlget

Func _my_vwrite($buffer, $size, $nmemb, $stream)
    Local $vData = DllStructCreate("byte[" & $size * $nmemb & "]", $buffer)
    $bMyData &= DllStructGetData($vData, 1)
    Return $size * $nmemb
EndFunc   ;==>_my_vwrite

Also you may want to take a look at the example for _curl_easy_setopt in the help-file, it shows how to save the output directly to a file.

Hope this helps :)

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By goku200
      I'm having an issue with my html paginated table. The script work as expected. It reads the html table and clicks on the Download button. However when it clicks on the next page its not iterating the items. instead it goes to the next URL from the spreadsheet and then iterates through the html table clicking the Download button and so on. Not sure why its doing that. I want it to click the next page and then continue iterating then after it has reached the end of the pagination go to the next url in the spreadsheet and repeat the process. Below is my script. Any help is appreciated ūüôā
    • By DevMode
      I need help dealing with a C++ DLL
      Specify that I need help with how to use this function
      struct libusb_device; typedef struct libusb_context libusb_context; ssize_t LIBUSB_CALL libusb_get_device_list(libusb_context *ctx,libusb_device ***list); My problem is how to deal with "libusb_device ***list"
      my code 
      $libusb_device = DllStructCreate("PTR") Func libusb_init() $Var = DllCall($libusb0, "ptr", "libusb_init",'ptr',$_libusb_device_handle_Ptr) If @error Then Return False Return $Var[1] EndFunc Func libusb_get_device_list() MsgBox(0,VarGetType($_libusb_device_handle_Ptr),$_libusb_device_handle_Ptr) $Var = DllCall($libusb0, "int", "libusb_get_device_list","ptr",$_libusb_device_handle_Ptr,"ptr*",POINTER(POINTER($libusb_device))) $Error = @error If $Error Then Return False MsgBox(0,VarGetType($Var),$Error&@CRLF&$Var) _ArrayDisplay($Var) Return $Var[1] EndFunc  
      Thank you
      iLibUSB.dll iLibUSB_32.dll libusb.h
    • By Hermes
      Hi, I have a site that has the following elements below: 
      <div>More element here</div> <div>More element here</div> <div>More element here</div> When I do this in Auto It:
      Local $oSelectDiv = _WD_FindElement($sSession, $_WD_LOCATOR_ByCSSSelector, "div") _WD_HighlightElement($sSession, $oSelectDiv, 1) I also tried to add [3], but it doesnt seems to work:
      Local $oSelectDiv = _WD_FindElement($sSession, $_WD_LOCATOR_ByCSSSelector, "div[3]") _WD_HighlightElement($sSession, $oSelectDiv, 1) It always highlight the first one, but I am trying to highlight the 3rd in the list. Is there anyway to select the 3rd div without having to add any class/id in the divs, and without using XPATH? The structure of the elements in that site were built that way.
    • By Pured
      I am looking to create a script which refreshes/reads a webpage every few seconds. My goal is to see if the page has changed, then I will send myself a notification that the webpage has been updated.
      However, rather than downloading the entire webpage every single time, is there a way to check when the webpage last updated?
      If not, is there away to partially download/read html source until a specific tag is hit?
      Goal: I would like to increase my poll rate and not excessively waste data.
    • By Mr_Microphone
      Alright, I may be an idiot.
      Three years ago, I wrote a program that pushed component information to a secure site via their API. I went back to add some attributes and (here's the idiot part) ended up losing the  source code and my modified code does not quite work. I have the compiled version that works minus the new attributes, so I know that their system has not changed. I stripped the larger program down from 3,000 lines to the part that is broken, but I am stumped. This was one of my first scripts, so it heavily leverages examples and isn't as pretty as I'd like it to be.
      Be gentle. 
      The program / script creates a new records as expected, but for some reason, I cannot access information in the response, which I need for a later step.
      I use Charles, a web debugging proxy tool so I can see the request and the response and both are as expected. Also, when I write to log file, the JSON reply is exactly what I expect and need, but when I try to do anything with the http body, it seems to be blank. 
      Here is the script minus  the URL and token:
      #include <Array.au3> #include <Curl.au3> #include <MsgBoxConstants.au3> #include <json.au3>  ; this was added as an alternate way to read the data Global $WM_serial_number = "WM20745001" Global $wm_component_status_id = "10" Global $wm_manufacturer ="Multi-Tech" Global $wm_model = "MTR-LAT1-B07" Global $cellular_carrier_id = "3" Global $iccid_esn = "89010303300012345678" Global $ip_address = "" Global $NewIDNumber     Local $Curl = Curl_Easy_Init()     Local $Html = $Curl ; any number as identify     Local $Header = $Curl + 1 ; any number as identify     Local $HtmlFile = "cURL_Request.html"     Local $File = FileOpen($HtmlFile, 2 + 16)     Local $Slist = Curl_Slist_Append(0, "content-type: multipart/form-data; boundary=---011000010111000001101001")     $Slist = Curl_Slist_Append($Slist, "authorization: Token token=" & $Token)     Curl_Easy_Setopt($Curl, $CURLOPT_PROXY, "") ; needed to use Charles web debugging proxy     Curl_Easy_Setopt($Curl, $CURLOPT_PROXYPORT, 8888) ; needed to use Charles     Curl_Easy_Setopt($Curl, $CURLOPT_HTTPHEADER, $Slist) ;     Curl_Easy_Setopt($Curl, $CURLOPT_URL, $Server & "wireless_module" & "s")     Curl_Easy_Setopt($Curl, $CURLOPT_SSL_VERIFYPEER, 0)     Curl_Easy_Setopt($Curl, $CURLOPT_TIMEOUT, 30)     Curl_Easy_Setopt($Curl, $CURLOPT_WRITEDATA, $Html)     Curl_Easy_Setopt($Curl, $CURLOPT_WRITEFUNCTION, Curl_FileWriteCallback())     Curl_Easy_Setopt($Curl, $CURLOPT_WRITEDATA, $File)     Local $HttpPost = ""     Local $LastItem = ""         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[serial_number]", $CURLFORM_COPYCONTENTS, $WM_serial_number, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[component_status_id]", $CURLFORM_COPYCONTENTS, $wm_component_status_id, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[manufacturer]", $CURLFORM_COPYCONTENTS, $wm_manufacturer, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[model]", $CURLFORM_COPYCONTENTS, $wm_model, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[cellular_carrier_id]", $CURLFORM_COPYCONTENTS, $cellular_carrier_id, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[iccid_esn]", $CURLFORM_COPYCONTENTS, $iccid_esn, $CURLFORM_END)         Curl_FormAdd($HttpPost, $LastItem, $CURLFORM_COPYNAME, "wireless_module" & "[ip_address]", $CURLFORM_COPYCONTENTS, $ip_address, $CURLFORM_END)         ; submit         Curl_Easy_Setopt($Curl, $CURLOPT_HTTPPOST, $HttpPost)         Local $Code = Curl_Easy_Perform($Curl)         If $Code = $CURLE_OK Then         ConsoleWrite("Content Type: " & Curl_Easy_GetInfo($Curl, $CURLINFO_CONTENT_TYPE) & @LF)         ConsoleWrite("Download Size: " & Curl_Easy_GetInfo($Curl, $CURLINFO_SIZE_DOWNLOAD) & @LF)         MsgBox(0, 'Html', BinaryToString(Curl_Data_Get($Html))) ; this is something I threw in for debugging, expecting to see SOMETHING. Returns nothing         MsgBox(0, 'Header', BinaryToString(Curl_Data_Get($Header))) ; this is something I threw in for debugging, expecting to see SOMETHING. Returns nothing         Local $response = Curl_Easy_GetInfo($Curl, $CURLINFO_RESPONSE_CODE)             If $response = "409" Then $response = "Failed due to a conflict."             If $response = "200" Then $response = "Was NOT created."             If $response = "201" Then $response = "Was created."             ; read the ID that was assigned and store it         $NewIDNumber = StringRight(StringLeft(BinaryToString(Curl_Data_Get($Html)), 10), 4) ; this DID work, but now it doesn't. An old compiled version still works ;~         Global $JsonObject = json_decode($Html); another debugging attempt. Did not use json functions previously and the program worked without it. ;~         Global $NewIDNumber = json_get($JsonObject, '.id')         ConsoleWrite(@CRLF &'! id:' & $NewIDNumber & @CRLF & @CRLF)    ; debugging feedback         MsgBox(0, $response, $wm_serial_number & " new ID = " & $NewIDNumber); debugging feedback         If $Code <> $CURLE_OK Then ConsoleWrite(Curl_Easy_StrError($Code) & @LF)             Local $Data = BinaryToString(Curl_Data_Get($Curl))             Curl_Easy_Cleanup($Curl)             Curl_Data_Cleanup($Curl)             Curl_Data_Cleanup($Header)             Curl_Data_Cleanup($Html)             Curl_FormFree($HttpPost)             Curl_slist_free_all($Slist)             curl_easy_reset($Curl)             FileClose($File)             ConsoleWrite(@LF)         EndIf  This is the captured request (minus the host and token)
      POST /api/v2/wireless_modules HTTP/1.1 Host: api. Accept: */* authorization: Token token= Content-Length: 942 Expect: 100-continue content-type: multipart/form-data; boundary=---011000010111000001101001; boundary=------------------------9adb0d87c7ea5061 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[serial_number]" WM20745001 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[component_status_id]" 10 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[manufacturer]" Multi-Tech --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[model]" MTR-LAT1-B07 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[cellular_carrier_id]" 3 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[iccid_esn]" 89010303300012345678 --------------------------9adb0d87c7ea5061 Content-Disposition: form-data; name="wireless_module[ip_address]" --------------------------9adb0d87c7ea5061-- and the captured response
      HTTP/1.1 201 Created Date: Sun, 04 Apr 2021 00:12:18 GMT Server: Apache Cache-Control: max-age=0, private, must-revalidate Access-Control-Allow-Origin: not-allowed Vary: Accept-Encoding Access-Control-Max-Age: 1728000 X-XSS-Protection: 1; mode=block X-Request-Id: 71cfcf36-6020-48a6-a822-d2b393a27b69 Access-Control-Allow-Credentials: true Access-Control-Allow-Methods: PUT, OPTIONS, GET, POST ETag: W/"25d97fe8a9387cb4b9029a9e62b0bfa2" X-Frame-Options: SAMEORIGIN, SAMEORIGIN X-Runtime: 0.344005 X-Content-Type-Options: nosniff Access-Control-Request-Method: * X-Powered-By: Phusion Passenger 5.2.1 Strict-Transport-Security: max-age=63072000; includeSubDomains; preload Location: /wireless_modules/3195 Status: 201 Created Connection: close Transfer-Encoding: chunked Content-Type: application/json; charset=utf-8 X-Charles-Received-Continue: HTTP/1.1 100 Continue {"id":3195,"model":"MTR-LAT1-B07","serial_number":"WM20745001","manufacturer":"Multi-Tech","mfg_date":null,"iccid_esn":"89010303300012345678","ip_address":"","purchase_order":null,"supplier":null,"cellular_carrier_id":3,"component_status_id":10,"component_status":{"id":10,"name":"Hold","description":"Available- Held for specific use"},"custom_attributes":[{"name":"Deactivated","type":"Boolean","value":false},{"name":"Port 3001","type":"Boolean","value":false}],"comments":[]}  
      Also attached is the log file. I need to read the id value. Clearly, it is arriving back to cURL, since it is being written out to the log, but I cannot seem to get to it within the code. 
      It is established that I may be an idiot, but this idiot has wasted days in non-billable hours trying to figure out what should be a simple glitch.
  • Create New...