Jump to content
seangriffin

cURL UDF - a UDF for transferring data with URL syntax

Recommended Posts

got same problem.

EDIT: was reading other examples and find that:

Dlls are 32bit. That means you have to run 32bit AutoIt when using them.

BTW is there any way to get content from a site that generate it from JS?

I'll update the top post to indicate Win 32-bit is a requirement thanks. Yes I only develop under 32-bit sorry, but my guess is if you downloadeded the libcurl 64-bit DLLs then it would work?

I'm no expert in cURL so not sure whether pulling dynamic JS is possible. I know the UDF is only simple right now, mainly as a solution to suit my own needs. Would love to put more effort into it.


Cheers, Sean.

See my other UDFs:

Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia

Share this post


Link to post
Share on other sites

V0.3 is now available.

The downloads on page 1 been updated.

New features include cURL multi-mode support. cURL_multi can be used in place of cURL_easy to avoid the issue where cURL easy mode blocks certain functions in Windows (such as opening a MS Office document).


Cheers, Sean.

See my other UDFs:

Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia

Share this post


Link to post
Share on other sites

...I'm finding easy mode is blocking certain activities in Windows 32-bit when it runs, and slows my computer such that I can't get much work done..

;) That shouldn't be.

In my experience, using Win7 32 and 64bit versions, the easy interface runs quite reliably (in harmony with MS office too), which OS are you using?

The poor performance is maybe because you download the entire internet resource into memory($curlout) with _WriteFunc() (even when saving to disk) so large files => massive memory usage. :oops:

Why not use the stream parameter, and write directly to file(when required), and so maintain a small buffer size, like the examples on libcurl's website? :bye:

With respect to the wrapper UDFs I have almost completed the easy interface (largely untested though) I will post it in a new thread in the coming week. (named libcurl UDF maybe? hope there's no confusion ;)).

I'm sure you'll see many ways to improve your functions using the libcurl UDF.

Now its just a matter of finding time to work on it :doh:

Edit: fixed typos

Edited by smartee

Share this post


Link to post
Share on other sites

V0.4 is now available.

The downloads on page 1 been updated.

This version includes HTTP Header support.


Cheers, Sean.

See my other UDFs:

Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia

Share this post


Link to post
Share on other sites

:bye: That shouldn't be.

In my experience, using Win7 32 and 64bit versions, the easy interface runs quite reliably (in harmony with MS office too), which OS are you using?

The poor performance is maybe because you download the entire internet resource into memory($curlout) with _WriteFunc() (even when saving to disk) so large files => massive memory usage. :oops:

Perhaps, though the HTTP responses I get here in the office aren't big. You may not notice the delay unless you are executing cURL in rapid succession many times over. I am using it to make HTTP requests to a RESTful web service that responds in pages. I have to repeatedly execute the same requests collecting each page one at a time. I need to make over 100 requests each day to collect the complete dataset I need. If I switch to cURL multi-mode I don't experience any blocking effects in Win XP, however the requests take quite a bit longer to finish (from 2 hours using cURL easy to about 5 hours using cURL multi). If you can come up with a better implementation I can trial it for you on my OS.

I'm sure you'll see many ways to improve your functions using the libcurl UDF.

No doubt, I this put together in a tremendous rush as I desparately needed it at work.

Edited by seangriffin

Cheers, Sean.

See my other UDFs:

Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia

Share this post


Link to post
Share on other sites

V0.5 is now available.

The downloads on page one been updated.

Just a minor update to increase the maximum length of URLs to 65,536 characters.


Cheers, Sean.

See my other UDFs:

Chrome UDF - Automate Chrome | SAP UDF - Automate SAP | Java UDF - Automate Java Applications & Applets | Tesseract (OCR) UDF - Capture text from applications, controls and the desktop | Textract (OCR) UDF - Capture text from applications and controls | FileSystemMonitor UDF - File, Folder, Drive and Shell Monitoring | VLC (Media Player) UDF - Creating and controlling a VLC control in AutoIT | Google Maps UDF - Creating and controlling Google Maps (inc. GE) in AutoIT | SAPIListBox (Speech Recognition) UDF - Speech Recognition via the Microsoft Speech (SAPI) ListBox | eBay UDF - Automate eBay using the eBay API | ChildProc (Parallel Processing) UDF - Parallel processing functions for AutoIT | HyperCam (Screen Recording) UDF - Automate the HyperCam screen recorder | Twitter UDF - Automate Twitter using OAuth and the Twitter API | cURL UDF - a UDF for transferring data with URL syntax

See my other Tools:

Rapid Menu Writer - Add menus to DVDs in seconds | TV Player - Automates the process of playing videos on an external TV / Monitor | Rapid Video Converter - A tool for resizing and reformatting videos | [topic130531]Rapid DVD Creator - Convert videos to DVD fast and for free | ZapPF - A tool for killing processes and recycling files | Sean's eBay Bargain Hunter - Find last minute bargains in eBay using AutoIT | Sean's GUI Inspector - A scripting tool for querying GUIs | TransLink Journey Planner with maps - Incorporating Google Maps into an Australian Journey Planner | Automate Qt and QWidgets | Brisbane City Council Event Viewer - See what's going on in Brisbane, Australia

Share this post


Link to post
Share on other sites

Hi Sean,

great piece of work. I am using this extensively. Can you add some way to timeout. I know there is a LIBCURL timeout easyopt. I access remote equipment with your curl udf. If that equipment is down the default timeout is very long.

Thanks again

Greg

Share this post


Link to post
Share on other sites

I need a little help, I found a php file that uses curl to pull the warranty info from a dell site. The php file uses a cookie to pass the service tag to dell and pulls the raw data back then processes that data. I'm still at the point of getting the cookie to pass to dell. I have tried several different things for the past several hours and can only seem to get this to work with php on linux; not autoit on windows ;)

If any one has any ideas please let me know; I'm so frustrated with this right now.

#include <cURL.au3>
cURL_initialise()
$url = "http://www.dell.com/support/troubleshooting/us/en/04/TroubleShooting/Display_Warranty_Tab?name=TroubleShooting_WarrantyTab"
$cookie_file = @ScriptDir & "cookie.txt"
$out = cURL_easy($url, $cookie_file, 1, 0, "", "", 0, 1, 1)
; Output the HTML to the console
;~ ConsoleWrite($out)
MsgBox(0, "", $out)
; Clean up cURL
cURL_cleanup()

the cookie in php

$strCookie = "name=OLRProduct; OLRProduct=".$ServiceTag."|;";

Share this post


Link to post
Share on other sites

Hi Sean,

Am getting an error for the code.

#include "cURL.au3"

cURL_initialise()
$url = 'http://www.google.com'
$cookie_file = 'cookie.txt'
$cookie_action = 2
$output_type = 1 ;binary
$output_file = 'output.txt'
$postfields = ''
$ssl_verifypeer = 0
$noprogress = 0
$followlocation = 1
$http_headers = 'Content-Encoding,Content-Language,Content-Length,Content-Location,Content-Disposition,Content-Type'
cURL_easy($url, $cookie_file, $cookie_action, $output_type, $output_file, $postfields, $ssl_verifypeer, $noprogress, $followlocation, $http_headers)
cURL_cleanup()

Error:

>Running:(3.3.8.1):C:\Program Files\AutoIt3\autoit3.exe "C:\a\cURL_test.au3"
C:\a\cURL.au3 (124) : ==> Subscript used with non-Array variable.:
$hCurlHandle = $hCurlHandle[0]
$hCurlHandle = $hCurlHandle^ ERROR
->12:53:10 AutoIT3.exe ended.rc:1

Share this post


Link to post
Share on other sites

Error!

cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,

Share this post


Link to post
Share on other sites

Can anyone help? It says

C:\Users\******\AutoIt\UDF\cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.

The line is

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle, "uint", $CURLOPT_HTTPHEADER, "ptr", $curl_slist_struct)

but i believe it should be

DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlEasyHandle, "uint", $CURLOPT_HTTPHEADER, "ptr", $curl_slist_struct)

because the previous lines have $hCurlEasyHandle and also since it is doing "curl_easy_setopt"

Also, when I run the example 1 code, it returns nothing but instead exits with exit code -1073741819. The script doesn't even finish, almost as if the dll made it exit. I tried adding a ConsoleWrite("Hi") at the end but it doesn't get printed. Thus, there's no way to check any @error variables or things like it. Any ideas?

Here's the code I ran:

#include 

; Initialise cURL
cURL_initialise()

; Return the HTML from "http://www.autoitscript.com/site"
$out = cURL_easy("http://www.autoitscript.com/site/")

; Output the HTML to the console
ConsoleWrite("-----BEGIN HTML-----"&@CRLF&$out&@CRLF&"-----END HTML-----")

; Clean up cURL
cURL_cleanup()

Output:

>"C:\Program Files (x86)\AutoIt3\SciTE\AutoIt3Wrapper\AutoIt3Wrapper.exe" /run /prod /ErrorStdOut /in "C:\Users\*****\Downloads\AutoIt\cURL test.au3" /UserParams   
+>14:19:06 Starting AutoIt3Wrapper v.2.1.0.33    Environment(Language:0409  Keyboard:00000409  OS:WIN_7/Service Pack 1  CPU:X64 OS:X64)
>Running AU3Check (1.54.22.0)  from:C:\Program Files (x86)\AutoIt3
+>14:19:06 AU3Check ended.rc:0
>Running:(3.3.8.1):C:\Program Files (x86)\AutoIt3\autoit3.exe "C:\Users\*****\Downloads\AutoIt\cURL test.au3"   
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
!>14:19:22 AutoIT3.exe ended.rc:-1073741819
>Exit code: -1073741819    Time: 18.048
Edited by Smiley43210

Share this post


Link to post
Share on other sites

Reading HTML from a webpage, and writing the reg File.

Hi.

Me Need Help For Read URL.com/text.txt and writing the Windows registry.

InetGet("http://www.test.com/test.txt", @TempDir & "\test.txt", 1, 1)

Thanks A lot.

Edited by AHRIMANSEFID

Share this post


Link to post
Share on other sites
Thank you seangriffin, your cURL UDF is very useful.

Could someone let me know what is wrong with the below AutoIt Syntax?

$out = cURL_easy("https://docs.google.com/forms/d/162F4JLfnnU1U61Y_ZhIYb7yV_ZV2y65y6BAcR2N9IeM/formResponse",,,,,"entry.156901506='google@gmail.com'&draftResponse=[]&pageHistory=0")

I've created a sample form in Google docs and passing on email value.

Share this post


Link to post
Share on other sites

Hey! 

Tring to use UDF like in example

#include <cURL.au3>

; Initialise cURL
cURL_initialise()

; Return the HTML from "http://www.autoitscript.com/site"
$out = cURL_easy("http://www.autoitscript.com/site/")

; Output the HTML to the console
ConsoleWrite($out)

; Clean up cURL
cURL_cleanup()

And getting 

+>21:19:54 Starting AutoIt3Wrapper v.2.1.3.0 SciTE v.3.3.6.0 ;  Keyboard:00000409  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86    Environment(Language:0419  Keyboard:00000409  OS:WIN_XP/Service Pack 3  CPU:X64 OS:X86)
>Running:(3.3.8.0):C:Program FilesAutoIt3autoit3.exe "C:Documents and SettingsAdminМои документыDropboxsmscurlcurl practice.au3"    
--> Press Ctrl+Alt+F5 to Restart or Ctrl+Break to Stop
!>21:19:56 AutoIt3.exe ended.rc:-1073741819
>Exit code: -1073741819    Time: 3.505
 
Well usng http://fossies.org/windows/www/curl-7.34.0-ssl-sspi-zlib-static-bin-w32.zip/ and after scipt said i need zlib downloaded it from http://www.zlib.net/ version 1.2.8. I guess i need another version of curl lib ? Can some one upload it plz..
 
UPD:
Last one was from ruiing on X, now after try it on Win 7 i got more information - 
Problem signature:
   Problem Event Name: APPCRASH
   Application Name: autoit3.exe
   Application Version: 3.3.10.2
   Application Timestamp: 52c196bd
   Name of the module with the error: StackHash_0a9e
   Version of the module with the error: 0.0.0.0
   The time stamp module with the error: 00000000
   Exception Code: c0000005
   Exception Offset: 00000115
   OS Version: 6.1.7601.2.1.0.256.1
   Language Code: 1049
   Additional Information 1: 0a9e
   Additional information 2: 0a9e372d3b4ad19135b953a78882e789
   Additional Information 3: 0a9e
   Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

 

and in autoit >Exit code: -1073741819  

Edited by babullika

Share this post


Link to post
Share on other sites

i'm trying to use this UDF however openDLL cannot open libcurl.dll and the entire script fails

libcurl.dll/libeay32.dll/sslea32.dll exist in the working dir and in system32

i tried both 32 and 64bit versions of the files...

running windows 7 ultimate x64

Share this post


Link to post
Share on other sites

Hey guys. I'm trying to figure out how to use this UDF. Each time I run it using the sample script, I keep getting an error.

>Running AU3Check (1.54.22.0)  from:C:\Program Files\AutoIt3
C:\Documents and Settings\salespc2\My Documents\websiteScripts\addudfs\cURL.au3(411,79) : WARNING: $hCurlHandle: possibly used before declaration.
        DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\Documents and Settings\salespc2\My Documents\websiteScripts\addudfs\cURL.au3(411,79) : ERROR: $hCurlHandle: undeclared global variable.
        DllCall($hDll_LibCurl, "uint:cdecl", "curl_easy_setopt", "ptr", $hCurlHandle,
        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
C:\curl-7.34.0-ssl-sspi-zlib-static-bin-w32\curltest.au3 - 1 error(s), 1 warning(s)
!>11:26:06 AU3Check ended. Press F4 to jump to next error.rc:2
>Exit code: 2    Time: 5.712

Am I doing something wrong? I placed the sample script in the DLL files folder like instructed.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...