Jump to content
Suan

Downloading web content using Autoit

Recommended Posts

Suan

Hi Autoit Experts,

I am pretty new to AutoIt, and have this requirement, where I have to download the content of a webpage using Autoit. I have tried InetGet() but the downloaded file contains HTML of the web page without the content of the webpage.

Is there a way I can achieve this. The download format I want to get it CSV or TXT files.

Thanks upfront.

Regards,
Suan.

Edited by Suan

Share this post


Link to post
Share on other sites
Melba23

Suan,

Welcome to the AutoIt forum.

But please pay attention to where you post - the "Dev Chat" section where you started this thread is not for general support questions.  I have moved it for you, but would ask you to be more careful in future.

M23


Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind._______My UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Share this post


Link to post
Share on other sites
Suan

Hi Melba23,

Thank you. I assure you, I will be careful in the future. My apology.

Suan.

Share this post


Link to post
Share on other sites
antfuentes87

I created this simple function to retrieve any web page. You can then do whatever you want with the data after that (write it to a file, etc...). Let me know if you have any questions.

Func _HTTP_ResponseText($URL)
   $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
   $oHTTP.Open("GET", $URL)
   $oHTTP.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10 (.NET CLR 4.0.20506)")
   $oHTTP.Send()
   return $oHTTP.ResponseText
EndFunc
Edited by antfuentes87
  • Like 1

Share this post


Link to post
Share on other sites
Suan

Hi A87,

Thanks for the response. The website from which I need to retrieve the info is password protected.

Is there a way that I can achieve this? I have found it for IE, but I want it for Chrome, Firefox etc.

If you could provide me this, I would be most grateful.

Thanks.

Suan

Share this post


Link to post
Share on other sites
Suan

I am now able to connect it using the below code. Thanks again A87.

 

#include <INet.au3>
#include <String.au3>

$URL = "url123456"
Global $loginBody = "username=myusername&password=myapassword&autologin=on"
$source = _HTTP_ResponseText($URL)
FileWrite(@ScriptDir & "\winhttp.winhttprequest.5.1.html",$source); 
$read = FileRead(@ScriptDir & "\winhttp.winhttprequest.5.1.html") 
$readtitle = _StringBetween($read, "<head>","</head>") ;read title from file
MsgBox(0,"",$readtitle[0]) ;display first found string

Func _HTTP_ResponseText($URL)
   $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
   $oHTTP.Open("POST", $URL)
   $oHTTP.SetRequestHeader("Content-Type","application/x-www-form-urlencoded")
   MsgBox (0, "", "URL : " &$URL)
   $oHTTP.Send($loginBody)
   return $oHTTP.ResponseText
EndFunc

Using the above code, I am able to log in. But only issue is it takes me to a different url location different from url123456. Any idea how to fix this?

 

Regards,

Suan

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×