Retrieving a Secure Webpage Source

[spooky101]

Hello!

I've used AutoIt off an on for about a year by automating mundane tasks for my job. Which by the way really got me noticed, so I want to thank basically everyone on this forum! I Mostly use window management, Send, and Mouse functions, so i only really know some of the basics. I have a lot of time on my hands and I want use it by creating more efficient scripts.

This may be a stretch, but is it possible to pull a webpage source from a secure website using _INetGetSource without having to login and load the webpage each time? I'll be grabbing specific information from thoasands of orders on the website. Each order is a different URL. The script I'm working with is listed below, but I cant seem to figure out how to get passed the login page. Is there a way I could modify Internet Explorer.. (I believe cookies?) to ignore this login page all together? I've built another script that would pull information by simply logging in and using IE_Create / Navigate to load the pages and retreive the source using _IEBodyReadHTML, but I want to see if I can make it even better. Any help is appreciated!

#include <Excel.au3>
#include <Array.au3>
#include <IE.au3>
#include <String.au3>
#include <INet.au3>
$loop = 0
$filepath = "C:\Documents and Settings\Desktop\donors.xlsx"
$exceldoc = _ExcelBookOpen ($filepath)
Do
$order = _ExcelReadCell($exceldoc, 2 + $loop, 1)
$HTML = _INetGetSource ("webpage.com/"&$order)
$aTrackItems = _StringBetween($HTML, 'href="http', "</a>")
$order = _ExcelWriteCell($exceldoc, $aTrackItems, 2 + $loop, 2)
$loop = $loop +1
Until $loop = 5

Michael

[SmOke_N]

Rather than _InetGetSource(), you're probably more interested in using WinHTTP :

Although, I'm quite confused on what or why you're trying to bypass "Login", kind of a security flaw in their design if it actually allowed you to wouldn't you think?

Their cookies probably are probably date stamped and have expiration as well.

----

However, I have used stored cookies using IE.au3 by passing those in a made header ( look at __IE_Navigate(), (it's not in the help file, not sure why the "Post" navigate is a worker function only, but it is in the IE.au3 file ).

You can probably get the cookies from _IEPropertyGet, unless you're familiar with using the DOM methods and properties directly.

----

WinHttp has a bit of a learning curve, but those guys went all out even providing a help file, if you have time, that's a route I suggest, but I've found it easier for those sites that use .aspx (.net) to just use the IE or IE object in my own gui to do what I need/want.

----

Good luck!

----

Edit:

Just thought of something I did in the past, ( I can't remember exactly, was a couple of years ago ).

• _IECreate() ( hidden mode I did this );
• Login with credentials ( You'll need to get the input id's or names ( take a look at _IEForm* funcs )
• _InetGetSource or InetRead() ( I believe it was InetRead because I downloaded an xls file ).

I kept the browser open while the rest of my script worked, the cookie created from that login seemed to work, then on logoff I closed all my browser objects.

However, I know for sure it doesn't work anymore (Valik rewrote InetRead I believe), as I had to rewrite it completely, and I used winhttp.au3 for that.

Edited October 14, 2012 by SmOke_N

[spooky101]

Thanks for the help Smoke_N. I'll look into WinHTTP and see what I can come up with. I'm not really trying to bypass the login feature, just more or less tell Autoit to login and pull the source from within the website, or some how remember that I've logged in before so _Inetgetsource doesnt pull the login page.

Thanks again,

Michael