Sign in to follow this  
Followers 0
leuce

How to InetGet a logged-in page

4 posts in this topic

#1 ·  Posted

G'day everyone

I'm trying to download thousands of pages (JSON, actually) from a web site that shows one thing to users who are not logged in, and another thing to users who are logged in.  I want to download the pages while "logged in".  When I visit these pages in my browser, I get the "logged in" version of the pages, but when I use InetGet, I get the "not logged in" version of the pages.  I thought that since AutoIt uses Internet Explorer, it would work if I simply log in to that web site using Internet Explorer, but that doesn't work.

Is there a way to download web pages, using AutoIt more or less directly, that will get the "logged in" pages?

Thanks

Samuel

PS. The test URL I'm using is https://groups.yahoo.com/api/v1/groups/omegat/messages/19034/raw (increment the number).  When not logged in, all e-mail addresses on the page have their domain names removed.

Share this post


Link to post
Share on other sites



#2 ·  Posted

Have you checked out the WinHTTP functions?

 

Share this post


Link to post
Share on other sites

#3 ·  Posted

Thanks for the reply.  In the end, I found another solution: curl with cookies.

Install this extension for Firefox:
https://addons.mozilla.org/en-US/firefox/addon/export-cookies/?src=api

Then log in to Yahoogroups with Firefox to create a cookie.  Then export 
your cookies to e.g. "cookies.txt".  Optionally, remove all lines that 
do not begin with "yahoo.com", but I think it's safe to use the entire 
cookie file.

Then test it:
curl -b cookies.txt -4 
https://groups.yahoo.com/api/v1/groups/omegat/messages/19034/raw > 
message19034.txt

Share this post


Link to post
Share on other sites

#4 ·  Posted

Hi.

I have a similar question. How can I use third-party cookies stored in the cookies.txt file in order to be authorized on the site?

For example, the contents of the cookies.txt file looks like this:

site.ru    TRUE    /    FALSE    2986620187    logged    c5f89049a73fe53449795c1495f8bb1e4fd62c9f%7E1
site.ru    TRUE    /    FALSE    1493224330    authautologin    535b44ec66ae8b23f536c71da82b505653bfd%7E26c8b957f0e2ba3ca2d53f8995b9e306e1119
site.ru    TRUE    /    FALSE    1492057931    session_name    o2v8nees199of2o30lqdmjtob7
site.ru    TRUE    /    FALSE    1578414737    last_visit    1492000337234::1492014737234

Can I use the function InetRead() or InetGet() in order to reading cookies and send a request to the server, so that the authorization will be considered passed?

Thank you

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0