Sign in to follow this  
Followers 0

Using http proxy

4 posts in this topic

Posted

I'm trying to make my ip appear to change, in order to avoid hitting quota limits for downloads from google trends in a single day. 

First I looked at the HttpSetProxy function, which I soon discovered only supports CERN type proxy servers, of which I can't find in use today. 

I have found a list of free proxies which should require no passwords at hidemyass.com. A csv list of these is available from https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=hide_my_ass_proxy_list_ip&query=select+*+from+`hidemyass`&apikey=

I copied code from and cut out a couple of bits I that look necessary for my task. 

My current code for a randomly selected HTTP proxy in the csv list is:

$ini_proxy = "213.152.173.137:8080";
$ini_proxy_name = ''
$ini_proxy_passw = ''

Const $HTTPREQUEST_PROXYSETTING_DEFAULT = 0
Const $HTTPREQUEST_PROXYSETTING_PRECONFIG = 0
Const $HTTPREQUEST_PROXYSETTING_DIRECT = 1
Const $HTTPREQUEST_PROXYSETTING_PROXY = 2
Const $HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0
Const $HTTPREQUEST_SETCREDENTIALS_FOR_PROXY = 1

$o_http = ObjCreate("winhttp.winhttprequest.5.1")

If $ini_proxy <> '' Then
    $o_http.SetProxy($HTTPREQUEST_PROXYSETTING_PROXY, $ini_proxy, "")
    If $ini_proxy_name <> '' Then
        $o_http.SetCredentials($ini_proxy_name, $ini_proxy_passw, $HTTPREQUEST_SETCREDENTIALS_FOR_PROXY)
    EndIf
EndIf


$o_HTTP.Open('GET', 'http://www.whatismyip.com/', False)
$o_HTTP.SetRequestHeader('Content-Type','application/x-www-form-urlencoded')
$o_HTTP.SetRequestHeader("Connection", "Close")
$o_HTTP.Send()

ConsoleWrite($o_HTTP.Responsetext & @CRLF)

The error mesage returned is:

(25) : ==> The requested action with this object has failed.:

$o_HTTP.Send()
$o_HTTP.Send()^ ERROR
>Exit code: 1    Time: 21.225
 
 
The execution time is pretty long so I guess it may have timed out.
 
I think I understand the majority of the code, but since the "SetRequestHeader" parts are simply copied from other people's scripts there's probably something missing from those.
 
If anybody can help it would be very useful. Alternatively if anybody knows of some CERN servers that are still in use today I'd be interested in using those instead 

Share this post


Link to post
Share on other sites



Posted

Just found a simpler example at http://brugbart.com/Tutorials/autoit-proxy-winhttprequest.

Using this code:

$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
$oHTTP.Open("GET", "http://brugbart.com/", False)
$oHttp.SetProxy(2,"84.20.84.33:8080")

$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status

If $oStatusCode == 200 then
 $file = FileOpen("Received.html", 2) ; The value of 2 overwrites the file if it already exists
 FileWrite($file, $oReceived)
 FileClose($file)
 EndIf

Still fails on the $oHTTP.Send() command.

I'm deinately picking http proxy servers, so not sure what the problem is with this. Again, execution time is greater than 21 seconds

Share this post


Link to post
Share on other sites

Posted

Just found another source (http://pastebin.com/JjQSLp4D) and adding a method "SertRequestHeader", which looks like it specifies the application to access the internet, finishes off the solutiion and allows the code to complete its job.

Allows the code to run successfully. So for anybody reading this in the future, working code looks like this:

$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")

$oHTTP.Open("GET", "http://www.whatismyip.com/", False)

$oHttp.SetProxy(2,"177.69.195.4:3128");"86.123.225.126:8080");"84.20.84.33:8080")

$oHttp.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5") ;


$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status

If $oStatusCode == 200 then
 $file = FileOpen("Received.html", 2) ; The value of 2 overwrites the file if it already exists
 FileWrite($file, $oReceived)
 FileClose($file)
EndIf

 

Share this post


Link to post
Share on other sites

Posted

Hmmm, still not perfect.

Trying to access the url: http://www.google.com/trends/fetchComponent?q=yahoo&geo=US&cid=TIMESERIES_GRAPH_0&export=3&date=today%203-m&cmpt=q,

data displayed when accessing in a browser is entirey different from the data returned through code.

code now looks like this:

$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")


$oHTTP.Open("GET", "http://www.google.com/trends/fetchComponent?q=yahoo&geo=US&cid=TIMESERIES_GRAPH_0&export=3&date=today%203-m&cmpt=q", False)


$oHttp.SetProxy(2,"177.69.195.4:3128");"86.123.225.126:8080");"84.20.84.33:8080")

$oHttp.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5") 


$oHTTP.Send()
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status

If $oStatusCode == 200 then
 $file = FileOpen("Received2.html", 2) ; The value of 2 overwrites the file if it already exists
 FileWrite($file, $oReceived)
 FileClose($file)

 EndIf

Consolewrite($oStatusCode & @crlf)
Consolewrite($oReceived & @crlf)

I had a similar issue yesterday ('?do=embed' frameborder='0' data-embedContent>>) that was fixed using XML, but am struggling applying a proxy with that method

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0

  • Recently Browsing   0 members

    No registered users viewing this page.