Jump to content

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Find out more here. X
X


Photo

Using http proxy


  • Please log in to reply
3 replies to this topic

#1 adamchapman

adamchapman

    Seeker

  • Active Members
  • 46 posts

Posted 21 June 2013 - 09:15 AM

I'm trying to make my ip appear to change, in order to avoid hitting quota limits for downloads from google trends in a single day. 

 

First I looked at the HttpSetProxy function, which I soon discovered only supports CERN type proxy servers, of which I can't find in use today. 

 

I have found a list of free proxies which should require no passwords at hidemyass.com. A csv list of these is available from https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=hide_my_ass_proxy_list_ip&query=select+*+from+`hidemyass`&apikey=

 

I copied code from http://www.autoitscript.com/forum/topic/139374-winhttp-proxyauth/?p=977467 and cut out a couple of bits I that look necessary for my task. 

 

My current code for a randomly selected HTTP proxy in the csv list is:

AutoIt         
  1. $ini_proxy = "213.152.173.137:8080";
  2. $ini_proxy_name = ''
  3. $ini_proxy_passw = ''
  4.  
  5. Const $HTTPREQUEST_PROXYSETTING_DEFAULT = 0
  6. Const $HTTPREQUEST_PROXYSETTING_PRECONFIG = 0
  7. Const $HTTPREQUEST_PROXYSETTING_DIRECT = 1
  8. Const $HTTPREQUEST_PROXYSETTING_PROXY = 2
  9. Const $HTTPREQUEST_SETCREDENTIALS_FOR_SERVER = 0
  10. Const $HTTPREQUEST_SETCREDENTIALS_FOR_PROXY = 1
  11.  
  12. $o_http = ObjCreate("winhttp.winhttprequest.5.1")
  13.  
  14. If $ini_proxy <> '' Then
  15.     $o_http.SetProxy($HTTPREQUEST_PROXYSETTING_PROXY, $ini_proxy, "")
  16.     If $ini_proxy_name <> '' Then
  17.         $o_http.SetCredentials($ini_proxy_name, $ini_proxy_passw, $HTTPREQUEST_SETCREDENTIALS_FOR_PROXY)
  18.     EndIf
  19.  
  20.  
  21. $o_HTTP.Open('GET', 'http://www.whatismyip.com/', False)
  22. $o_HTTP.SetRequestHeader('Content-Type','application/x-www-form-urlencoded')
  23. $o_HTTP.SetRequestHeader("Connection", "Close")
  24. $o_HTTP.Send()
  25.  
  26. ConsoleWrite($o_HTTP.Responsetext & @CRLF)

The error mesage returned is:

(25) : ==> The requested action with this object has failed.:

$o_HTTP.Send()
$o_HTTP.Send()^ ERROR
>Exit code: 1    Time: 21.225
 
 
The execution time is pretty long so I guess it may have timed out.
 
I think I understand the majority of the code, but since the "SetRequestHeader" parts are simply copied from other people's scripts there's probably something missing from those.
 
If anybody can help it would be very useful. Alternatively if anybody knows of some CERN servers that are still in use today I'd be interested in using those instead 








#2 adamchapman

adamchapman

    Seeker

  • Active Members
  • 46 posts

Posted 21 June 2013 - 10:33 AM

Just found a simpler example at http://brugbart.com/Tutorials/autoit-proxy-winhttprequest.

 

Using this code:

  1. $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
  2. $oHTTP.Open("GET", "http://brugbart.com/", False)
  3. $oHttp.SetProxy(2,"84.20.84.33:8080")
  4.  
  5. $oHTTP.Send()
  6. $oReceived = $oHTTP.ResponseText
  7. $oStatusCode = $oHTTP.Status
  8.  
  9. If $oStatusCode == 200 then
  10.  $file = FileOpen("Received.html", 2) ; The value of 2 overwrites the file if it already exists
  11.  FileWrite($file, $oReceived)
  12.  FileClose($file)

Still fails on the $oHTTP.Send() command.

 

I'm deinately picking http proxy servers, so not sure what the problem is with this. Again, execution time is greater than 21 seconds



#3 adamchapman

adamchapman

    Seeker

  • Active Members
  • 46 posts

Posted 21 June 2013 - 12:21 PM

Just found another source (http://pastebin.com/JjQSLp4D) and adding a method "SertRequestHeader", which looks like it specifies the application to access the internet, finishes off the solutiion and allows the code to complete its job.

Allows the code to run successfully. So for anybody reading this in the future, working code looks like this:

  1. $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
  2.  
  3. $oHTTP.Open("GET", "http://www.whatismyip.com/", False)
  4.  
  5. $oHttp.SetProxy(2,"177.69.195.4:3128");"86.123.225.126:8080");"84.20.84.33:8080")
  6.  
  7. $oHttp.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5") ;
  8.  
  9.  
  10. $oHTTP.Send()
  11. $oReceived = $oHTTP.ResponseText
  12. $oStatusCode = $oHTTP.Status
  13.  
  14. If $oStatusCode == 200 then
  15.  $file = FileOpen("Received.html", 2) ; The value of 2 overwrites the file if it already exists
  16.  FileWrite($file, $oReceived)
  17.  FileClose($file)

 

 



#4 adamchapman

adamchapman

    Seeker

  • Active Members
  • 46 posts

Posted 21 June 2013 - 03:06 PM

Hmmm, still not perfect.

 

Trying to access the url: http://www.google.com/trends/fetchComponent?q=yahoo&geo=US&cid=TIMESERIES_GRAPH_0&export=3&date=today%203-m&cmpt=q,

data displayed when accessing in a browser is entirey different from the data returned through code.

 

code now looks like this:

AutoIt         
  1. $oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
  2.  
  3.  
  4. $oHTTP.Open("GET", "http://www.google.com/trends/fetchComponent?q=yahoo&geo=US&cid=TIMESERIES_GRAPH_0&export=3&date=today%203-m&cmpt=q", False)
  5.  
  6.  
  7. $oHttp.SetProxy(2,"177.69.195.4:3128");"86.123.225.126:8080");"84.20.84.33:8080")
  8.  
  9. $oHttp.SetRequestHeader("User-Agent", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5")
  10.  
  11.  
  12. $oHTTP.Send()
  13. $oReceived = $oHTTP.ResponseText
  14. $oStatusCode = $oHTTP.Status
  15.  
  16. If $oStatusCode == 200 then
  17.  $file = FileOpen("Received2.html", 2) ; The value of 2 overwrites the file if it already exists
  18.  FileWrite($file, $oReceived)
  19.  FileClose($file)
  20.  
  21.  
  22. Consolewrite($oStatusCode & @crlf)
  23. Consolewrite($oReceived & @crlf)

I had a similar issue yesterday (http://www.autoitscript.com/forum/topic/151868-inetget-not-applying-filters-in-google-search/) that was fixed using XML, but am struggling applying a proxy with that method

 






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users