adamchapman Posted June 19, 2013 Share Posted June 19, 2013 (edited) I'm trying to download the search results at https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws My code looks like this: #include <Array.au3> #include <Inet.au3> Local $sKeyword = 'HSBC' Local $sSource = _INetGetSource('https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws') ConsoleWrite ( $sSource) Now if I copy the output text from ConsoleWrite() and save as an html file, opening that in the browser shows different search results from using the first url directly in the browser. If anybody can find the issue I'd be very grateful. I'm very inexperienced with AutoIt so apologies if looks silly. Edited June 19, 2013 by adamchapman Link to comment Share on other sites More sharing options...
JohnOne Posted June 19, 2013 Share Posted June 19, 2013 Try losing the session stuff, and just using "https://www.google.co.uk/search?q=HSBC" AutoIt Absolute Beginners Require a serial Pause Script Video Tutorials by Morthawt ipify Monkey's are, like, natures humans. Link to comment Share on other sites More sharing options...
adamchapman Posted June 19, 2013 Author Share Posted June 19, 2013 That's the problem, I need the date filter applied: '&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws' Link to comment Share on other sites More sharing options...
Solution bogQ Posted June 19, 2013 Solution Share Posted June 19, 2013 (edited) winhttp.winhttprequest.5.1 whud not work on google so i guess this can $data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws' $XML = ObjCreate("Microsoft.XMLHTTP") $XML.Open("GET", $data, 0) $XML.Send() $sSource = $XML.Responsetext() ConsoleWrite($sSource)whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better... Edited June 19, 2013 by bogQ TCP server and client - Learning about TCP servers and clients connectionAu3 oIrrlicht - Irrlicht projectAu3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related) There are those that believe that the perfect heist lies in the preparation.Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost. Link to comment Share on other sites More sharing options...
FireFox Posted June 19, 2013 Share Posted June 19, 2013 (edited) whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better... There must be a google search API somewhere... Edited June 19, 2013 by FireFox Link to comment Share on other sites More sharing options...
bogQ Posted June 19, 2013 Share Posted June 19, 2013 (edited) probably but if this works i dont see why shud he use google search API, advice is not to open it in browser after it save it to see end result, only coz source from the code will automatically redirect browser to original page instead to his saved source.<div style="display:block">Please click <a href="https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr:1,cd_min:15/06/2013,cd_max:16/06/2013&tbm=nws&gbv=1&sei=Ba_BUafRBuaI4AT81YDgCQ">here</a> if you are not redirected within a few seconds.No other reason. Edited June 19, 2013 by bogQ TCP server and client - Learning about TCP servers and clients connectionAu3 oIrrlicht - Irrlicht projectAu3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related) There are those that believe that the perfect heist lies in the preparation.Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost. Link to comment Share on other sites More sharing options...
FireFox Posted June 19, 2013 Share Posted June 19, 2013 Fine. Link to comment Share on other sites More sharing options...
adamchapman Posted June 19, 2013 Author Share Posted June 19, 2013 Thanks to all of you. bogQ, your solution was superb. Very impressive Link to comment Share on other sites More sharing options...
adamchapman Posted June 21, 2013 Author Share Posted June 21, 2013 winhttp.winhttprequest.5.1 whud not work on google so i guess this can $data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws' $XML = ObjCreate("Microsoft.XMLHTTP") $XML.Open("GET", $data, 0) $XML.Send() $sSource = $XML.Responsetext() ConsoleWrite($sSource) whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better... Is there a way to use a proxy server with this method? I've managed to set up an http proxy server today ('?do=embed' frameborder='0' data-embedContent>>), but am having a similar problem as before, where the data returned through code is not the same as the data displayed through a browser. I hope this XML method would fix the issue as it did before Link to comment Share on other sites More sharing options...
bogQ Posted June 21, 2013 Share Posted June 21, 2013 I would not know if it's possible, I never had any need to use proxy on google before. either way reading from that link I don't like what are you trying to do I did mention that winhttp.winhttprequest.5.1 will not work on google (but don't know why did you try to use it in that link) and larger automation on google site will lead to image verification that is needed to confirm that your not bot, if you are you talking about that quota limits I would suggest you to read forum rules - Part about: "Bypassing of security measures" TCP server and client - Learning about TCP servers and clients connectionAu3 oIrrlicht - Irrlicht projectAu3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related) There are those that believe that the perfect heist lies in the preparation.Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost. Link to comment Share on other sites More sharing options...
adamchapman Posted June 21, 2013 Author Share Posted June 21, 2013 Ok thanks for taking the time to have a look anyway. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now