Sign in to follow this  
Followers 0
adamchapman

INetGet not applying filters in google search

11 posts in this topic

#1 ·  Posted (edited)

I'm trying to download the search results at https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws

 

My code looks like this:

#include <Array.au3>
#include <Inet.au3>

Local $sKeyword = 'HSBC'
Local $sSource = _INetGetSource('https://www.google.co.uk/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws')

ConsoleWrite ( $sSource)

Now if I copy the output text from ConsoleWrite() and save as an html file, opening that in the browser shows different search results from using the first url directly in the browser.

If anybody can find the issue I'd be very grateful. I'm very inexperienced with AutoIt so apologies if looks silly.

Edited by adamchapman

Share this post


Link to post
Share on other sites



That's the problem, I need the date filter applied:

'&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'

Share this post


Link to post
Share on other sites

#4 ·  Posted (edited)

winhttp.winhttprequest.5.1 whud not work on google so i guess this can

 

$data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'
$XML = ObjCreate("Microsoft.XMLHTTP")
$XML.Open("GET", $data, 0)
$XML.Send()
$sSource = $XML.Responsetext()
ConsoleWrite($sSource)
whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better... Edited by bogQ

TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better...

There must be a google search API somewhere...

Edited by FireFox

 

OS : Win XP SP2 (32 bits) / Win 7 SP1 (64 bits) / Win 8 (64 bits) | Autoit version: latest stable / beta.
Hardware : Intel(R) Core(TM) i5-2400 CPU @ 3.10Ghz / 8 GiB RAM DDR3.

My UDFs : Skype UDF | TrayIconEx UDF | GUI Panel UDF | Excel XML UDF | Is_Pressed_UDF

My Projects : YouTube Multi-downloader | FTP Easy-UP | Lock'n | WinKill | AVICapture | Skype TM | Tap Maker | ShellNew | Scriptner | Const Replacer | FT_Pocket | Chrome theme maker

My Examples : Capture toolIP Camera | Crosshair | Draw Captured Region | Picture Screensaver | Jscreenfix | Drivetemp | Picture viewer

My Snippets : Basic TCP | Systray_GetIconIndex | Intercept End task | Winpcap various | Advanced HotKeySet | Transparent Edit control

 

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

probably but if this works i dont see why shud he use google search API, advice is not to open it in browser after it save it to see end result, only coz source from the code will automatically redirect browser to original page instead to his saved source.

<div style="display:block">Please click <a href="https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr:1,cd_min:15/06/2013,cd_max:16/06/2013&tbm=nws&gbv=1&sei=Ba_BUafRBuaI4AT81YDgCQ">here</a> if you are not redirected within a few seconds.
No other reason. Edited by bogQ

TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 

Share this post


Link to post
Share on other sites

Fine.


 

OS : Win XP SP2 (32 bits) / Win 7 SP1 (64 bits) / Win 8 (64 bits) | Autoit version: latest stable / beta.
Hardware : Intel(R) Core(TM) i5-2400 CPU @ 3.10Ghz / 8 GiB RAM DDR3.

My UDFs : Skype UDF | TrayIconEx UDF | GUI Panel UDF | Excel XML UDF | Is_Pressed_UDF

My Projects : YouTube Multi-downloader | FTP Easy-UP | Lock'n | WinKill | AVICapture | Skype TM | Tap Maker | ShellNew | Scriptner | Const Replacer | FT_Pocket | Chrome theme maker

My Examples : Capture toolIP Camera | Crosshair | Draw Captured Region | Picture Screensaver | Jscreenfix | Drivetemp | Picture viewer

My Snippets : Basic TCP | Systray_GetIconIndex | Intercept End task | Winpcap various | Advanced HotKeySet | Transparent Edit control

 

Share this post


Link to post
Share on other sites

Thanks to all of you.

bogQ, your solution was superb. Very impressive

Share this post


Link to post
Share on other sites

winhttp.winhttprequest.5.1 whud not work on google so i guess this can

 

$data = 'https://www.google.com/search?q=HSBC&safe=off&hl=en&gl=uk&biw=1920&bih=979&sa=X&ei=jKG8UYLfGYKqOrbIgOgJ&ved=0CCAQpwUoBg&source=lnt&tbs=cdr%3A1%2Ccd_min%3A15%2F06%2F2013%2Ccd_max%3A16%2F06%2F2013&tbm=nws'
$XML = ObjCreate("Microsoft.XMLHTTP")
$XML.Open("GET", $data, 0)
$XML.Send()
$sSource = $XML.Responsetext()
ConsoleWrite($sSource)
whud not advise you to save as an html file and open it with browser to see end results, after save try with some html editor like frontpage if you have it or something better...

 

 

Is there a way to use a proxy server with this method?

I've managed to set up an http proxy server today ('?do=embed' frameborder='0' data-embedContent>>), but am having a similar problem as before, where the data returned through code is not the same as the data displayed through a browser. I hope this XML method would fix the issue as it did before

Share this post


Link to post
Share on other sites

I would not know if it's possible, I never had any need to use proxy on google before. either way reading from that link I don't like what are you trying to do

I did mention that winhttp.winhttprequest.5.1 will not work on google (but don't know why did you try to use it in that link) and larger automation on google site will lead to image verification that is needed to confirm that your not bot, if you are you talking about that quota limits I would suggest you to read forum rules - Part about: "Bypassing of security measures"


TCP server and client - Learning about TCP servers and clients connection
Au3 oIrrlicht - Irrlicht project
Au3impact - Another 3D DLL game engine for autoit. (3impact 3Drad related)



460px-Thief-4-temp-banner.jpg
There are those that believe that the perfect heist lies in the preparation.
Some say that it’s all in the timing, seizing the right opportunity. Others even say it’s the ability to leave no trace behind, be a ghost.

 

Share this post


Link to post
Share on other sites

Ok thanks for taking the time to have a look anyway.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0