Jump to content
oemript

How to collect a list of searching result?

Recommended Posts

oemript

When I type "k" in google, it displays a list of potential results as return, I would like to know on how to collect them into text file and where to look for those potential results from HTML viewsource.

Does anyone have any suggestions?
Thanks in advance for any suggestions

 

Google.png

Share this post


Link to post
Share on other sites
jdelaney

Using my signature:

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

output:

speed test
typing test
iq test
broadband speed test
personality test
myers briggs test
theory test
depression test
pregnancy test
driving test

Although, if some are searches you have done in the past (they will have a 'remove' link next to them), you would need to do 2 calls...

the second would be like this:

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

If the first array is not ubound=10, then you can do the second search to get 'previous searches history' data.

Edited by jdelaney
  • Like 2

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
spudw2k

Looking at the HTML source after typing in "k", the results were found in a DIV tag with the ID sbfbl.  I'm not sure you can make a POSt request to generate the auto fill list without manipulating the browser, but if you use the _IE UDF functions, I suspect you can automate entering the form and collecting the DIV (source) contents , then parse it.

 

edit: or just try @jdelaney's example.  :P didn't refresh the page before I posted to see your message.

Edited by spudw2k

Share this post


Link to post
Share on other sites
oemript
On 2/2/2018 at 9:43 AM, jdelaney said:

Using my signature:

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

output:

speed test
typing test
iq test
broadband speed test
personality test
myers briggs test
theory test
depression test
pregnancy test
driving test

Although, if some are searches you have done in the past (they will have a 'remove' link next to them), you would need to do 2 calls...

the second would be like this:

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

If the first array is not ubound=10, then you can do the second search to get 'previous searches history' data.

I would like to know on whether it can be coded using PowerShell Scripts or not.

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

 

Share this post


Link to post
Share on other sites
jdelaney

Of course.   You'd need to convert everything, or make your own loops. 


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
oemript

 I would like to know on how to interpret following statement.  Does it from the viewsource in HTML code?

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"

Share this post


Link to post
Share on other sites
jdelaney

Google xpath query...2 slashes means to look as far into the html structure as needed.  The brackets show constraints of the current node you are looking for.   @ signifies you are looking for an attribute. 


IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites
oemript
On 2/6/2018 at 4:36 AM, jdelaney said:

Google xpath query...2 slashes means to look as far into the html structure as needed.  The brackets show constraints of the current node you are looking for.   @ signifies you are looking for an attribute. 

I cannot find class='sbqs_c' under following view-source, could you have any suggestions on where class='sbqs_c' is existed?

view-source:https://www.google.com/

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

 

Edited by oemript

Share this post


Link to post
Share on other sites
oemript

I find it, and would like to know on what PowerShell function should be used to perform similar task as shown below:

Step 1 Open "Google.com"

Step 2 Input "k" into searching bar

Step 3 Retrieve the display results

$r.SelectNodes('//li[@class="sbsb_c gsfs")]//div[@class="sbqs_c"]')

Step 4 Output the results into C:\Sample.txt file

$r.SelectNodes('//????')

Do you have any suggestions on what PowerShell function should be used?
Thank you very much for any suggestions (^v^)

 

sbsb_c.png

Edited by oemript

Share this post


Link to post
Share on other sites
oemript
On 2/2/2018 at 9:43 AM, jdelaney said:

 

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

I get an error when I run the execute file, please see attached image

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

Google1.png

 

I get the same problem for following code as well

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

 

 

Google2.png

Edited by oemript

Share this post


Link to post
Share on other sites
Subz

You need to go to the link @jdelaney posted above

Copy the code from the first post into a new script and save it as IEbyXPath.au3 in the same location as your script.

At the top of your script add:

#include "IEbyXPath.au3"
...rest of your code goes here...

The IEbyXPath includes the function 

BGe_IEGetDOMObjByXPathWithAttributes

Share this post


Link to post
Share on other sites
oemript

Please see following image as shown below, I get an error when I run the execute file, I already updated with latest version.

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

 

URL3.png

Edited by oemript

Share this post


Link to post
Share on other sites
Subz

Which linked image?

Share this post


Link to post
Share on other sites
oemript

When I attached an image, it automatically insert into content, as show above.

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

Share this post


Link to post
Share on other sites
Subz

Do you have the IEbyXpath.au3 in the same folder as your script?

Share this post


Link to post
Share on other sites
oemript

AutoIT3 is installed under C:\Program Files (x86)\AutoIt3, and I have searched C:\ drive, IEbyXpath.au3 is not existed at all.

My scripts are saved under D:\ drive

I already updated with latest version, should IEbyXpath.au3 be included within its own library under C:\Program Files (x86)\AutoIt3 directory ?

Do you have any suggestions on where to get this file?
Thank you very much for any suggestions (^v^)

Share this post


Link to post
Share on other sites
Subz

As I said you need to go to the first post of the link I provided above, copy the code in the first post to a new Scite document, save that as D:\Scripts\IEbyXPath.au3, you can copy it to C:\Program Files (x86)\AutoIt3\Include if you wanted.

Share this post


Link to post
Share on other sites
oemript

I copy the first post to a new Scite document, and delete following section and  save that as D:\Scripts\IEbyXPath.au3

#region SAMPLE
...
#endregion SAMPLE

After compile and run it, an error occurs on line 5399 within IEbyXPath.au3 file as shown on below image,

Furthermore, I also test IEbyXPath.au3 with given SAMPLE by compiling IEbyXPath.au3 and run IEbyXPath.exe, I also get the same error.

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

 

Google3.png

Edited by oemript

Share this post


Link to post
Share on other sites
Subz

Try:

#include "IEbyXPath.au3"
$oIE = _IECreate("Google.com")
_IELoadWait($oIE)
Sleep(5000) ;~ Wait 5 seconds, extend it if you still get an error
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
For $i = 0 To UBound($aReturnResults)-1
  $aReturnResults[$i] = $aReturnResults[$i].Innertext
Next
_ArrayDisplay($aReturnResults)

 

Share this post


Link to post
Share on other sites
oemript

I still get the same error, which come from IEbyXPath.au3 file.

Within IEbyXPath.au3 file, it contains following files, do I need to place those files under the same directory as well? if yes, where to get those files?

#include <ie.au3>
#include <array.au3

Do you have any suggestions?
Thank you very much for any suggestions (^v^)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×