Jump to content

How to collect a list of searching result?


oemript
 Share

Recommended Posts

When I type "k" in google, it displays a list of potential results as return, I would like to know on how to collect them into text file and where to look for those potential results from HTML viewsource.

Does anyone have any suggestions?
Thanks in advance for any suggestions

 

Google.png

Link to comment
Share on other sites

Using my signature:

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

output:

speed test
typing test
iq test
broadband speed test
personality test
myers briggs test
theory test
depression test
pregnancy test
driving test

Although, if some are searches you have done in the past (they will have a 'remove' link next to them), you would need to do 2 calls...

the second would be like this:

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

If the first array is not ubound=10, then you can do the second search to get 'previous searches history' data.

Edited by jdelaney
IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.
Link to comment
Share on other sites

Looking at the HTML source after typing in "k", the results were found in a DIV tag with the ID sbfbl.  I'm not sure you can make a POSt request to generate the auto fill list without manipulating the browser, but if you use the _IE UDF functions, I suspect you can automate entering the form and collecting the DIV (source) contents , then parse it.

 

edit: or just try @jdelaney's example.  :P didn't refresh the page before I posted to see your message.

Edited by spudw2k
Link to comment
Share on other sites

On 2/2/2018 at 9:43 AM, jdelaney said:

Using my signature:

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

output:

speed test
typing test
iq test
broadband speed test
personality test
myers briggs test
theory test
depression test
pregnancy test
driving test

Although, if some are searches you have done in the past (they will have a 'remove' link next to them), you would need to do 2 calls...

the second would be like this:

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

If the first array is not ubound=10, then you can do the second search to get 'previous searches history' data.

I would like to know on whether it can be coded using PowerShell Scripts or not.

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

 

Link to comment
Share on other sites

 I would like to know on how to interpret following statement.  Does it from the viewsource in HTML code?

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
Link to comment
Share on other sites

Google xpath query...2 slashes means to look as far into the html structure as needed.  The brackets show constraints of the current node you are looking for.   @ signifies you are looking for an attribute. 

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.
Link to comment
Share on other sites

On 2/6/2018 at 4:36 AM, jdelaney said:

Google xpath query...2 slashes means to look as far into the html structure as needed.  The brackets show constraints of the current node you are looking for.   @ signifies you are looking for an attribute. 

I cannot find class='sbqs_c' under following view-source, could you have any suggestions on where class='sbqs_c' is existed?

view-source:https://www.google.com/

Do you have any suggestions?
Thanks, to everyone very much for any suggestions (^v^)

 

Edited by oemript
Link to comment
Share on other sites

I find it, and would like to know on what PowerShell function should be used to perform similar task as shown below:

Step 1 Open "Google.com"

Step 2 Input "k" into searching bar

Step 3 Retrieve the display results

$r.SelectNodes('//li[@class="sbsb_c gsfs")]//div[@class="sbqs_c"]')

Step 4 Output the results into C:\Sample.txt file

$r.SelectNodes('//????')

Do you have any suggestions on what PowerShell function should be used?
Thank you very much for any suggestions (^v^)

 

sbsb_c.png

Edited by oemript
Link to comment
Share on other sites

  • 2 weeks later...
On 2/2/2018 at 9:43 AM, jdelaney said:

 

$oIE = _IECreate("Google.com")
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
_ArrayDisplay($aReturnResults)
For $i = 0 To UBound($aReturnResults)-1
    ConsoleWrite($aReturnResults[$i].Innertext & @CRLF)
Next

I get an error when I run the execute file, please see attached image

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

Google1.png

 

I get the same problem for following code as well

$sXpath = "//li[@class='sbsb_c gsfs')]//span[@class='sbpqs_a']"

 

 

Google2.png

Edited by oemript
Link to comment
Share on other sites

Please see following image as shown below, I get an error when I run the execute file, I already updated with latest version.

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

 

URL3.png

Edited by oemript
Link to comment
Share on other sites

AutoIT3 is installed under C:\Program Files (x86)\AutoIt3, and I have searched C:\ drive, IEbyXpath.au3 is not existed at all.

My scripts are saved under D:\ drive

I already updated with latest version, should IEbyXpath.au3 be included within its own library under C:\Program Files (x86)\AutoIt3 directory ?

Do you have any suggestions on where to get this file?
Thank you very much for any suggestions (^v^)

Link to comment
Share on other sites

I copy the first post to a new Scite document, and delete following section and  save that as D:\Scripts\IEbyXPath.au3

#region SAMPLE
...
#endregion SAMPLE

After compile and run it, an error occurs on line 5399 within IEbyXPath.au3 file as shown on below image,

Furthermore, I also test IEbyXPath.au3 with given SAMPLE by compiling IEbyXPath.au3 and run IEbyXPath.exe, I also get the same error.

Do you have any suggestions on what wrong it is?
Thank you very much for any suggestions (^v^)

 

Google3.png

Edited by oemript
Link to comment
Share on other sites

Try:

#include "IEbyXPath.au3"
$oIE = _IECreate("Google.com")
_IELoadWait($oIE)
Sleep(5000) ;~ Wait 5 seconds, extend it if you still get an error
$oInput = _IEGetObjById($oIE,'lst-ib')
_IEFormElementSetValue($oInput,"test")
_IEAction($oInput,"blur")

$sXpath = "//li[@class='sbsb_c gsfs')]//div[@class='sbqs_c']"
$aReturnResults = BGe_IEGetDOMObjByXPathWithAttributes($oIE,$sXpath,5000)
For $i = 0 To UBound($aReturnResults)-1
  $aReturnResults[$i] = $aReturnResults[$i].Innertext
Next
_ArrayDisplay($aReturnResults)

 

Link to comment
Share on other sites

I still get the same error, which come from IEbyXPath.au3 file.

Within IEbyXPath.au3 file, it contains following files, do I need to place those files under the same directory as well? if yes, where to get those files?

#include <ie.au3>
#include <array.au3

Do you have any suggestions?
Thank you very much for any suggestions (^v^)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...