Jump to content

WebDriver click on text to load content

maninternet
 Share

Recommended Posts

maninternet Seeker

maninternet

Posted
Posted

Hi

I try to use WebDriver over Chrome to retrieve information from a website. I have problem that there is a "Word" that normally need to click to let the website show more content.

The HTML before click on the "Word" is below. 

<div role="button" class="px-3 d-flex bg-white">
<span class="title-font-family fs-24px me-2">More</span>
<div class="icon-circle ms-2">
<svg xmlns="http://www.w3.org/2000/svg" width="1em" height="1em" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather feather-plus plus">
<line x1="12" y1="5" x2="12" y2="19"></line><line x1="5" y1="12" x2="19" y2="12">
</line></svg> <!----></div></div>

I try to use _WD_ElementAction / _WD_ElementActionEx / _WD_LinkClickByText but it cannot active click to let the website show content. (I already check in HTML and confirmed that the content not just hide by script but will loaded after the code above was clicked.)

Can anyone advise for the correct code to use? Below is code to for FindElement, it already confirm can use to get element <span> in above code. 

$CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//div[@class='news-finance-state mb-4']/div[2]/div[6]/div/span")

 

Link to comment
Share on other sites
maninternet Seeker

maninternet

Posted
  • Author
Posted

Thank you for your suggestion. I try to use action on element DIV instead but it still cannot click to load content. 

$CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//div[@class='news-finance-state mb-4']/div[2]/div[6]/div")
        _WD_ElementActionEx($sSession,$CheckText,"CLICK")

 

Link to comment
Share on other sites
maninternet Seeker

maninternet

Posted
  • Author
Posted

Sorry as the result still same previous so I didn't explain much for the details.

Correct that the code can detect and didn't show any error. The output show as below. 

_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//div[@class='news-finance-state mb-4']/div[2]/div[6]/div   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
_WD_JsonActionPointer ==> Success [0] : {"type":"pointerDown","button":0}
_WD_JsonActionPointer ==> Success [0] : {"type":"pointerUp","button":0}
__WD_Post ==> Webdriver Exception [10] : HTTP status = 500
_WD_Action ==> Webdriver Exception [10] : Parameters:   Command=actions   Option={"actions":[{"id":"hover","type":"pointer","parameters":{"pointerType":"mouse"},"actions":[{"type":"pointerMove","duration":100,"x":0,"y":0,"origin":{"ELEMENT":"065df1d6-e22a-4cdc-a918-3d8ac2f62c1f","element-6066-11e4-a52e-4f735466cecf":"065df1d6-e22a-4cdc-a918-3d8ac2f62c1f"}},{"type":"pointerDown","button":0},{"type":"pointerUp","button":0}]}]}
_WD_ElementActionEx ==> Webdriver Exception [10] : Parameters:    Element=065df1d6-e22a-4cdc-a918-3d8ac2f62c1f    Command=Click    XOffset=Default    YOffset=Default    Button=Default    HoldDelay=Default    Modifier=Default

Can you advise how to use Chrome's Dev Tools to check the event attached. Sorry I really less knowledge on this. I try to search and read in internet but I cannot get result as the suggestion from those website.

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
Posted
10 minutes ago, maninternet said:

Correct that the code can detect and didn't show any error. The output show as below.

Ahh... but there are errors shown. Please take another look.

Also, please post the output related to _WD_Startup, which should look like this -- 

_WD_Startup: OS:    WIN_11 WIN32_NT 22621
    _WD_Startup: AutoIt:    3.3.16.1
    _WD_Startup: Webdriver UDF:    0.11.0 (Up to date)
    _WD_Startup: WinHTTP:    1.6.4.2
    _WD_Startup: Driver:    geckodriver.exe (64 Bit)
    _WD_Startup: Params:    --log trace
    _WD_Startup: Port:    4444
    _WD_Startup: Command:    "geckodriver.exe" --log trace
    _WD_Startup ==> Success [0]

 

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
maninternet Seeker

maninternet

Posted
  • Author
Posted

Oh, i always understand that as I still get element ID it shouldn't be any error occur. Thank you for suggestion.

Below is full output from my code. 

_WD_Option ==> Success [0] : Parameters:   Option=Driver   Value=chromedriver.exe
_WD_Option ==> Success [0] : Parameters:   Option=Port   Value=9515
_WD_Option ==> Success [0] : Parameters:   Option=DriverParams   Value=--verbose --log-path="E:\My\SET\New web RROA\chrome.log"
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesDump: JSON structure starts below: 11
{
    "capabilities":{
        "alwaysMatch":{
            "goog:chromeOptions":{
                "w3c":true,
                "excludeSwitches":[
                    "enable-automation"
                ]
            }
        }
    }
}
_WD_CapabilitiesDump: JSON structure ends above.
_WD_IsLatestRelease ==> Success [0] : True
_WD_Startup: OS:    WIN_11 WIN32_NT 22621 
_WD_Startup: AutoIt:    3.3.16.1
_WD_Startup: Webdriver UDF: 0.11.0 (Up to date)
_WD_Startup: WinHTTP:   1.6.4.1 (Download latest source at <https://raw.githubusercontent.com/dragana-r/autoit-winhttp/master/WinHttp.au3>)
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --verbose --log-path="E:\My\SET\New web RROA\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --verbose --log-path="E:\My\SET\New web RROA\chrome.log" 
_WD_Startup ==> Success [0]
__WD_Post ==> Success [0] : HTTP status = 200
_WD_CreateSession ==> Success [0] : 7ca194b236f41e11a2a5e6ea56dcfd34
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Window ==> Success [0] : Parameters:   Command=MAXIMIZE   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_Navigate ==> Success [0] : Parameters:   URL=https://www.set.or.th/en/market/product/stock/quote/LPF/news
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//h1[@class='symbol text-white mb-0 me-2']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Get ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=TEXT   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//span[@class='fs-12px col-10 text-color-grey']   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Get ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=TEXT   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//div[@class='news-finance-state mb-4']/div[2]/div[6]/div/span   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
__WD_Get ==> Success [0] : HTTP status = 200
_WD_ElementAction ==> Success [0] : Parameters:   Command=TEXT   Option=Default
__WD_Post ==> Success [0] : HTTP status = 200
_WD_FindElement ==> Success [0] : Parameters:   Strategy=xpath   Selector=//div[@class='news-finance-state mb-4']/div[2]/div[6]/div   StartNodeID=Default   Multiple=Default   ShadowRoot=Default
_WD_JsonActionPointer ==> Success [0] : {"type":"pointerDown","button":0}
_WD_JsonActionPointer ==> Success [0] : {"type":"pointerUp","button":0}
__WD_Post ==> Webdriver Exception [10] : HTTP status = 500
_WD_Action ==> Webdriver Exception [10] : Parameters:   Command=actions   Option={"actions":[{"id":"hover","type":"pointer","parameters":{"pointerType":"mouse"},"actions":[{"type":"pointerMove","duration":100,"x":0,"y":0,"origin":{"ELEMENT":"a0914ca9-f8d8-48f3-9ec1-3880623c3752","element-6066-11e4-a52e-4f735466cecf":"a0914ca9-f8d8-48f3-9ec1-3880623c3752"}},{"type":"pointerDown","button":0},{"type":"pointerUp","button":0}]}]}
_WD_ElementActionEx ==> Webdriver Exception [10] : Parameters:    Element=a0914ca9-f8d8-48f3-9ec1-3880623c3752    Command=Click    XOffset=Default    YOffset=Default    Button=Default    HoldDelay=Default    Modifier=Default

 

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
Posted (edited)
57 minutes ago, maninternet said:

_WD_Startup: WinHTTP:   1.6.4.1 (Download latest source at <https://raw.githubusercontent.com/dragana-r/autoit-winhttp/master/WinHttp.au3>)

Please update your source as indicated above, which may resolve the issue you are encountering.

P.S. The target website is accessible to everyone so I don't understand why you haven't posted a reproducer script, which would have made this much simpler to resolve. 🤔

Edited by Danp2

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
Posted
7 hours ago, maninternet said:

Can you advise how to use Chrome's Dev Tools to check the event attached. Sorry I really less knowledge on this. I try to search and read in internet but I cannot get result as the suggestion from those website.

Check the FAQ section of the wiki. One of the FAQs links to a bunch of Youtube videos covering this topic.

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
maninternet Seeker

maninternet

Posted
  • Author
Posted
14 hours ago, Danp2 said:

Please update your source as indicated above, which may resolve the issue you are encountering.

P.S. The target website is accessible to everyone so I don't understand why you haven't posted a reproducer script, which would have made this much simpler to resolve. 🤔

I already update as your suggest it not suggest to update version but the error still be the same. 

_WD_Option ==> Success [0] : Parameters:   Option=Driver   Value=chromedriver.exe
_WD_Option ==> Success [0] : Parameters:   Option=Port   Value=9515
_WD_Option ==> Success [0] : Parameters:   Option=DriverParams   Value=--verbose --log-path="E:\My\SET\New web RROA\chrome.log"
_WD_CapabilitiesAdd ==> Success [0] : Successfully used [alwaysMatch]  with specified browser: chrome
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesAdd ==> Success [0] : Successfully added capability
_WD_CapabilitiesDump: JSON structure starts below: 11
{
    "capabilities":{
        "alwaysMatch":{
            "goog:chromeOptions":{
                "w3c":true,
                "excludeSwitches":[
                    "enable-automation"
                ]
            }
        }
    }
}
_WD_CapabilitiesDump: JSON structure ends above.
_WD_IsLatestRelease ==> Success [0] : True
_WD_Startup: OS:    WIN_11 WIN32_NT 22621 
_WD_Startup: AutoIt:    3.3.16.1
_WD_Startup: Webdriver UDF: 0.11.0 (Up to date)
_WD_Startup: WinHTTP:   1.6.4.2
_WD_Startup: Driver:    chromedriver.exe (32 Bit)
_WD_Startup: Params:    --verbose --log-path="E:\My\SET\New web RROA\chrome.log"
_WD_Startup: Port:  9515
_WD_Startup: Command:   "chromedriver.exe" --verbose --log-path="E:\My\SET\New web RROA\chrome.log"

  

__WD_Post ==> Webdriver Exception [10] : HTTP status = 500
_WD_Action ==> Webdriver Exception [10] : Parameters:   Command=actions   Option={"actions":[{"id":"hover","type":"pointer","parameters":{"pointerType":"mouse"},"actions":[{"type":"pointerMove","duration":100,"x":0,"y":0,"origin":{"ELEMENT":"c39af9eb-8632-4acf-b340-73ed3caa0de0","element-6066-11e4-a52e-4f735466cecf":"c39af9eb-8632-4acf-b340-73ed3caa0de0"}},{"type":"pointerDown","button":0},{"type":"pointerUp","button":0}]}]}
_WD_ElementActionEx ==> Webdriver Exception [10] : Parameters:    Element=c39af9eb-8632-4acf-b340-73ed3caa0de0    Command=Click    XOffset=Default    YOffset=Default    Button=Default    HoldDelay=Default    Modifier=Default

My code is below but as the code must read another .txt file, You can create the file with only text "LPF" to test. 

#include "wd_helper.au3"
#include "wd_capabilities.au3"
#include "array.au3"
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')
_WD_CapabilitiesStartup()
_WD_CapabilitiesAdd('alwaysMatch', 'chrome')
_WD_CapabilitiesAdd('w3c', True)
_WD_CapabilitiesAdd('excludeSwitches', 'enable-automation')
_WD_CapabilitiesDump(@ScriptLineNumber)
Local $sCapabilities = _WD_CapabilitiesGet()
_WD_Startup()
$sSession = _WD_CreateSession($sCapabilities)
_WD_Window($sSession,"MAXIMIZE")
$File = FileOpen("StockRROA.txt")
$Symbol = FileReadToArray($File)
FileClose($File)
For $i = 0 To UBound($Symbol) - 1
    _WD_Navigate($sSession,"https://www.set.or.th/en/market/product/stock/quote/"&$Symbol[$i]&"/news")
    $Count = 0
    Do
        $LoadSymbol = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//h1[@class='symbol text-white mb-0 me-2']")
        $ReadSymbol = _WD_ElementAction($sSession,$LoadSymbol,"TEXT")
        $Count = $Count + 1     
        If $Count >9 Then
            _WD_Navigate($sSession,"https://www.set.or.th/en/market/product/stock/quote/"&$Symbol[$i]&"/financial-statement/company-highlights")
            $Count = 0
        EndIf
    Until $ReadSymbol == $Symbol[$i]
    Do
        $CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//span[@class='fs-12px col-10 text-color-grey']")
        $TextCheck = _WD_ElementAction($sSession,$CheckText,"TEXT")
    Until StringInStr($TextCheck,"important")
    $CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//div[@class='news-finance-state mb-4']/div[2]/div[6]/div/span")
    $TextCheck = _WD_ElementAction($sSession,$CheckText,"TEXT")
    If StringInStr($TextCheck,"more") Then
        $CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//div[@class='news-finance-state mb-4']/div[2]/div[6]/div")
        _WD_ElementActionEx($sSession,$CheckText,"Click")
    EndIf
Next
_WD_DeleteSession($sSession)
_WD_Shutdown()

 

Link to comment
Share on other sites
AlfredF Seeker

AlfredF

Posted
Posted

Hi. Danp2 already gave you the solution, you need to find the correct element path and click on the DIV (<div role="button" class="px-3 d-flex bg-white">).
This works: document.querySelector('div[class="px-3 d-flex bg-white"]').click(), meaning that all you need is to put the correct XPath (something like this maybe: //div[@class='px-3 d-flex bg-white']) and make the click. Remember that there are 2 buttons with that same class name, if you need to click on the second one you can use xpath indexing option.

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
Posted

When I try to run your code, it generates an error on this line -- 

$CheckText = _WD_FindElement($sSession,$_WD_LOCATOR_ByXPath,"//div[@class='news-finance-state mb-4']/div[2]/div[6]/div/span")

Here is the related console output -- 

    __WD_Post ==> No match [8] : HTTP status = 404
    _WD_FindElement ==> No match [8] : Parameters:   Strategy=xpath   Selector=//div[@class='news-finance-state mb-4']/div[2]/div[6]/div/span   StartNodeID=Default   Multiple=Default   ShadowRoot=Default

so it is unable to locate the target element.

I will try to post a much simpler example in a bit.

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
Danp2 Universalist

Danp2

Posted
Posted

Here's my "simplified" version -- 

#include "wd_helper.au3"
#include "wd_capabilities.au3"
#include "array.au3"
_WD_Option('Driver', 'chromedriver.exe')
_WD_Option('Port', 9515)
_WD_Option('DriverParams', '--verbose --log-path="' & @ScriptDir & '\chrome.log"')
_WD_CapabilitiesStartup()
_WD_CapabilitiesAdd('alwaysMatch', 'chrome')
_WD_CapabilitiesAdd('w3c', True)
_WD_CapabilitiesAdd('excludeSwitches', 'enable-automation')
_WD_CapabilitiesDump(@ScriptLineNumber)
Local $sCapabilities = _WD_CapabilitiesGet()
_WD_Startup()
$sSession = _WD_CreateSession($sCapabilities)
_WD_Window($sSession,"MAXIMIZE")
Local $aSymbol = 'LPF'

_WD_Navigate($sSession,"https://www.set.or.th/en/market/product/stock/quote/"& $aSymbol &"/news")
_WD_LoadWait($sSession)

; Accept cookies
Local $sElement = _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//div[@class='align-self-md-center mb-3 mb-md-0']/button")
If @error = $_WD_Error_Success Then
    _WD_ElementAction($sSession, $sElement, "Click")
EndIf

Local $sElement = _WD_WaitElement($sSession, $_WD_LOCATOR_ByXPath, "//div[contains(@class, 'news-finance-state mb')]/div[2]/div[6]/div/span")
If @error = $_WD_Error_Success Then
    If _WD_ElementAction($sSession, $sElement, "text") = 'more' Then
        ; scroll to bottom
        _WD_ExecuteScript($sSession, "window.scrollTo({ top: document.body.scrollHeight, behavior: 'smooth' })")
        Sleep(1000)

        $sElement = _WD_FindElement($sSession, $_WD_LOCATOR_ByXPath, "//div[contains(@class, 'news-finance-state mb')]/div[2]/div[6]/div")
        _WD_ElementAction($sSession, $sElement, "Click")
    EndIf
EndIf

_WD_DeleteSession($sSession)
_WD_Shutdown()

Some items of note --

  • The cookie banner is accepted to avoid any issues due to items being covered by it
  • Forcibly scroll to the bottom of the webpage. This avoids issue with trying to click on "more" when it hasn't fully scrolled into view

Latest Webdriver UDF Release

Webdriver Wiki FAQs

Link to comment
Share on other sites
SOLVE-SMART Universalist

SOLVE-SMART

Posted
Posted

Really nice simplified example @Danp2 . I simply had to mention it 😀 .

Best regards
Sven

Stay innovative!

Spoiler

🌍 Au3Forums

🎲 AutoIt (en) Cheat Sheet

📊 AutoIt limits/defaults

💎 Code Katas: [...] (comming soon)

🎭 Collection of GitHub users with AutoIt projects

🐞 False-Positives

🔮 Me on GitHub

💬 Opinion about new forum sub category

📑 UDF wiki list

✂ VSCode-AutoItSnippets

📑 WebDriver FAQs

👨‍🏫 WebDriver Tutorial (coming soon)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...