IE DOM Elements visual selector tool

[Gianni]

The DOM allows to do anything with elements and their contents, but first we need to reach the corresponding DOM object, get it into a variable, and then we are able to modify it. *
Well, this little tool (although it is not very nice aesthetically) allows you to get visually a "selector" usable to reference DOM objects.
Once you have the "selector" of an element you can pass it to the javascript querySelector() function that will return a reference to that element.
To use this tool you have to:
1) open the web page you want to inspect into IE browser
2) run this script (if it find more instances of IE running, it allows you to chose one)
3) move the mouse over the browser. The "selector" of the element below the pointer is catched automatically while hovering. To copy the selector in the clipboard just right click on the element.
As you can see, while hovering, the element pointed by the mouse is highlighted with a thin red dotted frame to allow you to better "take aim"
when the selector is copied to the clipboard a little acoustic signal is emitted as a confirm, then you can paste it in your listing where you need it.

I hope it can come in handy and save you time when you need to automate a site .... have fun (debugged on Sept. 30 2018)

expandcollapse popup

#include <IE.au3>
#include <GUIConstantsEx.au3>
#include <GuiListBox.au3>
#include <WindowsConstants.au3>
#include <Misc.au3> ; for _IsPressed (23 END key)
Global $hDLL = DllOpen("user32.dll")

; following global variables are automatically updated by events from the browser
; -------------------------------------------------------------------------------------
Global $g_iMouseX, $g_iMouseY ; coordinates of the mouse while mooving over the browser
Global $bCopySelector = False ; becomes True when you right click on wanted element
; -------------------------------------------------------------------------------------
Global $oIE = _Get_IE() ; get IE instance to inspect
If IsObj($oIE) Then
    $hIE = _IEPropertyGet($oIE, "hwnd")
    WinActivate($hIE)
    _InspectElements()
EndIf
DllClose($hDLL)
Exit

Func _InspectElements() ; it uses the global variable $oIE as source

    ; --- set IE to interact with AutoIt ---
    Local $oDocument
    Do ; wait for document
        Sleep(250)
        $oDocument = $oIE.document
    Until IsObj($oDocument)

    Local $oWindow = $oDocument.ParentWindow

    ; create a reference to the javascript eval method
    ; in the body section of the dovument
    $oWindow.setTimeout("document.body.JSeval = eval; ", 0)

    ; attach the $JSeval variable to the javascript eval method
    Local $JSeval
    Do
        $JSeval = Execute('$oIE.Document.body.JSeval')
    Until IsObj($JSeval)

    ; ---------------------------------------------
    ; Inject Javascript functions/elements  to $oIE
    ; ---------------------------------------------

    ; Get the DOM path of an element (a CSS selector)
    ; -----------------------------------------------
    ; This javascript function returns the CSS selector of the passed element.
    ; You can then use the returned path to get a reference to the pointed
    ; element by the QuerySelector() javascript function
    ; function copied from the following link:
    ; https://stackoverflow.com/questions/5728558/get-the-dom-path-of-the-clicked-a
    ; see answer by "Aleksandar Totic" (thanks to him)
    Local $sJScript = "" & _
            " function getDomPath(el) {" & _
            "  if (!el) {" & _
            "    return;" & _
            "  }" & _
            "   var stack = [];" & _
            "   var isShadow = false;" & _
            "   while (el.parentNode != null) {" & _
            "     var sibCount = 0;" & _
            "     var sibIndex = 0;" & _
            "     for ( var i = 0; i < el.parentNode.childNodes.length; i++ ) {" & _
            "       var sib = el.parentNode.childNodes[i];" & _
            "       if ( sib.nodeName == el.nodeName ) {" & _
            "         if ( sib === el ) {" & _
            "           sibIndex = sibCount;" & _
            "         }" & _
            "         sibCount++;" & _
            "       }" & _
            "     }" & _
            "    var nodeName = el.nodeName.toLowerCase();" & _
            "    if (isShadow) {" & _
            "      nodeName += ""::shadow"";" & _
            "      isShadow = false;" & _
            "    }" & _
            "    if ( sibCount > 1 ) {" & _
            "      stack.unshift(nodeName + ':nth-of-type(' + (sibIndex + 1) + ')');" & _
            "    } else {" & _
            "      stack.unshift(nodeName);" & _
            "    }" & _
            "    el = el.parentNode;" & _
            "    if (el.nodeType === 11) {" & _
            "      isShadow = true;" & _
            "      el = el.host;" & _
            "    }" & _
            "   }" & _
            "   stack.splice(0,1);" & _
            "   return stack.join(' > ');" & _
            " }"
    ; more infos here: https://www.kirupa.com/html5/finding_elements_dom_using_querySelector.htm

    ; Inject the above javascript function contained in the $sJScript variable into the document
    _JS_Inject($oIE, $sJScript)
    Local $_getDomPath ; a reference to call above function from AutoIt
    Do
        Sleep(250)
        $_getDomPath = $jsEval("getDomPath")
    Until IsObj($_getDomPath)
    ;
    ; -------------------
    ; hook some IE events
    ; -------------------
    Local $oEventObjects[2], $oEventsSource
    $oEventsSource = $oIE.document.documentElement ; element we want catch events from
    ; https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa769636(v=vs.85)
    $oEventObjects[0] = ObjEvent($oEventsSource, "_HTMLElementEvents2_", "HTMLElementEvents2")

    ; https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa768283(v%3dvs.85)
    $oEventObjects[1] = ObjEvent($oIE, "_IEEvent_", "DWebBrowserEvents2")

    ; open a GUI where to show some element's properties
    ; --------------------------------------------------
    Local $hGUIMain = GUICreate("Info", 500, 140, -1, -1, -1, $WS_EX_TOPMOST)
    Local $hProperties = GUICtrlCreateEdit("", 0, 0, 500, 140)
    GUICtrlSetFont(-1, 9, -1, -1, "Courier New")
    GUISetState() ;Show GUI
    ; --------------------------------------------------

    ; ---------
    ; Main loop
    ; ---------
    Local $iMouseX, $iMouseY, $oElement, $oNewElement, $sSelector
    Local $oGotElement, $sElementInfos
    Local $sSaved_StyleOutline, $sSaved_StyleOutline2

    ; Loop until the user exits.
    While IsObj($oIE)
        Switch GUIGetMsg()
            Case $GUI_EVENT_CLOSE
                ExitLoop ; ---> end
        EndSwitch

        If ($g_iMouseX <> $iMouseX) Or ($g_iMouseY <> $iMouseY) Then
            $iMouseX = $g_iMouseX
            $iMouseY = $g_iMouseY
            ; $oElement = $oIE.document.elementFromPoint($iMouseX, $iMouseY) ; <-- this way is slower
            $oNewElement = $JSeval('document.elementFromPoint(' & $iMouseX & ',' & $iMouseY & ');')
            If $oNewElement <> $oElement Then
                If IsObj($oElement) Then $oElement.style.outline = $sSaved_StyleOutline
                $oElement = $oNewElement
                ; $bSelfie = False
                ; $iSelf_Timer = TimerInit()
                $sSaved_StyleOutline = $oElement.style.outline ; save new element's original outline style
                $sSelector = $_getDomPath($oElement) ; get CSS path

                If $sSelector <> "" Then
                    ; We could use the $oNewElement, but just to proof that $sSelector is OK
                    ; we get again a reference to the new pointed element using it's $sSelector
                    $oGotElement = $JSeval('document.querySelector("' & $sSelector & '");') ; <-- how to use a selector
                    $oGotElement.style.outline = "1px dashed red" ; mark new pointed element
                    ; https://css-tricks.com/
                    $sElementInfos = "" & _
                            "nodeName: " & $oGotElement.nodeName & @CRLF & _
                            "id:       " & $oGotElement.getAttribute('id') & @CRLF & _
                            "class:    " & $oGotElement.getAttribute('class') & @CRLF & _
                            "type:     " & $oGotElement.getAttribute('type') & @CRLF & _
                            "---------" & @CRLF & _
                            $sSelector
                    ControlSetText($hGUIMain, "", $hProperties, $sElementInfos)
                EndIf
            EndIf
        EndIf

        ; $bCopySelector is setted to True by the right-click event on an element,
        ; see Volatile Func _HTMLElementEvents2_onContextmenu($oEvent) near script bottom
        If $bCopySelector And ($sSelector <> "") Then ; And (TimerDiff($iSelf_Timer) > $bSelfie_Delay) Then
            ;
            $sSaved_StyleOutline2 = $oGotElement.style.outline
            $oGotElement.style.outline = "5px dotted #ff0066" ; mark copied element
            ClipPut($sSelector)
            $sElementInfos &= @CRLF & "selector copied to ClipBoard"
            ControlSetText($hGUIMain, "", $hProperties, $sElementInfos)
            Beep(2000, 50)
            $bCopySelector = False
            Sleep(250)
            $oGotElement.style.outline = $sSaved_StyleOutline2
            ; ToolTip('')
        EndIf

        If _IsPressed("23", $hDLL) Then ; END key pressed
            If IsObj($oElement) Then $oElement.style.outline = $sSaved_StyleOutline
            WinActivate($hGUIMain)
            ; WinSetState($hGUIMain, "", @SW_SHOW)
            $aWin = WinGetPos($hGUIMain)
            MouseMove($aWin[0] + $aWin[2] / 2, $aWin[1] + $aWin[3] / 2, 0)
        EndIf
    WEnd

    ; the end
    ; ------------------------------------------
    For $i = 0 To UBound($oEventObjects) - 1
        ; Tell IE we don't want to receive events.
        $oEventObjects[$i] .Stop
        $oEventObjects[$i] = 0
    Next
    $oIE = 0 ; Remove IE from memory
    GUIDelete($hGUIMain) ; Remove GUI
    ; ------------------------------------------
EndFunc   ;==>_InspectElements

Func _Get_IE()
    ; Example 5 from the _IEAttach help
    ; Create an array of object references to all current browser instances
    ; The first array element will contain the number of instances found
    Local $aIE[1]
    $aIE[0] = 0

    Local $i = 1, $oIEx
    While 1
        $oIEx = _IEAttach("", "instance", $i)
        If @error = $_IEStatus_NoMatch Then ExitLoop
        ReDim $aIE[$i + 1]
        $aIE[$i] = $oIEx
        $aIE[0] = $i
        $i += 1
    WEnd

    If $aIE[0] > 0 Then
        If $aIE[0] = 1 Then Return $aIE[1] ; only one IE is running, return this then
        ;
        ; Create a little list box to choose the IE instance from
        Local $hChoose_IE = GUICreate("IE Instances", 600, 350)
        Local $Label1 = GUICtrlCreateLabel($aIE[0] & " running Instances of IE browser found, click the one you want to attach to then click on 'ok'", 5, 5, 590, 20)
        Local $List1 = GUICtrlCreateList("", 5, 30, 590, 300, BitOR($LBS_STANDARD, $LBS_EXTENDEDSEL))
        Local $hButton_choosed = GUICtrlCreateButton("OK", 5, 325, 590, 20)

        For $i = 1 To $aIE[0]
            GUICtrlSetData($List1, $i & ") " & _IEPropertyGet($aIE[$i], "locationurl"))
        Next
        GUISetState(@SW_SHOW)

        While 1 ; wait for a selection
            Switch GUIGetMsg()
                Case $GUI_EVENT_CLOSE
                    GUIDelete($hChoose_IE)
                    Return False
                Case $hButton_choosed
                    $aSelected = _GUICtrlListBox_GetSelItems($List1)
                    If $aSelected[0] Then
                        GUIDelete($hChoose_IE)
                        Return $aIE[$aSelected[1] + 1]
                    Else
                        MsgBox(0, "Info", "Please select an item")
                    EndIf
            EndSwitch
        WEnd
    Else
        MsgBox(0, 'error', "Sorry" & @CRLF & @CRLF & "no running IE instances found")
    EndIf
EndFunc   ;==>_Get_IE

; this function creates a javascript script into the html document
; of the passed $oIE object using the createElement method.
Func _JS_Inject($oIE, $sJScript, $bIsUrl = False)
    ;
    ; get a reference to the document object
    Local $objDocument = $oIE.document
    ;
    Local $oScript = $objDocument.createElement('script') ;
    $oScript.type = 'text/javascript'
    If $bIsUrl Then
        $oScript.src = $sJScript ; works if $sJScript is a link to a js listing (url)
    Else
        ; (https://stackoverflow.com/questions/35213147/difference-between-text-content-vs-inner-text)
        ;  $oScript.innerText = $sJScript
        $oScript.TextContent = $sJScript ; works if $sJScript contains the listing itself
    EndIf
    ;
    $objDocument.getElementsByTagName('head').item(0).appendChild($oScript)
    ; $objDocument.getElementsByTagName('head').item(0).removeChild($oScript);
    ;
EndFunc   ;==>_JS_Inject

; -------------------------------------------------------------------
; following function(s) are called by registered $oIE elements events
; -------------------------------------------------------------------
;
; The function automatically fired by an event
; will receive as parameter an Event Obj.
; This obj has properties related to
; the object that fired the event.
; See following link:
; https://msdn.microsoft.com/en-us/library/aa703876(v=vs.85).aspx

; function called by the mousemove event
; we use this to update 2 global variables:
Volatile Func _HTMLElementEvents2_onMousemove($oEvent)
    $g_iMouseX = $oEvent.clientX
    $g_iMouseY = $oEvent.clientY
EndFunc   ;==>_HTMLElementEvents2_onMousemove

; function called by the contextmenu event
; we use this to update 1 global variable
; and we also neutralize this event:
Volatile Func _HTMLElementEvents2_onContextmenu($oEvent)
    $oEvent.cancelBubble = True ; event propagation cancelled
    $oEvent.returnValue = False ; prevent default behaviour
    $bCopySelector = True ; when True, selector will be copied to clipboard in main loop
EndFunc   ;==>_HTMLElementEvents2_onContextmenu

; https://docs.microsoft.com/en-us/previous-versions/windows/internet-explorer/ie-developer/platform-apis/aa768280%28v%3dvs.85%29
Func _IEEvent_BeforeNavigate2($oIEpDisp, $sIEURL, $iIEFlags, $sIETargetFrameName, $sIEPostData, $iIEHeaders, $bIECancel)
    ;ConsoleWrite("Debug: navigate away cancelled." & @CRLF)
    ; https://stackoverflow.com/questions/6526876/how-to-cancel-or-dispose-current-navigation-at-webbrowser-element
    $oIE.stop
EndFunc   ;==>_IEEvent_BeforeNavigate2

Here is a simple example on how a "selector" can be used in AutoIt.
suppose we want automate the login to the AutoIt site with our username and password.
I've already prepared a very simple "template" where are missing some important parts without which the script can't work. Missing parts are the references to the elements of the AutoIt web page that we have to manage by our script.
well, here is where the tool I have just posted here above comes to our help.
follow this steps:
1) in IE open the AutoIt site at the forum page (https://www.autoitscript.com/forum/)
2) run the above tool (select the IE instance and/or bring it to front if needed)
3) when the script is "ready", move the mouse over the "Existing user? Sign In" string and right click the mouse button. Doing so the "selector" of that element is copied to the clipboard. Now we can paste it in our AutoLogIt.au3 script as value of the $sSignIn variable.
4) now click on the "Existing user? Sign In"  to open the "Sig In" session from where we will copy selectors of each of the 2 input box Username and Password, in the same way as we have already done in step 3, and paste those selectors to the $sInputUserId and $sInputPasswd variables respectively.
5) do the same for the "Sign In" Button and paste it's selector to the $sSignInButn variable
6) of course also fill the $sMyUserId and $sMyPasswd variables with your data.
That's It. Run the AutoLogIt script and it should Log you on automatically to the forum.

AutoLogIt.au3

expandcollapse popup

#include <ie.au3>

$sMyUserId = "" ; <-- your userid here
$sMyPasswd = "" ; <-- your password here

; set selectors here
$sSignIn = "" ; <-- SigIn element selector here
$sInputUserId = "" ; <-- UserId input selector here
$sInputPasswd = "" ; <-- Password input selector here
$sSignInButn = "" ; <-- Sig In button selector here

$oIE = _IECreate("https://www.autoitscript.com/forum/")

; here is how to use the QuerySelector javascript function
$hDOM_Element = $oIE.document.QuerySelector($sSignIn) ; get the "sign in" link element

; perform a click action on the above element
$hDOM_Element.click() ; or  _IEAction($hDOM_Element, "click") as well

; fill the username input
$hDOM_Element = $oIE.document.QuerySelector($sInputUserId)
$hDOM_Element.value = $sMyUserId

; fill the password input
$hDOM_Element = $oIE.document.QuerySelector($sInputPasswd)
$hDOM_Element.value = $sMyPasswd

; .... or also using the dot notation directly ....
$oIE.document.QuerySelector($sSignInButn).click()

Sleep(5000)

; this should  logout
$sMenu = "body > div:nth-of-type(2) > header > div > ul > li:nth-of-type(6) > a:nth-of-type(2)"

$oIE.document.QuerySelector($sMenu).click()

$sLogOut = "body > ul > li:nth-of-type(9) > a"
$oIE.document.QuerySelector($sLogOut).click()

 

Edited September 30, 2018 by Chimp
replaced main script with a "debugged" one

[taylansan]

Hi Chimp, 

This is very handy and really good tool. The tool works perfectly, captures the red dotted area and copies to the clipboard. My question is about the example: 

Line-18: Clicking on "Existing user? Sign In" works fine 

But in Line-22 setting the value didn't work. As you can see from the screenshot, the input box is not filled. 

I tried to replace Line-22 as the following (with my limited knowledge), but yet didn't work either: 

; fill the username input
$hDOM_Element = $oIE.document.QuerySelector($sInputUserId)
;$hDOM_Element.value = $sMyUserId
_IEFormElementSetValue($hDOM_Element, "$sMyUserId")

As a reference, these are the values that I read from the tool: 

$sSignIn = "body > div:nth-of-type(2) > header > div > ul > li:nth-of-type(1) > a" ; <-- SigIn element selector here
$sInputUserId = "body > div:nth-of-type(5) > div > form > ul > li:nth-of-type(1) > input" ; <-- UserId input selector here
$sInputPasswd = "body > div:nth-of-type(5) > div > form > ul > li:nth-of-type(2) > input" ; <-- Password input selector here
$sSignInButn = "body > div:nth-of-type(5) > div > form > ul > li:nth-of-type(5) > button" ; <-- Sig In button selector here

 

 

[Gianni]

Hi @taylansan
Thank you for appreciation,
...when I tried the example the first time, it worked well and the login was prformed regularly, now, trying it again I see that, as you are reporting, some selectors does not find the reference to the input elements .... however I also see that by changing the index from 5 to 4 in the component 'div:nth-of-type(4)' of the following selectors, the script works again, 

$sInputUserId = "body > div:nth-of-type(4) > div > form > ul > li:nth-of-type(1) > input" ; <-- UserId input selector here
$sInputPasswd = "body > div:nth-of-type(4) > div > form > ul > li:nth-of-type(2) > input" ; <-- Password input selector here
$sSignInButn = "body > div:nth-of-type(4) > div > form > ul > li:nth-of-type(5) > button" ; <-- Sig In button selector here

...but this is strange... maybe the web page has some hidden part that dynamically changes some DOM hierarchy .. (?)
I will look into the matter to see if I understand where the cause of this strangeness lies ...
thanks again for reporting.

Edited September 24, 2018 by Chimp

[Gianni]

Hi @taylansan and also to all of you who have come to read ...

after some debugging I've found the cause of the issue you reported.
The 'subtle' problem was not in the javascript function that retrieves the CSS selector, it was instead caused by a "div" element that I injected into the page, to be used as an highlighter on the element where the right-click was performed.

To highlight the right-clicked element, Instead of poisoning the page by injecting the div element, now I use an innocuous colored 5px dotted outline

well, the "debuged" listing is now posted in first post replacing the previuos one.

Bug reports, suggestion and improovements are welcome.
Thank You

[taylansan]

Hi Chimp, 

Yeah, now it works very fine. This is very handy and useful tool which makes automation a lot easier. 

Thank you very much for providing this and update. 

[adegard]

Hi Chimp,

Thanks you so much for this tool!

I'm using it for Autohotkey automation to get selectors... and it works well. (very handy) I also put it in a RPA editor I create, so I'm ask you if for you it's ok I'm using it and also if it 's possibile to convert it in ahk language, and to integrate it in this tool package.

Thanks you for your answer

Arnaud

[Gianni]

Hi Arnaud, I'm glad You find it useful,  of course feel free to use it as you better like.
P.S.
A note to keep in mind is that the derived selector is a "brute force" selector, ie, the returned selector contains the complete path from the root element of the web page down to the target element. So in the eventuality that the page structure is modified, or is dynamically modified by some script, the selector obtained before any changes may no longer be valid after the changes.

Edited December 3, 2018 by Chimp

[adegard]

On 2/12/2018 at 11:20 PM, Chimp said:

Hi Arnaud, I'm glad You find it useful,  of course feel free to use it as you better like.
P.S.
A note to keep in mind is that the derived selector is a "brute force" selector, ie, the returned selector contains the complete path from the root element of the web page down to the target element. So in the eventuality that the page structure is modified, or is dynamically modified by some script, the selector obtained before any changes may no longer be valid after the changes.

Thanks you so much Chimp!

If I understand It use ids... and if it's a page with dynamics ids, it won't run.

I'm also using iMacros to record selectors, and there is an'option to not use ids in the path. Then useful in this case.

Do you think it's possibile to than in your code: "HTML path without ids"?

[Gianni]

Hi @adegard,

 the selector returned by the embedded javascript function doesn't relies on any ID, it returns a "full path" selector that makes no use of intermediate IDs, (this is why I called it a "brute force" selector).
This kind of selector, if from one side has the advantage of being immune to the dynamic ID naming, from the other side is vulnerable to the changes performed on the html tree  structure.

[adegard]

Hi @chimp ,

Thanks you so much again for your answer (I understoud now)

I'm trying to modify your script to use it as Macro Recorder: so when I Left Click, I save the selector in a Edit form, and then continue to navigate, instead of staying on the same web-page...

 

[EDIT]: Maybe I find the trick... see file. I reload the GUI each time page in loading. It's working!

 

IE_selector_Recorderv3.au3

Edited December 6, 2018 by adegard