How to detect when Start menu opens (Win10/Win11)

[WildByDesign]

I am surprisingly more perplexed by this than with any previous challenge. 🙃

So perplexed that I don't have any code examples to share here to begin with.

I want to detect the moment that the Start menu opens.

I've tried WinEventHook, shell hook, and also simple loops checking active process.

The problem is, at least on Win11, all built-in functions to check active window and all UDF WinAPI functions to check active window, window with focus, and foreground window all show the same:

Class name: Windows.UI.Core.CoreWindow  Window title: Search

A window title with "Start" as the title never becomes foregorund/active.

Although "Start" does exist and another problem is the "Start" window stays visible (attribute) 100% of the time. So I can't even check based on visibility.

Another problem: There are over a dozen windows using class name Windows.UI.Core.CoreWindow

Some light near the end of the tunnel:

TAGPOINT with _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointHov), $GA_ROOT) (thanks to @Nine here)is the only thing that I've seen name the correct window. But this TAGPOINT was obtained with _WinAPI_GetMousePos and therefore required clicking on the Start button and hovering over the Start menu to get it.

My last attempt was with _WinAPI_GetCurrentPosition because it returns with a TAGPOINT. So there may be potential there but I am not very familiar with it and there are zero examples in the forum.

#include <WinAPI.au3>

$hStart = 0x101dc
$hDC = _WinAPI_GetDC($hStart)
$tPointStart = _WinAPI_GetCurrentPosition($hDC)
$hWndStart = _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointStart), $GA_ROOT)
ConsoleWrite("title: " & _WinAPI_GetWindowText($hWndStart) & @CRLF)

As you can see, I was exhausted after spending almost two hours on this issue and resorted to just manually copying the handle for the Start menu window. That is the exact moment when I realized that I should ask for help. 😆

I am still pretty new to AutoIt. Lots to learn. But I will always put in a significant amount of effort and attempts before I eventually ask for help here. That is because I value everyone's time and don't want to ask without putting in some effort on my own first. I am legitimately stuck on this one.

This is something that would have been simple on Windows 7 and prior.

Edited Sunday at 12:52 PM by WildByDesign

[WildByDesign]

So apparently none of the functions seem to find these "modern" windows.

For example:

Local $aEnumWindows = _WinAPI_EnumWindows(False)
For $n = 1 To UBound($aEnumWindows) - 1
    If $aEnumWindows[$n][1] <> "Start" Then ContinueLoop
    ConsoleWrite("start menu?" & @CRLF)
Next

There are 2 windows titled "Start". It does find the win32 "Start" related to the actual Start button on the taskbar. But it does not find the "Start" window which is active and visible.

This also seems to explain why things like this don't work:

WinExists("[Class:Windows.UI.Core.CoreWindow]", "Start")

What a thorough and interesting mess. I will continue to try more things and hopefully not lose my marbles in the madness. 😃

WindowFromPoint seems to still be the only function that can see these modern windows.

Edited Sunday at 12:55 PM by WildByDesign

[Nine]

You would need to use UIAutomation :

#include "..\UIAutomation\Includes\CUIAutomation2.au3"
#include <Array.au3>

Opt("MustDeclareVars", True)

; Create UI Automation object
Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
If Not IsObj($oUIAutomation) Then Exit ConsoleWrite("$oUIAutomation ERR" & @CRLF)
;ConsoleWrite("$oUIAutomation OK" & @CRLF)

; Get Desktop element
Local $pDesktop, $oDesktop, $sValue
$oUIAutomation.GetRootElement($pDesktop)
$oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oDesktop) Then Exit ConsoleWrite("$oDesktop ERR" & @CRLF)
;ConsoleWrite("$oDesktop OK" & @CRLF)

; Get Start button
Local $pCondition, $pStart, $oStart, $sValue, $aRect
$oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, "StartButton", $pCondition)
$oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pStart)
$oStart = ObjCreateInterface($pStart, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oStart) Then Exit ConsoleWrite("$oStart ERR" & @CRLF)
;ConsoleWrite("$oStart OK" & @CRLF)
$oStart.GetCurrentPropertyValue($UIA_NamePropertyId, $sValue)
ConsoleWrite($sValue & @CRLF)

$oStart.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $aRect)
_ArrayDisplay($aRect)

 

[WildByDesign]

11 hours ago, Nine said:

You would need to use UIAutomation :

Thank you. I am marking your response as the solution because I believe that UIAutomation is the sane and proper method for this scenario. Anyone with a similar scenario should use UIAutomation.

However, I personally have struggled to learn and understand UIAutomation. It almost feels like a language of its own.

I ended up figuring out a way to do it within a block of code that I already have setup (also a solution from @Nine) from here.

Within the IsPressed section that I added, I just had to make sure that classname = "Windows.UI.Core.CoreWindow" And window text = "DesktopWindowXamlSource". If both matched at the same time, I just had to run a quick _WinAPI_GetWindowText(_WinAPI_GetForegroundWindow()) to make sure that = "Start" Or "Search". This captured the Start button click 100% of the time.

Since I had this block of code already functioning in my script for detecting when "Progman" had been clicked, it just made sense to adapt it quite easily in the end. Well, not exactly easily. But it worked out great at the end of the day.

Thank you again for your time.

Edited Monday at 12:42 AM by WildByDesign

[ioa747]

; https://www.autoitscript.com/forum/topic/213297-how-to-detect-when-start-menu-opens-win10win11/#findComment-1547177

#include <WinAPISysWin.au3>

HotKeySet("{ESC}", "GoToExit")

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start")
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

Func CheckForTitle($sTitle, $sClass = "")
    Local Static $tStruct = DllStructCreate($tagPOINT) ; Create a structure that defines the point to be checked.
    DllStructSetData($tStruct, "x", MouseGetPos(0))    ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    DllStructSetData($tStruct, "y", MouseGetPos(1))    ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    Local $hWnd = _WinAPI_WindowFromPoint($tStruct)    ; Retrieves the handle of the window that contains the specified point.
    Local $sActiveTitle = WinGetTitle($hWnd)           ; Retrieves the full title from a window.
    If $sActiveTitle = $sTitle Then
        Local $sActiveClass = _WinAPI_GetClassName($hWnd)
        If $sClass = "" Then
            Return $hWnd
        Else
            If $sActiveClass = $sClass Then Return $hWnd
        EndIf
    EndIf
    Return 0
EndFunc   ;==>CheckForTitle

Func GoToExit()
    Exit
EndFunc   ;==>GoToExit

 

[WildByDesign]

@ioa747 Holy Guacamole! That is short and sweet. 🙏

I just tested your example and it does exactly what I need and it does it beautifully and with a small amount of code.

I am definitely going to try to swap out my existing functionality and use yours instead because it's more accurate and should be way more efficient as well compared to what I was doing.

[Werty]

2 hours ago, ioa747 said:

win10

I'm on win10 and have hidden taskbar, your script triggers when I only hover over the start button, without clicking and opening it.

[ioa747]

Try adding the class as well, to narrow down the results 

[Werty]

Same thing, atleast if I'm doing it right, could you post an example of what exactly you mean?

[ioa747]

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start", "Windows.UI.Core.CoreWindow")
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

 

[Werty]

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window.

[WildByDesign]

2 minutes ago, Werty said:

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window

Same, I noticed that as well. That is why mine checks the foreground/active window 20ms after to confirm Start menu activated.

But that reminds me, I never did end up sharing my working solution. I am not at my PC right now but will share it in a few hours just in case it’s worthwhile.

[ioa747]

1 hour ago, Werty said:

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window.

This seems logical to me, since it uses _WinAPI_WindowFromPoint  

Edit:
this could be fixed with an offset parameter

#include <WinAPISysWin.au3>

HotKeySet("{ESC}", "GoToExit")

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start", "Windows.UI.Core.CoreWindow", 100, -100)
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

Func CheckForTitle($sTitle, $sClass = "", $iXOffSet = 0, $iYOffSet = 0)
    Local Static $tStruct = DllStructCreate($tagPOINT) ; Create a structure that defines the point to be checked.
    DllStructSetData($tStruct, "x", MouseGetPos(0) + $iXOffSet) ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    DllStructSetData($tStruct, "y", MouseGetPos(1) + $iYOffSet) ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    Local $hWnd = _WinAPI_WindowFromPoint($tStruct) ; Retrieves the handle of the window that contains the specified point.
    Local $sActiveTitle = WinGetTitle($hWnd)        ; Retrieves the full title from a window.
    If $sActiveTitle = $sTitle Then
        Local $sActiveClass = _WinAPI_GetClassName($hWnd)
        If $sClass = "" Then
            Return $hWnd
        Else
            If $sActiveClass = $sClass Then Return $hWnd
        EndIf
    EndIf
    Return 0
EndFunc   ;==>CheckForTitle

Func GoToExit()
    Exit
EndFunc   ;==>GoToExit

 

Edited Tuesday at 08:55 PM by ioa747

[WildByDesign]

1 hour ago, ioa747 said:

Edit:
this could be fixed with an offset parameter

This works really well with the offset parameter. Very fast too. 

For the sake of sharing ideas and since I forgot to share what I ended up using initially, here was my working example:

Works on Win11 but have not testing on Win10.

expandcollapse popup

#include <WinAPIvkeysConstants.au3>
#include <WinAPIMisc.au3>
#include <WinAPISysWin.au3>
#include <Misc.au3>

Global $hUser32 = DllOpen('user32.dll')
HotKeySet("{ESC}", Terminate)

Example()

Func Example()
    Local $tPoint, $hWndCur, $hWndCur2, $hWndChildCur, $sChild, $sChildText, $sParent, $sActiveText
    While 1
        If _IsPressed($VK_LBUTTON, $hUser32) Then
            $tPoint = _WinAPI_GetMousePos()
            $hWndCur = _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPoint), $GA_ROOT)
            $hWndCur2 = _WinAPI_WindowFromPoint($tPoint)
            $hWndChildCur = GetRealChild($hWndCur2)
            $sChild = _WinAPI_GetClassName($hWndChildCur)
            $sChildText = _WinAPI_GetWindowText($hWndChildCur)
            $sParent = _WinAPI_GetClassName($hWndCur)
            If $sChild = "Windows.UI.Core.CoreWindow" And $sChildText = "DesktopWindowXamlSource" Then
                ; wait until key is released
                While _IsPressed($VK_LBUTTON, $hUser32)
                    Sleep(5)
                WEnd
                Sleep(20)
                $sActiveText = _WinAPI_GetWindowText(_WinAPI_GetForegroundWindow())
                If $sActiveText = "Start" Or $sActiveText = "Search" Then
                    ConsoleWrite("Start button has been pushed." & @CRLF)
                EndIf
            EndIf
        EndIf
        Sleep(100)
    WEnd
EndFunc

Func Terminate()
    Exit
EndFunc   ;==>Terminate

Func _WinAPI_RealChildWindowFromPoint($hWnd, $tPoint)
  Local $aRet = DllCall($hUser32, 'hwnd', 'RealChildWindowFromPoint', 'hwnd', $hWnd, 'struct', $tPoint)
  If @error Then Return SetError(@error, @extended, 0)
  Return $aRet[0]
EndFunc   ;==>_WinAPI_RealChildWindowFromPoint

Func GetRealChild($hWnd)
  Local $tPoint, $hRoot = _WinAPI_GetAncestor($hWnd, $GA_ROOT)
  If $hWnd = $hRoot Then
    $tPoint = _WinAPI_GetMousePos(True, $hWnd)
    Return _WinAPI_ChildWindowFromPointEx($hWnd, $tPoint)
  EndIf
  Local $hParent = _WinAPI_GetAncestor($hWnd, $GA_PARENT)
  Local $aChild = _WinAPI_EnumChildWindows($hParent)
  If @error Then Return 0

  Local $hFound

  For $i = 1 To $aChild[0][0]
    $hParent = _WinAPI_GetParent($aChild[$i][0])
    $tPoint = _WinAPI_GetMousePos(True, $hParent)
    $hFound = _WinAPI_RealChildWindowFromPoint($hParent, $tPoint)
    If $hFound = $aChild[$i][0] Then Return $hFound
  Next
  Return 0
EndFunc   ;==>GetRealChild

 

[Nine]

I kind of misunderstood the initial issue.  I first thought you wanted to locate the Start button and then act upon it.  With the recent posts, you just seem to detect the appearance of the Start window.  So here a solution with UIAutomation that will detect window creation event with rightful window title.

expandcollapse popup

#include "..\UIAutomation\UIAEHEvents\Includes\UIAEH_AutomationEventHandler.au3"
#include <Constants.au3>

Opt("MustDeclareVars", True)

Global $oWinStart

HotKeySet("{END}", Terminate)

Example()

Func Example()
  ; Create UI Automation object
  Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtag_IUIAutomation)
  If Not IsObj($oUIAutomation) Then Exit ConsoleWrite("$oUIAutomation ERR" & @CRLF)
  ;ConsoleWrite("$oUIAutomation OK" & @CRLF)

  ; Get Desktop element
  Local $pDesktop, $oDesktop
  $oUIAutomation.GetRootElement($pDesktop)
  $oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
  If Not IsObj($oDesktop) Then Exit ConsoleWrite("$oDesktop ERR" & @CRLF)
  ;ConsoleWrite("$oDesktop OK" & @CRLF)

  UIAEH_AutomationEventHandlerCreate()
  If Not IsObj($oUIAEH_AutomationEventHandler) Then Exit ConsoleWrite("$oUIAEH_AutomationEventHandler ERR" & @CRLF)
  ;ConsoleWrite("$oUIAEH_AutomationEventHandler OK" & @CRLF)

  $oUIAutomation.AddAutomationEventHandler($UIA_Window_WindowOpenedEventId, $pDesktop, $TreeScope_Subtree, 0, $oUIAEH_AutomationEventHandler)

  Local $hWnd, $bTreated

  While Sleep(100)
    If IsObj($oWinStart) Then
      If Not $bTreated Then
        ConsoleWrite("Start Window showed" & @CRLF)
        $bTreated = True
        $oWinStart.GetCurrentPropertyValue($UIA_NativeWindowHandlePropertyId, $hWnd)
      ElseIf Not WinExists($hWnd) Then
        $oWinStart.Release()
        $oWinStart = 0
        $bTreated = False
        $hWnd = 0
        ConsoleWrite("Start window closed" & @CRLF)
      EndIf
    EndIf
  WEnd
EndFunc   ;==>Example

Func UIAEH_AutomationEventHandler_HandleAutomationEvent($pSelf, $pSender, $iEventId)
  Local $oSender = ObjCreateInterface($pSender, $sIID_IUIAutomationElement, $dtag_IUIAutomationElement)
  $oSender.AddRef()
  Local $sTitle, $sClass
  $oSender.GetCurrentPropertyValue($UIA_NamePropertyId, $sTitle)
  $oSender.GetCurrentPropertyValue($UIA_ClassNamePropertyId, $sClass)

  If $sTitle = "Démarrer" And $sClass = "Windows.UI.Core.CoreWindow" Then $oWinStart = $oSender
EndFunc   ;==>UIAEH_AutomationEventHandler_HandleAutomationEvent

Func ConsoleWriteEx($sString)
  ConsoleWrite(BinaryToString(StringToBinary($sString, $SB_UTF8), $SB_ANSI))
EndFunc   ;==>ConsoleWriteEx

Func Terminate()
  UIAEH_AutomationEventHandlerDelete()
  Exit
EndFunc   ;==>Terminate

The good thing about it, is that you do not need to check for "Search" window which does not relate to start button...

[WildByDesign]

33 minutes ago, Nine said:

I kind of misunderstood the initial issue.  I first thought you wanted to locate the Start button and then act upon it.  With the recent posts, you just seem to detect the appearance of the Start window.  So here a solution with UIAutomation that will detect window creation event with rightful window title.

This is by far the best, most accurate and proper solution. I will definitely have to switch the solution to this post so that any users needing something similar can be directed to the most appropriate solution.

You example also catches the Start menu when the Win button is pressed which mine was lacking.

Now that I see the power of UIAutomation, I'm going to spend some time reading and playing with older scripts and tools such as some posts by LarsJ and others. This forum is so rich with information.

Thank you!