How to detect when Start menu opens (Win10/Win11)

[WildByDesign]

I am surprisingly more perplexed by this than with any previous challenge. 🙃

So perplexed that I don't have any code examples to share here to begin with.

I want to detect the moment that the Start menu opens.

I've tried WinEventHook, shell hook, and also simple loops checking active process.

The problem is, at least on Win11, all built-in functions to check active window and all UDF WinAPI functions to check active window, window with focus, and foreground window all show the same:

Class name: Windows.UI.Core.CoreWindow  Window title: Search

A window title with "Start" as the title never becomes foregorund/active.

Although "Start" does exist and another problem is the "Start" window stays visible (attribute) 100% of the time. So I can't even check based on visibility.

Another problem: There are over a dozen windows using class name Windows.UI.Core.CoreWindow

Some light near the end of the tunnel:

TAGPOINT with _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointHov), $GA_ROOT) (thanks to @Nine here)is the only thing that I've seen name the correct window. But this TAGPOINT was obtained with _WinAPI_GetMousePos and therefore required clicking on the Start button and hovering over the Start menu to get it.

My last attempt was with _WinAPI_GetCurrentPosition because it returns with a TAGPOINT. So there may be potential there but I am not very familiar with it and there are zero examples in the forum.

#include <WinAPI.au3>

$hStart = 0x101dc
$hDC = _WinAPI_GetDC($hStart)
$tPointStart = _WinAPI_GetCurrentPosition($hDC)
$hWndStart = _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointStart), $GA_ROOT)
ConsoleWrite("title: " & _WinAPI_GetWindowText($hWndStart) & @CRLF)

As you can see, I was exhausted after spending almost two hours on this issue and resorted to just manually copying the handle for the Start menu window. That is the exact moment when I realized that I should ask for help. 😆

I am still pretty new to AutoIt. Lots to learn. But I will always put in a significant amount of effort and attempts before I eventually ask for help here. That is because I value everyone's time and don't want to ask without putting in some effort on my own first. I am legitimately stuck on this one.

This is something that would have been simple on Windows 7 and prior.

Edited Sunday at 12:52 PM by WildByDesign

[WildByDesign]

So apparently none of the functions seem to find these "modern" windows.

For example:

Local $aEnumWindows = _WinAPI_EnumWindows(False)
For $n = 1 To UBound($aEnumWindows) - 1
    If $aEnumWindows[$n][1] <> "Start" Then ContinueLoop
    ConsoleWrite("start menu?" & @CRLF)
Next

There are 2 windows titled "Start". It does find the win32 "Start" related to the actual Start button on the taskbar. But it does not find the "Start" window which is active and visible.

This also seems to explain why things like this don't work:

WinExists("[Class:Windows.UI.Core.CoreWindow]", "Start")

What a thorough and interesting mess. I will continue to try more things and hopefully not lose my marbles in the madness. 😃

WindowFromPoint seems to still be the only function that can see these modern windows.

Edited Sunday at 12:55 PM by WildByDesign

[Nine]

You would need to use UIAutomation :

#include "..\UIAutomation\Includes\CUIAutomation2.au3"
#include <Array.au3>

Opt("MustDeclareVars", True)

; Create UI Automation object
Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
If Not IsObj($oUIAutomation) Then Exit ConsoleWrite("$oUIAutomation ERR" & @CRLF)
;ConsoleWrite("$oUIAutomation OK" & @CRLF)

; Get Desktop element
Local $pDesktop, $oDesktop, $sValue
$oUIAutomation.GetRootElement($pDesktop)
$oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oDesktop) Then Exit ConsoleWrite("$oDesktop ERR" & @CRLF)
;ConsoleWrite("$oDesktop OK" & @CRLF)

; Get Start button
Local $pCondition, $pStart, $oStart, $sValue, $aRect
$oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, "StartButton", $pCondition)
$oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pStart)
$oStart = ObjCreateInterface($pStart, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oStart) Then Exit ConsoleWrite("$oStart ERR" & @CRLF)
;ConsoleWrite("$oStart OK" & @CRLF)
$oStart.GetCurrentPropertyValue($UIA_NamePropertyId, $sValue)
ConsoleWrite($sValue & @CRLF)

$oStart.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $aRect)
_ArrayDisplay($aRect)

 

[WildByDesign]

11 hours ago, Nine said:

You would need to use UIAutomation :

Thank you. I am marking your response as the solution because I believe that UIAutomation is the sane and proper method for this scenario. Anyone with a similar scenario should use UIAutomation.

However, I personally have struggled to learn and understand UIAutomation. It almost feels like a language of its own.

I ended up figuring out a way to do it within a block of code that I already have setup (also a solution from @Nine) from here.

Within the IsPressed section that I added, I just had to make sure that classname = "Windows.UI.Core.CoreWindow" And window text = "DesktopWindowXamlSource". If both matched at the same time, I just had to run a quick _WinAPI_GetWindowText(_WinAPI_GetForegroundWindow()) to make sure that = "Start" Or "Search". This captured the Start button click 100% of the time.

Since I had this block of code already functioning in my script for detecting when "Progman" had been clicked, it just made sense to adapt it quite easily in the end. Well, not exactly easily. But it worked out great at the end of the day.

Thank you again for your time.

Edited yesterday at 12:42 AM by WildByDesign

[ioa747]

; https://www.autoitscript.com/forum/topic/213297-how-to-detect-when-start-menu-opens-win10win11/#findComment-1547177

#include <WinAPISysWin.au3>

HotKeySet("{ESC}", "GoToExit")

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start")
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

Func CheckForTitle($sTitle, $sClass = "")
    Local Static $tStruct = DllStructCreate($tagPOINT) ; Create a structure that defines the point to be checked.
    DllStructSetData($tStruct, "x", MouseGetPos(0))    ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    DllStructSetData($tStruct, "y", MouseGetPos(1))    ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    Local $hWnd = _WinAPI_WindowFromPoint($tStruct)    ; Retrieves the handle of the window that contains the specified point.
    Local $sActiveTitle = WinGetTitle($hWnd)           ; Retrieves the full title from a window.
    If $sActiveTitle = $sTitle Then
        Local $sActiveClass = _WinAPI_GetClassName($hWnd)
        If $sClass = "" Then
            Return $hWnd
        Else
            If $sActiveClass = $sClass Then Return $hWnd
        EndIf
    EndIf
    Return 0
EndFunc   ;==>CheckForTitle

Func GoToExit()
    Exit
EndFunc   ;==>GoToExit

 

[WildByDesign]

@ioa747 Holy Guacamole! That is short and sweet. 🙏

I just tested your example and it does exactly what I need and it does it beautifully and with a small amount of code.

I am definitely going to try to swap out my existing functionality and use yours instead because it's more accurate and should be way more efficient as well compared to what I was doing.

[Werty]

2 hours ago, ioa747 said:

win10

I'm on win10 and have hidden taskbar, your script triggers when I only hover over the start button, without clicking and opening it.

[ioa747]

Try adding the class as well, to narrow down the results 

[Werty]

Same thing, atleast if I'm doing it right, could you post an example of what exactly you mean?

[ioa747]

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start", "Windows.UI.Core.CoreWindow")
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

 

[Werty]

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window.

[WildByDesign]

2 minutes ago, Werty said:

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window

Same, I noticed that as well. That is why mine checks the foreground/active window 20ms after to confirm Start menu activated.

But that reminds me, I never did end up sharing my working solution. I am not at my PC right now but will share it in a few hours just in case it’s worthwhile.

[ioa747]

1 hour ago, Werty said:

Ok that works, kinda, it doesnt trigger until I moved the mouse from the start button and into the window.

This seems logical to me, since it uses _WinAPI_WindowFromPoint  

Edit:
this could be fixed with an offset parameter

#include <WinAPISysWin.au3>

HotKeySet("{ESC}", "GoToExit")

;**********************************
While Sleep(100)
    Local $hStart = CheckForTitle("Start", "Windows.UI.Core.CoreWindow", 100, -100)
    If $hStart Then MsgBox(0, " 👮 I caught you.", "The handle to the 'Start' window: " & $hStart)
WEnd
;**********************************

Func CheckForTitle($sTitle, $sClass = "", $iXOffSet = 0, $iYOffSet = 0)
    Local Static $tStruct = DllStructCreate($tagPOINT) ; Create a structure that defines the point to be checked.
    DllStructSetData($tStruct, "x", MouseGetPos(0) + $iXOffSet) ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    DllStructSetData($tStruct, "y", MouseGetPos(1) + $iYOffSet) ; Update the X and Y elements with the X and Y co-ordinates of the mouse.
    Local $hWnd = _WinAPI_WindowFromPoint($tStruct) ; Retrieves the handle of the window that contains the specified point.
    Local $sActiveTitle = WinGetTitle($hWnd)        ; Retrieves the full title from a window.
    If $sActiveTitle = $sTitle Then
        Local $sActiveClass = _WinAPI_GetClassName($hWnd)
        If $sClass = "" Then
            Return $hWnd
        Else
            If $sActiveClass = $sClass Then Return $hWnd
        EndIf
    EndIf
    Return 0
EndFunc   ;==>CheckForTitle

Func GoToExit()
    Exit
EndFunc   ;==>GoToExit

 

Edited 29 minutes ago by ioa747