How to detect when Start menu opens (Win10/Win11)

[WildByDesign]

I am surprisingly more perplexed by this than with any previous challenge. 🙃

So perplexed that I don't have any code examples to share here to begin with.

I want to detect the moment that the Start menu opens.

I've tried WinEventHook, shell hook, and also simple loops checking active process.

The problem is, at least on Win11, all built-in functions to check active window and all UDF WinAPI functions to check active window, window with focus, and foreground window all show the same:

Class name: Windows.UI.Core.CoreWindow  Window title: Search

A window title with "Start" as the title never becomes foregorund/active.

Although "Start" does exist and another problem is the "Start" window stays visible (attribute) 100% of the time. So I can't even check based on visibility.

Another problem: There are over a dozen windows using class name Windows.UI.Core.CoreWindow

Some light near the end of the tunnel:

TAGPOINT with _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointHov), $GA_ROOT) (thanks to @Nine here)is the only thing that I've seen name the correct window. But this TAGPOINT was obtained with _WinAPI_GetMousePos and therefore required clicking on the Start button and hovering over the Start menu to get it.

My last attempt was with _WinAPI_GetCurrentPosition because it returns with a TAGPOINT. So there may be potential there but I am not very familiar with it and there are zero examples in the forum.

#include <WinAPI.au3>

$hStart = 0x101dc
$hDC = _WinAPI_GetDC($hStart)
$tPointStart = _WinAPI_GetCurrentPosition($hDC)
$hWndStart = _WinAPI_GetAncestor(_WinAPI_WindowFromPoint($tPointStart), $GA_ROOT)
ConsoleWrite("title: " & _WinAPI_GetWindowText($hWndStart) & @CRLF)

As you can see, I was exhausted after spending almost two hours on this issue and resorted to just manually copying the handle for the Start menu window. That is the exact moment when I realized that I should ask for help. 😆

I am still pretty new to AutoIt. Lots to learn. But I will always put in a significant amount of effort and attempts before I eventually ask for help here. That is because I value everyone's time and don't want to ask without putting in some effort on my own first. I am legitimately stuck on this one.

This is something that would have been simple on Windows 7 and prior.

Edited 15 hours ago by WildByDesign

[WildByDesign]

So apparently none of the functions seem to find these "modern" windows.

For example:

Local $aEnumWindows = _WinAPI_EnumWindows(False)
For $n = 1 To UBound($aEnumWindows) - 1
    If $aEnumWindows[$n][1] <> "Start" Then ContinueLoop
    ConsoleWrite("start menu?" & @CRLF)
Next

There are 2 windows titled "Start". It does find the win32 "Start" related to the actual Start button on the taskbar. But it does not find the "Start" window which is active and visible.

This also seems to explain why things like this don't work:

WinExists("[Class:Windows.UI.Core.CoreWindow]", "Start")

What a thorough and interesting mess. I will continue to try more things and hopefully not lose my marbles in the madness. 😃

WindowFromPoint seems to still be the only function that can see these modern windows.

Edited 15 hours ago by WildByDesign

[Nine]

You would need to use UIAutomation :

#include "..\UIAutomation\Includes\CUIAutomation2.au3"
#include <Array.au3>

Opt("MustDeclareVars", True)

; Create UI Automation object
Local $oUIAutomation = ObjCreateInterface($sCLSID_CUIAutomation, $sIID_IUIAutomation, $dtagIUIAutomation)
If Not IsObj($oUIAutomation) Then Exit ConsoleWrite("$oUIAutomation ERR" & @CRLF)
;ConsoleWrite("$oUIAutomation OK" & @CRLF)

; Get Desktop element
Local $pDesktop, $oDesktop, $sValue
$oUIAutomation.GetRootElement($pDesktop)
$oDesktop = ObjCreateInterface($pDesktop, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oDesktop) Then Exit ConsoleWrite("$oDesktop ERR" & @CRLF)
;ConsoleWrite("$oDesktop OK" & @CRLF)

; Get Start button
Local $pCondition, $pStart, $oStart, $sValue, $aRect
$oUIAutomation.CreatePropertyCondition($UIA_AutomationIdPropertyId, "StartButton", $pCondition)
$oDesktop.FindFirst($TreeScope_Descendants, $pCondition, $pStart)
$oStart = ObjCreateInterface($pStart, $sIID_IUIAutomationElement, $dtagIUIAutomationElement)
If Not IsObj($oStart) Then Exit ConsoleWrite("$oStart ERR" & @CRLF)
;ConsoleWrite("$oStart OK" & @CRLF)
$oStart.GetCurrentPropertyValue($UIA_NamePropertyId, $sValue)
ConsoleWrite($sValue & @CRLF)

$oStart.GetCurrentPropertyValue($UIA_BoundingRectanglePropertyId, $aRect)
_ArrayDisplay($aRect)

 

[WildByDesign]

11 hours ago, Nine said:

You would need to use UIAutomation :

Thank you. I am marking your response as the solution because I believe that UIAutomation is the sane and proper method for this scenario. Anyone with a similar scenario should use UIAutomation.

However, I personally have struggled to learn and understand UIAutomation. It almost feels like a language of its own.

I ended up figuring out a way to do it within a block of code that I already have setup (also a solution from @Nine) from here.

Within the IsPressed section that I added, I just had to make sure that classname = "Windows.UI.Core.CoreWindow" And window text = "DesktopWindowXamlSource". If both matched at the same time, I just had to run a quick _WinAPI_GetWindowText(_WinAPI_GetForegroundWindow()) to make sure that = "Start" Or "Search". This captured the Start button click 100% of the time.

Since I had this block of code already functioning in my script for detecting when "Progman" had been clicked, it just made sense to adapt it quite easily in the end. Well, not exactly easily. But it worked out great at the end of the day.

Thank you again for your time.

Edited 3 hours ago by WildByDesign