UIAutomation issue

[fraizor]

Hello

I'm currently working with your UIAutomation_Extended.au3 

I'm trying to achieve a specific goal: automate the process of extracting OCR result text from the Windows 11 Photos app (after clicking the "Scan Text" button on an image).

So far, I’ve been able to:

• Locate and click the “Scan Text” button using UIA (specifically, the OcrButton AutomationId).

• Wait for the OCR to complete, and detect that a new element labeled img (1).jpg appears in the UIA tree, with ControlType 50032 (Text).

However, I'm running into an issue:
Even though this element appears after OCR has been triggered, and seems like it should contain the result, but i cannot figure out how to fetch the result, here is my code

 

#include <UIAutomation_Extended.au3>
;~ ; Initialize UI Automation
Local $oUIA = _UIA_InitInterface()
Local $oRoot = _UIA_GetRootElement($oUIA)

; Find Photos window element by Name
Local $oCondition = _UIA_CreatePropertyCondition($oUIA, $UIA_ClassNamePropertyId , "WinUIDesktopWin32WindowClass")
Local $oCalc = _UIA_FindFirst($oRoot, $oCondition, $TreeScope_Descendants)

; Find the "Scan Text" button using its AutomationId
$oCondition = _UIA_CreatePropertyCondition($oUIA, $UIA_AutomationIdPropertyId, "OcrButton")
Local $oBtn1 = _UIA_FindFirst($oCalc, $oCondition, $TreeScope_Descendants)

;~ ; Invoke the button (Toggle)
Local $oPattern = _UIA_GetCurrentPattern($oBtn1, $UIA_TogglePatternId)
$oPattern.Toggle()

---

Would you happen to know if:

1. The Photos app renders OCR text in a way that's not accessible via standard TextPattern or ValuePattern?

2. There’s a workaround with your UDF (perhaps using LegacyIAccessible, RawViewWalker, or something low-level)?

3. You’ve seen similar behavior with modern UWP/XAML apps using Canvas-like rendering?

Any advice or insight you can provide would be greatly appreciated.

Thanks

Edited July 31 by fraizor

[junkew]

If spytools are not showing it most likely you cannot retrieve with api.

For sure you need to look not at the button for the text.

Best chance to send ctrl a and ctrl c and get it from the clipboard with the clipboard functions