How to automate OpenJDK app ?

[mLipok]

As so far I try to use UIASpy to inspect this app but with no success.
The desired element are not visible for UIASpy

How to automate OpenJDK app ?
I need to click button and provide text in INPUT like field.

 

As so far I found:

Please confirm if this is the only one resonable way.

Edited November 27, 2023 by mLipok

[junkew]

The others are

• Findbmp and click based on a percentagecoordinate like
• percentageMouseclick(50%,50%) to click relative. I have to look for a function thats doing that

• testpercentagemove("50%","50%")
  
  func testpercentagemove($x,$y)
      if stringinstr($x,"%") Then
          $realX=stringreplace($x,"%","") * (@DesktopWidth / 100)
          $realy=stringreplace($y,"%","") * (@DesktopHeight / 100)
          
          mousemove($realX ,$realy)
      EndIf
  EndFunc

 

 

Edited November 27, 2023 by junkew

[mLipok]

As for now I uses @Danyfirex UDF:

 

With such example:

expandcollapse popup

#include <array.au3>
#include <GDIPlus.au3>
#include <ScreenCapture.au3>
#include "..\UWPOCR.au3"


_Example()

Func _Example()
    If Not WinExists("[CLASS:SunAwtDialog]") Then
        ConsoleWrite("! JAVA APP Window not exist" & @CRLF)
        Return
    EndIf
    WinActivate("[CLASS:SunAwtDialog]")

    #Region - create example bitmap - take a screenshot
    _GDIPlus_Startup()
    ;hImage/hBitmap GDI
    Local $hTimer = TimerInit()
    Local $hHBitmap = _ScreenCapture_CaptureWnd("", WinWait("[CLASS:SunAwtDialog]"), 300, 500) ; Get Text from some JAVA application
;~  Local $hHBitmap = _ScreenCapture_CaptureWnd("", WinWait("[CLASS:SunAwtDialog]")) ; Get Text from some JAVA application
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBitmap)
    #EndRegion - create example bitmap  - take a screenshot

    #Region - do the OCR Stuff
    ;Get OCR Text From hImage/hBitmap
;~  _UWPOCR_Log(__UWPOCR_Log) ;Enable Log
    Local $sOCRTextResult = _UWPOCR_GetText($hBitmap, Default, True)
    ClipPut($sOCRTextResult)
    MsgBox($MB_TOPMOST, "Time Elapsed: " & TimerDiff($hTimer), $sOCRTextResult)
    #EndRegion - do the OCR Stuff

    $hTimer = TimerInit()
    ; do the OCR stuff with getting coordinates for each word
    Local $aWords = _UWPOCR_GetWordsRectTo2DArray($hBitmap)
    _ArrayDisplay($aWords, "Time Elapsed: " & TimerDiff($hTimer))

    #Region - bitmap clean up
    _WinAPI_DeleteObject($hHBitmap)
    _GDIPlus_BitmapDispose($hBitmap)
    _GDIPlus_Shutdown()
    #EndRegion - bitmap clean up

EndFunc   ;==>_Example

 

Edited November 28, 2023 by mLipok