I want to modify this object detection code to detect any program on screen

[yalil]

 

 

sorry for my english and i was try to learn autoit for 5 days    never write a programe before

this is the code

 

#Region ;**** Directives created by AutoIt3Wrapper_GUI ****
#AutoIt3Wrapper_UseX64=y
#AutoIt3Wrapper_AU3Check_Parameters=-w 1 -w 2 -w 3 -w 4 -w 5
#EndRegion ;**** Directives created by AutoIt3Wrapper_GUI ****
#include <ButtonConstants.au3>
#include <EditConstants.au3>
#include <GUIConstantsEx.au3>
#include <StaticConstants.au3>
#include <WindowsConstants.au3>
#include <AutoYOLO3.au3>
#include <GDIPlus.au3>

ConsoleWrite("Using autoyolo version:" & _AutoYolo3_GetVersion() & @CRLF)

_GUIExample1()

Func _GUIExample1()
    _GDIPlus_Startup()

    Local $iPicW=609
    Local $iPicH=385

    #Region ### START Koda GUI section ### Form=
    GUICreate(@ScriptName&" - Object Detection with AutoIt!", 625, 506)
    GUISetFont(14, 400, 0, "Calibri")
    Local $Button1 = GUICtrlCreateButton("browse", 472, 8, 145, 33)
    Local $Input1 = GUICtrlCreateInput(@ScriptDir & "\people-2557408_1920.jpg", 8, 8, 457, 31)
    Local $Button2 = GUICtrlCreateButton("detect objects!", 228, 448, 169, 49)
    Local $Pic1 = GUICtrlCreatePic("", 8, 48, $iPicW, $iPicH, $SS_BITMAP)
    GUISetState(@SW_SHOW)
    #EndRegion ### END Koda GUI section ###

    Local $hImage = _LoadImage($Input1, $Pic1, 0, $iPicW,$iPicH)

    While 1
        $nMsg = GUIGetMsg()
        Switch $nMsg
            Case $GUI_EVENT_CLOSE
                Exit
            Case $Button1
                GUICtrlSetData($Input1, FileOpenDialog("Please select image", @ScriptDir, "All (*.*)"))
                $hImage = _LoadImage($Input1, $Pic1, $hImage, $iPicW,$iPicH)
            Case $Button2
                GUICtrlSetState($Button2, $GUI_DISABLE)
                GUICtrlSetData($Button2, "working..")
                Local $hHBITMAP = _GDIPlus_BitmapCreateHBITMAPFromBitmap($hImage, 0x00000000)
                Local $aRes = _AutoYolo3_GetObjInHBitmap($hHBITMAP)
                ;_ArrayDisplay($aRes)
                If @error Then
                    Local $iErr = @error
                    Local $asDllErrors[6] = ["No error", "unable to use the DLL file", "unknown ""return type""", """function"" not found in the DLL file", "bad number of parameters", "bad parameter"]
                    If $iErr < 6 Then
                        MsgBox(32, @ScriptName, "DLL Error, " & $asDllErrors[$iErr])
                    Else
                        MsgBox(32, @ScriptName, "DLL Error, " & _AutoYolo3_GetLastError())
                    EndIf
                ElseIf _AutoYolo3_GetLastError() <> "" Then
                    MsgBox(32, @ScriptName, _AutoYolo3_GetLastError())
                Else
                    ;nice thick green boxes
                    Local $hPen = _GDIPlus_PenCreate(0xFF00FF00, 2, 2)

                    Local $hBrush = _GDIPlus_BrushCreateSolid(0xFF000000)
                    Local $hFormat = _GDIPlus_StringFormatCreate()
                    Local $hFamily = _GDIPlus_FontFamilyCreate("Calibri")
                    Local $hFont = _GDIPlus_FontCreate($hFamily, 12, 0)
                    Local $tLayout = _GDIPlus_RectFCreate(140, 110, 100, 20)
                    Local $hBrushBack = _GDIPlus_BrushCreateSolid(0x7FFFFFFF) ;color format AARRGGBB (hex)
                    $hGraphics = _GDIPlus_ImageGetGraphicsContext($hImage)
                    _GDIPlus_GraphicsSetSmoothingMode($hGraphics, $GDIP_SMOOTHINGMODE_HIGHQUALITY) ;sets the graphics object rendering quality (antialiasing)
                    For $i = 1 To $aRes[0][0]
                        ; Draw a frame around the object
                        _GDIPlus_GraphicsDrawRect($hGraphics, $aRes[$i][2], $aRes[$i][3] , ($aRes[$i][4] - $aRes[$i][2]) , ($aRes[$i][5] - $aRes[$i][3]) , $hPen)
                        ; Label it
                        $sLabel = $aRes[$i][0] & " " & StringLeft($aRes[$i][1], 6)
                        $tLayout = _GDIPlus_RectFCreate($aRes[$i][2] , $aRes[$i][3] , 200, 20)
                        $iBackLen = 0
                        If (StringLen($sLabel) * 8) > $iBackLen Then
                            $iBackLen = StringLen($sLabel) * 8
                        EndIf

                        _GDIPlus_GraphicsFillRect($hGraphics, $aRes[$i][2] , $aRes[$i][3], $iBackLen, 20, $hBrushBack)
                        _GDIPlus_GraphicsDrawStringEx($hGraphics, $sLabel, $hFont, $tLayout, $hFormat, $hBrush)
                    Next
                    $hHBITMAP = _GDIPlus_BitmapCreateHBITMAPFromBitmap($hImage, 0x00000000)
                    GUICtrlSendMsg($Pic1, $STM_SETIMAGE, $IMAGE_BITMAP, $hHBITMAP)
                    _GDIPlus_PenDispose($hPen)
                    _GDIPlus_FontDispose($hFont)
                    _GDIPlus_FontFamilyDispose($hFamily)
                    _GDIPlus_StringFormatDispose($hFormat)
                    _GDIPlus_BrushDispose($hBrush)
                    _GDIPlus_BrushDispose($hBrushBack)
                    _GDIPlus_GraphicsDispose($hGraphics)
                EndIf
                GUICtrlSetData($Button2, "detect objects!")
                GUICtrlSetState($Button2, $GUI_ENABLE)
        EndSwitch
    WEnd

    If $hImage <> 0 Then
        _GDIPlus_ImageDispose($hImage)
    EndIf
    _GDIPlus_Shutdown()
EndFunc   ;==>_GUIExample1

Func _LoadImage($Input1, $Pic1, $hImage, $iPicW, $iPicH)
    If $hImage <> 0 Then
        _GDIPlus_ImageDispose($hImage)
    EndIf

    Local $sImagePath = GUICtrlRead($Input1)
    $hImage = _GDIPlus_ImageLoadFromFile($sImagePath)

    Local $iWidth = _GDIPlus_ImageGetWidth($hImage)
    Local $iHeight = _GDIPlus_ImageGetHeight($hImage)

    Local $sFactor = 1
    If $iWidth > $iPicW Then
        $sFactor = $iPicW / $iWidth
    EndIf
    If $iHeight > $iPicH And ($iHeight * $sFactor) > $iPicH Then
        $sFactor = $iPicH / $iHeight
    EndIf
    $iHeight = $iHeight * $sFactor
    $iWidth = $iWidth * $sFactor

    $hResizedImage = _GDIPlus_ImageResize($hImage, $iWidth, $iHeight)

    _GDIPlus_BitmapDispose($hImage)

    Local $hHBITMAP = _GDIPlus_BitmapCreateHBITMAPFromBitmap($hResizedImage, 0x00000000)
    GUICtrlSendMsg($Pic1, $STM_SETIMAGE, $IMAGE_BITMAP, $hHBITMAP)
    Return $hResizedImage
EndFunc   ;==>_LoadImage

 

[AutoBert]

AutoYOLO3.au3 isn't a standard-udf and so i got error:

+>         SciTEDir => C:\Program Files (x86)\AutoIt3\SciTE   UserDir => C:\Users\(len)Bert\AppData\Local\AutoIt v3\SciTE\AutoIt3Wrapper   SCITE_USERHOME => C:\Users\(len)Bert\AppData\Local\AutoIt v3\SciTE 
!->Includefile <AutoYOLO3.au3> not found.

 

[junkew]

Put the full path in there

#include <AutoYOLO3.au3>

otherwise read the helpfile to put correct entries in registry on where to find includefiles

https://www.autoitscript.com/autoit3/docs/keywords/include.htm

Edited February 2, 2023 by junkew

[yalil]

go to this page 

 

 

 

[AutoBert]

Following the link: i get:

Sichere Website nicht verfügbar.html

 

[yalil]

if you do it right you will get somthing like this

[yalil]

and i found someone on youtube did it and i just want to change code to detect on screen not form image so

helpppppppppppppppppppppppppppppppppppp!!!!!!

 

[AutoBert]

3 minutes ago, yalil said:

if you do it right you will get somthing like this

I didn't it because the linked page is HTTP and not HTTPS, so my browser blocks the page

[yalil]

search "object detection autoit" on google 

[yalil]

here is the line that i think i have problem wtih 

 

can you example how to change the gui to detect on screen not form image

[Jos]

10 hours ago, yalil said:

and i found someone on youtube did it and i just want to change code to detect on screen not form image so

helpppppppppppppppppppppppppppppppppppp!!!!!!

 

Looking at these screen captures, it seems this is for automating a Game., so I advice you to now first read our forum rules and not to further discuss this topic.

*click*