Sign in to follow this  
Followers 0

Image streaming

2 posts in this topic

I am working on a desktop remote application so I made a simple server and client to send binary screen captures from client to server. All it's good if I test both scripts on my computer but when the client is in other network many screen shots are corrupted and some of them looks good. Have any idea why?



#include <ScreenCapture.au3>
#include <Memory.au3>
#include <WinAPI.au3>
#include <GDIPlus.au3>

$Client = TCPConnect(@IPAddress1,12100)


While True
    Local $hHBitmap = _ScreenCapture_Capture('')
    Local $hBitmap = _GDIPlus_BitmapCreateFromHBITMAP($hHBitmap)
    $bData = _GDIPlus_StreamImage2BinaryString($hBitmap)
    TCPSend($Client,'~stream:' & BinaryLen($bData))
    While BinaryLen($bData)
        $a = TCPSend($Client, $bData)
        $bData = BinaryMid($bData, $a+1, BinaryLen($bData)-$a)


Func _GDIPlus_StreamImage2BinaryString($hBitmap, $sFormat = "JPG", $iQuality = 100)  ;UEZ
    Local $sImgCLSID, $tGUID, $tParams, $tData
    Switch $sFormat
        Case "JPG"
            $sImgCLSID = _GDIPlus_EncodersGetCLSID($sFormat)
            $tGUID = _WinAPI_GUIDFromString($sImgCLSID)
            $tData = DllStructCreate("int Quality")
            DllStructSetData($tData, "Quality", $iQuality) ;quality 0-100
            Local $pData = DllStructGetPtr($tData)
            $tParams = _GDIPlus_ParamInit(1)
            _GDIPlus_ParamAdd($tParams, $GDIP_EPGQUALITY, 1, $GDIP_EPTLONG, $pData)
        Case "PNG", "BMP", "GIF", "TIF"
            $sImgCLSID = _GDIPlus_EncodersGetCLSID($sFormat)
            $tGUID = _WinAPI_GUIDFromString($sImgCLSID)
        Case Else
            Return SetError(1, 0, 0)
    Local $hStream = _WinAPI_CreateStreamOnHGlobal() ;
    If @error Then Return SetError(2, 0, 0)
    _GDIPlus_ImageSaveToStream($hBitmap, $hStream, DllStructGetPtr($tGUID), DllStructGetPtr($tParams))
    If @error Then Return SetError(3, 0, 0)
    Local $hMemory = _WinAPI_GetHGlobalFromStream($hStream) ;
    If @error Then Return SetError(4, 0, 0)
    Local $iMemSize = _MemGlobalSize($hMemory)
    If Not $iMemSize Then Return SetError(5, 0, 0)
    Local $pMem = _MemGlobalLock($hMemory)
    $tData = DllStructCreate("byte[" & $iMemSize & "]", $pMem)
    Local $bData = DllStructGetData($tData, 1)
    _WinAPI_ReleaseStream($hStream) ;
    Return $bData
EndFunc   ;==>_GDIPlus_StreamImage2BinaryString



Global $Buffer, $BufferSize
Global $Count = 0


$Server = TCPListen(@IPAddress1,12100)
If @error Then MsgBox(0,'',@error)

    $Socket = TCPAccept($Server)
Until $Socket <> -1

While True
    $Recv = TCPRecv($Socket,10240)
    If $Recv = -1 Then
    ElseIf $Recv Then
        If StringLeft($Recv,7) = '~stream' Then
            $BufferSize = StringSplit($Recv,':')[2]
            $Buffer = '0x'
                $Recv = TCPRecv($Socket,10240)
                If BinaryLen($Recv) <> 0 Then
                    $BufferSize -= BinaryLen($Recv)
                    $Buffer &= StringTrimLeft($Recv,2)
            Until $BufferSize = 0
            $hFile = FileOpen(@ScriptDir & '\Screen' & $Count & '.jpeg',18)
            $Buffer = Null
            $Count += 1
        If $Count = 100 Then ExitLoop




When the words fail... music speaks

Share this post

Link to post
Share on other sites

I can only imagine it is a timing issue, but I am no expert, especially with Network stuff.

AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Make sure brain is in gear before opening mouth!
Remember, what is not said, can be just as important as what is said.


What is the Secret Key? Life is like a Donut

If I put effort into communication, I expect you to read properly & fully, or just not comment.
Ignoring those who try to divert conversation with irrelevancies.
If I'm intent on insulting you or being rude, I will be obvious, not ambiguous about it.
I'm only big and bad, to those who have an over-active imagination.

I may have the Artistic Liesense ;) to disagree with you. TheSaint's Toolbox (be advised many downloads are not working due to ISP screwup with my storage)


Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  
Followers 0

  • Similar Content

    • DeathChicken
      By DeathChicken
      If possible please add or edit the comments to explain how this works.
      ;includes functions from other things
      #include <GDIPlus.au3>
      #include <ScreenCapture.au3>
      HotKeySet("{ESC}", _exit)
      HotKeySet("{F1}", _scan)
      ;global variables
      Global $win_title                    ;name of the window
      Global $area_x                        ;
      Global $area_y                        ;
      Global $area_w                       ;
      Global $area_h                        ;
      Global $cursor                         ;
      Global $rect_file                      ;
      Global $hbmpscreen              ;
      Global $i=  1043                      ;moves the rect
      Global $ii=    378                      ;moves the rect
      Global $x=1044                        ;inner rect offset
      Global $y=501                          ;inner rect offset
      Global $hbmprect                    ;image inside rect
      Global $hscreen                        ;image whole screen
      _GDIPlus_Startup()                    ;?
      ;display hotkeys on screen
      ToolTip("Press F1 to scan | Press ESC to Exit",0,0)
                                          ;infinite loop to keep prog running
      While 1
      Func _scan()
      ;~ reads your screen area:
      ;~ converts screen captured into bmp
      ;~ .  Loads converted bmp to be read
      ;~ compares the bmp of your scanned screen to the actual screen that is being displayed
                                          ;reads screen
      Func _read()
          $hscreen = _ScreenCapture_CaptureWnd("", WinGetHandle($win_title), $area_x, $area_y, $area_x + $area_w, $area_y + $area_h, $cursor)
                                          ;converts screen into bmp
      Func _convert()
                                          ;loads converted screen
      Func _loadBMP()
                                          ;compares savedBMP to current screen
      Func _compare()
      ;If(                              below code                                     ) = (                   below code           ) Then
      If _GDIPlus_BitmapGetPixel($hbmpscreen, ($i + $x) - $area_x, ($ii + $y) - $area_y) = _GDIPlus_BitmapGetPixel($hbmprect, $x, $y) Then
      ;display message box titled found with a message of found
      ;if above is not correct then
                                          ;exit func
      Func _exit()
    • GordonFreeman
      By GordonFreeman
      Hi, i searched some topics but cannot found a simple way to do that:
      #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> $Form1 = GUICreate("Form1", 615, 438, 192, 124) GUISetState(@SW_SHOW) ; I tried this but user reported that it work in 2006, not seems work actually $filemenu = GUICtrlCreateMenu ("&File") $fileitem = GUICtrlCreateMenuitem ("Open",$filemenu) GuiCtrlSetImage($fileitem, "shell32.dll", 4) While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit EndSwitch WEnd Also tried:
      #include <GUIConstantsEx.au3> #include <WindowsConstants.au3> $Form1 = GUICreate("Form1", 615, 438, 192, 124) GUISetState(@SW_SHOW) $filemenu = TrayCreateMenu("&File") $fileitem = TrayCreateItem("Open",$filemenu) ;GuiCtrlSetImage($fileitem, "shell32.dll", 4) ;GuiCtrlSetImage(TrayItemGetHandle($fileitem), "shell32.dll", 4) While 1 $nMsg = GUIGetMsg() Switch $nMsg Case $GUI_EVENT_CLOSE Exit EndSwitch WEnd Well.. thanks in advance!
      Ok. I found this working example from Yashied. If i cannot find a simple then i will go with it
      #Include <GUIConstantsEx.au3> #Include <GUIMenu.au3> #Include <Constants.au3> #Include <WinAPI.au3> #Include <WindowsConstants.au3> Opt('MustDeclareVars', 1) Global $hMenu, $hForm, $hFile = 1000, $idNew, $idExit $hForm = GUICreate('Menu', 400, 300) $hFile = _GUICtrlMenu_CreateMenu() _GUICtrlMenu_InsertMenuItem ($hFile, 0, ' &Favorites', $idNew) _GUICtrlMenu_InsertMenuItem ($hFile, 1, '', 0) _GUICtrlMenu_InsertMenuItem($hFile, 2, ' E&xit', $idExit) $hMenu = _GUICtrlMenu_CreateMenu() _GUICtrlMenu_InsertMenuItem($hMenu, 0, '&File', 0, $hFile) _GUICtrlMenu_SetMenu($hForm, $hMenu) _GUICtrlMenu_SetItemBmp($hFile, 0, _CreateBitmapFromIcon(_WinAPI_GetSysColor($COLOR_MENU), @SystemDir & '\shell32.dll', 43, 16, 16)) _GUICtrlMenu_SetItemBmp($hFile, 2, _CreateBitmapFromIcon(_WinAPI_GetSysColor($COLOR_MENU), @SystemDir & '\shell32.dll', 27, 16, 16)) GUISetState() Do Until GUIGetMsg() = $GUI_EVENT_CLOSE Func _CreateBitmapFromIcon($iBackground, $sIcon, $iIndex, $iWidth, $iHeight) Local $hDC, $hBackDC, $hBackSv, $hIcon, $hBitmap $hDC = _WinAPI_GetDC(0) $hBackDC = _WinAPI_CreateCompatibleDC($hDC) $hBitmap = _WinAPI_CreateSolidBitmap(0, $iBackground, $iWidth, $iHeight) $hBackSv = _WinAPI_SelectObject($hBackDC, $hBitmap) $hIcon = _WinAPI_PrivateExtractIcon($sIcon, $iIndex, $iWidth, $iHeight) If Not @error Then _WinAPI_DrawIconEx($hBackDC, 0, 0, $hIcon, 0, 0, 0, 0, $DI_NORMAL) _WinAPI_DestroyIcon($hIcon) EndIf _WinAPI_SelectObject($hBackDC, $hBackSv) _WinAPI_ReleaseDC(0, $hDC) _WinAPI_DeleteDC($hBackDC) Return $hBitmap EndFunc ;==>_CreateBitmapFromIcon Func _WinAPI_PrivateExtractIcon($sIcon, $iIndex, $iWidth, $iHeight) Local $hIcon, $tIcon = DllStructCreate('hwnd'), $tID = DllStructCreate('hwnd') Local $Ret = DllCall('user32.dll', 'int', 'PrivateExtractIcons', 'str', $sIcon, 'int', $iIndex, 'int', $iWidth, 'int', $iHeight, 'ptr', DllStructGetPtr($tIcon), 'ptr', DllStructGetPtr($tID), 'int', 1, 'int', 0) If (@error) Or ($Ret[0] = 0) Then Return SetError(1, 0, 0) EndIf $hIcon = DllStructGetData($tIcon, 1) If ($hIcon = Ptr(0)) Or (Not IsPtr($hIcon)) Then Return SetError(1, 0, 0) EndIf Return $hIcon EndFunc ;==>_WinAPI_PrivateExtractIcon Ok, i noticed that is a pain create submenu with _GUICtrlMenu_CreatePopup to do like in Tray, and Tray better located, then i need images/icon with Tray functions, some workaround?? 
    • SimTheNo1
      By SimTheNo1
      I am busy with building a solution for change monitoring of VOIP call program and to be properly automated means among others need for some limited OCR functionality. Current works of others are way too much overkill for this case what makes the need to build it myself. But to do it properly I significantly have to increase my knowledge about digital graphics management.
      For now I already have discovered here and there some mind blowing  Autoit miracles what can be achieved with Windows own possibilities to manipulate that what is output to the monitor. And as far I can judge there are 2 options to process graphics without use of any external libraries like ImageMagick, FreeImage and so on. These are:
       WinAPI  GDIPlus It is for me quite obvious to have various holes in general understanding of graphics and it is once more very clear what advantages gives consistent general study in 1 or another official institutions like University.  Cause there you are introduced into certain domain of knowledge in a way which usually has been perfected over long period of time. So you are not overloaded by stuff which requires a certain amount of information to be initially clear for you. For example, before starting to solve physics you first learn to read, count and so on and then move to subjects like physics.
      Though in my case opportunity to study in such educational system I had only for 8 years, from my 7th to 15th year of age in the country that was falling apart now Ukraine but used to be USSR, was all what it was. After have immigrated to Netherlands possibilities to study further haven't occur. And this therefor causes often various implications when going deep in that or another field of practical knowledge acquiring for any needed physical result, like programming to perform enormous amount of tasks. In this particular case automating VOIP call program.
      Anyway, right now I think the best direction to move is to concentrate on as basic as possible image management and if someone would maybe explain in general what is a pixel will definitely help. Particularly I am very curious about how to do picture manipulations in Autoit. Especially would help a lot to produce eventually following functions:
      createImage($imageFileName, $width, $height, $color) readImagePixel($imageFileName, $x, $y)  writeImagePixel($imageFileName, $x, $y) I do not know exactly how image is handled in computer but preferably above mentioned functions should deal with so far possible origin of graphics creation on computer. But nevertheless I definitely would love to hear any proposition for solution.
      The problem with explaining screenshots:
       VOIPConnect full Window  Part of Window with control to monitor for changes  Exact location of area where actual changes occur and have to be processed  It comes down to a rectangle of approximately 51 pixel wide and 7 pixel high. In fact if I get to learn as far as to be able exactly read, write and compare 2 images consistently across different computers I could narrow down then even further the area to watch as little as a square of 2-5 pixels wide. 
      To finish here is last detail about particularly no need for ultra fast solution at all. This because it is needed only once when it is first run on a new computer and to have to wait few minutes while it is being set instead of just a few seconds make no sence.
      This is it and what I too think to do beside this very particular case is to purify out beautiful generic Autoit functions for core image manipulation by using WinAPI or/and GDIPlus. 
    • tarretarretarre
      By tarretarretarre
      Yep yep, this is pretty much an attempt to port the existing project's concept to AutoIt's Codebase. So i will not go in to so much detail.
      This is how the communication is done Each client is isolated to the server Features
      Easy API VarType Translation (Example: If the server sends an int, the client will receive an int and vice versa) Fully featured examples Data encryption (Using Autoit's UDF Crypt.au3) Limitations / Drawbacks
      Every Broadcast/Emit is limited to a maximum of 16 parameters Every Callback Function has to have the same number of parameters as the Broadcasted/Emited event It is not possible to Broadcast/Emit objects Only 1D-arrays are allowed to be Broadcasted/Emitted (2D arrays will probably never be supported) Changelog
      Version 1.4.0 (This update DOES NOT break scripts)
      Added a new server method: _Io_getSockets which will return an array of all sockets. See more in the doc Added a banning-system, see more at: _Io_getBanlist, _Io_Ban, _Io_Sanction, _Io_IsBanned Added a new default event for clients banned. See more at default events Added two new client and server methods _Io_setEventPreScript And _Io_setEventPostScript. The intent for these is to not DRY when doing debug \ tasks that requires to be ran before or after events. Added a new client and server method _Io_ClearEvents. Added a third optional parameter to _Io_On called $socket, you may only pass the socket returned from _Io_Listen or _Io_Connect. The intent for this change is to allow for server + client in the same envoirment. Added a second parameter to _Io_Loop called $WhoAmI which should used with the new enums $_IO_SERVER and $_IO_CLIENT. The intent for this change is to allow for server + client in the same envoirment. Added a new client method _Io_TransferSocket. Added a new server method _Io_getActiveSocketCount. Optimations, avoiding Redims and unnecessary nested arrays as good as possible etc. Version 1.3.0 (This update DOES NOT break scripts)
      Got rid of unnecessary Redims with sockets and subscriptions in the main loop (This increased write performence greatly) Changed $iMaxDeadSocketsBeforeTidy from 100 to 1000 Changed _Io_setRecvPackageSize($nPackageSize = 2048) to _Io_setRecvPackageSize($nPackageSize = 4096) because 2017. Added Tests for both subscriptions and the automatic TidyUp Added a new server method: _Io_getMaxConnections Added a new server method: _Io_getMaxDeadSocketsCount Added a fifth parameter to the _Io_Listen method called $iMaxConnections which defaults to 100000. If the iMaxConnection + 1 user connects, they will be instantly disconnected. Added a parameter to _Io_Disconnect called $socket which defaults to null. If the iMaxConnections + 1 client connects, they will be instantly disconnected. Version 1.2.0 (This update DOES NOT break scripts)
      Added an option to set the packet-size of TCP-transports, see _Io_setRecvPackageSize Got rid of unnecessary StringLen's in _Io_loop Changed __Io_TidyUp to _Io_TidyUp and added it to the public Api reference list. Changed $iMaxDeadSocketsBeforeTidy default value from 1000 to 100 and added an option to disable it, read more at _Io_Listen Changed $bAutoReconnect from False to True. Fixed gitignore epicZ fail Improvemend Documentation Version 1.1.0 (This update DOES NOT break scripts)
      Fixed bug when Emitting / Broadcasting without any parameters causing a $fCallback crash Optimized Package-handling once again. Added 1D-Array support (Endless nestning). Added Subscriptions (See _Io_Subscribe _Io_Unsubscribe and _Io_BroadcastToRoom). Added new example for subscriptions (Be sure to use different room names when joining with clients) Added Unit testing (See Tests\Runner.au3 and Tests\Tests.au3, to run tests you need a udf found here: Version 1.0.0
      (This update DOES NOT break scripts) Added data encryption (Using Autoit's UDF Crypt.au3) See more at _Io_EnableEncryption Added new method _Io_Disconnect which can be used with both servers and clients Improved package-handling to increase performance Increased the limit of Broadcasted/Emit parameters from 10 to 16 Api methods
      Server methods
      _Io_Listen($iPort, $iAddress = @IPAddress1, $iMaxPendingConnections = Default, $iMaxDeadSocketsBeforeTidy = 1000, $iMaxConnections = 100000) _Io_Subscribe(ByRef $socket, $sRoomName) _Io_Unsubscribe(ByRef $socket, $sRoomName = null) _Io_Broadcast(ByRef $socket, $sEventName, $p1, $p2, ...$p16)  
      _Io_BroadcastToAll(ByRef $socket, $sEventName, $p1, $p2, ...$p16) _Io_BroadcastToRoom(ByRef $socket, $sDesiredRoomName, $sEventName, $p1, $p2, ...$p16) _Io_socketGetProperty(ByRef $socket, $sProp = Default) _Io_getSockets($bForceUpdate = False, $socket = $__g_io_mySocket, $whoAmI = $__g_io_whoami) _Io_getDeadSocketCount() _Io_getSocketsCount() _Io_getActiveSocketCount() _Io_getMaxConnections() _Io_getMaxDeadSocketsCount() _Io_getBanlist($iEntry = Default) _Io_Ban($socketOrIp, $nTime = 3600, $sReason = "Banned", $sIssuedBy = "system") _Io_Sanction($socketOrIp) _Io_IsBanned($socketOrIp) _Io_TidyUp() Client methods
      _Io_Connect($iAddress, $iPort, $bAutoReconnect = True) _Io_Reconnect(ByRef $socket) Server and Client methods
      _Io_setEventPreScript($fCallback) _Io_setEventPostScript($fCallback) _Io_getVer() _Io_On(Const $sEventName, Const $fCallback, $socket = $__g_io_mySocket) _Io_Emit(ByRef $socket, $sEventName, $p1, $p2, ...$p16) _Io_Loop(ByRef $socket, $whoAmI = $__g_io_whoami) _Io_LoopFacade() _Io_EnableEncryption($sFileOrKey, $CryptAlgId = $CALG_AES_256) _Io_Disconnect($socket = null) _Io_setRecvPackageSize($nPackageSize = 4096) _Io_ClearEvents() _Io_TransferSocket(ByRef $from, ByRef $to) Default events
      Server events
      connection Client events
      banned Server and Client events
      disconnect View source on github
     (OLD!) (OLD) (OLD) (NEWEST 2017-08-11)
    • timmalos
      By timmalos
      Hello all.
      In case this is interesting few of you, I share my AMCP 2.1 protocol UDF in AutoIT. This protocol is used by CasparCG server, which is a Windows and Linux software used to play out professional graphics, audio and video to multiple outputs as a layerbased real-time compositor. It has been in 24/7 broadcast production since 2006. It's free and opensource.
      The UDF I share allows communication between an AutoIt based client and the CasparCG, based on following documentation :
      If you want more details on CasparCG :  official WebSite or have a look to this video
      I'm currently building a full Client based on AutoIt, with many features like drag-and-drop layers, but sadly I can't share it right now, might come later. Don't hesitate to ask questions if you have any or need a basic example.
      The only requirement for this UDF is the other Event-driven TCP UDF by Kip