alien4u

Android Visual Control by USB. NO SDK Needed, NO SCREEN or TOUCH Screen Needed.

7 posts in this topic

#1 ·  Posted (edited)

This is an small script to Control Android Devices by USB using adb, you can use it with phones with a Cracked Screen or not working Touch Screen.

What it Does:
- Show your Android Screen on your Desktop.
- Allow you to Swipe or Tap on the screen.
- Allow you to send BACK button event.
- Allow you to send WakeUp event.
- Allow you to move in List Menu using Swipe gestures or Mouse Wheel Forward or Backward.
- Refresh Screen with Wheel Click(Wheel Down)(To update your device screen on your desktop view).
- Allow you to unlock your Device with Pattern only 3 points pattern for now.

What it does not:
- Complex Unlock Pattern not supported yet I'm working on that.
- Complex Swipe gesture not supported yet, only straight lines.
- PC Keyboard to write SMS or Email not Supported yet, I'm working on that.
- Volume UP or Down not supported yet I will add it soon.
- No configuration interface or file yet, keys and actions are pre set on script code. I will change that.

What it Requires:
- In order to Compile the Script you need 3 files:
adb.exe, AdbWinApi.dll, AdbWinUsbApi.dll
- In order to use the script without these files or without compile it you need to copy that files to user Temp Dir, or windows Temp dir I mean @TempDir.
 

Keys Used and Notes:
- Left Click will tap on the screen.
- Keep Left Click down, move the mouse and release it with perform a Swipe operation.(Mouse Drag = Swipe).
- Right Click will go back.
- Enter on the keyboard will send WakeUp event(Power Button)
- Wheel Down will Refresh the screen on your desktop, also F5.(Take last pictures of your android device screen)
 

IMPORTANT: There is an $scale variable on the script that scale down your Phone screenshot image to a good resolution to show on your Desktop, I use 1.5 of scale factor because my phone resolution is 480x800 and my Desktop Screen is 1366x768 and that scale factor is good enough for me.(I will calculate that in the future to make it automatic).

IMPORTANT 2: If you feel is very slow feel free to decrease Sleep() values from 400 to 200, that will be faster but you will need to Refresh with Wheel Click after you perform certain actions.

Thanks and Credits:
UDFs\MouseControl.au3 Base on Low Level Mouse Hook by @_Kurt all credits to him.
UDFs\GUICtrlPicPNG.au3 Based on PNG work around by @UEZ from the help file, All credits to him.

Kind Regards
Alien.
 

AndroidControl.zip

Edited by alien4u
Adding Swipe Operation Instruction.
1 person likes this

Share this post


Link to post
Share on other sites



cool , how can i integrat it inside an application am developing to help ime in my word?

Share this post


Link to post
Share on other sites

#4 ·  Posted

On 10/10/2016 at 5:22 AM, WormIT said:

Using screen capture to show Android's screens and control will make your Android system lag. :D

One of the purposes of this code is to learn and also access to an Android Device with a broken screen, is not really about efficiency.

Share this post


Link to post
Share on other sites

#5 ·  Posted

Use the TeamViewer UDF instead of screen cap?


2015 - Still no flying cars, instead blankets with sleeves.

Share this post


Link to post
Share on other sites

#6 ·  Posted

Bad idea to use Team Viewer as an intermediator, not only does it not work while the phone is not connected to the internet, you have to totally trust them with your device. I think there are better ways to do this... something like a VNC Server, I only know droidVNC but it does not work on my phone... But screen cap does work for now :)


AutoIt.4.Life Clubrooms - Life is like a Donut (secret key)

Spoiler

My contributions to the AutoIt Community

Some messages & Apologizes:

If I hurt you, Please accept my apologies, I never (regardless of the situation) mean to hurt anybody!!!

Also, I am very busy with my project so I will appear in the last row of the online list, if you want to contact me: Email@TheDcoder.xyz

Or you can have a nice chat with me in freenode, I use the same nick on freenode too!

3fHNZJ.gif

PLEASE JOIN ##AutoIt AND HELP THE IRC AUTOIT COMMUNITY!

Share this post


Link to post
Share on other sites

#7 ·  Posted

8 hours ago, Lakes said:

Use the TeamViewer UDF instead of screen cap?

@Lakes
The idea here is you will not need any software installed on your Android Device with the broken screen.
You only need USB Debugging Enable that all.

As an example, a family:
Man, Woman, Kid they all enable USB debugging and connect at least once to this computer or any computer they can run the Script.
6 months or a year from that moment, if the computer operating system was not re-installed or the phone system, if they get a not working touch-screen or display they will be able to connect to their phones and save data or do some things... regardless what they have installed on their phones.

Kind Regards
Alien. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Similar Content

    • robcull
      By robcull
      Hello all! I have had some issues reading text from different types of windows, occasionally, specifically with controlgettext. 
      **Before I begin, I know there are better ways to do what I attempt in the example below. That's not the point of this post. The point is my issues with controlgettext. 
      I am about to cite an example with an application you may be familiar with called SpeedFan (v4.52). My problem is not specific to speedfan, it is simply the most recent and easily reproducible example I can think of. 
      So, the goal of the script below is to get a string of text containing the current fan RPMs from the highlighted control in the screenshot below (see "speedfan_control_details.png").

      Now, here's a simple script for grabbing the window handle and reading the text from that control: 
      $wintitle = "SpeedFan 4.52" $controlID = "197934" ;will be reformatted as "[ID:######]" $hwnd = wingethandle($wintitle) if @error<>0 then msgbox(0, "WinGetHandle", "FAILURE. @error="&@error) Exit EndIf $text = ControlGetText($hwnd, "", "[ID:"&$controlID&"]") if @error=1 then msgbox(0, "ControlGetText", "FAILURE. @error="&@error) ;failure returns "" and @error=1 Exit EndIf msgbox (0, "ControlGetText", "SUCCESS. @error="&@error &@CRLF& "$text="&$text) ;success returns string and @error=0 You'll see that the ControlGetText operation runs without error, however it does not capture any text from the control. If you explore the other controls in this one window, you'll find mixed results across the board. Neither the temps nor voltages can be read, while the log field and some other elements can be read. Even when you read the text from the whole window, those elements are not included in the visible nor hidden texts. 
       
      I have run into this issue many times in the past- inconsistencies in the ability of autoit to interact with certain controls. What is it which makes this text different than any other readable texts? Is there an alternate method of reading the text in the window/control which could work? Any and all info to help me solve this mystery and satisfy my curiosity would be greatly appreciated. 
      Thanks  -Rob C
      PS: Running Autoit v3.3.14.2 on Win7 Ultimate x64
    • Skeletor
      By Skeletor
      Hi Guys, 
      So with me working in my Virtual Machines in full screen, I often have my Host OS playing music. Now the issue I had was trying to control the Host OS without having to minimise the Guest OS.
      So I thought of this small program. I call it the Volume Control. It resides in the bottom right hand corner of your screen just above the system clock. 
      I'm still working on getting the volume buttons to increase/decrease when you hold down the button.
      Currently, you will need to tap on the volume buttons numerous times to decrease/increase the volume. 
      Any other improvements let me know. 
      The source code, icon and compiled program are already in the zip. 
       

       
      Volume Controlv1.zip
      Screenshot.bmp
    • SAM1386
      By SAM1386
      Hello,
      I can easily find my modem comport number using the code below:-
       
      #include <FileConstants.au3> #include <MsgBoxConstants.au3> #include <WinAPIFiles.au3>    Global $key = RegRead("HKLM\HARDWARE\DEVICEMAP\SERIALCOMM","\Device\ssudmdm0000")     Local $iError = 0     If @error Then        ; The server is probably offline/port is not opened on the server.        $iError = @error         MsgBox(BitOR($MB_SYSTEMMODAL, $MB_ICONHAND), "", "Client:" & @CRLF & "Could not connect, Error code: " & $iError)         Else         MsgBox($MB_ SYSTEMMODAL, "MODEM:", $key)         EndIf  
      but the thing is I am planning to design a small tool by which it can read the modem COM port and set the COM number and send AT command to modem port using UDF CommMG.au3 created by Martin. I added some below code to martins file :-
      ;====================================================================================================================== ;CommSendStandard,a new function for sending information to a serial port. ;====================================================================================================================== Func _CommSendStandard($port,$error,$command)    _CommSetPort($port,$error,9600,8,0,1,2)  if $error <> '' Then      MsgBox(262144,'Port Error = ',$error)  EndIf  _CommSendString($command & @CR) EndFunc   ;==>_CommGetLineStates ;=============================================================================================== After adding I wrote the below code for sending AT command to modem port but its not working.
      Please check the below code and let me know for the issue.
      ;===================================================================================== #include "C:\Program Files (x86)\AutoIt3\SciTE\CommMG.au3" #include <FileConstants.au3> #include <MsgBoxConstants.au3> #include <WinAPIFiles.au3>    Global $key = RegRead("HKLM\HARDWARE\DEVICEMAP\SERIALCOMM","\Device\ssudmdm0000")     Local $iError = 0     If @error Then        ; The server is probably offline/port is not opened on the server.        $iError = @error         MsgBox(BitOR($MB_SYSTEMMODAL, $MB_ICONHAND), "", "Client:" & @CRLF & "Could not connect, Error code: " & $iError)         Else         MsgBox($MB_SYSTEMMODAL, "MODEM:", $key)         EndIf Local $port=$key Local $porterror _CommSendStandard($port,$portError,"AT+CFUN=1,1") ;================================================================================================ but If I set the port number by my own in the code (for eg :- port number 8 or any value  instead of $key) it works fine and the modem gets reset.I want to know if there is any way to get the port number to a variable $port and discarding so that my above code works fine...In that case user donot have to take care of entering the port manually everytime and the script will automatically read the port number and set itself.
       
      Please help
       
      Regards,
      SAM
       
       
    • CADMonkey
      By CADMonkey
      Hi,
      I have created a bit of code to enable a script to find named controls on a userform, even though the MS CLASS description changes between machines.
      e.g. The CLASS "WindowsForms10.EDIT.app.0.24b689f_r14_ad1" for a text box on one PC is different on another.
      The below code, (a cut-down version of the code from jdelaney original post)  loops through all the controls and finds the first one with the desired string in the control class name - in this case "EDIT".  This name is then usable for setting the text using ControlSetText.
       
      #include <array.au3> WinActivate("Edit Part Rule") $TheClassName = GetAllWindowsControls(WinGetHandle("Edit Part Rule"), "EDIT") ;Sleep(500) ControlSetText("Edit Part Rule", "", "[CLASS:" & $TheClassName & "; INSTANCE:3]", $CmdLine[1]) ControlSetText("Edit Part Rule", "", "[CLASS:" & $TheClassName & "; INSTANCE:4]", $CmdLine[2]) Func GetAllWindowsControls($hCallersWindow, $sStringIncludes) ; Get all list of controls $sClassList = WinGetClassList($hCallersWindow) ; Create array $aClassList = StringSplit($sClassList, @CRLF, 2) ; Sort array _ArraySort($aClassList) _ArrayDelete($aClassList, 0) ; Loop For $i = 0 To UBound($aClassList) - 1 If StringInStr($aClassList[$i], $sStringIncludes) Then Return $aClassList[$i] EndIf Next EndFunc ;==>GetAllWindowsControls If anyone has any suggestions to improve it, or a better way to achieve the same thing, please let me know.
       
    • rcmaehl
      By rcmaehl
      A UDF with Extended Functions for Window Management

      DISCLAIMER:
      This UDF is still in the development phase. All code works and should be generally bug free, however function names and parameters are subject to change.
       
      Notes:
      Fixes _WindowGetClassList's barbaric returning of a @LF separated string instead of an array.
       
      Potential Uses:
      Automating applications that change their controls' handles/classes on each launch (e.g. half of Cisco's programs)
       
      Functions:
      _WinGetClassList
      _WinGetClassNNList
      _WindowGetHandleList
      _WindowGetHandleListFromPos
       
      Download: 
      WindowEx.zip  (v0.4)
       
      Changelog:
      10/04/2016 (v0.4): _WinGetClassNNList Fixed : Not Returning an Index when using $2D_ARRAY _WinGetClassNNList Fixed : Not Properly returning $aArray[x][1] on Classes with instances > 9 when using $2D_ARRAY 10/03/2016 (v0.3): _WinGetClassList Added : Exactly the same as WinGetClassList but returns a more civilized Array _WinGetClassNNList Added : Returns Classes and their instances in either a 1D or 2D array depending on Flags _WindowGetHandleList Renamed: _WinGetHandleList SCRIPT BREAKING! _WindowGetHandleListFromPos Renamed: _WinGetHandleListFromPos SCRIPT BREAKING! 10/01/2016 (v0.2): WindowsExConstants.au3 Added : Flags in _WindowGetHandleListFromPos _WindowGetHandleListFromPos Removed: ConsoleWrite left in during debug _WindowGetHandleListFromPos Added : Flag for if part of a Control is at $X, $Y return it as well. 10/01/2016 (v0.1): _WindowGetHandleList Added : Retrieves the handles of classes from a window. _WindowGetHandleListFromPos Added : Retrieves the handles of classes at a specific position from a window. Known and Reported Bugs:
      None reported To Do:
      To Be Decided. Opinions welcome! Upcoming Changes:
      To Be Decided.