Jump to content
Sign in to follow this  
MarkRobbins

Smuggling AutoIt into IE embedded

Recommended Posts

MarkRobbins

In my >previous adventures, I got a bare-bones AutoIt proxy object injected into MSScriptControl.ScriptControl.

The proxy had just enough functionality that a basic form application could be coded from Javascript. That was kind of fun but the ScriptControl is old and iffy, its environment is like using an old version of JScript, sometimes mere comments would trigger non-compilation.

What would be really good is to get a full-fledged proxy injected into IE, so that you had ECMA5-ish language features and a more stable platform. I had no idea how to do this though, so I was shocked when it worked, which is why I call it 'smuggling'.

Once you had a decked out proxy in there, you could code your entire app in Js, and use IE as a GUI (or not). I am most interested in coding in an OOP language, so that is what is driving my attempts.

Long story short, it worked. So now I have an initial framework for doing this, and I put together this stress test of an application and demo.

The clip from the header:

; OVERVIEW
;  Name: Jsaioie -- Javascript, AutoItObject, Internet Explorer
;
;  This is an advanced demo of using an embedded IE control as a GUI.
;  What is noteworthy is that the main application code is in javascript, which makes calls
;  to core AutoIt functions via an imported AutoItObject which has been constructed.
;  This means that virtually all of the application can be coded as JS, which means OOP.
;  JS OOP is really not demonstrated here within this demo however.
;
;  What is demonstrated in the JS code is how JS can be persuaded to behave asynch, and emulate
;  a threaded application.
;
;  When the application starts, it will analyze all the files in the users AutoIt include directory in order
;  to build a series of collapsable divs with information on the constants and functions found.
;  While this process is occuring, the UI is still relatively responsive, and you can watch the nested div
;  items being added, or even filter them while it is loading.
;
;  The application will pick up about 14k identifiers from the include directory, each having its own div
;  and tt tag containing documentation. Some research shows that IE8 has problems when the div count exceeds
;  1000, but I am using IE 10. Your results will depend on your IE version. It is said that IE9+ should have
;  not much problem with 100k divs. The real lag is filtering them, which could be optimized.
;
;  Parsing the files, loading and building html and listeners for 14k items takes 5.5 minutes on my machine.
;  That breaks down to 42 items per second, or 24k bytes per second. Tweaking seems to have no effect. The
;  goal is to keep the thing responsive during load, the html I mean, but the application will hourglass
;  when it encounters the bigger files, probably a IO thing.
;
;  The application consists of the following files:
;   Jsaioie.au3         - Backbone code file
;   Jsaioie.au3.ini     - Config, will be created
;   Jsaioie.au3.js      - Application code
;   Jsaioie.au3.html    - Simple html skeleton
;   Jsaio-inc-AiFn.au3  - Constructs the AutoIt proxy AutoItObject
;   Jsaio-utils.au3     - Some misc utils
;   jquery-1.10.2.js    - For dom manipulation
;   Jsaioie.au3.log.txt - Output log for debugging
;   Jsaio-loaded.js     - A preview of the javascipt to be loaded after preprocessing
;
;  OPERATION
;   1. $AutoItForCom AutoItObject is created and core AutoIt functionality appended to it
;   2. Small amount of application specific functionality is also appended
;   3. Form with Embedded IE constructed
;   4. Html is constructed from assets, preprocessed and injected into IE
;   5. $AutoItForCom is injected into IE (the real magic part)
;   6. Javascript init() function is called which grabs the injected $AutoItForCom and assigns it to window.A var
;   7. Javascript Main() function is called which begins the application
;
;  JAVASCRIPTS USE OF $AutoItForCom
;   1. Getting file list from include directory
;   2. Getting file contents
;   3. Logging
;   4. Running a specified editor to edit file
;   5. Placing text on clipboard
;   6. Reading config ini
;   7. Browse for folder
;   8. Browse for file
;   9. Getting ScriptFullPath
;  10. Checking @Error
;
;  FEATURES
;    1. Dynamic html nested accordian browser
;    2. Filter for file, or for identifier
;    3. Up Down, Page Up Page Down keys while in filter boxes will scroll document
;    4. Clear filter will collapse all
;    5. Click on Icon to copy signatures to clip
;    6. Unfold Items to see documentation if exists
;    7. Click on file link to edit file
;    8. Specify your editor, saved in config
;    9. Specify your include dir, saved in config
;   10. Asynch loading, file icons change color
;
;  NOTES
;   This is not intended to be a finished app, but a stess test and demo. A lot of the JS coding has been
;   unrolled and exposed in a simplistic manner for illustration purposes. Familiarity with Javascript and
;   jQuery are neccessary for understanding how the application works. All the AutoIt code is essentially
;   a framework so that it is possible to code the app in the browser, and have access to autoit.

And a pretty pic:

post-78716-0-18165200-1375948336_thumb.p

And the zip:

Jsaioie.zip

 

Share this post


Link to post
Share on other sites
eimhym

This >old project use ScriptControl, but already has wrapper for AutoIt and JScript to communicate with ease (calling methods, passing variables, arrays etc). The bridge is closure compiled but the unpacked version is included, hopefully readable :)
And FYI Chakra JScript Engine can be used directly without ScriptControl nor IE, see here for hint, use Chakra's GUID instead.

Share this post


Link to post
Share on other sites
MarkRobbins

This >old project use ScriptControl, but already has wrapper for AutoIt and JScript to communicate with ease (calling methods, passing variables, arrays etc). The bridge is closure compiled but the unpacked version is included, hopefully readable :)

And FYI Chakra JScript Engine can be used directly without ScriptControl nor IE, see here for hint, use Chakra's GUID instead.

 

Yes. Your code is what got me started! I even made a helper so that you can set the property of an AutoItObject using a property name string : >Index into an AutoItObject? Back then I was just using the JSOs for Assoc Arrays.

I tried to get AIO proxy into Moz, but it wouldn't accept the same technique.

Is Chakra the same obj as ScriptControl, or a newer version. My coding experience with ScriptControl has been too full of gotchas for me to use it.

Long term, what needs to be done is wrap the imported AutoIt proxy with OO oriented facade, so all the AI UI controls can be dealt with as objects. Then the entire app can be coded in Js. Though moving across com has a slight delay, my testing shows that Js is going to be much faster as an overall language.

Can you give any pointers as to capturing Js errors, and debugging? That part seems really bad, especially when you have a syntax error in js.

Share this post


Link to post
Share on other sites
eimhym

I tried to get AIO proxy into Moz, but it wouldn't accept the same technique.

 

If you refer Mozilla Firefox then we can't use AIO with javascript other than IE.

Is Chakra the same obj as ScriptControl

 

Chakra (used since IE9) is the core JS engine, M$ Active Scripting version of V8. ScriptControl and IE are Active Scripting Hosts, where you can select the engine you like (JS, VB, Python, etc).

But Chakra by default is not visible to ScriptControl, only for IE. By implementing IActiveScriptSite you can instantiate it.

You can make script host wrapper dll for AutoIt, or implement it directly (yes AutoIt CAN implement Interface :) ) I'm onto the later but my responsibilities lately hold me from messing with the idea.

Can you give any pointers as to capturing Js errors, and debugging? That part seems really bad, especially when you have a syntax error in js.

 

Try google for 'javascript stacktrace', to get detailed info about exception/error thrown. As for syntax checking you can first pass the code into JSLint or similar.

Good luck digging :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • FMS
      By FMS
      Hello,
      I try to get all the text from a news site around a subject.
      The first run I get all the text inside a attribute in an array.
      When i try to go back and reload another page it chrashes and think it because "$oIE.GoBack"
      I couldn't find anything in the help/forum around this subject.
      Do I need to reload the $oIE or something afther an X.GoBack?
      The error i got is :
      if $oTag2.GetAttribute("class") == "NewsDetail" Then if $oTag2^ ERROR I'm not shure why I got this error, maybe someone could explain?
      Also I'm open for some pointers in this test script because I'm pretty new in working whit the IE.UDF
      Maybe there is an simpler way to get the same results?

      test script:
      #include <IE.au3> #include <MsgBoxConstants.au3> #include <Array.au3> HotKeySet("{ESC}", "Terminate") Global $oIE = _IECreate("https://www.iex.nl/Zoeken/Nieuws.aspx?q=air%20france") ;get first subject Global $oLink1 = _IEGetObjById($oIE, "ctl00_ctl00_Content_LeftContent_NewsSearch_repNews_ctl00_linkNews") Sleep(500) _IEAction($oLink1, "click") Sleep(500) Global $oTags = _IETagNameGetCollection($oIE, "div") Global $aResults[1] For $oTag In $oTags if $oTag.GetAttribute("class") == "NewsDetail" Then _ArrayAdd($aResults, $oTag.innerTEXT) EndIf Next $aResults[0] = UBound($aResults) - 1 _ArrayDisplay($aResults, "Episodelist") ConsoleWrite($aResults[1] & @CRLF) $oIE.GoBack ;get second subject Global $oLink2 = _IEGetObjById($oIE, "ctl00_ctl00_Content_LeftContent_NewsSearch_repNews_ctl01_linkNews") Sleep(500) _IEAction($oLink2, "click") Sleep(500) Local $oTags2 = _IETagNameGetCollection($oIE, "div") Local $aResults2[1] For $oTag2 In $oTags2 if $oTag2.GetAttribute("class") == "NewsDetail" Then _ArrayAdd($aResults2, $oTag2.innerTEXT) EndIf Next $aResults2[0] = UBound($aResults2) - 1 ConsoleWrite($aResults2[1] & @CRLF) Func Terminate() _IEQuit($oIE) Exit EndFunc ;==>Terminate  
    • SkysLastChance
      By SkysLastChance
      I am having a hard time understanding why this is not working. I was hoping some one could help explain it to me. 
      $tags = $oIE.document.GetElementsByTagName("input") For $tag in $tags $class_value = $tag.GetAttribute("class") If string($class_value) = "fTs-p3298-l0 wplEditControl" Then $target = $tag ExitLoop EndIF Next MsgBox(0,"",$target) If $target = "fTs-p3298-l0 wplEditControl" THEN MsgBox(0,"","itworked") I have tried 
       MsgBox(0,"",$target.Attribute)  MsgBox(0,"",$target.Value)  MsgBox(0,"",$target.InnerText) I would expect to see this in the msgbox
      fTs-p3298-l0 wplEditControl  
    • SkysLastChance
      By SkysLastChance
      I am trying to grab the id "in2xk_26" however it the characters before the underscore always change. (in the name too)
      Is there a way I can find a id or name by the last 3 charcters?
      Using something like "stringright?"
      So I would want to search for just "_26" in this case.
      This is assuming that there are no other _26
       

      #include <Excel.au3> #include <IE.au3> #include <GUIConstantsEx.au3> Global $iMousespeed = 25,$target = "",$TagName = "",$Value = "",$Atrribute = "" $oIE = _IEAttach("MEDITECH") $TagName = "input" $Value = "in2xk_26" $Attribute = "id" $tags = $oIE.document.GetElementsByTagName($TagName) ;TagName ------ MAKE SURE TO NAME THESE For $tag in $tags $class_value = $tag.GetAttribute($Attribute) ;Attribute ------ MAKE SURE TO NAME THESE If string($class_value) = $Value Then ;Value ------ MAKE SURE TO NAME THESE $target = $tag $iScreenX = _IEPropertyGet($target, "screenx") $iScreenY = _IEPropertyGet($target, "screeny") $iWidth = _IEPropertyGet($target, "width") $iHeight = _IEPropertyGet($target, "height") $oMouseCords = MouseMove($iScreenX + $iWidth / 2, $iScreenY + $iHeight / 2, $iMousespeed) MouseClick($MOUSE_CLICK_LEFT) ExitLoop EndIf Next  
       
    • SkysLastChance
      By SkysLastChance
      I was wondering how I read data inside of a iframe. I would like to be able to click something inside a iframe. However, I can't even find the tag. 
      The id and name change all the time so I can't use those. 
      The code highlighted in blue is what I am trying to click. (second picture)
      Here is what I have tried.
      $target = "" $tags = $oIE.document.GetElementsByTagName("div") For $tag in $tags $class_value = $tag.GetAttribute("class") If string($class_value) = "s_92 altstyle s_93 s_94" Then $target = $tag ConsoleWrite("Tag Found " & $target.outerText&@CRLF) ExitLoop EndIf Next  

       
       

    • SkysLastChance
      By SkysLastChance
       
      WinActivate("MEDITECH - Internet Explorer") Sleep (500) $oIE = _IEAttach("MEDITECH") $oDiv1 = _IEGetObjById($oIE, "sysmenu-searchbarbutton") _IEAction($oDiv1, "click") I am just trying to click the little magnifying glass, next to the gear button with no luck. I was hoping someone might have an idea why this is not working?
       

×