AutoIt plugin architecture

[jvanegmond]

After several months pondering how to make the best possible plugin architecture, I have come up with an answer. The code is still in an infancy stage and will require work before it can be applied in "real" projects (PM me if you do want to use this and I'll tag along).

What I've made allows you to load an Au3 script file at runtime and call methods in it. This script doesn't have to implement or change anything. Combine this with other functionality such as what Eval offers and (shameless self promotion ftw) and this has many different possibilities.

There often seems to be some differences in what people expect a plugin architecture should support, but here is what I support:

Sometimes the services interface is not included because it's not needed (say other people). That is what I call a one-way plugin architecture, only the main script can call plugin functions. The architecture in the picture is what I call a two-way plugin architecture and that's what I will be demonstrating in this thread.

Firstly, I have created a one way plugin architecture. With the provided libraries, one can do this:

plugins\hello world.au3:

Func HelloWorld($str)
    MsgBox(0x40, @ScriptName, $str)
EndFunc

plugin loader.au3:

#include "lib\reflection.au3"

; Demonstrating simple plugin loading, one-way

; This can be used when the main script knows which functions are in the plugin and how to use them
; and the plugin does not need to use any functions in the main script (more rare than you think)

$script = _RDoFile("plugins\hello world.au3")
$script.HelloWorld("Something different this time")

$script.Quit()

The _R prefix stands for "_Reflection", because that's what I hope to do eventually fully support with this library. What happens in this example is that the reflection library: Reads the script from the .au3 file, modifies the script so that it can export its function, stores the au3 file as temp.au3, runs it.

For curious minds, the temp.au3 file that is generated looks as following. This is a technique that I call "COM server injection". (And here you can see why I said the project is in its infancy)

#NoTrayIcon ;tehee
#include "lib\AutoItObject.au3"
Global $oError = ObjEvent("AutoIt.Error", "_ErrFunc")
Func _ErrFunc()
    Return
EndFunc
_AutoItObject_StartUp()
Global $oObject = _SomeObject()
Global $hObj = _AutoItObject_RegisterObject($oObject, "Manadar.AutoIt.Plugin")
While 1
    Sleep(100)
WEnd
_QuitObject(0)

Func _SomeObject()
    Local $oClassObject = _AutoItObject_Class()
    $oClassObject.AddMethod("HelloWorld", "HelloWorld")

    $oClassObject.AddMethod("Quit", "_QuitObject")
    Return $oClassObject.Object
EndFunc
Func _QuitObject($oSelf)
    _AutoItObject_UnregisterObject($hObj)
    $oObject = 0
    Exit
EndFunc


; ============= ACTUAL SCRIPT ============= 

Func HelloWorld($o___________________o, $str)
    MsgBox(0x40, @ScriptName, $str)
EndFunc

The reflection library _RDoFile function uses a method to extract all the functions from the original script, and generates the correct $oClassObject.AddMethod calls when generating the COM object.

The while-loop that is added before the script is optional. It can be disabled with another parameter in _RDoFile and eventually there will be various ways to initialize scripts this way, either partly or completely. This parameter is necessary for two-way plugins, but we'll see that later on.

This shows how simply it is to load an Au3 file and do a bit of "reflection" magic on the code. It greatly increases the versatility and capabilities of AutoIt scripts. The function in its current state can not load Au3 files when the plugin loader script is compiled but that is almost trivial to add.

Now we take a look at two-way plugins, here is the plugin loader script:

; Demonstrating plugin loading, two-way
; Both the plugin and the main script provide an interface
; This is the most common way to deal with plugins

; You simply load the plugins and the plugin defines the additional behavior,
; by adding items to your menus and handling the functionality for them etc.

$o = _RImplementInterface("pluginterface.au3", "Manadar.AutoIt.Main")

$called = False
$script2 = _RDoFile("plugins\hello forums.au3", True)

While Not $called
    Sleep(100)
WEnd

Func Print($obj, $s)
    $called = True
    MsgBox(0x30, "", $s)
EndFunc

We see in this example that the plugin loader script wants to expose a function called 'Print', but this could be anything and any number of functions. The plugin is written as follows (most ideal case, and actually works):

plugins\hello forums.au3

#include "pluginterface.au3"

Print("Hello AutoIt forums!")

Via the one-way plugin architecture method we can load the plugin, we read the contents, apply COM injection magic, run the script, done. But the same method does not work the other way around because our main script is already running (and COM injection requires restart). What we can do, is let the main script create a COM server which does not require a restart. We do this via the call to _RImplementInterface, this function gets the functions from the pluginterface.au3 file and creates a COM server which exposes these methods to the outside. The pluginterface.au3 file functions connect to the plugin loader script and forward any calls from the plugin there.

The pluginterface.au3 file looks like this and it is possible to entirely automatically generate this file (just when you were thinking: I need to write dumb code for this to work? Lame.):

#include "lib\reflection.au3"

$main = _RAttach("Manadar.AutoIt.Main") ; just a call to _AutoItObject_ObjCreate() with cbi:

Func Print($s)
    $main.Print($s)
EndFunc

With this approach, we have built a fully two-way interface. The code "needs work" and I put this in various comments through the attached files, but the needs work comments are not complete.

Some ideas for the future:

- COM server injection also exposes global variables like: $script.g_sVar

- Automatic generation of the services interface

- Very complete reflection library, with the library itself also object oriented ($script.GetMethods()[0].Call() capability)

- Support of these features (or similar) in native AutoIt: #1931

Attached I have the examples used in this thread, the reflection library and a version of AutoItObject that supports this (v1.2.4.0 I believe). The examples should work "out of the box" with AutoIt v3.3.6.1 (latest stable atm).

Edit: Forgot to attach file. (Really, Manadar, Really?)

Reflection Au3.zip

Edited June 8, 2011 by Manadar

[MrCreatoR]

You don't want to compile this (main.au3) and run it

As know line sais, “Curiosity killed the cat”, well, i just almost have killed my system

Other than that, looks very good.

Edited June 8, 2011 by MrCreatoR

[jvanegmond]

You don't want to compile this (main.au3) and run it

As know line sais, “Curiosity killed the cat”, well, i just almost have killed my system

Other than that, looks very good.

I mentioned that there is no support at the moment for loading from a compiled file. You could have guessed that the behavior when compiled could lead to unexpected results. In this case, the script restarting itself hundreds of times.

[MrCreatoR]

If it's not supported as compiled, then i really don't see the use for it, or am i wrong?

P.S

Probably you should add some kind of @compiled checking, in case that someone curios as me will try to run it as compiled executable.

[Mat]

Am I right in saying there could be a compiled version, that would mean just doing the server injection before you compile it? You could then comment out the bit that deals with the injection and it should work (if you are lucky)

[jvanegmond]

The plugin architecture works whether you compile or not. The reflection library doesn't, as it dynamically gets function names from scripts. Ideally, someone would never compile plugins (The idea is that they can be easily changed, remember?).

[wraithdu]

This is seriously cool The plugins should work if compiled to a3x though shouldn't they? Or is the library directly parsing the script text?

Edited June 8, 2011 by wraithdu

[jvanegmond]

This is seriously cool The plugins should work if compiled to a3x though shouldn't they? Or is the library directly parsing the script text?

Yes, I'm parsing the script text but it should be possible to write the plugins in a certain way so that they can be loaded as plugins. I think running the scripts uncompiled, though, has more merit as a useful application.

@Everyone: Might I ask, what is the obsession with compilation here?

Edit: I don't think compiling more than one script to include the interpreter makes sense, hence uncompiled plugins. But tbh, I have no clue what a3x files are and how they work. Maybe I can something with that.

Edited June 9, 2011 by Manadar

[wraithdu]

a3x files are the "compiled" script data that is tagged to the end of the interpreter in a compiled script EXE. It is (I assume) already lexed and tokenized. I figured running the plugin in this form might yield better performance, but it would probably be negligible.

[Mat]

a3x files are the "compiled" script data that is tagged to the end of the interpreter in a compiled script EXE. It is (I assume) already lexed and tokenized. I figured running the plugin in this form might yield better performance, but it would probably be negligible.

Interestingly enough you get a larger file... I would not have expected that at all.

The only good thing IMO is that you can have a build process like normal that does the preprocessing... That way you don't need to worry about includes on the client system (which there won't be). Not being human-readable has some advantages as well.

I'd certainly say a3x is better than exe for a plugin... Whether au3 beats that is another question. I can't see many benefits at the moment for going either way.