_RemoteThreadX - Memory MultiThreading

[Ascend4nt]

..

Edited August 6, 2009 by Ascend4nt

[monoceres]

You obviously spent much time with this. Great job!

However I'm curious if it's any real difference from the assembly stuff me and mostly trancexx has done lately. Things like this:

#include <winapi.au3>


$msgbox = DllCall("Kernel32.dll", "ptr", "GetProcAddress", "ptr", _WinAPI_GetModuleHandle("user32.dll"), "str", "MessageBoxW")
$msgbox = $msgbox[0]

Local $codebuffers[5]

For $i = 0 To UBound($codebuffers)-1

    $codebuffers[$i] = DllStructCreate("byte[1024]")
    DllStructSetData($codebuffers[$i], 1, "0x6800000000680000000068000000006800000000b8" & SwapEndian($msgbox) & "ffd0c3")

    $hThread = DllCall("Kernel32.dll", "ptr", "CreateThread", "ptr", 0, "int", 0, "ptr", DllStructGetPtr($codebuffers[$i]), "ptr", 0, "dword", 0, "ptr", 0)
    $hThread = $hThread[0]
Next

While True
    Sleep(100)
WEnd


Func SwapEndian($iValue)
    Return Hex(BinaryMid($iValue, 1, 4))
EndFunc   ;==>SwapEndian

[Ascend4nt]

Thanks for the compliment.

I haven't seen that code around, I didn't know you and trancexx were working on something like that.

I'd have to say that, in a way its similar - the Assembly code itself is capable (and does) look up the proc addresses for any API function using the same basic method. The only two things the Assembly code needs is:

1. A 'real' DLL handle to Kernel32.dll

2. A pointer to 'GetProcAddress'

From those two things it can load any DLL and grab the procedure address. I'm pretty sure the AutoIT code must do the same thing (it cant store exports for EVERY DLL function ever made)..

However, I've added a send/recv msg area, the ability to pass a user parameter (a pointer to another DLLStruct, or a memory allocated buffer, for example), and the whole thing operates as a program (I even set up the data pointer correctly to access variables within the code).

Plus in the x64 version I had to tackle with the way the new stack setup and alignment works, etc. I also close cleanup the threads properly etc.

But yeah, the code you and trancexx have is definitely a small start in the general direction that I took mine.

I'm thinking of making an 'asynchronous DLL call function for the next expansion - which would take regular DLLCall() strings and convert it to something the Assembly code can take and work with. This would free code from API calls which require you to wait for their return.

Also, another thing I've considered is adding a Callback function, but I'm still not certain how the Asynchronous ability would work with AutoIT code.. I guess I'll have to test it out!

P.S. Where have you and trancexx worked on that project? or is it a behind-the-scenes type of thing that hasnt been posted?

[monoceres]

Surely you have seen it!

trancexx have made tons of amazing work here & here.

And I made a post that got trancexx started here.

The problem I see is that the created threads will not run in the same "namespace" and same condition as the main autoit thread (and simplicity). Thus making it too complicated for the average autoit user (remember the casual autoit programmer usually doesn't even understand pointers).

Don't expect anything good too come out of the callback thing. It will only lead to problems and deadlocks, see more here.

Anyways, it's still great and working directly with machine code is always a thrill

[Ascend4nt]

Surely you have seen it!

Actually, no - I figured out all the Thread\GetProcAddress stuff on my own, but as far as executing code in a buffer - that was definitely something I learned from trancexx and others. I just happened to wake up one morning think 'hmm.. I wonder if I can create Threads in those buffers'

Then, a bit of MSDN searching/reading, a LOT of Assembly coding/recoding (especially in x64, da*#$&nit!!), and a good chunk of time creating a good interrface between AutoIT and the code, and wa-la, Remote.Thread.X

Thank you for the links though, I'll have to take a look at this stuff.

That'd be a shame about the callbacks..

[Guest LulKutwijfKankerop]

do i have to use binary for my functions or what?? this doesn't look useful aside from executing other code.

[Datenshi]

" I just happened to wake up one morning think 'hmm.. I wonder if I can create Threads in those buffers' "

Makes me smile knowing I'm not alone with these urges to create and achieve something

[Ascend4nt]

@Monoceres: update: I took a look at those links - some pretty cool stuff going on! When I've done searches on multithreading before, I always came up with UDF's where people are running a 2nd script or creating new 'Adlib' replacements and pretending its multithreading. If only those posts you linked came up in my searches.

However, I have to say - I do not prefer the opcode-mixed-with-address-insertion method used by trancexx (though it's quite ingenious). I prefer to code in straight Assembly language and pass info in parameters or a Structure with everything I need in it. Or, if possible, do the extra work inside the Assembly source myself (like for mine - loading/calling DLL's that are needed etc). Plus, the x64 issue will result in the code crashing, which would mean you'd have to recode it in x64 Assembly, find all the opcodes and offsets again and redo the function - much more difficult that using Assembly code with parameter(s).

By the way, if you haven't seen it, look up Flat Assembler - its perfect for generating binary code. You can disassemble the binary output if you like and insert addresses and whatnot in the right places (there's a few Disassemblers I've come across that let you see hex opcodes and offsets relatively easily), and the official Intel Manuals help with opcodes.

[and you're right - coding in machine code is fun as heck.]

@LulKutwijfKankerop: Thanks for creating account to tell me my code is useless. haha. You do have to create binary output - but you can do it using either Assembly language or something which allows you to generate Assembly language (certain C/C++ compilers can do this, such as Sphinx C--) - which can then be compiled to binary code.

@Datenshi - I'm having too many of those moments which causes projects I'm in the middle of to get put on hold! hehe

[FireFox]

@Ascend4nt

It's a good start, I have seen some other examples wich only work for simples things

I want to know if it will be possible to create a "part of script" in thread mode for example a "While" wich you will be able to interact with (for example you change a variable and the loop will stop for a moment)

Edit1 : I applaud you in the way I understand anything in DllStructs

Cheers, FireFox.

Edited July 25, 2009 by FireFox

[Ascend4nt]

@FireFox,

A 'while' in Assembly is simple, but it depends on what the 'While' is - what are you checking exactly? And is it being checked by your script or do you envision the 'while' being used in the Assembly code?

Right now, all of my Assembly examples check a 'communication state' item used by the UDF when 'RequestStop' is called - which is a kind of 'While' (sort of like While 'RequestStopFlag<>-1'). You can easily add functionality for the Thread to check on another variable set by your code also, by sending a pointer to a DLLStruct as a parameter to 'Activate'.

Also, if you want the Thread to stop, 'Suspend' is the easiest method; however if you don't want to stop it until certain operation(s) are completed, then flagging it to stop through a DLLStruct is easiest, then checking if it stopped by a 'return' flag before continuing would work.

Thanks about the DLLStruct knowledge compliment - its not too hard to understand if you get your head around the idea its just a bit of memory allocated and accessed through a certain 'type-defined' interface. (Alignment of variables can be confusing sometimes though)

Let me know what you're looking to achieve - I'll see if its easily doable in code. I'm sorta looking for good example usage anyway.

Take it easy - A

[FireFox]

@FireFox,

A 'while' in Assembly is simple, but it depends on what the 'While' is - what are you checking exactly? And is it being checked by your script or do you envision the 'while' being used in the Assembly code?

Right now, all of my Assembly examples check a 'communication state' item used by the UDF when 'RequestStop' is called - which is a kind of 'While' (sort of like While 'RequestStopFlag<>-1'). You can easily add functionality for the Thread to check on another variable set by your code also, by sending a pointer to a DLLStruct as a parameter to 'Activate'.

Also, if you want the Thread to stop, 'Suspend' is the easiest method; however if you don't want to stop it until certain operation(s) are completed, then flagging it to stop through a DLLStruct is easiest, then checking if it stopped by a 'return' flag before continuing would work.

Thanks about the DLLStruct knowledge compliment - its not too hard to understand if you get your head around the idea its just a bit of memory allocated and accessed through a certain 'type-defined' interface. (Alignment of variables can be confusing sometimes though)

Let me know what you're looking to achieve - I'll see if its easily doable in code. I'm sorta looking for good example usage anyway.

Take it easy - A

The "While" is a good example because without multithreading you can't run some whiles in the same time..

If you can put a whole script in another thread and run them in same time it will be good (not that only, onevent functions and other things must work like real multithread)

Cheers, FireFox.

Cheers, FireFox.

[Ascend4nt]

@Firefox:

Running AutoIT script in a separate thread isn't possible, but accessing a lot of the same Windows API calls it uses and communicating back and forth with one running script through structs is.

As monoceres pointed out, calling the script from a thread is a bad idea too, so truly the Thread has to be separate from the AutoIT code.

Perhaps I should have been clear about that from the start... AutoIT script is an interpreted language (interpreter:AutoIT3.exe), my Threads work with pure non-interpreted code - however, they can communicate with your script through DLLStructs or allocated memory..

If you need just script code to run in a multithreaded way, then you'll have to choose one of the created coprocess/multi process or interscript communication UDFs..

Let me know if you have something specific in mind though, there are definite script elements that can be emulated in code - you can PM me if its something you don't want to disclose here though.