multi threading

Jefrey · August 14, 2016

Hi all!

I've made this small UDF to emulate multithreading for AutoIt3.

As we know, AU3 is singlethread and multithreading is only possible through direct ASM. This UDF provides a multithreading emulation through running multiple instances of your script. It also provides an easy way for sending and receiving messages between threads (temporary files are used).

Example:

Reveal hidden contents

Start by including the lib onto your script.

#include 'authread.au3'

Now call _AuThread_Startup(). This function is responsible for calling the callback functions if the currently running process is a "thread", or saving important data to temporary files if it's the main thread.

_AuThread_Startup()

Now create your thread function. You can choose any name for it or even create multiple functions (for multiple threads). Note that if you want your thread to last an undetermined time, you must include an infinite loop inside the function.

Func myThreadFunction()
    While True
        ; In this example, we will get a message sent by the main thread (if any)
        $sMsg = _AuThread_GetMessage()
        If $sMsg Then
            ; Message was sent, let's display it
            MsgBox(0, "Message from the main thread to the sub thread", $sMsg)
        EndIf

        MsgBox(0, "Thread example", "Hey, I am a thread!")
        Sleep(1000)
    WEnd
EndFunc

Now start the thread by calling:

$hThread = _AuThread_StartThread("myThreadFunction")

This function returns the PID of the process-thread. You'll use it for sending messages to threads.

You can continue writing your script on the same file:

While True
    MsgBox(0, "", "Hey, I am the main thread!")
    _AuThread_SendMessage($hThread, "Hello, sub thread! How are you?")
    Sleep(1000)
WEnd

By default, Windows API (and therefore AutoIt) allows only 1 MessageBox per thread. However, by running the script we've just written, you'll see that we're capable of running two message boxes at once on the same script.

Also, when you close the main script, all the threads are also closed.

You cannot start a thread directly without opening the main thread.

Final code:

#include 'authread.au3'

_AuThread_Startup()

Func myThreadFunction()
    While True
        ; In this example, we will get a message sent by the main thread (if any)
        $sMsg = _AuThread_GetMessage()
        If $sMsg Then
            ; Message was sent, let's display it
            MsgBox(0, "Message from the main thread to the sub thread", $sMsg)
        EndIf

        MsgBox(0, "Thread example", "Hey, I am a thread!")
        Sleep(1000)
    WEnd
EndFunc

$hThread = _AuThread_StartThread("myThreadFunction")

While True
    MsgBox(0, "", "Hey, I am the main thread!")
    _AuThread_SendMessage($hThread, "Hello, sub thread! How are you?")
    Sleep(1000)
WEnd

Example 2:

Reveal hidden contents

#include 'authread.au3'

_AuThread_Startup()

$hThread = _AuThread_StartThread("sendalert")

Func sendalert()
    While 1
        $msg = _AuThread_GetMessage()
        If $msg Then
            MsgBox(0, "Alert from thread", $msg)
        EndIf

        ; randomly sends (or not) messages to the main thread
        $rand = Random(1, 10, 1)
        If $rand = 2 Then
            _AuThread_SendMessage(_AuThread_MainThread(), "Hey!")
        EndIf
    WEnd
EndFunc

; main thread
While True
    ; randomly sends (or not) messages to the main thread
    $rand = Random(1, 3, 1)
    TrayTip("Main thread window", "Sorted number: " & $rand, 1)
    If $rand = 2 Then
        _AuThread_SendMessage($hThread, $rand & " was sorted")
    EndIf

    ; get some message sent to the main PID
    $msg = _AuThread_GetMessage()
    If $msg Then
        MsgBox(0, "Wayback message", "The thread sent: " & $msg)
    EndIf
    Sleep(1000)
WEnd

; The line below is just an example,
; as once the main thread is closed or killed, the UDF will automatically close all the started threads
_AuThread_CloseThread($hThread)

Docs:

Reveal hidden contents

Download:

Github ZIP

Fork me on Github

Edited August 15, 2016 by Jefrey

jaberwacky · August 14, 2016

Does this use multiple cores if one's computer were so equipped?

Andreik · August 14, 2016

It shouldn't be so hard to use _WinAPI_SetProcessAffinityMask().

JiBe · August 15, 2016

Hi,.

But, use of 'AdlibRegister' is not the true multi threading.

everything from useful.

Good Week!

TheDcoder · August 15, 2016

On 8/14/2016 at 5:10 PM, jaberwacky said:

Does this use multiple cores if one's computer were so equipped?

Expand

No, this UDF launches the same executable multiple times to emulate multi-threading, As AutoIt is not able to use Multiple Cores, this UDF does not use them too!

On 8/15/2016 at 7:53 AM, JiBe said:

But, use of 'AdlibRegister' is not the true multi threading.

Expand

Huh? , Where did the UDF use Adlib functions?

JLogan3o13 · August 15, 2016

@JiBe just because you disagree with what TheDcoder states, is not a reason to report the thread. Think it through next time...

TheDcoder · August 15, 2016

On 8/15/2016 at 1:54 PM, JLogan3o13 said:

just because you disagree with what TheDcoder states, is not a reason to report the thread.

Expand

He reported me? :blink: ... First, sorry if I have said anything wrong @JiBe :sweating: , it is not my intention to hurt anyone! (The same is mentioned in my signature)

I would welcome any counter-argument , but please do it here, not in the report box! .

JiBe · August 15, 2016

Sorry, just a mistake of using the forum.

I wanted to just posted this:

excuse my English, I'm French. but the Autoit Code used in the UDF is very clear. and my interpretation of this code is the use of the Adlibregister. should you detail your assertion.

argumentum · August 15, 2016

Why not just call it a method of forking ?

http://stackoverflow.com/questions/2483041/what-is-the-difference-between-fork-and-thread

Jefrey · August 15, 2016

On 8/14/2016 at 5:10 PM, jaberwacky said:

Does this use multiple cores if one's computer were so equipped?

Expand

Hi, @jaberwacky! It looks like Andreik and TheDcoder have answered to your question. I've done no tests so far with _WinAPI_SetProcessAffinityMask, tho.

On 8/14/2016 at 8:39 PM, Andreik said:

It shouldn't be so hard to use _WinAPI_SetProcessAffinityMask().

Expand

On 8/15/2016 at 8:45 AM, TheDcoder said:

No, this UDF launches the same executable multiple times to emulate multi-threading, As AutoIt is not able to use Multiple Cores, this UDF does not use them too!

Expand

----------------------

On 8/15/2016 at 7:53 AM, JiBe said:

Hi,.

But, use of 'AdlibRegister' is not the true multi threading.

everything from useful.

Good Week!

Expand

Hi, JiBe!

Actually, the AdlibRegister is not the true way it runs the "threads". Truly it really creates new processes and uses Adlib just to check for new messages (from the UDF _AuThread_SendMessage()) or closing signal.

Also, it's really not the true multi threading. It's just an emulation. It's "forking"

----------------------

On 8/15/2016 at 3:41 PM, argumentum said:

Why not just call it a method of forking ?

http://stackoverflow.com/questions/2483041/what-is-the-difference-between-fork-and-thread

Expand

Hi, argumentum!

Yes, this is the right name hehe. It came out of my mind at the time of the creation of this topic. I've added it to the topic title (I chose to keep "multi thread emulation" for helping users who are using the search engine, as some might not know what's forking). Thanks for poiting out

---------------

Guys, I've started another project that, according to what my latest tests are poiting, might provide some kind of true multithreading for AutoIt without external processes. It's not fully working yet (so far it works just with basic tasks). This is an example:

#include 'thread.au3'

CreateThread("helloworld")

Func helloworld($h)
    MsgBox(0, "", "Hello from the sub thread")
EndFunc

MsgBox(0, "", "Hello from the main thread")

Exit

If you want to help, here's the repo: https://github.com/jesobreira/thread.au3

argumentum · August 16, 2016

On 8/15/2016 at 6:42 PM, Jefrey said:

I've done no tests so far with _WinAPI_SetProcessAffinityMask, tho.

Expand

I did

If Not StringInStr($CmdLineRaw, "/ErrorStdOut") And Not @Compiled Then Exit MsgBox(262144, @ScriptName, "...please run from editor.", 5)

#include <Array.au3>
#include <WinAPI.au3>
#include <WinAPISys.au3>
#include <Process.au3>
#include <WinAPIShPath.au3>

SetProcessAffinityMaskForCurrentProcess(0x05)
; For example, if you pass a mask of 0x05, processors 0 and 2 ( 1st and 3th ) are allowed to run.

Global $aProcessAffinityMask = GetProcessAffinityMask(_WinAPI_PathStripPath(@AutoItExe))
_ArrayDisplay($aProcessAffinityMask, "$aProcessAffinityMask") ; show the affinity masks

Func SetProcessAffinityMaskForCurrentProcess($i = 0)
    Local $hCurrentProcess = _WinAPI_GetCurrentProcess() ; process handle of currrent process
    Local $n = _WinAPI_SetProcessAffinityMask($hCurrentProcess, $i) ; Success: True, Failure: False
    Local $iErr = _WinAPI_GetLastError() ; call _WinAPI_GetLastError() to get extended error information
    Local $affiity = _WinAPI_GetProcessAffinityMask($hCurrentProcess)
    ConsoleWrite("--- > SetProcessAffinityMaskForCurrentProcess( 0x" & Hex($i, 2) & " ) = Success : " & _
            $affiity[0] & @TAB & "Proc Mask : " & $affiity[1] & @TAB & "Sys Mask : " & $affiity[2] & @CRLF)
    Return SetError($iErr, 0, $n)
EndFunc   ;==>SetProcessAffinityMaskForCurrentProcess

Func GetProcessAffinityMask($sProgram = Default)
    Local $a, $n, $m, $hProcess
    If $sProgram Then
        $a = ProcessList($sProgram)
    Else
        $a = ProcessList()
    EndIf
    ReDim $a[UBound($a)][4]
    $a[0][1] = "PID"
    $a[0][2] = "Process affinity cores ( mask )"
    $a[0][3] = "System affinity cores ( mask )"
    For $n = 1 To $a[0][0]
        $a[$n][2] = "failed"
        $a[$n][3] = "_WinAPI_OpenProcess"
        If _WinAPI_GetVersion() >= 6.0 Then
            $hProcess = _WinAPI_OpenProcess($PROCESS_QUERY_LIMITED_INFORMATION, 0, $a[$n][1]) ; process handle of external process
        Else
            $hProcess = _WinAPI_OpenProcess($PROCESS_QUERY_INFORMATION, 0, $a[$n][1]) ; process handle of external process
        EndIf
        If @error Then ContinueLoop
        $a[$n][3] = "_WinAPI_GetProcessAffinityMask"
        $b = _WinAPI_GetProcessAffinityMask($hProcess)
        If @error Then ContinueLoop
        For $m = 1 To 2 ; I only have 8 cores, adjust to your needs. You may need hex 8, this is just a demo.
            $a[$n][1 + $m] = AffinityMaskToCores($b[$m]) & " ( 0x" & Hex($b[$m], 2) & " )"
        Next
    Next
    Return $a
EndFunc   ;==>GetProcessAffinityMask

Func AffinityMaskToCores($i)
    Local $s = "", $n = 1, $c = 0 ; processors are "hardware", therefore start at 0 ( zero )
    Do
        If BitAND($i, $n) Then $s &= "," & $c
        $n += $n
        $c += 1
    Until $n > $i
    Return StringTrimLeft($s, 1)
EndFunc   ;==>AffinityMaskToCores

So here is a nice help to show how to set the process affinity and how to get it.

... on the other hand,

#NoTrayIcon

Global $g_UniqueInAWay = "Wasted away again in Margaritaville"
If $CmdLine[0] Then
    f_WasteAway()
    Exit
EndIf
Func exitInAMinuteJustInCase()
    Exit
EndFunc   ;==>exitInAMinuteJustInCase
Func f_WasteAway()
    AutoItWinSetTitle($g_UniqueInAWay)
    AdlibRegister("exitInAMinuteJustInCase", 60000)
    Local $watever
    Do
        $watever = 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123
    Until 1 = 2
EndFunc   ;==>f_WasteAway

main()
Func main()
    Local $nop = Int(EnvGet("NUMBER_OF_PROCESSORS"))
    For $n = 1 To $nop ; or more, or less, windows will handle it.
        If @Compiled Then
            Run('"' & @ScriptFullPath & '" /wasteAway')
        Else
            Run('"' & @AutoItExe & '" "' & @ScriptFullPath & '" /wasteAway')
        EndIf
    Next
    MsgBox(262144, @ScriptName, "you've got " & $nop & " running processes" & @CR & _
            "as you can see in task manager > performance" & @CR & _
            "windows handle the cpu core assignation quite well" & @CR & @CR & "press ok to stop them.", 60)
    While WinExists($g_UniqueInAWay)
        WinClose($g_UniqueInAWay)
    WEnd
EndFunc   ;==>main

that is not needed.

Edited August 16, 2016 by argumentum
changed the example a bit

TheDcoder · August 16, 2016

On 8/15/2016 at 6:42 PM, Jefrey said:

Guys, I've started another project that, according to what my latest tests are poiting, might provide some kind of true multithreading for AutoIt without external processes.

Expand

Ooohhhh , I will be following the changes on GitHub .

argumentum · August 16, 2016

hmmm, to fork or to thread, this is a good read even if meant for linux.
In AutoIt3 forking is stable, threading is not.
Working with threads is not a holy grail, loading a script ( compiled or not ) in my PC takes less than 10 mSec., memory nowadays is abundant, so forking is not a burden. Ingenuity is the key here.
Using MailSlots for IPC, is sequential, the message can be of unlimited* size and will not be overwritten, no great semaphore brainstorming needed. Forks can be pre-loaded and put to work when needed, and would even allow for distributed processing ( meaning that with this technique, you are not even limited to a single PC, forking is distributed processing ).
ok, café cubano time.

PS: I modified an example from another thread to kind of showcase what I mean.

Edited August 16, 2016 by argumentum
expand on my point

jchd · August 16, 2016

Exactly. And never forget that threads are evil. I mean here that multithreading "Hello World!" is one thing, but multithreading a non-trivial task is completely another one, especially with a high-level language which is explicitely not designed to cope with any drop of multithreading.

Unless you gamble blindly about what AutoIt is doing under your feet when you invoke complex or high-level functions deeply interacting with the OS, you'd better stay away from MT in serious code, else you unduly multiply the risks of potential disaster.

ultimateheadcenter · August 17, 2016

Hi (sorry for english errors i'm french)

Is it not better to use mailslot or other communication udf ? Imagine i got a

100mb memory use script ( the main script)

and one

1mb memory use (update or other function script)

if i use your udf my script will use 200 mb memory

if i use a communication udf it will use 101 mb

why is it better to use forking ? in wich situations ?

(in a hard-protected computer it can be cool no ? you just need to bypass one app in the antivirus / firewall and all the script will work)

i'm really interessed by your udf for some projects i'm already testing it.

Good bye and thanks for this useful udf

RTFC · August 18, 2016

@ultimateheadcenter: MailSlot payload on localhost is restricted to slightly less than 64KB (I'm using 65,507 as upper bound in my Pool UDF); across a LAN, the maximum MailSlot size is a datagram (424 bytes; same as for UDP); for TCP it's 1,472 bytes per packet. So sending a 100MB MailSlot msg is out of the question.

Also, if you're going to be using MailSlots, please be aware that some antivirus software (e.g., Sophos) can block MailSlot messages.

Edited August 18, 2016 by RTFC
AV caveat

Biatu · August 23, 2016

lol, for the past week i have been working on a similar project, using STDIO. my goal is to create a modular include system.
Im working on tailoring it like the apt-get system in linux where dependencies and versioning is tracked.
So far i have a working demo, and it work as follows:
ur code starts up with my script as an include, on start modules are detected, then called for identity, then once you load the module, it is initialized and I have an array if info for each module, along with nested arrays for each STD[IN|ERR|OUT] everything is queued in an adlib timer

rcmaehl · August 23, 2016

Hey @Jefrey,

I believe you'd see significant performance improvement if you spawned the multiple instances using the Run function with STDIO enabled and passing commands through that method. I have an example around my computer somewhere with a sample AV program that uses this method to separate the scanning process from it to allow interacting with the GUI while scanning.

WormIT · October 10, 2016

Thanks for sharing but i think call this "multi process" is clearly than "multi thread emulation"

sr my bad english.

TheDcoder · October 10, 2016

@WormIT This one is different, it does not use multiple processes! Genuine threads are created by calling the Windows API, have a look at the code: https://github.com/jesobreira/authread/blob/master/authread.au3

Oops , I just realized that this is another UDF, The UDF which I was talking about is this:
https://github.com/jesobreira/thread.au3

Edited October 10, 2016 by TheDcoder
Oops

Sign In

multi threading

Recommended Posts

Jefrey

void _AuThread_Startup()

int _AuThread_MainThread()

int _AuThread_StartThread(string $sCallback [, string $sMsg ] )

string _AuThread_GetMessage( [ $iPID = @AutoItPID ] )

void _AuThread_SendMessage($iPid, $sMessage)

void _AuThread_CloseThread( [ $iPID = @AutoItPID ] )

Top Posters In This Topic

Top Posters In This Topic

jaberwacky

Andreik

JiBe

TheDcoder

JLogan3o13

TheDcoder

JiBe

argumentum

Jefrey

argumentum

TheDcoder

argumentum

jchd

ultimateheadcenter

RTFC

Biatu

rcmaehl

WormIT

TheDcoder

Create an account or sign in to comment

Create an account

Sign in

Recently Browsing 0 members

Browse

AutoIt Resources

Release

Beta