Jump to content

multi threading


 Share

Recommended Posts

Hi all!

I've made this small UDF to emulate multithreading for AutoIt3.

As we know, AU3 is singlethread and multithreading is only possible through direct ASM. This UDF provides a multithreading emulation through running multiple instances of your script. It also provides an easy way for sending and receiving messages between threads (temporary files are used).

Example:

Spoiler

 

Start by including the lib onto your script.

#include 'authread.au3'

Now call _AuThread_Startup(). This function is responsible for calling the callback functions if the currently running process is a "thread", or saving important data to temporary files if it's the main thread.

_AuThread_Startup()

Now create your thread function. You can choose any name for it or even create multiple functions (for multiple threads). Note that if you want your thread to last an undetermined time, you must include an infinite loop inside the function.

Func myThreadFunction()
    While True
        ; In this example, we will get a message sent by the main thread (if any)
        $sMsg = _AuThread_GetMessage()
        If $sMsg Then
            ; Message was sent, let's display it
            MsgBox(0, "Message from the main thread to the sub thread", $sMsg)
        EndIf

        MsgBox(0, "Thread example", "Hey, I am a thread!")
        Sleep(1000)
    WEnd
EndFunc

Now start the thread by calling:

$hThread = _AuThread_StartThread("myThreadFunction")

This function returns the PID of the process-thread. You'll use it for sending messages to threads.

You can continue writing your script on the same file:

While True
    MsgBox(0, "", "Hey, I am the main thread!")
    _AuThread_SendMessage($hThread, "Hello, sub thread! How are you?")
    Sleep(1000)
WEnd

By default, Windows API (and therefore AutoIt) allows only 1 MessageBox per thread. However, by running the script we've just written, you'll see that we're capable of running two message boxes at once on the same script.

Also, when you close the main script, all the threads are also closed.

You cannot start a thread directly without opening the main thread.

Final code:

#include 'authread.au3'

_AuThread_Startup()

Func myThreadFunction()
    While True
        ; In this example, we will get a message sent by the main thread (if any)
        $sMsg = _AuThread_GetMessage()
        If $sMsg Then
            ; Message was sent, let's display it
            MsgBox(0, "Message from the main thread to the sub thread", $sMsg)
        EndIf

        MsgBox(0, "Thread example", "Hey, I am a thread!")
        Sleep(1000)
    WEnd
EndFunc

$hThread = _AuThread_StartThread("myThreadFunction")

While True
    MsgBox(0, "", "Hey, I am the main thread!")
    _AuThread_SendMessage($hThread, "Hello, sub thread! How are you?")
    Sleep(1000)
WEnd

 

Example 2:

Spoiler
#include 'authread.au3'

_AuThread_Startup()

$hThread = _AuThread_StartThread("sendalert")

Func sendalert()
    While 1
        $msg = _AuThread_GetMessage()
        If $msg Then
            MsgBox(0, "Alert from thread", $msg)
        EndIf

        ; randomly sends (or not) messages to the main thread
        $rand = Random(1, 10, 1)
        If $rand = 2 Then
            _AuThread_SendMessage(_AuThread_MainThread(), "Hey!")
        EndIf
    WEnd
EndFunc

; main thread
While True
    ; randomly sends (or not) messages to the main thread
    $rand = Random(1, 3, 1)
    TrayTip("Main thread window", "Sorted number: " & $rand, 1)
    If $rand = 2 Then
        _AuThread_SendMessage($hThread, $rand & " was sorted")
    EndIf

    ; get some message sent to the main PID
    $msg = _AuThread_GetMessage()
    If $msg Then
        MsgBox(0, "Wayback message", "The thread sent: " & $msg)
    EndIf
    Sleep(1000)
WEnd

; The line below is just an example,
; as once the main thread is closed or killed, the UDF will automatically close all the started threads
_AuThread_CloseThread($hThread)

 

Docs:

Spoiler

void _AuThread_Startup()

Verifies if the currently running script is a thread being called by a mainthread or if it's the main thread itself. If it's being called by a main thread, it calls the specified callback function. Else, it will just save required data for the UDF.

int _AuThread_MainThread()

This function can be called in both threads and mainthreads. It will always return the PID of the main thread.

int _AuThread_StartThread(string $sCallback [, string $sMsg ] )

Starts a thread and returns the new thread PID. $sCallback must be a callable function name. If $sMsg is specified,_AuThread_SendMessage($sMsg) will be automatically called.

string _AuThread_GetMessage( [ $iPID = @AutoItPID ] )

Gets the last message sent to the current thread (or main thread, if called on it), returns and deletes it. If no message is found, False is returned.

void _AuThread_SendMessage($iPid, $sMessage)

Sends a message to a thread (main or sub). This message can be retrieved through _AuThread_GetMessage().

void _AuThread_CloseThread( [ $iPID = @AutoItPID ] )

Closes a thread or the current thread (if no thread PID is specified).

Download:

Github ZIP

Fork me on Github

Edited by Jefrey

My stuff

Spoiler

My UDFs  _AuThread multithreading emulation for AutoIt · _ExtInputBox an inputbox with multiple inputs and more features · forceUTF8 fix strings encoding without knowing its original charset · JSONgen JSON generator · _TCPServer UDF multi-client and multi-task (run on background) event-based TCP server easy to do · _TCPClient_UDF multi-server and multi-task (runs on background) event-based TCP client easy to do · ParseURL and ParseStr functions ported from PHP · _CmdLine UDF easily parse command line parameters, keys or flags · AutoPHP Create documents (bills, incomes) from HTML by sending variables/arrays from AutoIt to PHP · (Un)Serialize Convert arrays and data into a storable string (PHP compatible) · RTTL Plays and exports to MP3 Nokia-format monophonic ringtones (for very old cellphones) · I18n library Simple and easy to use localization library · Scripting.Dictionary OOP and OOP-like approach · Buffer/stack limit arrays to N items by removing the last one once the limit is reached · NGBioAPI UDF to work with Nitgen fingerprint readers · Serial/Licensing system require license key based on unique machine ID from your users · HTTP a simple WinHTTP library that allows GET, POST and file uploads · Thread true AutoIt threads (under-dev) · RC4 RC4 encryption compatible with PHP and JS ·  storage.au3 localStorage and sessionStorage for AutoIt Classes _WKHtmlToX uses wkhtmlto* to convert HTML files and webpages into PDF or images (jpg, bmp, gif, png...) Snippets _Word_DocFindReplaceByLongText replace strings using Word UDF with strings longer than 255 characters (MSWord limit) rangeparser parser for printing-like pages interval (e.g.: "1,2,3-5") EnvParser parse strings/paths with environment variables and get full path GUICtrlStaticMarquee static text scrolling Random stuff Super Mario beep sound your ears will hurt

 

Link to comment
Share on other sites

  • Replies 60
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

15 hours ago, jaberwacky said:

Does this use multiple cores if one's computer were so equipped?

No, this UDF launches the same executable multiple times to emulate multi-threading, As AutoIt is not able to use Multiple Cores, this UDF does not use them too!

50 minutes ago, JiBe said:

But, use of 'AdlibRegister' is not the true multi threading.

Huh? :huh:, Where did the UDF use Adlib functions?

 

EasyCodeIt - A cross-platform AutoIt implementation - Fund the development! (GitHub will double your donations for a limited time)

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to comment
Share on other sites

  • Moderators

@JiBe just because you disagree with what TheDcoder states, is not a reason to report the thread. Think it through next time...

"Profanity is the last vestige of the feeble mind. For the man who cannot express himself forcibly through intellect must do so through shock and awe" - Spencer W. Kimball

How to get your question answered on this forum!

Link to comment
Share on other sites

4 minutes ago, JLogan3o13 said:

just because you disagree with what TheDcoder states, is not a reason to report the thread.

He reported me? :blink:... First, sorry if I have said anything wrong @JiBe :sweating:, it is not my intention to hurt anyone! (The same is mentioned in my signature)

I would welcome any counter-argument :), but please do it here, not in the report box! :D.

EasyCodeIt - A cross-platform AutoIt implementation - Fund the development! (GitHub will double your donations for a limited time)

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to comment
Share on other sites

Sorry, just a mistake of using the forum.
I wanted to just posted this:

 

excuse my English, I'm French. but the Autoit Code used in the UDF is very clear. and my interpretation of this code is the use of the Adlibregister. should you detail your assertion.
Link to comment
Share on other sites

On 14/08/2016 at 2:10 PM, jaberwacky said:

Does this use multiple cores if one's computer were so equipped?

Hi, @jaberwacky! It looks like Andreik and TheDcoder have answered to your question. I've done no tests so far with _WinAPI_SetProcessAffinityMask, tho.

21 hours ago, Andreik said:

It shouldn't be so hard to use _WinAPI_SetProcessAffinityMask().

 

9 hours ago, TheDcoder said:

No, this UDF launches the same executable multiple times to emulate multi-threading, As AutoIt is not able to use Multiple Cores, this UDF does not use them too!

----------------------

10 hours ago, JiBe said:
Hi,.

But, use of 'AdlibRegister' is not the true multi threading.

everything from useful.

Good Week!

Hi, JiBe!

Actually, the AdlibRegister is not the true way it runs the "threads". Truly it really creates new processes and uses Adlib just to check for new messages (from the UDF _AuThread_SendMessage()) or closing signal.

Also, it's really not the true multi threading. It's just an emulation. It's "forking" ;)

----------------------

2 hours ago, argumentum said:

Hi, argumentum!

Yes, this is the right name hehe. It came out of my mind at the time of the creation of this topic. I've added it to the topic title (I chose to keep "multi thread emulation" for helping users who are using the search engine, as some might not know what's forking). Thanks for poiting out :)

---------------

Guys, I've started another project that, according to what my latest tests are poiting, might provide some kind of true multithreading for AutoIt without external processes. It's not fully working yet (so far it works just with basic tasks). This is an example:

#include 'thread.au3'

CreateThread("helloworld")

Func helloworld($h)
    MsgBox(0, "", "Hello from the sub thread")
EndFunc

MsgBox(0, "", "Hello from the main thread")

Exit

If you want to help, here's the repo: https://github.com/jesobreira/thread.au3 :D 

My stuff

Spoiler

My UDFs  _AuThread multithreading emulation for AutoIt · _ExtInputBox an inputbox with multiple inputs and more features · forceUTF8 fix strings encoding without knowing its original charset · JSONgen JSON generator · _TCPServer UDF multi-client and multi-task (run on background) event-based TCP server easy to do · _TCPClient_UDF multi-server and multi-task (runs on background) event-based TCP client easy to do · ParseURL and ParseStr functions ported from PHP · _CmdLine UDF easily parse command line parameters, keys or flags · AutoPHP Create documents (bills, incomes) from HTML by sending variables/arrays from AutoIt to PHP · (Un)Serialize Convert arrays and data into a storable string (PHP compatible) · RTTL Plays and exports to MP3 Nokia-format monophonic ringtones (for very old cellphones) · I18n library Simple and easy to use localization library · Scripting.Dictionary OOP and OOP-like approach · Buffer/stack limit arrays to N items by removing the last one once the limit is reached · NGBioAPI UDF to work with Nitgen fingerprint readers · Serial/Licensing system require license key based on unique machine ID from your users · HTTP a simple WinHTTP library that allows GET, POST and file uploads · Thread true AutoIt threads (under-dev) · RC4 RC4 encryption compatible with PHP and JS ·  storage.au3 localStorage and sessionStorage for AutoIt Classes _WKHtmlToX uses wkhtmlto* to convert HTML files and webpages into PDF or images (jpg, bmp, gif, png...) Snippets _Word_DocFindReplaceByLongText replace strings using Word UDF with strings longer than 255 characters (MSWord limit) rangeparser parser for printing-like pages interval (e.g.: "1,2,3-5") EnvParser parse strings/paths with environment variables and get full path GUICtrlStaticMarquee static text scrolling Random stuff Super Mario beep sound your ears will hurt

 

Link to comment
Share on other sites

On 8/15/2016 at 2:42 PM, Jefrey said:

I've done no tests so far with _WinAPI_SetProcessAffinityMask, tho.

I did :) 

If Not StringInStr($CmdLineRaw, "/ErrorStdOut") And Not @Compiled Then Exit MsgBox(262144, @ScriptName, "...please run from editor.", 5)

#include <Array.au3>
#include <WinAPI.au3>
#include <WinAPISys.au3>
#include <Process.au3>
#include <WinAPIShPath.au3>

SetProcessAffinityMaskForCurrentProcess(0x05)
; For example, if you pass a mask of 0x05, processors 0 and 2 ( 1st and 3th ) are allowed to run.

Global $aProcessAffinityMask = GetProcessAffinityMask(_WinAPI_PathStripPath(@AutoItExe))
_ArrayDisplay($aProcessAffinityMask, "$aProcessAffinityMask") ; show the affinity masks

Func SetProcessAffinityMaskForCurrentProcess($i = 0)
    Local $hCurrentProcess = _WinAPI_GetCurrentProcess() ; process handle of currrent process
    Local $n = _WinAPI_SetProcessAffinityMask($hCurrentProcess, $i) ; Success: True, Failure: False
    Local $iErr = _WinAPI_GetLastError() ; call _WinAPI_GetLastError() to get extended error information
    Local $affiity = _WinAPI_GetProcessAffinityMask($hCurrentProcess)
    ConsoleWrite("--- > SetProcessAffinityMaskForCurrentProcess( 0x" & Hex($i, 2) & " ) = Success : " & _
            $affiity[0] & @TAB & "Proc Mask : " & $affiity[1] & @TAB & "Sys Mask : " & $affiity[2] & @CRLF)
    Return SetError($iErr, 0, $n)
EndFunc   ;==>SetProcessAffinityMaskForCurrentProcess

Func GetProcessAffinityMask($sProgram = Default)
    Local $a, $n, $m, $hProcess
    If $sProgram Then
        $a = ProcessList($sProgram)
    Else
        $a = ProcessList()
    EndIf
    ReDim $a[UBound($a)][4]
    $a[0][1] = "PID"
    $a[0][2] = "Process affinity cores ( mask )"
    $a[0][3] = "System affinity cores ( mask )"
    For $n = 1 To $a[0][0]
        $a[$n][2] = "failed"
        $a[$n][3] = "_WinAPI_OpenProcess"
        If _WinAPI_GetVersion() >= 6.0 Then
            $hProcess = _WinAPI_OpenProcess($PROCESS_QUERY_LIMITED_INFORMATION, 0, $a[$n][1]) ; process handle of external process
        Else
            $hProcess = _WinAPI_OpenProcess($PROCESS_QUERY_INFORMATION, 0, $a[$n][1]) ; process handle of external process
        EndIf
        If @error Then ContinueLoop
        $a[$n][3] = "_WinAPI_GetProcessAffinityMask"
        $b = _WinAPI_GetProcessAffinityMask($hProcess)
        If @error Then ContinueLoop
        For $m = 1 To 2 ; I only have 8 cores, adjust to your needs. You may need hex 8, this is just a demo.
            $a[$n][1 + $m] = AffinityMaskToCores($b[$m]) & " ( 0x" & Hex($b[$m], 2) & " )"
        Next
    Next
    Return $a
EndFunc   ;==>GetProcessAffinityMask

Func AffinityMaskToCores($i)
    Local $s = "", $n = 1, $c = 0 ; processors are "hardware", therefore start at 0 ( zero )
    Do
        If BitAND($i, $n) Then $s &= "," & $c
        $n += $n
        $c += 1
    Until $n > $i
    Return StringTrimLeft($s, 1)
EndFunc   ;==>AffinityMaskToCores

So here is a nice help to show how to set the process affinity and how to get it.

... on the other hand, 

#NoTrayIcon

Global $g_UniqueInAWay = "Wasted away again in Margaritaville"
If $CmdLine[0] Then
    f_WasteAway()
    Exit
EndIf
Func exitInAMinuteJustInCase()
    Exit
EndFunc   ;==>exitInAMinuteJustInCase
Func f_WasteAway()
    AutoItWinSetTitle($g_UniqueInAWay)
    AdlibRegister("exitInAMinuteJustInCase", 60000)
    Local $watever
    Do
        $watever = 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123 * 123
    Until 1 = 2
EndFunc   ;==>f_WasteAway

main()
Func main()
    Local $nop = Int(EnvGet("NUMBER_OF_PROCESSORS"))
    For $n = 1 To $nop ; or more, or less, windows will handle it.
        If @Compiled Then
            Run('"' & @ScriptFullPath & '" /wasteAway')
        Else
            Run('"' & @AutoItExe & '" "' & @ScriptFullPath & '" /wasteAway')
        EndIf
    Next
    MsgBox(262144, @ScriptName, "you've got " & $nop & " running processes" & @CR & _
            "as you can see in task manager > performance" & @CR & _
            "windows handle the cpu core assignation quite well" & @CR & @CR & "press ok to stop them.", 60)
    While WinExists($g_UniqueInAWay)
        WinClose($g_UniqueInAWay)
    WEnd
EndFunc   ;==>main

that is not needed.

Edited by argumentum
changed the example a bit

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Link to comment
Share on other sites

11 hours ago, Jefrey said:

Guys, I've started another project that, according to what my latest tests are poiting, might provide some kind of true multithreading for AutoIt without external processes.

Ooohhhh :o, I will be following the changes on GitHub :D.

EasyCodeIt - A cross-platform AutoIt implementation - Fund the development! (GitHub will double your donations for a limited time)

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to comment
Share on other sites

hmmm, to fork or to thread, this is a good read even if meant for linux.
In AutoIt3 forking is stable, threading is not.
Working with threads is not a holy grail, loading a script ( compiled or not ) in my PC takes less than 10 mSec., memory nowadays is abundant, so forking is not a burden. Ingenuity is the key here.
Using MailSlots for IPC, is sequential, the message can be of unlimited* size and will not be overwritten, no great semaphore brainstorming needed. Forks can be pre-loaded and put to work when needed, and would even allow for distributed processing ( meaning that with this technique, you are not even limited to a single PC, forking is distributed processing ).
ok, café cubano time.

PS: I modified an example from another thread to kind of showcase what I mean.

Edited by argumentum
expand on my point

Follow the link to my code contribution ( and other things too ).
FAQ - Please Read Before Posting.
autoit_scripter_blue_userbar.png

Link to comment
Share on other sites

Exactly. And never forget that threads are evil. I mean here that multithreading "Hello World!" is one thing, but multithreading a non-trivial task is completely another one, especially with a high-level language which is explicitely not designed to cope with any drop of multithreading.

Unless you gamble blindly about what AutoIt is doing under your feet when you invoke complex or high-level functions deeply interacting with the OS, you'd better stay away from MT in serious code, else you unduly multiply the risks of potential disaster.

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to comment
Share on other sites

Hi (sorry for english errors i'm french)

Is it not better to use mailslot or other communication udf ? Imagine i got a

100mb memory use script ( the main script)

and one

1mb memory use (update or other function script)

 

if i use your udf my script will use 200 mb memory 

 

if i use a communication udf it will use 101 mb 

 

why is it better to use forking ? in wich situations ? 

(in a hard-protected computer it can be cool no ? you just need to bypass one app in the antivirus / firewall and all the script will work) 

i'm really interessed by your udf for some projects i'm already testing it.

Good bye and thanks for this useful udf 

Link to comment
Share on other sites

@ultimateheadcenter: MailSlot payload on localhost is restricted to slightly less than 64KB (I'm using 65,507 as upper bound in my Pool UDF); across a LAN, the maximum MailSlot size is a datagram (424 bytes; same as for UDP); for TCP it's 1,472 bytes per packet. So sending a 100MB MailSlot msg is out of the question.

Also, if you're going to be using MailSlots, please be aware that some antivirus software (e.g., Sophos) can block MailSlot messages.

Edited by RTFC
AV caveat
Link to comment
Share on other sites

lol, for the past week i have been working on a similar project, using STDIO. my goal is to create a modular include system. 
Im working on tailoring it like the apt-get system in linux where dependencies and versioning is tracked.
So far i have a working demo, and it work as follows:
ur code starts up with my script as an include, on start modules are detected, then called for identity, then once you load the module, it is initialized and I have an array if info for each module, along with nested arrays for each STD[IN|ERR|OUT] everything is queued in an adlib timer
 

What is what? What is what.

Link to comment
Share on other sites

Hey @Jefrey,

I believe you'd see significant performance improvement if you spawned the multiple instances using the Run function with STDIO enabled and passing commands through that method. I have an example around my computer somewhere with a sample AV program that uses this method to separate the scanning process from it to allow interacting with the GUI while scanning.

My UDFs are generally for me. If they aren't updated for a while, it means I'm not using them myself. As soon as I start using them again, they'll get updated.

My Projects

WhyNotWin11
Cisco FinesseGithubIRC UDFWindowEx UDF

 

Link to comment
Share on other sites

  • 1 month later...

@WormIT This one is different, it does not use multiple processes! Genuine threads are created by calling the Windows API, have a look at the code: https://github.com/jesobreira/authread/blob/master/authread.au3

Oops :>, I just realized that this is another UDF, The UDF which I was talking about is this:
https://github.com/jesobreira/thread.au3

Edited by TheDcoder
Oops

EasyCodeIt - A cross-platform AutoIt implementation - Fund the development! (GitHub will double your donations for a limited time)

DcodingTheWeb Forum - Follow for updates and Join for discussion

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...