emendelson

Allow multiple instances, but wait for others to finish?

29 posts in this topic

#1 ·  Posted (edited)

I have a script that performs various operations on files. I use it with a folder watcher that launches the script when a PCL file arrives in a folder. It works perfectly well I'm only watching one folder, but if I watch two different folders, then two instances of the script try to run at the same time and only one completes its task.

Is there a way to make the script - when it starts up - test for a second (or third) instance of itself, and wait until the second or third instance exits before actually doing anything?

I think I can see that I should be testing for ProcessExists by name and by PID, and testing for a situation where the name of the process exists, but the PID is different from the PID of the script that is doing the testing. But I can't figure out the logic of it - and of course I'm not sure that I've got the basic technique right.

Again, I want to allow multiple instances, because the script will be run automatically, and I can't go back and run it again if it doesn't run the first time. But I want the second instance to wait until the first exits, then the third instance to wait until the second exits, and so on.

I'll be very grateful to learn if anyone has an answer to this.

EDIT: I think the answer may be a modification of this; but if anyone has a better idea, I'll be grateful.

 

 

 

 

Edited by emendelson

Share this post


Link to post
Share on other sites



I think you're right: I didn't look closely enough at _Singleton and didn't realize that I could use the option "1" - that would presumably let me use a While/Wend loop that would run _Singleton until there was only one instance running.

But I wonder if this would let me wait until two other instances finish before proceeding. Will experiment and report back. Meanwhile, thank you!

Share this post


Link to post
Share on other sites

I use a modified version of _Singleton to do exactly what you want to do.
Will post tomorrow.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

#5 ·  Posted (edited)

Thank you, water. I'll be very glad to have this. Without knowing what I'm doing, I tried to experiment with this code, but it didn't work, and left one instance of my compiled script running so that it had to be shut down in the Task Manager. So something that works will be very welcome! Thank you!

Local $x = 1
While 1
    Local $list = ProcessList()
    For $i = 1 To $list[0][0]
        If $list[$i][0] = @ScriptName Then
            If $list[$i][1] <> @AutoItPID Then
                $x = $x + 1
            EndIf
        EndIf
    Next
    If $x = 1 Then ExitLoop
    Sleep(200)
WEnd

 

Edited by emendelson
Correcting lower-case name

Share this post


Link to post
Share on other sites

#6 ·  Posted (edited)

Simplified:

While UBound(ProcessList(@ScriptName)) > 2
    Sleep(200)
WEnd

You can add in a timer to start killing the processes (not = your pid) within the loop.

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

Interesting simplification! I'll experiment with that one too. Thank you!

Share this post


Link to post
Share on other sites

#8 ·  Posted (edited)

Here is one example of your way, corrected

While 1
    Local $x = 0
    Local $list = ProcessList(@ScriptName)
    For $i = 1 To $list[0][0]
        If $list[$i][1] <> @AutoItPID Then
            $x += 1
            ExitLoop

        EndIf

    Next
    If $x = 0 Then ExitLoop

    Sleep(200)
WEnd

The key is to reset $x within the loop.

You can even continueloop...such as this one:

While 1
    Local $list = ProcessList(@ScriptName)
    For $i = 1 To $list[0][0]
        If $list[$i][1] <> @AutoItPID Then
            Sleep(200)
            ContinueLoop 2
        EndIf
    Next
    ExitLoop
WEnd

But I would stick with my first example, in previous post, since it would be most efficient...though I don't base that on any actual analysis.

Edited by jdelaney

IEbyXPATH-Grab IE DOM objects by XPATH IEscriptRecord-Makings of an IE script recorder ExcelFromXML-Create Excel docs without excel installed GetAllWindowControls-Output all control data on a given window.

Share this post


Link to post
Share on other sites

Simplified:

While UBound(ProcessList(@ScriptName)) > 2
    Sleep(200)
WEnd

You can add in a timer to start killing the processes (not = your pid) within the loop.

Won't work.
Example:
Imagine that (while one instance is running) > 2 new instances get started. Now > 3 instances are active or waiting. When the active instance ended still > 2 instances are waiting. No new instance will start running and all instances will wait forever.
 


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

@jdelaney, declaring variables in a tight loop is a performance bottleneck.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Actually, as for the problem, look in my signature for _AppMonEx(), that might be of some use to you.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

Agreed. But as he is doing a Sleep in the loop that won't notice the small delay ;)


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

I am thinking for the future when they're not using sleep.


_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 04/09/2015

Share this post


Link to post
Share on other sites

I have a script that performs various operations on files. I use it with a folder watcher that launches the script when a PCL file arrives in a folder. It works perfectly well I'm only watching one folder, but if I watch two different folders, then two instances of the script try to run at the same time and only one completes its task.

What is the reason for this?


AutoIt Absolute Beginners    Require a serial    Pause Script    Video Tutorials by Morthawt   ipify 

Monkey's are, like, natures humans.

Share this post


Link to post
Share on other sites

#15 ·  Posted (edited)

I use the following modified version of function _Singleton to queue requests and process one script after the other.

While 1
    If _SingletonEx("Your Scriptname or something unique", 1) = 0 Then
        $iSleep = @AutoItPID * 10
        Sleep($iSleep)
    Else
        ExitLoop
    EndIf
WEnd

; #FUNCTION# ====================================================================================================================
; Name...........: _SingletonEx
; Description ...: Enforce a design paradigm where only one instance of the script may be running.
; Syntax.........: _SingletonEx($sOccurenceName[, $iFlag = 0])
; Parameters ....: $sOccurenceName - String to identify the occurrence of the script.  This string may not contain the \ character unless you are placing the object in a namespace (See Remarks).
;                  $iFlag          - Behavior options.
;                  |0 - Exit the script with the exit code -1 if another instance already exists.
;                  |1 - Return from the function without exiting the script.
; Return values .: Success - The handle to the object used for synchronization (a mutex).
;                  Failure - 0
; Author ........: Valik
; Modified.......:
; Remarks .......:
; Related .......:
; Link ..........:
; Example .......: Yes
; ===============================================================================================================================
Func _SingletonEx($sOccurenceName, $iFlag = 0)

    Local Const $ERROR_ALREADY_EXISTS = 183
    Local $tSecurityAttributes = 0
    Local $handle = DllCall("kernel32.dll", "handle", "CreateMutexW", "struct*", $tSecurityAttributes, "bool", 1, "wstr", $sOccurenceName)
    If @error Then Return SetError(@error, @extended, 0)
    Local $lastError = DllCall("kernel32.dll", "dword", "GetLastError")
    If @error Then Return SetError(@error, @extended, 0)
    If $lastError[0] = $ERROR_ALREADY_EXISTS Then
        DllCall("kernel32.dll", "bool", "CloseHandle", "handle", $handle[0])
        If @error Then Return SetError(@error, @extended, 0)
        If BitAND($iFlag, 1) Then
            Return SetError($lastError[0], $lastError[0], 0)
        Else
            Exit -1
        EndIf
    EndIf
    Return $handle[0]

EndFunc   ;==>_SingletonEx

The first script to run creates a Mutex. Every script checks for the existance of the Mutex and if it still exists waits for a random amount of time. When the first script ends the Mutex is being released and the next script checking for existance then starts.

Edited by water

My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

First, many thanks for all these suggestions. I'll be able to report results on Monday. In answer to an earlier question: I'm not sure exactly why the script doesn't work when two instances are running simultaneously, but I think it is because both instances are trying to write or read a file with the same name in a temporary directory; a user in the UK (I'm in the US) reported this and will test the solution (but he can't test it on the weekend). That user is working in a system that watches two (or more) directories for PCL files and uses my script to convert them to PDF files.

If anyone is interested, here is my totally amateurish and hopelessly inefficient code (with lots of things thrown in awkwardly when people asked for them). The $mydir variable works in the temporary directory, and seems to be the source of the problem.

https://www.dropbox.com/s/hx204irwcdrnec7/WinPCLtoPDF.au3?dl=0

And here is the web page where I describe the compiled script:

http://www.columbia.edu/~em36/pcltopdf.html

As you can see in the acknowledgments section of that page, I already owe a lot to this forum. I know I should rewrite the script in an efficient and intelligent way, but I'm afraid of breaking it altogether as a result of ignorance and incompetence.

PS: It occurs to me that one way to avoid this problem is for each instance to create a temporary directory with a unique name based on the date-time, and then delete that directory when it's finished processing the files. But maybe it's easier just to wait for other instances to finish.

 

Share this post


Link to post
Share on other sites

#17 ·  Posted (edited)

@water - I'd like to confirm that your mutex method (modified _Singleton) works absolutely perfectly, and is exactly what I was looking for. Thank you.

I also tried another method, which didn't work: instead of letting the script output to the same temporary filename, I made each instance generate a unique name using the time in milliseconds and a random number. But that didn't work either, apparently (I'm guessing) because both instances were trying to use the same executable at the same time.

Again, thank you for this ingenious and effective method.

Edited by emendelson

Share this post


Link to post
Share on other sites

:) Glad to be of service!

I do not think using the same executable is the problem - they are executed in separate processes. The bottleneck is always a shared ressource (file etc.)


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

I do not think using the same executable is the problem - they are executed in separate processes. The bottleneck is always a shared ressource (file etc.)

Thanks for explaining this. Now that I've got your method, it isn't urgent that I figure out what's going wrong, but I'd like to sort it out anyway, and will report back here if I do - just for the sake of anyone else who wants to do something similar in the future. Meanwhile, thank you again.

Share this post


Link to post
Share on other sites

I think it is always a good idea to limit the number of parallel processes. My problem once was caused by a server starting up to 50 processes at once all running PowerShell which would use all available memory. So I needed a queueing system which - after some discussion - was a modified version of _Singleton.


My UDFs and Tutorials:

Spoiler

UDFs:
Active Directory (NEW 2017-04-18 - Version 1.4.8.0) - Download - General Help & Support - Example Scripts - Wiki
OutlookEX (NEW 2017-02-27 - Version 1.3.1.0) - Download - General Help & Support - Example Scripts - Wiki
ExcelChart (2015-04-01 - Version 0.4.0.0) - Download - General Help & Support - Example Scripts
Excel - Example Scripts - Wiki
Word - Wiki
PowerPoint (2015-06-06 - Version 0.0.5.0) - Download - General Help & Support

Tutorials:
ADO - Wiki

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now