FileSystemMonitor UDF

[johnmcloud]

Hi,

Seems this script not working for me, or better to say not working what i need.

I need to monitoring a single folder ( Like C:Test, not all the HDD ), without subfolder, for check it out if new files added ( only files, don't need folder ) and if yes i'd like a simply ConsoleWrite. Nothing else.

Someone can post an example? I don't need ant GUI or something.

Thanks

EDIT: Solved, stupid thing, sorry

Edited October 9, 2012 by johnmcloud

[aguynamedray]

Is there a way to pull out the user name of the specified domain using the example script?

Thanks in advance.

[RetroComp]

This thread seems to be dead by now.

I have trouble with this UDF, and started a new topic here

bye.

[VAN0]

Hello.

Am I doing something wrong or Floppy drives not supported by this UDF?

It doesn't send any events when floppy is inserted or disconnected..(yeah, I know, "what's a floppy?" )

Thanks

[Iczer]

is it possible to monitor multiple folders with this UDF in the same time?

It seems UDF has many global variables...

[tonycst]

#cs ----------------------------------------------------------------------------

 AutoIt Version: 3.3.12.0
 Author:         myName

 Script Function:
    Template AutoIt script.

#ce ----------------------------------------------------------------------------

; Script Start - Add your code below here
#include <GUIConstantsEx.au3>
#include <GuiListView.au3>
#include <FileSystemMonitor.au3>

$Title = "Window Title"

$Gui = GUICreate ($Title,600,200)
GUICtrlCreateGroup ("",5,0,510,115)
$Listview = _GUICtrlListView_Create ($Gui,"Backups",10,10,500,100)

GUISetState (@SW_SHOW,$Gui)
While 1
   $MSG = GUIGetMsg()
   If $MSG = $GUI_EVENT_CLOSE Then ExitLoop
WEnd

FileSystemMonitor.au3"(287,74) : error: _FileSysMonActionEvent(): undefined function.

What am i doing wrong ? I have not called any functions yet of this UDF and its already failing. Do i need to include something ?

[Melba23]

tonycst,

You need to define the _FileSysMonActionEvent function within your own script - this is the function called by the UDF when an event occurs and you need to determine what happens in that case as it will vary depending on the script.

M23

[GhostLine]

Hi guys,

sorry for the grave digging, but I'm looking for a way to see, in real time and without too much CPU and memory load, what happen to the files in a folder of my server.

The library in this topic work well, but not for the deletion and addition of files (they doesn't seem to be seemed by the program) when I use the File System Monitor proposed as an example.

My guess is that, in Windows 7 and next versions, the "addresses" aren't the good ones :

        Case 0
            $fs_event.item(Hex(0x00000001)) = "file added to the directory|FILE_ACTION_ADDED"
            $fs_event.item(Hex(0x00000002)) = "file removed from the directory|FILE_ACTION_REMOVED"
            $fs_event.item(Hex(0x00000003)) = "file was modified|FILE_ACTION_MODIFIED"
            $fs_event.item(Hex(0x00000004)) = "file was renamed old name|FILE_ACTION_RENAMED_OLD_NAME"
            $fs_event.item(Hex(0x00000005)) = "file was renamed new name|FILE_ACTION_RENAMED_NEW_NAME"

When I try to look after the $fs_event value, it show me "1","2","3" in the middle of "true" values as "0x00000001", "0x00040000" etc.).

Could you help me on this matter ?

Thank you in advance

[easywind]

thanks a lot!

[llewxam]

GhostLine:

 

When you say "files in a folder on my server", do you mean things like media, or system files?  Just curious, as I would personally design the 2 ideas rather differently...

 

Anyway, I have not used this UDF but I think I would just use _FileListToArrayRec in an AblibRegister'd function to run every 10-15 seconds, seems real-time enough for me.  Run _FileListToArrayRec and save the array, run again to a temporary array, and then use _ArraySearch to find duplicate entries.  What is not found in the previous array is new, deleted would go the other way around.  You could then add things like time stamps, file sizes, even MD5 hashes if it meets your needs, for the comparison of the files to look for changes.  A simple GUI with a continuous ListBox could work for simple ongoing output of changes found, though the ways to display the output could go so many ways.  Also saving the file array to a file/SQLite could be nice for persistence after reboots of the machine doing the monitoring..........

 

Just thinking out loud about your question, hopefully this gives you a few ideas.

 

Ian

[GhostLine]

Hi !

My exact problem is taht I've to watch my filesystem for curious behaviors, like suddent files deleting, during the execution of a very ressource demanding program.

As you can guess, that's why I've to know the name, timestamp (and eventually name of the deleter process), but this UDF only show me if a file is modified or if the size of a folder vary. I don't have any indication of a new file with a specific message, nor the deletion.

Thanks for your concern !

[stealthmsgr]

Brilliant. I'm going to test (along with many others it seems) and supply feedback. Thanks for sharing a great idea. Cheers.

[bobmcrae]

Excellent work Sean.  This monitoring approach seems to be far superior to the WMI query approach as discussed in this thread: 

 

[t0nZ]

Hi bobmcrae , i was curious about this monitoring system, I use the WMI query for folders with 10000+ files, and I use a custom made sqllite approach for folders with few hundreds files, for performance reasons.

But I want to ask you, you use this UDF in the original version (dated 2010..) with no issues ? Is it reliable ?

Cheers,

[bobmcrae]

t0nz: no issues whatsoever.  While I have not stress tested it; it's performance has been extraordinary and never have a I noticed a "missed" file event

[Taneeda]

Hi,

is there any newer version available? I try to run the script on WIn10 x64 with AutoIt 3.3.14.5, unfortunately without success yet. The program crashes with:

!>23:53:09 AutoIt3.exe ended.rc:-1073741819

I did some research and it seems that the parameters of some DLL-calls must be changed. Unfortunately it wasn't possible to me yet to create a running version of the UDF...

Note: I modified the MOD version from ...

 

expandcollapse popup

#include-once

#cs
    Title:           File System Monitoring UDF Library for AutoIt3
    Filename:          FileSystemMonitor.au3
    Description:     A collection of functions for monitoring the Windows File System
    Author:           seangriffin
    Version:          V0.4
    Last Update:     02/05/10
    Requirements:     AutoIt3 3.2 or higher
#ce

#AutoIt3Wrapper_au3check_parameters=-d -w 1 -w 2 -w 3 -w 4 -w 5 -w 6

; #INCLUDES# =========================================================================================================
#include <WinAPIShellEx.au3>

; #GLOBAL VARIABLES# =================================================================================================
Global $pFSM_DirEvents, $pFSM_Dir, $pFSM_Overlapped, $tFSM_FNI, $pFSM_Buffer, $sFSM_Filename, $aFSM_Register, $iFSM_Buffersize, $tFSM_Overlapped
Global $tFSM_Buffer, $tFSM_DirEvents, $iFSM_DirEvents, $hFSM_Event, $hFSM_ShellMonGUI = GUICreate("")

; #FUNCTION# ;===============================================================================
;
; Name...........: _FileSysMonSetup()
; Description ...: Setup File System Monitoring.
; Syntax.........: _FileSysMonSetup($iMonitor_Type = 3, $sDirMon_Path = "C:\", $sShellMon_Path = "")
; Parameters ....: $iMonitor_Type - Optional: The type of monitoring to use.
;                      1 = directory monitoring only
;                      2 = shell monitoring only
;                      3 = both directory and shell monitoring
;                  $sDirMon_Path - Optional: The path to use for directory monitoring.
;                      The path "C:\" is used if one isn't provided.
;                  $sShellMon_Path - Optional: The path to use for shell monitoring.
;                      The blank path is used if one isn't provided. This
;                      denotes that system-wide shell events will be monitored.
; Return values .: On Success            - Returns True.
;                  On Failure            - Returns False.
; Author ........: seangriffin
; Modified.......:
; Remarks .......: A call to this function should be inserted in a script prior to calling other
;                  functions in this UDF.  Ideally the function should be placed before
;                  the main message loop in a GUI-based script.
; Related .......:
; Link ..........:
; Example .......: Yes
; ;==========================================================================================
Func _FileSysMonSetup($iMonitor_Type = 3, $sDirMon_Path = "C:\", $sShellMon_Path = "")

    If BitAnd($iMonitor_Type, 1) Then ; Setup the Directory Event Handler
        Local $sdir = $sDirMon_Path
        $tFSM_Buffer = DllStructCreate("byte[4096]")
        $pFSM_Buffer = DllStructGetPtr($tFSM_Buffer)
        $iFSM_Buffersize = DllStructGetSize($tFSM_Buffer)
        $tFSM_FNI = 0
        $pFSM_Dir = DllCall("kernel32.dll", "hwnd", "CreateFile", "Str", $sdir, "Int", 0x1, "Int", BitOR(0x1, 0x4, 0x2), "ptr", 0, "int", 0x3, "int", BitOR(0x2000000, 0x40000000), "int", 0)
        $pFSM_Dir = $pFSM_Dir[0]
        $tFSM_Overlapped = DllStructCreate("Uint OL1;Uint OL2; Uint OL3; Uint OL4; hwnd OL5")
        For $i = 1 To 5
            DllStructSetData($tFSM_Overlapped, $i, 0)
        Next
        $pFSM_Overlapped = DllStructGetPtr($tFSM_Overlapped)
        $tFSM_DirEvents = DllStructCreate("hwnd DirEvents")
        $pFSM_DirEvents = DllStructGetPtr($tFSM_DirEvents)

        Local $hFSM_Event = DllCall("kernel32.dll", "hwnd", "CreateEvent", "UInt", 0, "Int", True, "Int", False, "UInt", 0)
        If @error Then ConsoleWrite("DLLCall(CreateEvent) --> @error = " & @error & @CRLF)
        If Not IsArray($hFSM_Event) Then ConsoleWrite("Err: Returned array expected from DLLCall(CreateEvent)" & @CRLF)

        DllStructSetData($tFSM_Overlapped, 5, $hFSM_Event[0])
        If @error Then ConsoleWrite("DllStructSetData($tFSM_Overlapped) --> @error = " & @error & @CRLF)

        DllStructSetData($tFSM_DirEvents, 1, $hFSM_Event[0])
        If @error Then ConsoleWrite("DllStructSetData($tFSM_DirEvents) --> @error = " & @error & @CRLF)
        ConsoleWrite("FSM_Log_1" & @CRLF)

        DllCall("kernel32.dll", "Int", "ReadDirectoryChangesW", "hwnd", $pFSM_Dir, "ptr", $pFSM_Buffer, "dword", $iFSM_Buffersize, _
            "bool", False, "dword", BitOR(0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x100), "PTR", 0, "PTR", $pFSM_Overlapped, "PTR", 0)
        ConsoleWrite("FSM_Log_2" & @CRLF)
        If @error Then ConsoleWrite("DLLCall(ReadDirectoryChangesW) --> @error = " & @error & @CRLF)

        $sFSM_Filename = ""
    EndIf

    If BitAND($iMonitor_Type, 2) Then ; Setup the Shell Event Handler
        ; Register a window message to associate an AutoIT function with the change notification events
        Local $aRet = DllCall("user32.dll", "uint", "RegisterWindowMessageW", "wstr", "shchangenotifymsg")
        If @error Then Return SetError(@error, @extended, 0)
        Local $SHNOTIFY = $aRet[0]
        GUIRegisterMsg($SHNOTIFY, "_FileSysMonShellEventHandler")
        ; Setup the structure for registering the gui to receive shell notifications
        If StringCompare($sShellMon_Path, "") <> 0 Then
            Local $ppidl = DllCall("shell32.dll", "ptr", "ILCreateFromPath", "wstr", $sShellMon_Path)
        EndIf
        Local $shnotifystruct = DllStructCreate("ptr pidl; int fRecursive")
        If StringCompare($sShellMon_Path, "") <> 0 Then
            DllStructSetData($shnotifystruct, "pidl", $ppidl[0])
        Else
            DllStructSetData($shnotifystruct, "pidl", 0)
        EndIf
        DllStructSetData($shnotifystruct, "fRecursive", 0)
        ; Register the gui to receive shell notifications
        $aFSM_Register = DllCall("shell32.dll", "int", "SHChangeNotifyRegister", "hwnd", $hFSM_ShellMonGUI, "int", BitOR(0x0001, 0x0002), "long", 0x7FFFFFFF, "uint", $SHNOTIFY, "int", 1, "ptr", DllStructGetPtr($shnotifystruct))
        If StringCompare($sShellMon_Path, "") <> 0 Then
            DllCall("ole32.dll", "none", "CoTaskMemFree", "ptr", $ppidl[0])
        EndIf
    EndIf

    Return True
EndFunc   ;==>_FileSysMonSetup

; #FUNCTION# ;===============================================================================
;
; Name...........: _FileSysMonSetDirMonPath()
; Description ...: Change the path of Directory Monitoring
; Syntax.........: _FileSysMonSetDirMonPath($sDirMon_Path = "C:\")
; Parameters ....: $sDirMon_Path - Optional: The path to use for directory monitoring.
;                      The path "C:\" is used if one isn't provided.
; Return values .: On Success - Returns True.
;                  On Failure - Returns False.
;
; Author ........: seangriffin
; Modified.......:
; Remarks .......: For an unknown reason, after this function is called the
;
; Related .......:
; Link ..........:
; Example .......: Yes
; ;==========================================================================================
Func _FileSysMonSetDirMonPath($sDirMon_Path = "C:\")

    Local $sdir = $sDirMon_Path
    $pFSM_Dir = DllCall("kernel32.dll", "hwnd", "CreateFile", "Str", $sdir, "Int", 0x1, "Int", BitOR(0x1, 0x4, 0x2), "ptr", 0, "int", 0x3, "int", BitOR(0x2000000, 0x40000000), "int", 0)
    $pFSM_Dir = $pFSM_Dir[0]
    For $i = 1 To 5
        DllStructSetData($tFSM_Overlapped, $i, 0)
    Next
    Local $hFSM_Event = DllCall("kernel32.dll", "hwnd", "CreateEvent", "UInt", 0, "Int", True, "Int", False, "UInt", 0)
    DllStructSetData($tFSM_Overlapped, 5, $hFSM_Event[0])
    DllStructSetData($tFSM_DirEvents, 1, $hFSM_Event[0])
    DllCall("kernel32.dll", "Int", "ReadDirectoryChangesW", "hwnd", $pFSM_Dir, "ptr", $pFSM_Buffer, "dword", $iFSM_Buffersize, "int", False, "dword", BitOR(0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x100), "Uint", 0, "Uint", $pFSM_Overlapped, "Uint", 0)

    Return True
EndFunc   ;==>_FileSysMonSetDirMonPath

; #FUNCTION# ;===============================================================================
;
; Name...........: _FileSysMonSetShellMonPath()
; Description ...: Change the path of Shell Monitoring
; Syntax.........: _FileSysMonSetShellMonPath($sDirMon_Path = "")
; Parameters ....: $sDirMon_Path    - Optional: The path to use for shell monitoring.
;                      The path "" is used if one isn't provided.
; Return values .: On Success - Returns True.
;                  On Failure - Returns False.
;
; Author ........: seangriffin
; Modified.......:
; Remarks .......:
;
; Related .......:
; Link ..........:
; Example .......: Yes
; ;==========================================================================================
Func _FileSysMonSetShellMonPath($sShellMon_Path = "")

    ; De-Register the gui from receiving shell notifications
    DllCall("shell32.dll", "int", "SHChangeNotifyDeregister", "ulong", $aFSM_Register[0])
    ; Register a window message to associate an AutoIT function with the change notification events
    ;Local $SHNOTIFY = _WinAPI_RegisterWindowMessage("shchangenotifymsg")
    Local $aRet = DllCall("user32.dll", "uint", "RegisterWindowMessageW", "wstr", "shchangenotifymsg")
    If @error Then Return SetError(@error, @extended, 0)
    Local $SHNOTIFY = $aRet[0]
    GUIRegisterMsg($SHNOTIFY, "_FileSysMonShellEventHandler")
    ; Setup the structure for registering the gui to receive shell notifications
    If StringCompare($sShellMon_Path, "") <> 0 Then
        Local $ppidl = DllCall("shell32.dll", "ptr", "ILCreateFromPath", "wstr", $sShellMon_Path)
    EndIf
    Local $shnotifystruct = DllStructCreate("ptr pidl; int fRecursive")
    If StringCompare($sShellMon_Path, "") <> 0 Then
        DllStructSetData($shnotifystruct, "pidl", $ppidl[0])
    Else
        DllStructSetData($shnotifystruct, "pidl", 0)
    EndIf
    DllStructSetData($shnotifystruct, "fRecursive", 0)
    ; Register the gui to receive shell notifications
    $aFSM_Register = DllCall("shell32.dll", "int", "SHChangeNotifyRegister", "hwnd", $hFSM_ShellMonGUI, "int", BitOR(0x0001, 0x0002), "long", 0x7FFFFFFF, "uint", $SHNOTIFY, "int", 1, "ptr", DllStructGetPtr($shnotifystruct))
    If StringCompare($sShellMon_Path, "") <> 0 Then
        DllCall("ole32.dll", "none", "CoTaskMemFree", "ptr", $ppidl[0])
    EndIf

    Return True

EndFunc   ;==>_FileSysMonSetShellMonPath

; #FUNCTION# ;===============================================================================
;
; Name...........: _FileSysMonDirEventHandler()
; Description ...: Monitors the file system for changes to a given directory.  If a change event occurs,
;                      the user-defined "_FileSysMonActionEvent" function is called.
; Syntax.........: _FileSysMonDirEventHandler()
; Parameters ....: none
; Return values .: On Success - Returns True.
;                  On Failure - Returns False.
;
; Author ........: seangriffin
; Modified.......:
; Remarks .......: This function utilises the "ReadDirectoryChangesW" Win32 operating system function to
;                  monitor the a directory for changes.
;
;                  The ReadDirectoryChangesW function appears to queue events, such that whenever
;                  it is called, all unprocessed events are retrieved one at a time.
;
;                  The function "_FileSysMonSetup" must be called, with a $iMonitor_Type
;                  of either 1 or 3, prior to calling this    function.
;
;                  A call to this function should be inserted within the main message loop of a GUI-based script.
;
;                  A user-defined function to action the events is required to be created by the user
;                  in the calling script, and must be defined as follows:
;
;                  Func _FileSysMonActionEvent($event_type, $event_id, $event_value)
;
;                  EndFunc
;
; Related .......:
; Link ..........:
; Example .......: Yes
; ;==========================================================================================
Func _FileSysMonDirEventHandler()

    Local $aRet, $iOffset, $nReadLen, $tStr, $iNext, $ff

    $aRet = DllCall("User32.dll", "dword", "MsgWaitForMultipleObjectsEx", "dword", 1, "ptr", $pFSM_DirEvents, "dword", 100, "dword", 0x4FF, "dword", 0x6)

    If $aRet[0] = 0 Then
        $iOffset = 0
        $nReadLen = 0

        DllCall("kernel32.dll", "Uint", "GetOverlappedResult", "hWnd", $pFSM_Dir, "Uint", $pFSM_Overlapped, "UInt*", $nReadLen, "Int", True)
        While 1
            $tFSM_FNI = DllStructCreate("dword Next;dword Action;dword FilenameLen", $pFSM_Buffer + $iOffset)
            $tStr = DllStructCreate("wchar[" & DllStructGetData($tFSM_FNI, "FilenameLen") / 2 & "]", $pFSM_Buffer + $iOffset + 12)
            $sFSM_Filename = DllStructGetData($tStr, 1)
            _FileSysMonActionEvent(0, DllStructGetData($tFSM_FNI, "Action"), $sFSM_Filename)
            $iNext = DllStructGetData($tFSM_FNI, "Next")
            If $iNext = 0 Then ExitLoop
            $iOffset += $iNext
        WEnd
        $ff = DllStructGetData($tFSM_Overlapped, 5)
        DllCall("kernel32.dll", "Uint", "ResetEvent", "UInt", $ff)
        DllCall("kernel32.dll", "Int", "ReadDirectoryChangesW", "hwnd", $pFSM_Dir, "ptr", $pFSM_Buffer, "dword", $iFSM_Buffersize, "int", False, "dword", BitOR(0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x100), "Uint", 0, "Uint", $pFSM_Overlapped, "Uint", 0)
    EndIf

    Return True

EndFunc   ;==>_FileSysMonDirEventHandler

; #FUNCTION# ;===============================================================================
;
; Name...........: _FileSysMonShellEventHandler()
; Description ...: Monitors the file system for shell events.
; Syntax.........: _FileSysMonShellEventHandler()
; Parameters ....: $hWnd   - The Window handle of the GUI in which the message appears.
;                  $iMsg    - The Windows message ID.
;                  $wParam - The first message parameter as hex value.
;                  $lParam - The second message parameter as hex value.
; Return values .: On Success - Returns True.
;                  On Failure - Returns False.
;
; Author ........: seangriffin
; Modified.......:
; Remarks .......: If a directory was provided in "_FileSysMonSetup" then only events in
;                  that directory will be caught.  If no directory was provided, then
;                  system-wide events will be caught.
;
;                  This function utilises the "SHChangeNotifyRegister" Win32 operating system functionality
;                  monitor a system or directory for changes relating to the Windows shell.
;
;                  The function "_FileSysMonSetup" must be called, with a $iMonitor_Type
;                  of either 2 or 3, prior to calling this    function.
;
;                  A call to this function is not required.  It is triggered automatically
;                  for each new shell event.
;
;                  A user-defined function to action the events is required to be created by the user
;                  in the calling script, and must be defined as follows:
;
;                  Func _FileSysMonActionEvent($event_type, $event_id, $event_value)
;
;                  EndFunc
;
; Related .......:
; Link ..........:
; Example .......: Yes
; ;==========================================================================================
Func _FileSysMonShellEventHandler($hWnd, $iMsg, $wParam, $lParam)

    #forceref $hWnd, $iMsg

    Local $tDestination, $wHighBit, $strData

    Local $tPath = DllStructCreate("dword dwItem1; dword dwItem2", $wParam)
;~  Local $tPath = DllStructCreate("USHORT cb;BYTE abID[1]", $wParam)

    ; Crash happens here
    Local $aRet = DllCall("shell32.dll", "int", "SHGetPathFromIDList", "ptr", DllStructGetData($tPath, "dwItem1"), "str", $strData)
;~  Local $aRet[3] = [ "","","" ]
;~  $aRet[2] = _WinAPI_ShellGetPathFromIDList ( $wParam )
    ; Get the drive for which free space has changed
    If $lParam = 0x00040000 Then
        $tDestination = DllStructCreate("long")
        DllCall("kernel32.dll", "none", "RtlMoveMemory", "ptr", DllStructGetPtr($tDestination), "ptr", (DllStructGetData($tPath, "dwItem1") + 2), "int", 4) ; CopyMemory
        $wHighBit = Int(Log(DllStructGetData($tDestination, 1)) / Log(2))
        $aRet[2] = Chr(65 + $wHighBit)
    EndIf
    If $lParam <> 0x00000002 And $lParam <> 0x00000004 Then ; FILE_ACTION_ADDED & FILE_ACTION_REMOVED skipped due to a deadlock with Directory_Event_Handler()
        _FileSysMonActionEvent(1, $lParam, $aRet[2])
    EndIf

    Return True

EndFunc   ;==>_FileSysMonShellEventHandler

 

[Taneeda]

Can anybody help with the DllStruct...() calls? At the moment, there is the problem. I can't get them working and unfortunately I didn't found helpful documentation for that :/

[BrewManNH]

3 hours ago, Taneeda said:

I can't get them working

What's not working? Without some idea where you are failing it's hard to help you. Also post the code you're using, or a reproducer script that shows the problem.

[Taneeda]

See my post above with the code file included...

 

The DllXXX() calls must be adjusted in any way

On 10/23/2018 at 3:10 PM, Taneeda said:

Can anybody help with the DllStruct...() calls? At the moment, there is the problem. ...unfortunately I didn't found helpful documentation for that :/

At last, I tried to modify the Dll...() calls within ...

On 9/27/2018 at 12:01 AM, Taneeda said:

Func _FileSysMonShellEventHandler($hWnd, $iMsg, $wParam, $lParam)

... but It wasn't yet able to me to get the code running.

It seems, that the API of the related windows sdk function within the called dlls have changed over the last years...

[Taneeda]

Any ideas?