File Monitoring Example

[arcker]

yes i understand your point

btw, i don't know if you know, but a lot of this monitor shareware use wmi

so when we can do the same free, why not ?

[Andrew Peacock]

yes i understand your point

btw, i don't know if you know, but a lot of this monitor shareware use wmi

so when we can do the same free, why not ?

You're right :-)

I just needed it quickly, and it was taking me to long to pester you with question :-)

I'd still like to do it eventually in AutoIt, as it will fit very neatly with the app I'm selling. Plus it will remove the dependency on a piece of software that is not supported any more, and hasn't been updated in a LOOONNNNNGGGG time.

Regards,

Andy

[arcker]

yes, and you know, wmi is really important in the new system (vista and longhorn) so learning it is a really good benefit

[Zomp]

ok here the script

it can monitor the file and more, it can report file modification

if you want sub monitoring juste replace targetinstance.path = '//1//' by targetinstance.path like '//1//%'

BUT IT CONSUMES A LOT OF RESSOURCES, EVEN IF YOU SET WITHIN 60

i've an alernative for it: it's to create a script that will sink everydirectories

i send you an example as soon as possible

here is my current work

[cut]

Thanks for your excellent example.

I have some questions:

1) Executing your script, on the console I see the messages about the modification of all the already existing files in the monitored folder. Is there any method to avoid it?

2) I have tried to monitor a folder and its subfolders as you have suggested, using "%" but it does not work. Is there any mistake?

3) I have only a vague idea of what you mean for "sink everydirectory". What about if someone, after having executed the "silk everydir" script, creates a new folder? Is such a newly created folder monitored or not?

Thanks for your patience.

[arcker]

Thanks for your excellent example.

I have some questions:

1) Executing your script, on the console I see the messages about the modification of all the already existing files in the monitored folder. Is there any method to avoid it?

2) I have tried to monitor a folder and its subfolders as you have suggested, using "%" but it does not work. Is there any mistake?

3) I have only a vague idea of what you mean for "sink everydirectory". What about if someone, after having executed the "silk everydir" script, creates a new folder? Is such a newly created folder monitored or not?

Thanks for your patience.

arf i've just leave work

i can post the script with sink on Monday

i see what you mean about modificated files, it is when you launch it right ? they're all listed on launch

maybe you can create a file list in a tab an filter it out, but i would use a counter to filter first event.

for the % it should work, i can't test it for now, but i will asap

[Zomp]

i see what you mean about modificated files, it is when you launch it right ? they're all listed on launch

maybe you can create a file list in a tab an filter it out, but i would use a counter to filter first event.

A counter? That is, I have to count the files in the folder before launching the script and then omit the number of first events so calculated? Or do you mean something else?

for the % it should work, i can't test it for now, but i will asap

Thanks for your very appreciated help. I have also searched in google about recursion or subfolder, but withot success. Even a link could be useful.

[VeeDub]

I have a need to monitor for file changes in a directory tree and I was originally going to perform a snapshot of the directory tree and then perform a comparison periodically; but WMI looks interesting and I am trying (so far without success) to use it instead to do the directory monitoring.

If I can't get WMI to work, then FileNotify looks like a decent alternative (note I have not used this as yet) I base this assessment solely on the user guide. But it looks like FileNotify can be installed transparently on the fly, and it claims to have low overhead and is freeware.

If FileNotify turns out to be a trojan, don't blame me ...

Also from my reading it seems that while WMI may be suitable for monitoring 'small' selections, if you want to monitor an entire logical drive then it may not be the best method. Another approach would be to use the API ReadDirectoryChangesW however that looks reasonably involved - and using FileNotify in theory gives you access to the ReadDirectoryChangesW API without having to master all the intricate detail - with the tradeoff being you need to install the app rather than having an AutoIt only solution.

For my needs I think WMI will be sufficient if I can get it to work.

VW

[Zomp]

If I can't get WMI to work, then FileNotify looks like a decent alternative (note I have not used this as yet) I base this assessment solely on the user guide. But it looks like FileNotify can be installed transparently on the fly, and it claims to have low overhead and is freeware.

The best should be to have a FileNotify function in AutoIt. Even if it seems to me a not much complicated task, I have not the knowledge to build it.

[Zomp]

Also from my reading it seems that while WMI may be suitable for monitoring 'small' selections, if you want to monitor an entire logical drive then it may not be the best method. Another approach would be to use the API ReadDirectoryChangesW however that looks reasonably involved - and using FileNotify in theory gives you access to the ReadDirectoryChangesW API without having to master all the intricate detail - with the tradeoff being you need to install the app rather than having an AutoIt only solution.

I have found this sample script in AUtoHotKey which uses ReadDirectoryChangeW

http://www.autohotkey.com/forum/topic22862.html

but I am not able to say if it is possible to convert easily in AutoIt.

Edit: In the meantime I have finally had success in monitoring a folder recursively using the hint

if you want sub monitoring juste replace targetinstance.path = '//1//' by targetinstance.path like '//1//%'

but it seems to me a not feasible method. The "ExecNotificationQuery ("SELECT * FROM ..." instruction takes just one hour to be completed on my pc, even for a path wich contains only ten files. Edited April 8, 2008 by Zomp

[arcker]

i think it's possible to use the same API in autoit.

for the script, i can post the script that uses one sink for each subfolder that you need to monitor.

here it is

change the path in find folders.

expandcollapse popup

WMI monitoring
#include<array.au3>
#include<date.au3>
#include<guiconstants.au3>
Dim $arrComputers, $strQuery, $SINK, $objContext, $objWMIService, $objAsyncContextItem, $return, $account
$arrComputers = _ArrayCreate(".")


Sink_Start($sink)

_find_folders("C:\WINDOWS\Microsoft.NET\Framework")
;SINK_Add("C:","\\1\\",$sink)
;SINK_Add("C:","\\1\\Nouveau dossier\\",$sink)

func _find_folders($path)
    $monitordrive  = stringleft($path,2)
    $monitorpath = StringReplace(stringtrimleft($path,2),"\","\\") & "\\"
    SINK_Add($monitordrive,$monitorpath,$sink)
    ConsoleWrite($path & @crlf)
    local $first=FileFindFirstFile($path & "\*.*")
    while 1
        $fichier = FileFindNextFile($first)
        if @error then ExitLoop
        ;ConsoleWrite($fichier & @tab & FileGetAttrib($path & "\" & $fichier)  & @crlf  )
        if stringinstr(FileGetAttrib($path & "\" & $fichier),"D") then _find_folders($path & "\" & $fichier)
    WEnd        
EndFunc 

While 1
    Sleep(10000)
WEnd

func Sink_Start(byref $Sink)
    $SINK = ObjCreate("WbemScripting.SWbemSink")
    ObjEvent($SINK, "SINK_")
EndFunc

func SINK_Add($wmidrive,$wmipath,$sinkref)
    $objContext = ObjCreate("WbemScripting.SWbemNamedValueSet")
    $strQuery = "SELECT * FROM __InstanceOperationEvent WITHIN 20 WHERE TargetInstance ISA 'CIM_DataFile' and TargetInstance.Path = '" & $wmipath & "' and TargetInstance.drive = '" & $wmidrive & "'"
    $objWMIService = ObjGet("winmgmts:!\\.\root\cimv2")
    $objWMIService.ExecNotificationQueryAsync ($sinkref,$strQuery, Default, Default, Default, $objContext)
EndFunc

;******************************************************************************
Func SINK_OnObjectReady ($objLatestEvent, $objAsyncContext)    ;Trap asynchronous events.
    Local $essai1, $essai2
    ;$objAsyncContextItem = $objAsyncContext.Item ("hostname")
    $filename= $objLatestEvent.TargetInstance.Properties_.item("Name").value
    ConsoleWrite($filename & @CRLF)
EndFunc   ;==>SINK_OnObjectReady
Func sink_onprogress ($iUpperBound, $iCurrent, $strMessage, $objWbemAsyncContext)
    ConsoleWrite("progress ... " & @CRLF)
    ConsoleWrite($iUpperBound & @CRLF & $iCurrent & @CRLF & $strMessage & @CRLF & $objWbemAsyncContext & @CRLF)
EndFunc   ;==>sink_onprogress

[VeeDub]

@Arcker

i think it's possible to use the same API in autoit.

for the script, i can post the script that uses one sink for each subfolder that you need to monitor

I don't doubt that using WMI can be made to "work" in the sense that you can recurse a directory tree, however it is not a scalable solution if you're considering monitoring say a directory tree with many 1,000's of files - or to take it to the extreme - the root of the filesystem (e.g. C:\).

The WMI approach uses polling and a snapshot approach, which for a small selection - say a single directory - ala ptrex's original example, is attractive because with a very small amount of code you can get WMI to do the work which otherwise would require comparitively more code if you're reading the directory yourself and performing the compare.

However once you go past a certain point, then clearly an approach where the filesystem notifies you of filesystem changes as they occur is clearly (or at least in my mind is going to be) superior. This for instance is how Explorer knows to "refresh" when filesystem activity is occurring to the directory structure being viewed. Or while FileMon is so efficient.

For me anyway, the attraction of using WMI was that I could hopefully refine ptrex's original example and monitor a "small" directory structure with minimal coding effort. Once you have to start monitoring each individual folder with a separate polling/snapshot process, I think the "simplicity" is lost and technically there are better approaches.

For now I am going to have a closer look at using FileNotify, and there are some other similar utilities around.

Longer term I think it would be interesting to see if a FileNotify/DirWatcher.udf could be written using calls to either ReadDirectoryChangesW and/or FindFirstChangeNotification, FindNextChangeNotification and FindCloseChangeNotification or perhaps some other API's. There are quite a few VB and C examples around - it is a matter of having the knowledge to translate these into a native AutoIt version (presumably via DllCall?).

VW

[Zomp]

Longer term I think it would be interesting to see if a FileNotify/DirWatcher.udf could be written using calls to either ReadDirectoryChangesW and/or FindFirstChangeNotification, FindNextChangeNotification and FindCloseChangeNotification or perhaps some other API's. There are quite a few VB and C examples around - it is a matter of having the knowledge to translate these into a native AutoIt version (presumably via DllCall?).

I have made an attempt to build a monitor folder script using ReadDirectoryChangeW. If you have the time to see it and help me, you can find it here:

http://www.autoitscript.com/forum/index.php?showtopic=69044

Thanks for your attention.

[okdewit]

Interesting topic,

I have been playing around with WMI for a couple of days now, and though my scripting skills are not that great I managed to create a GUI so I can create "rules" for each folder. That way, when a file matches certain properties inside a monitored folder, it is moved or copied.

At the moment I just run one script which controls, configures and launches a monitoring script for each seperate folder, so when I monitor a lot of folders the process list gets kinda full.

Fun to play around with, made me start researching COM and especially WMI.

[hot202]

anyway to make it detect if folders are created to?

[ptrex]

@hot202

This will get you going.

Func CreateFolder($FolderPath) 
$StrQuery_FolderCreate = "SELECT $* $FROM __InstanceCreationEvent $WITHIN 10 WHERE " _ 
                        & "TargetInstance ISA 'Win32_SubDirectory' AND " _ 
                        & "TargetInstance.GroupComponent=" _ 
                        & "'Win32_Directory.Name=""" & $FolderPath & """'" 

 $FolderCreateSink = Objcreate("WBemScripting.SWBemSink", "FolderCreateSink_") 

If @error <> 0 Then 
ConsoleWrite ("FolderCreateSink : " & @error )
EndIf

Rgds

ptrex

[Zeqalox]

This thread has helped me a lot!

I'm doing well with a script, but now I'm kinda stuck.

How do I make the script check in certain folders?

The script I am working on needs to check the current users homepath (like C:\Documents And Settings\User01).

I've tried so many combinations, I just can't get it to work!

Here's the part of the script I'm using.

$colMonitoredEvents = $objWMIService.ExecNotificationQuery _
("Select * From __InstanceOperationEvent WITHIN 5 WHERE " _
& "TargetInstance Isa 'Cim_DataFile' And " _
& "TargetInstance.Path = '\\1\\' And " _
& "TargetInstance.Drive = 'C:'")

How can I make it check the current userprofile and another directory like C:\folder1\folder2\folder3?

Thanks in advance!!

- Zeq

Edited July 7, 2011 by Zeqalox

[Meitantei]

Hello guys.

You can control an entire drive with this UDF? If so, how?

Even if the CPU utilization increases in an uncontrolled manner (100%).

PS: thanks for this UDF

EDIT: add information

Edited August 14, 2011 by Meitantei

[aguynamedray]

Hello Guys,

This script is awesome as well as the new edited version. How can I use it through server? I mean, I have to specify the current directory of the said server:

server01applications

Thanks.

[bobmcrae]

Folks,

After implementing my second file system monitor -- this time an asynchronous one (ExecNotificationQueryAsync), I learned a few VERY important things.  First, I was fooled that my script & this method was computationally efficient.  When looking at the script run, I was seeing 0-3% CPU utilization, with mostly idle.  However, I noticed my PC's overall CPU seemed to spike periodically -- with the frequency of the polling period.  In fact, viewing ALL processes, I now see that the ExecNotificationQueryAsync initiates a separate process: WmiPrvSE.exe.  As I had my polling frequency set to 1/sec, I would see this process consume about 35% of my CPU during each cycle -- non-trivial!  Moreover, and more importantly, once I exited the script cleanly, the WmiPrvSE.exe continued on as if it were still polling.  This process would not terminate on its own -- despite what MS indicates in the documentation.  So, I came across a method that allows one to terminate/cancel the query.  After implementing this single line of code in in my exit function, the WmiPrvSE.exe process CPU went to ZERO and eventually closed all together.  In short, when using the ExecNotificationQueryAsync approach, I HIGHLY recommend including the cancel method on the sink after you are done polling ($sink.Cancel()).  Partial code below.

OnAutoItExitRegister('_exit')
_pollQFApp()


Func _pollQFApp()

    $wmiObj     = ObjGet("winmgmts:\\localhost\root\CIMV2")
    $sinkObj    = ObjCreate("WbemScripting.SWbemSink")
                  ObjEvent($sinkObj, '_fileMonSink_')
    $wmiObj.ExecNotificationQueryAsync($sinkObj, "SELECT * FROM __InstanceCreationEvent Within 5 Where TargetInstance ISA 'Cim_DataFile' " & _
            "and TargetInstance.Drive= 'C:' and TargetInstance.Path = '\\Users\\Public\\Desktop\\'")

    While 1
        If ProcessExists($qfEXE)=0 Or ProcessExists($parentPID)=0 Then Exit ; terminate if one of the parent apps is terminated
        Sleep(250)
    WEnd
EndFunc ; _pollQFApp()

Func _fileMonSink_OnObjectReady($obj)
    $fileName  = $obj.TargetInstance.Name
    ; do something
EndFunc ;SINK_OnObjectReady()

Func _exit()
    $sinkObj.Cancel()
EndFunc ; _exit()

 

Edited January 21, 2016 by bobmcrae

[bobmcrae]

While happy with the solution of controlling the rouge WMI query, I was still not happy with the trade-off between polling frequency and CPU load.  So, I just implemented in the same manner (same directory monitored) the approach by seangriffin.  WOW!  Not only do I get ***instantaneous*** notifications, the CPU load is non-existent.  (Of course I checked all other processes as well.)  The approach Mr Griffin implements in his UDF is FAR more efficient and comes without the tradeoff of polling frequency & load.  Nice work Sean!