Jump to content
Sign in to follow this  
Ascend4nt

Disk and Device Read/Write Statistics

Recommended Posts

Disk and Device Read/Write (I/O) Statistics


This is an example of getting and displaying Read/Write Activity for all Disks and Device I/O.  The actual UDF function _NTQuerySysPerfInfo() returns a whole lot more Performance Information for the computer; however, we are focused on I/O activitiy here.

This was written as a faster alternative to WMI, as well as an alternative to >Performance Counters (see my >Physical Disk Read/Write activity example).

Update 2013-07-06:

Changed: Renamed module

Changed: _AddCommas now uses a nicer PCRE

Fixed - Copy and paste errors

Added - Comma separators

DiskDeviceIOStatistics.au3

; ========================================================================================================
; <DiskDeviceIOStatistics.au3>
;
; Example of reading and displaying Disk and Device I/O Statistics, using a 'Spash' window
;  NOTE There is alot more data returned from _NTSysPerformanceInfo(), but
;  for our purposes, we focus only on I/O data
;
; Functions:
;    _NTSysPerformanceInfo()        ; Returns I/O and a whole slew of other Performance Info
;
; Author: Ascend4nt
; ========================================================================================================

; ===================================================================================================================
; Func _NTSysPerformanceInfo()
;
; Returns Disk & Device I/O Statistics (and much more) from NtQuerySystemInformation.
;
; NOTE: ZWReadFile and ZWWriteFile are low-level File I/O that are called by higher-level File I/O
;    (such as ReadFile and WriteFile)
;
; Returns:
;  Success: Array of I/O and Performance Stats:
;    [0]  = Idle Time of all CPU's on the system (in 100-nanosecond intervals)
;    [1]  = # of Bytes Read through ZWReadFile calls
;    [2]  = # of Bytes Written through ZWWriteFile calls
;    [3]  = # of Bytes for all Other I/O Operations (such as Device I/O)
;    [4]  = # of calls made to ZWReadFile (NOT the # of bytes - see [1])
;    [5]  = # of calls made to ZWWriteFile (NOT the # of bytes - see [2])
;    [6]  = # of calls to other I/O Operations (such as Device I/O) - see [3]
;    [7]  = # of Physical Pages Available to Processes
;    [8]  = # of Commited Pages of Virtual Memory
;    [9]  = Commit Limit - # of Pages that can be committed before extending PageFile
;    [10] = Peak # of Pages of Committed Virtual Memory
;    [11] = # of Page Faults (soft and hard)
;    [12] = # of Write Faults (attempts to write to copy-on-write pages)
;    [13] = # of Soft Page Faults [TransitionFaults]
;    [14] = # of Demand Zero Faults
;    [15] = # of Pages Read from Disk to Resolve Page Faults
;    [16] = # of Read Operations to Resolve Page Faults [not # of pages]
;    [17] = # of Pages Written to System's Pagefiles
;    [18] = # of Write operations performed on System's Pagefiles
;    .. alot more info (see SYSTEM_PERFORMANCE_INFO structure) ..
;    [73] = # of System Calls
;
;  Failure: "" with @error set:
;     @error = 2: DLLCall error, @extended = error returned from DLLCall
;    @error = 3: NTSTATUS returned error code, @extended contains error code
;
; Author: Ascend4nt
; ===================================================================================================================

Func _NTQuerySysPerfInfo()
#cs
    ; SYSTEM_PERFORMANCE_INFORMATION Defined as size 312 in latest <winternl.h>,
    ;  but on Win7 this is 328 (both 32-bit and 64-bit modes).
    ; For now I've added a buffer at the end of 16 bytes (+ 16 more for future issues).
    ; Hopefully eventually I'll be able to track down what the extra data is.
    ; Its obviously not a 32-bit vs 64-bit size difference..
#ce
    Local Const $tagSYSTEM_PERFORMANCE_INFORMATION = "uint64 IdleTime;uint64 ReadTransferCount;uint64 WriteTransferCount;uint64 OtherTransferCount;" & _
        "ULONG ReadOperationCount;ULONG WriteOperationCount;ULONG OtherOperationCount;" & _
        "ULONG AvailablePages;ULONG TotalCommittedPages;ULONG TotalCommitLimit;ULONG PeakCommitment;" & _
        "ULONG PageFaults;ULONG WriteCopyFaults;ULONG TransitionFaults;ULONG Reserved1;ULONG DemandZeroFaults;ULONG PagesRead;ULONG PageReadIos;" & _
        "ULONG Reserved2[2];ULONG PagefilePagesWritten;ULONG PagefilePageWriteIos;ULONG MappedFilePagesWritten;ULONG MappedFilePageWriteIos;" & _
        "ULONG PagedPoolUsage;ULONG NonPagedPoolUsage;ULONG PagedPoolAllocs;ULONG PagedPoolFrees;ULONG NonPagedPoolAllocs;ULONG NonPagedPoolFrees;" & _
        "ULONG TotalFreeSystemPtes;ULONG SystemCodePage;ULONG TotalSystemDriverPages;ULONG TotalSystemCodePages;" & _
        "ULONG SmallNonPagedLookasideListAllocateHits;ULONG SmallPagedLookasideListAllocateHits;ULONG Reserved3;" & _
        "ULONG MmSystemCachePage;ULONG PagedPoolPage;ULONG SystemDriverPage;" & _
        "ULONG FastReadNoWait;ULONG FastReadWait;ULONG FastReadResourceMiss;ULONG FastReadNotPossible;ULONG FastMdlReadNoWait;" & _
        "ULONG FastMdlReadWait;ULONG FastMdlReadResourceMiss;ULONG FastMdlReadNotPossible;" & _
        "ULONG MapDataNoWait;ULONG MapDataWait;ULONG MapDataNoWaitMiss;ULONG MapDataWaitMiss;" & _
        "ULONG PinMappedDataCount;ULONG PinReadNoWait;ULONG PinReadWait;ULONG PinReadNoWaitMiss;ULONG PinReadWaitMiss;" & _
        "ULONG CopyReadNoWait;ULONG CopyReadWait;ULONG CopyReadNoWaitMiss;ULONG CopyReadWaitMiss;" & _
        "ULONG MdlReadNoWait;ULONG MdlReadWait;ULONG MdlReadNoWaitMiss;ULONG MdlReadWaitMiss;" & _
        "ULONG ReadAheadIos;ULONG LazyWriteIos;ULONG LazyWritePages;ULONG DataFlushes;ULONG DataPages;" & _
        "ULONG ContextSwitches;ULONG FirstLevelTbFills;ULONG SecondLevelTbFills;ULONG SystemCalls;" & "ULONG Buffer[8];"

    Local $stSysPerfInfo,$iSysInfoLen

    $stSysPerfInfo = DllStructCreate($tagSYSTEM_PERFORMANCE_INFORMATION)
    $iSysInfoLen = DllStructGetSize($stSysPerfInfo)

;~     ConsoleWrite("SYSTEM_PERFORMANCE_INFORMATION structure size:" & $iSysInfoLen & @CRLF)

    ; SystemPerformanceInformation = class 2
    Local $aRet=DllCall("ntdll.dll","long","NtQuerySystemInformation","int",2,"ptr",DllStructGetPtr($stSysPerfInfo),"ulong",$iSysInfoLen,"ulong*",0)
    If @error Then Return SetError(2, @error, "")

    ; NTSTATUS of something OTHER than success?
    If $aRet[0] Then
;~         If $aRet[0] = 0xC0000004 Then ConsoleWrite("STATUS_INFO_LENGTH_MISMATCH, size required: " &$aRet[4] & @CRLF)
        Return SetError(3, $aRet[0], "")
    EndIf

;~     If $aRet[4]<>$iSysInfoLen Then ConsoleWriteError("Size mismatch: $stInfo struct length="&$iSysInfoLen&", ReturnLength="&$aRet[4]&@LF)

;~     _DLLStructDisplay($stSysPerfInfo, $tagSYSTEM_PERFORMANCE_INFORMATION)

    Dim $aRet[74]
    For $i = 1 To 74
        $aRet[$i - 1] = DllStructGetData($stSysPerfInfo, $i)
    Next
    Return $aRet
EndFunc


; ====================================================================================================
; Func _AddCommas($sString)
;
; Simple PCRE to add commas - borrowed from StackOverflow
; "regex - Insert commas into number string" - answer by toolkit:
; @ http://stackoverflow.com/a/721415
;
; Author: Ascend4nt
; ====================================================================================================

Func _AddCommas($sString)
    Return StringRegExpReplace($sString, "(\d)(?=(\d{3})+$)", "$1,")
EndFunc


;   --------------------    HOTKEY FUNCTION & VARIABLE --------------------

Global $bHotKeyPressed = False

Func _EscPressed()
    $bHotKeyPressed=True
EndFunc

;   --------------------    MAIN PROGRAM CODE   --------------------

HotKeySet("{Esc}", "_EscPressed")

Local $hSplash, $sSplashText
Local $aIOStats


$aIOStats = _NTQuerySysPerfInfo()
If @error Then
    Exit ConsoleWrite("@error = " &@error & ", @extended = " &Hex(@extended) & @CRLF)
EndIf

ConsoleWrite(" I/O Stats: Read = " & _AddCommas($aIOStats[1]) & "; Write = " & _AddCommas($aIOStats[2]) & "; Other = " & _AddCommas($aIOStats[3]) & @CRLF)

$hSplash=SplashTextOn("Hard Drive Usage Information", "", 740, 24 + (4 * 15), Default, Default, 16+4, "Lucida Console", 11)

; Start loop
Do
    $aIOStats = _NTQuerySysPerfInfo()
    $sSplashText  = StringFormat("%50s", "== I/O Stats ==") & @CRLF
    $sSplashText &= StringFormat("Read = %20s | Write = %20s | Other = %20s", _AddCommas($aIOStats[1]), _AddCommas($aIOStats[2]), _AddCommas($aIOStats[3])) & @CRLF

    $sSplashText &= @CRLF & StringFormat("%48s", "[ESC] Exits") & @CRLF

    ControlSetText($hSplash, "", "[CLASS:Static; INSTANCE:1]", $sSplashText)

    Sleep(500)
Until $bHotKeyPressed
Edited by Ascend4nt

Share this post


Link to post
Share on other sites

Small changes - renamed module, changed _AddCommas() function. See 1st post


Share this post


Link to post
Share on other sites

Very nice. I also like the regular expression you found.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Very nice. I also like the regular expression you found.

 

[reference: StringRegExpReplace($sString, "(d)(?=(d{3})+$)", "$1,")]

You know, that regular expression was difficult to understand at first.  I got caught up on thinking about look-aheads as very definitive expressions, but this one allows any number of groups-of-3-digits - or maybe I should say,  a multiple of 3. It's actually quite clever when you understand how it moves from left to right picking off only those numbers followed by 3^x digits.  Seems there's always something new to learn in regular expressions :)


Share this post


Link to post
Share on other sites

True, I read it as though it was d{3,} but yeah I learn something new everyday about regular expressions.


UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

  • Similar Content

    • By rudi
      Hi.
      The final question first: Is there a reason, that Autoit doesn't clean up RAM usage as a standard feature?
       
      This posting by @guinness was pointing me to the solution for solving my problem:
      https://www.autoitscript.com/forum/topic/131315-accumulating-memory-usage/?do=findComment&comment=914208
      DllCall("psapi.dll", "int", "EmptyWorkingSet", "long", -1) this simple, single line called on a regular basis stopped the script consuming more and more RAM.
      CU, Rudi.
    • By simy8891
      Hi guys,
      It's been a while since I wrote my last message here and a while since I used AutoIt. I'm currently sort of desperate and I'm trying to find some help in regards of getting the network usage per process!
      I'm not interested in the total network usage of the NIC, but only on a specific PID's network utilization. They idea is to collect the amount of traffic uploaded and downloaded by a list of specific processes. So far Process Hacker and Process Explorer are capable of getting what I need, but I need to use these numbers in another script so they're sort of useless to me. I can't seem to find a way around it.
      Any idea, help is greatly appreciated.
      Thanks
    • By dzlee
      hello autoit developers community
      i have some question , i hope to find the answers of it :
      - in large project or even in small ones
      how to make my compiled script use less memory ?
      - how to test my script and be sure that there is no memory leak , or whatsoever reduce my script performance ?
      - where can i find anything talks about autoit script architecture to avoid bad script design ?

      last thing i find these script
      can anyone describe how these scripts work and is these script actually work fine
      thanks
      Func _ReduceMemory() Local $ai_GetCurrentProcessId = DllCall('kernel32.dll', 'int', 'GetCurrentProcessId') Local $ai_Handle = DllCall("kernel32.dll", 'int', 'OpenProcess', 'int', 0x1f0fff, 'int', False, 'int', $ai_GetCurrentProcessId[0]) Local $ai_Return = DllCall("psapi.dll", 'int', 'EmptyWorkingSet', 'long', $ai_Handle[0]) DllCall('kernel32.dll', 'int', 'CloseHandle', 'int', $ai_Handle[0]) Return $ai_Return[0] EndFunc Func _SelfReduceMemory() DllCall("psapi.dll", "int", "EmptyWorkingSet", "long", -1) EndFunc  
×
×
  • Create New...