Jump to content

How to speed up response time of 'FileExists()'?


Recommended Posts

Hi,

I have a little script that looks into a local drive/path or (mapped) network drive/path.

The local drive/path (USB drive) or the network drive/path could become unavailable or unstable -

this is not an error... It is "behavior by design". :)

Every time a drive/path is unavailable or unstable the function 'FileExists()' tends to hang for about 10-20 seconds.

I've tried to work around this by using 'DriveStatus()' or 'Ping()' before 'FileExists()'. But this doesn't really solve anything.

Is there a better way?

Greets,

-supersonic.

Edited by supersonic
Link to comment
Share on other sites

I've been trying to get around the same issue. There seems to be no way to adjust the time-out value. Like you, I tried to skirt the issue by doing some checks before calling FileExists()

I've been using this _ValidPath() function, but I still get lags. I have a recent folder launcher. To avoid errors when a folder that is not available is selected for launch, I screen the list as I pop up the window with the ListBox. Invalid network drives especially cause a pause before the window opens.

Func _ValidPath($path)
    ;filter out obvious junk
    If StringLen($path) < 2 Then Return False

    Local $sysDrive = StringLeft(@WindowsDir, 2)
    Local $drive = StringLeft($path, 2)
    If $sysDrive = $drive Or $drive = "" Then
        Return FileExists($path)
    EndIf

    Local $driveType = DriveGetType($drive)
    If @error Then Return False
    Switch $driveType

        Case "Unknown"
            Return False

        Case "Network"
            If DriveStatus($drive) = "Invalid" Then Return False

        Case "CDROM", "Removable"
            If DriveStatus($drive) = "NotReady" Then Return False

    EndSwitch

    Return FileExists($path)
EndFunc   ;==>_ValidPath
Edited by MilesAhead
Link to comment
Share on other sites

btw my recent files launcher allows a max of 64 entries. If there are some invalid entries in the list typically the lag is only a second or so. Feel free to try my _ValidPath() function and see if it helps. I'm looking into this issue again because I want to increase the list max to 128 entries. But I don't want the user to experience a long lag. Screening the list in the background doesn't really help because the user could map a drive or power up a USB dock at any time. I really have to filter the list as it is coming up.

Link to comment
Share on other sites

If you call the DOS dir command, it will return almost immediately for local drives. There is not much you can do to get around network timeout. The time may vary from network to network, tests on my network for shares that are no longer available average around 11 seconds with the dir command.

Run a dir /b filepathname, and capture the output to see if it exists.

Not elegant, but may speed things up for you.

Link to comment
Share on other sites

Why does_StringIsValid look weird?

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

You found it here >>

But I think you need to re-edit your post as for some reason the syntax has been messed. You'll understand what I mean if you copy the code above and paste into SciTE, do you see any errors?

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

Try toggling the edit mode, select the icon next to the eraser.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

Here's my current version of '_FileExists()'.

I have rearranged some code in order to handle network drives/paths better.

@guinness: Indeed, it is not your original version of '_StringIsValid()'. The function is provided as example 2. I will keep using the modified version in order to use @error.

#include-once


; Example.
Local $iTimerInit = TimerInit()
Local $iFileExists = _FileExists(@TempDir)
ConsoleWrite("_FileExists() -> [" & $iFileExists & "][" & @error & "][" & @extended & "][" & Round(TimerDiff($iTimerInit), 0) & "]" & @CRLF)


Func _FileExists($sFE_Path)
    Local $iFE_Return = 0
    If StringLen($sFE_Path) > 1 Then
        _StringIsValid($sFE_Path, '/*?"<>|')
        If Not @error Then
            Local $sFE_Drive = StringLeft($sFE_Path, 2)
            If StringInStr(@HomeDrive, $sFE_Drive, 0, 1, 1, 2) > 0 Then
                $iFE_Return = FileExists($sFE_Path)
                SetExtended(1)
            Else ; Non-error.
                Local $iFE_UNC = 0
                Local $sFE_UNC = ""
                Switch ($sFE_Drive <> "")
                    Case False ; UNC [http://en.wikipedia.org/wiki/Path_(computing)].
                        $sFE_UNC = $sFE_Path
                        ContinueCase
                    Case True ; Network drive/path?
                        If $sFE_UNC = "" Then
                            Local $sFE_DriveMapGet = DriveMapGet($sFE_Drive)
                            If Not @error Then
                                $iFE_UNC = 1
                                $sFE_UNC = $sFE_DriveMapGet
                            Else ; Error.
                                ContinueCase
                            EndIf
                        EndIf
                        If $sFE_UNC <> "" Then
                            Local $sFE_Host = StringMid($sFE_UNC, 3, StringInStr($sFE_UNC, "", 0, 1, 3) - 3)
                            If $sFE_Host <> "" Then
                                Local $sFE_NSLOOKUP = ""
                                Local $iFE_PID = Run(@ComSpec & " /C " & @SystemDir & "nslookup.exe " & $sFE_Host, @TempDir, @SW_HIDE, 0x8)
                                While 1
                                    $sFE_NSLOOKUP &= StdoutRead($iFE_PID, False, False)
                                    If @error Then
                                        ExitLoop
                                    EndIf
                                WEnd
                                If $sFE_NSLOOKUP <> "" And Not StringInStr($sFE_NSLOOKUP, "***", 0) Then
                                    Ping($sFE_Host, 1000)
                                    If Not @error Then
                                        $iFE_Return = FileExists($sFE_Path)
                                        If $iFE_UNC = 0 Then
                                            SetExtended(2)
                                        Else ; Non-error.
                                            SetExtended(3)
                                        EndIf
                                    Else ; Error.
                                        SetError(6, 0)
                                    EndIf
                                Else ; Error.
                                    SetError(5, 0)
                                EndIf
                            Else ; Error.
                                SetError(4, 0)
                            EndIf
                        Else ; Non-error.
                            SetError(3, 0)
                        EndIf
                    Case Else ; Local drive/path.
                        Switch DriveGetType($sFE_Drive)
                            Case "CDROM", "FIXED", "RAMDISK", "REMOVABLE"
                                If DriveStatus($sFE_Drive) = "READY" Then
                                    $iFE_Return = FileExists($sFE_Path)
                                    SetExtended(4)
                                Else ; Error.
                                    SetError(7, 0)
                                EndIf
                            Case "UNKNOWN"
                                $iFE_Return = FileExists($sFE_Path)
                                SetExtended(5)
                            Case Else ; "NETWORK".
                                SetError(8, 0)
                        EndSwitch
                EndSwitch
            EndIf
        Else ; Error.
            SetError(2, 0)
        EndIf
    Else ; Error.
        SetError(1, 0)
    EndIf
    Return $iFE_Return
EndFunc   ;==>_FileExists


Func _StringIsValid($sSIV_String, $sSIV_Pattern = '/:*?"<>|')
    $sSIV_Pattern = StringRegExpReplace($sSIV_Pattern, "E", "E", 0)
    Local $iSIV_Result = BitAND(Not StringRegExp($sSIV_String, '[Q' & $sSIV_Pattern & 'E]', 0, 1), 1)
    Return SetError(Not $iSIV_Result, 0, $iSIV_Result)
EndFunc   ;==>_StringIsValid

Please post any suggestions or problems. :)

Edited by supersonic
Link to comment
Share on other sites

Every time a drive/path is unavailable or unstable the function 'FileExists()' tends to hang for about 10-20 seconds.

Run it*, if possible, as none blocking function and add your own timeout to it. (Adlib)

*) whatever takes longer than you care to wait for it to finish.

Edited by MvGulik

"Straight_and_Crooked_Thinking" : A "classic guide to ferreting out untruths, half-truths, and other distortions of facts in political and social discussions."
"The Secrets of Quantum Physics" : New and excellent 2 part documentary on Quantum Physics by Jim Al-Khalili. (Dec 2014)

"Believing what you know ain't so" ...

Knock Knock ...
 

Link to comment
Share on other sites

I changed one line in _ValidPath() so that it should handle unc paths better.

But, more to the point, I see in your code you are going out to the internet. That's a different situation than my utility. My recent folders would most likely all be on the local machine or LAN. So _ValidPath() can keep the time-out to an acceptable level unless there are a bunch of invalid unc folders in the list. For a large network naturally some better solution is needed.

Link to comment
Share on other sites

Please, can you provide some sample code?

Mmm ... not sure on what part.

See help example code on "StdoutRead" for how to deal with a non blocking function.

Adding your own waiting/bailout time to it could be done with something like this.

Local $line
Local $timeout = 3000 ;; 3 sec.
Local $time = timerinit() ;; get/set timer data.
While timerdiff($time) < $timeout ;; quit waiting after 3 seconds. (don't forget to cleanup after something like this, if needed.)
    $line = StdoutRead($foo)
    If @error Then ExitLoop
    MsgBox(0, "STDOUT read:", $line)
WEnd

Note (forgot to add this after I though about this): The adlib part is only needed if you like your code to also do some other stuff while its waiting for that other part to finish. (not needed in this case I think.)

Edited by MvGulik

"Straight_and_Crooked_Thinking" : A "classic guide to ferreting out untruths, half-truths, and other distortions of facts in political and social discussions."
"The Secrets of Quantum Physics" : New and excellent 2 part documentary on Quantum Physics by Jim Al-Khalili. (Dec 2014)

"Believing what you know ain't so" ...

Knock Knock ...
 

Link to comment
Share on other sites

@MvGulik

I'm guessing what you mean... But if it comes to actually check a local/network/UNC drive or path even a simple DIR command will hang at least for some seconds... Particularly with unclear/unstable local or network connections. Even with your approach you will need further checking. This is where my function comes in...

Edited by supersonic
Link to comment
Share on other sites

@MilesAhead

In fact, my function tries to access a DNS server (NSLOOKUP.EXE), which could be local - so an internet connection is not urgently required. If the computer is part of a domain and the domain is well-configured then there should be at least one DNS server.

You are right, my function is optimized for use in a domain environment. To use it in a simple LAN you probably have to configure your DNS server (internet router, etc.). In an environment without a DNS server the function can't probably resolve the name of the server. In this case the function will return 0.

If you are in a domain environment, please give the function a try... :)

Edited by supersonic
Link to comment
Share on other sites

I just noticed this post by guinness for a FileExists replacement using an API call:

I tried it on my Lan. Mapped a network drive, got it in the cache, then disconnected. The delay for path validation is about 1/20th of my function that used FileExists. I don't know if it's applicable to your program. May be worth a look.

Link to comment
Share on other sites

Strange, but if it works then great.

UDF List:

 
_AdapterConnections()_AlwaysRun()_AppMon()_AppMonEx()_ArrayFilter/_ArrayReduce_BinaryBin()_CheckMsgBox()_CmdLineRaw()_ContextMenu()_ConvertLHWebColor()/_ConvertSHWebColor()_DesktopDimensions()_DisplayPassword()_DotNet_Load()/_DotNet_Unload()_Fibonacci()_FileCompare()_FileCompareContents()_FileNameByHandle()_FilePrefix/SRE()_FindInFile()_GetBackgroundColor()/_SetBackgroundColor()_GetConrolID()_GetCtrlClass()_GetDirectoryFormat()_GetDriveMediaType()_GetFilename()/_GetFilenameExt()_GetHardwareID()_GetIP()_GetIP_Country()_GetOSLanguage()_GetSavedSource()_GetStringSize()_GetSystemPaths()_GetURLImage()_GIFImage()_GoogleWeather()_GUICtrlCreateGroup()_GUICtrlListBox_CreateArray()_GUICtrlListView_CreateArray()_GUICtrlListView_SaveCSV()_GUICtrlListView_SaveHTML()_GUICtrlListView_SaveTxt()_GUICtrlListView_SaveXML()_GUICtrlMenu_Recent()_GUICtrlMenu_SetItemImage()_GUICtrlTreeView_CreateArray()_GUIDisable()_GUIImageList_SetIconFromHandle()_GUIRegisterMsg()_GUISetIcon()_Icon_Clear()/_Icon_Set()_IdleTime()_InetGet()_InetGetGUI()_InetGetProgress()_IPDetails()_IsFileOlder()_IsGUID()_IsHex()_IsPalindrome()_IsRegKey()_IsStringRegExp()_IsSystemDrive()_IsUPX()_IsValidType()_IsWebColor()_Language()_Log()_MicrosoftInternetConnectivity()_MSDNDataType()_PathFull/GetRelative/Split()_PathSplitEx()_PrintFromArray()_ProgressSetMarquee()_ReDim()_RockPaperScissors()/_RockPaperScissorsLizardSpock()_ScrollingCredits_SelfDelete()_SelfRename()_SelfUpdate()_SendTo()_ShellAll()_ShellFile()_ShellFolder()_SingletonHWID()_SingletonPID()_Startup()_StringCompact()_StringIsValid()_StringRegExpMetaCharacters()_StringReplaceWholeWord()_StringStripChars()_Temperature()_TrialPeriod()_UKToUSDate()/_USToUKDate()_WinAPI_Create_CTL_CODE()_WinAPI_CreateGUID()_WMIDateStringToDate()/_DateToWMIDateString()Au3 script parsingAutoIt SearchAutoIt3 PortableAutoIt3WrapperToPragmaAutoItWinGetTitle()/AutoItWinSetTitle()CodingDirToHTML5FileInstallrFileReadLastChars()GeoIP databaseGUI - Only Close ButtonGUI ExamplesGUICtrlDeleteImage()GUICtrlGetBkColor()GUICtrlGetStyle()GUIEventsGUIGetBkColor()Int_Parse() & Int_TryParse()IsISBN()LockFile()Mapping CtrlIDsOOP in AutoItParseHeadersToSciTE()PasswordValidPasteBinPosts Per DayPreExpandProtect GlobalsQueue()Resource UpdateResourcesExSciTE JumpSettings INISHELLHOOKShunting-YardSignature CreatorStack()Stopwatch()StringAddLF()/StringStripLF()StringEOLToCRLF()VSCROLLWM_COPYDATAMore Examples...

Updated: 22/04/2018

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...