Is it time to update/refactor _DateIsValid in Date.au3?

[mrider]

Greetings - first a bit of background:

I am updating some code of my own where a user has to deal with a datetime in UTC format - namely yyyy-MM-ddTHH:mm:ss.SSS (using the definitions of the characters from Java's SimpleDateFormat).  I had two requirements that caused me to not use _DateIsValid:  1) I have to do an incremental log of what is wrong in the event there is a problem, and 2) It has to include milliseconds.

_DateIsValid does neither.

So I thought - no problem, I'll just see how that code works and modify it for my own use.  When I got there, I couldn't help but wonder if it couldn't be improved.  Almost certainly that version was written with a much older version of AutoIt without some of the modern conveniences we now love.  So I whipped up the version you see below - and I'm wondering if it is worthwhile to push this into the include instead of the current version.  You'll note that I left the leading underbar off the function name so that I could test this along side the current version:

expandcollapse popup

#include <Date.au3>
; Obviously the previous line isn't needed if this code makes it into the include...

; Don't forget to change the function name to _DateIsValid...
Func DateIsValid($sDate)
    Local $iDateLen = StringLen($sDate)
    Local $sPattern
    Switch $iDateLen
        Case 10
            ; 'yyyy-MM-dd'
            $sPattern = "(\d{4})[-/.](\d{2})[-/.](\d{2})"
        Case 16
            ; 'yyyy-MM-dd HH:mm'
            $sPattern = "(\d{4})[-/.](\d{2})[-/.](\d{2})[ T](\d{2}):(\d{2})"
        Case 19
            ; 'yyyy-MM-dd HH:mm:ss'
            $sPattern = "(\d{4})[-/.](\d{2})[-/.](\d{2})[ T](\d{2}):(\d{2}):(\d{2})"
        Case 23
            ; 'yyyy-MM-dd HH:mm:ss.SSS'
            $sPattern = "(\d{4})[-/.](\d{2})[-/.](\d{2})[ T](\d{2}):(\d{2}):(\d{2})\.(\d{3})"
        Case Else
            ; Length is wrong, can't possibly be a date
            Return False
    EndSwitch
    Local $aiMatches = StringRegExp($sDate, $sPattern, 1)
    If @error Then
        ; Does not match RE, so it's invalid format
        Return False
    Else
        ; At this point we KNOW we have a year, month and day
        If $aiMatches[0] < 1000 Or $aiMatches[0] > 2999 Then Return False   ; year
        If $aiMatches[1] < 1 Or $aiMatches[1] > 12 Then Return False        ; month
         ; If we're here, then year and month are both valid so we can use them immediately
        $aiDaysInMonth = _DaysInMonth($aiMatches[0])
        If $aiMatches[2] < 1 Or $aiMatches[2] > $aiDaysInMonth[$aiMatches[1]] Then Return False ; day

        ; We may or may not have have hours and minutes
        If $iDateLen > 10 Then
            If $aiMatches[3] > 23 Then Return False ; hour
            If $aiMatches[4] > 59 Then Return False ; minute
            ; We may or may not have seconds or milliseconds
            If $iDateLen > 16 Then
                If $aiMatches[5] > 59 Then Return False ; seconds
                ; No point in testing milliseconds, since we know it's 3 digits...
            EndIf
        EndIf
    EndIf
    Return True
EndFunc

In addition to that, I cooked up some tests of both good and bad data and then tested for both speed and correctness.  Both pass the correctness tests.  I didn't doubt for a minute that the current version would pass, but I wanted to make sure the new algorithm had the same behavior as the old algorithm.  So far as speed goes - the new version is about 25% faster with valid dates, and a whopping 50% faster at recognizing bad dates.

Here are the tests that I used:

expandcollapse popup

#include <Date.au3>

; test data
Local $timer
Local $good[15] = [ _
    '2014-02-28', '2014/02/28', '2014.02.28', _
    '2014-02-28 12:15', '2014/02/28 12:15', '2014.02.28 12:15', _
    '2014-02-28T12:15', '2014/02/28T12:15', '2014.02.28T12:15', _
    '2014-02-28 12:15:15', '2014/02/28 12:15:15', '2014.02.28 12:15:15', _
    '2014-02-28T12:15:15', '2014/02/28T12:15:15', '2014.02.28T12:15:15' _
]
Local $bad[15] = [ _
    '2014x02x28', '2014/02/30', '2014.13.28', _
    '0014-02-28 12:15', '2014/00/28 12:15', '2014.02.28 25:15', _
    '2014-02-28R12:15', '2014/02/28T12:65', '2014.02.28T12.15', _
    '2014-02-28 12:15:75', '2014/02/28 12:1x:15', '2014.02.28 12:15:', _
    '2014-02-28_12:15:15', 'Foo', '2014.02.28T99999999' _
]
; test data ends

; Speed tests
$timer = TimerInit()
For $i = 1 To 1000
    For $j = 0 To UBound($good) - 1
        _DateIsValid($good[$j])
    Next
Next
ConsoleWrite("Current good:" & @TAB & TimerDiff($timer) & @LF)
$timer = TimerInit()
For $i = 1 To 1000
    For $j = 0 To UBound($good) - 1
        DateIsValid($good[$j])
    Next
Next
ConsoleWrite("New good:" & @TAB & TimerDiff($timer) & @LF)

$timer = TimerInit()
For $i = 1 To 1000
    For $j = 0 To UBound($bad) - 1
        _DateIsValid($bad[$j])
    Next
Next
ConsoleWrite("Current bad:" & @TAB & TimerDiff($timer) & @LF)
$timer = TimerInit()
For $i = 0 To 1000
    For $j = 0 To UBound($bad) - 1
        DateIsValid($bad[$j])
    Next
Next
ConsoleWrite("New bad:" & @TAB & TimerDiff($timer) & @LF)
$timer = TimerInit()

; Correctness tests
For $i = 0 to UBound($good) - 1
    If Not _DateIsValid($good[$i]) Then
        ConsoleWriteError("_DateIsValid incorrectly flagged " & $good[$i] & " as a bad date" & @LF)
    EndIf
    If Not DateIsValid($good[$i]) Then
        ConsoleWriteError("DateIsValid incorrectly flagged " & $good[$i] & " as a bad date" & @LF)
    EndIf
Next
For $i = 0 to UBound($bad) - 1
    If _DateIsValid($bad[$i]) Then
        ConsoleWriteError("_DateIsValid incorrectly flagged " & $bad[$i] & " as a good date" & @LF)
    EndIf
    If DateIsValid($bad[$i]) Then
        ConsoleWriteError("DateIsValid incorrectly flagged " & $bad[$i] & " as a good date" & @LF)
    EndIf
Next
; Correctness tests end

I hereby release this code under the terms of the WTFPL (http://en.wikipedia.org/wiki/WTFPL) should you decide that the version I wrote is worthy of inclusion in Date.au3 .

[EDIT] OOPS! I didn't realize that I pasted a version with a bug that I had since corrected.  I guess my Ctrl + C didn't work the second time...

Edited February 28, 2014 by mrider

[guinness]

This is the regular expression I came up with sometime ago.

(?x)^\d{4}(?:[.\-/]\d{2}){2}   (?:     (?:T|\h)\d{2}:\d{2}     (?::\d{2})?     )?$

[mrider]

I kind of liked the switch by length version since 1) we get a much less complicated regex, and 2) the switch inherently tests for string length without invoking the regex, and 3) it's much clearer what we're testing with the regex.

However, I'm totally willing to release this into the wild without even acknowledgement, so feel free to modify it if it looks better.

[EDIT] I didn't realize that sticking "B" and ")" together produced this ->

Edited February 28, 2014 by mrider

[guinness]

A couple of comments would alleviate the regexp understanding issue. You can make a Trac request if you feel this should be added to the UDFs.

[mrider]

Well - this will be my 89th post when I'm done, despite having created my account years ago.  So I'm kind of soliciting the opinions of others before I add this to Trac, seeing as how I don't have any "street cred" (for want of a better term).

[EDIT]

I should also add that your regex correctly identifies the pattern of characters, but it does not capture the digits.  So one would have to take additional steps to check e.g.: '2014-12-29'.

Local $sPattern = '(?x)^d{4}(?:[.-/]d{2}){2}   (?:     (?:T|h)d{2}:d{2}     (?::d{2})?     )?$'

Local $sDate = '2014-02-28'

Local $aMatches = StringRegExp($sDate, $sPattern, 1) ; '3' also produces the same result

ConsoleWrite(UBound($aMatches) & @LF)

Writes out "1"

Edited February 28, 2014 by mrider

[guinness]

My regexp is just to check, I didn't add any capturing groups.

[mrider]

Well, it took me a while to cook up a regex that would validate, capture, and add the ability to test milliseconds all on one line.  I used the following (which may not be optimal):

(?x)^(\d{4})[-/.](\d{2})[-/.](\d{2})  (?:   (?:T|\h)(\d{2}):(\d{2})   (?::(\d{2})  (?:\.(\d{3}))? )?  )?$

Using that versus the one that I wrote I got the following timings (where "complex" means the complex regex) on my admittedly underpowered computer:

Current good:    1134.98372218279
Complex good:    884.724858009665
Simple good:    840.117068884218

Current bad:    877.502325158039
Complex bad:    538.502495566972
Simple bad:    453.490679863207
 

So my version is a bit faster, but your regex is all together in one line...

Edited February 28, 2014 by mrider

[guinness]

You should have asked. I would have added capturing for you as well. 

Cheers for testing by the way. I am hoping people will join in the discussion as well and it just won't be you vs me.

[mrider]

You should have asked. I would have added capturing for you as well. 

Cheers for testing by the way. I am hoping people will join in the discussion as well and it just won't be you vs me.

 

I didn't mind adding the capturing.  I don't use regular expressions as often as I would like, so it was good practice.  [EDIT] I should add that what kept tripping me up was that I had to make sure the milliseconds were non-greedy, otherwise the regex would fail on yyyy-MM-dd?HH:mm:ss .  Not quite sure I understand that one.

I also hope others will chime in with some opinions.  For one thing, it wouldn't surprise me in the least to have someone come up with even better code still.  BTW, here's the latest version with the more complex but all-in-one regex:

#include <Date.au3>
; Obviously the previous line isn't needed if this code makes it into the include...

; Don't forget to change the function name to _DateIsValid...
Func DateIsValid($sDate)
    Local $iDateLen = StringLen($sDate)
    Local $sPattern = '(?x)^(\d{4})[-/.](\d{2})[-/.](\d{2})  (?:   (?:T|\h)(\d{2}):(\d{2})   (?::(\d{2})  (?:\.(\d{3}))? )?  )?$'
    Local $aiMatches = StringRegExp($sDate, $sPattern, 1)
    If @error Then
        ; Does not match RE, so it's invalid format
        Return False
    Else
        ; At this point we KNOW we have a year, month and day
        If $aiMatches[0] < 1000 Or $aiMatches[0] > 2999 Then Return False   ; year
        If $aiMatches[1] < 1 Or $aiMatches[1] > 12 Then Return False        ; month
         ; If we're here, then year and month are both valid so we can use them immediately
        $aiDaysInMonth = _DaysInMonth($aiMatches[0])
        If $aiMatches[2] < 1 Or $aiMatches[2] > $aiDaysInMonth[$aiMatches[1]] Then Return False ; day

        ; We may or may not have have hours and minutes
        If $iDateLen > 10 Then
            If $aiMatches[3] > 23 Then Return False ; hour
            If $aiMatches[4] > 59 Then Return False ; minute
            ; We may or may not have seconds or milliseconds
            If $iDateLen > 16 Then
                If $aiMatches[5] > 59 Then Return False ; seconds
                ; No point in testing milliseconds, since we know it's 3 digits...
            EndIf
        EndIf
    EndIf
    Return True
EndFunc

Edited February 28, 2014 by mrider