Jump to content

FileGetTime() uses an improper TZ+DST shift at today's date


Go to solution Solved by LaurentC,

Recommended Posts

Hi

FileGetTime() uses an improper TZ+DST shift at today's date ...
... and not a TZ+DST shift at the date of the file itself !
Thus, +/-1h error on the time...

Here a sample :
Filename                                   : 20190115_180000_test.jpg
FileGetTime                              : 20190115190000

And the display file properties inside Windows 10 shows the correct 18:00 for Date Modified and not the AutoIt erroneous 19:00...   🙄  
That causes errors when setting faraway date/time with a string to a file with FileSetTime !

Regards

Link to post
Share on other sites
  • Replies 46
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

Absolute time doesn't exist and every piece of software chooses it's starting point, e.g. Windows, Unix, Excel, you-name-it. And as shown, local time poison gets in the way. Even zulu time isn't perfe

Off-Topic : Here a list of user groups and rights : https://www.autoitscript.com/forum/topic/37739-forum-information-and-faqs/ Excerpt (modified)  : Possibly of interest for y

Are you sure?

Local $sFile = _TempFile()
FileWrite($sFile, "grnbuigrenpim")
ConsoleWrite(FileGetTime($sFile, 0, 1) & @LF)

Works as intended for me.

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites
  • Moderators

jchd,

As I read the OP the problem is that the function uses the current timezone difference and not the difference at the time of file creation. The timestamps he shows (2019...) offer support to this interpretation.

M23

Public_Domain.png.2d871819fcb9957cf44f4514551a2935.png Any of my own code posted anywhere on the forum is available for use by others without any restriction of any kind

Open spoiler to see my UDFs:

Spoiler

ArrayMultiColSort ---- Sort arrays on multiple columns
ChooseFileFolder ---- Single and multiple selections from specified path treeview listing
Date_Time_Convert -- Easily convert date/time formats, including the language used
ExtMsgBox --------- A highly customisable replacement for MsgBox
GUIExtender -------- Extend and retract multiple sections within a GUI
GUIFrame ---------- Subdivide GUIs into many adjustable frames
GUIListViewEx ------- Insert, delete, move, drag, sort, edit and colour ListView items
GUITreeViewEx ------ Check/clear parent and child checkboxes in a TreeView
Marquee ----------- Scrolling tickertape GUIs
NoFocusLines ------- Remove the dotted focus lines from buttons, sliders, radios and checkboxes
Notify ------------- Small notifications on the edge of the display
Scrollbars ----------Automatically sized scrollbars with a single command
StringSize ---------- Automatically size controls to fit text
Toast -------------- Small GUIs which pop out of the notification area

 

Link to post
Share on other sites
  • Developers
Posted (edited)
1 hour ago, LaurentC said:

FileGetTime() uses an improper TZ+DST shift at today's date ...
... and not a TZ+DST shift at the date of the file itself !

The FileGetTime() retrieves the Date/Time stamp of the creation time and then convert to Local time with FileTimeToLocalFileTime().
I tested with a picture created in the USA EST and the returned time is indeed 6 hours later as I am in CET.

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

Jos

Edited by Jos

SciTE4AutoIt3 Full installer Download page   - Beta files       Read before posting     How to post scriptsource   Forum etiquette  Forum Rules 
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Link to post
Share on other sites
Posted (edited)

Ah OK, didn't see where the issue was.

Unfortunately TZ+DST offsets are a huge can of worms. Not only DST are changing over time due to political decisions based on thin air, they also change over places which decide (again due to "administrative" reasons) to belong this side or the other side of a TZ boundary, which themselves are moved from times to times after "international" consensus or disagreement.

Furthermore, OS versions evolving, they aren't always consistent: DOS file times are not consistent with current Windows versions.

Hence computing the correct TZ+DST in the past is a nightmare.

The page https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times explains what a Windows filetime is:

 

A file time is a 64-bit value that represents the number of 100-nanosecond intervals that have elapsed since 12:00 A.M. January 1, 1601 Coordinated Universal Time (UTC). The system records file times when applications create, access, and write to files.

The NTFS file system stores time values in UTC format, so they are not affected by changes in time zone or daylight saving time. The FAT file system stores time values based on the local time of the computer. For example, a file that is saved at 3:00pm PST in Washington is seen as 6:00pm EST in New York on an NTFS volume, but it is seen as 3:00pm EST in New York on a FAT volume.

Time stamps are updated at various times and for various reasons. The only guarantee about a file time stamp is that the file time is correctly reflected when the handle that makes the change is closed.

Not all file systems can record creation and last access times, and not all file systems record them in the same manner. For example, the resolution of create time on FAT is 10 milliseconds, while write time has a resolution of 2 seconds and access time has a resolution of 1 day, so it is really the access date. The NTFS file system delays updates to the last access time for a file by up to 1 hour after the last access.

Read the full MS page for details. At the end of the day, you can see why it's almost impossible for any OS to go back in the past and compute what TZ and DST were in force when an event occured in the past.

Edited by jchd

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites

Look at how complex things are in practice: the tz database is modified several times a year and it's really a huge thing!

https://en.wikipedia.org/wiki/Tz_database

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites
  • Developers

Edited my comments a bit:

25 minutes ago, Jos said:

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

 

SciTE4AutoIt3 Full installer Download page   - Beta files       Read before posting     How to post scriptsource   Forum etiquette  Forum Rules 
 
Live for the present,
Dream of the future,
Learn from the past.
  :)

Link to post
Share on other sites
Hi Jos — Jchd — Melba23
 
I did not imagine that so many answers will come so quickly with so much insight : incredible !
 
1/
The fact is that before W7 with WinXP and older OS, the DST date/time was in accordance to the Windows properties displayed, as said in comments above
With W7 to W10, there is a change that corrects the properties displayed by Windows to be in coherency with the date of the filename, but yet not true for the old DOS DIR command...   (I don't know for PowerShell...)  
Thus, Windows 10 displays 18:00 for a JPG tagged 1800 = Good from the user point of view now !
 
2/
Quote

Jos : So in your case: the picture was taken during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken.

 

Yes, it's exactly the subject   😉  
This was, let's say, OK in the past with old DOS/Windows settings, but NOK with regards to Windows new behavior
So, Jos, you're definitely right in your explanations
The fact is at time of the 20190115_180000_test.jpg picture, the time was really 18:00, and now, Windows displays 18:00, but FileGetTime shows 19:00 using current DST
Not understandable from the user's point of view since W7...
 
3/
But, as said above in the link proposed : https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times
I understand this is certainly a pure Windows mismatch !
These AutoIt functions only reflect Windows functions : these are not original written AutoIt libraries, but a useful mapping to Windows API (?).
If FileGetTime is a pure mapping of Win32 DLL functions and shall be kept as this, Jos, I understand your proposition is becoming even the rule...   😏  
 
4/ Question : Jos, does FileGetTime is a pure Windows library mapping in AutoIt or is that adjustable ?
 
5/ Question : Without even discussing code correction, would it be possible, at least, to add a remark in the AutoIt Help AutoIt.chm ?
Quote

 

...
FindFirstFile retrieves the local time from the FAT file system and converts it to UTC by using the current settings for the time zone and daylight saving time. Therefore, if it is daylight saving time, FindFirstFile takes daylight saving time into account, even if the file time you are converting is in standard time.
...

 

 

This will help AutoIt users not to wrongly manipulates dates with FileSetTime()...  
I would have been happy last week to read these forum  explanations in the documentation of AutoIt take care of DST...
 
Regards
Link to post
Share on other sites

Absolute time doesn't exist and every piece of software chooses it's starting point, e.g. Windows, Unix, Excel, you-name-it. And as shown, local time poison gets in the way. Even zulu time isn't perfect: not well defined before 1960 and even then the question remains: how to deal with leap seconds?

I agree that things aren't simple for the unsuspecting casual user and as always the devil is in the details.

Granted, a short note or warning could be beneficial in the help but that's as true for almost any language/system where the details are implied or just untold.

To make things even more subtle, time and time differences (durations) aren't what most people think they are: time can only be well defined inside a referential in space-time. Due to relativistic effects, your time isn't mine when one of us changes position in space. Two very precise synchronized cesium fountain atomic clocks get out of sync if one is raised only 50cm and reset at it's former position! The latest atomic clocks are very precise: a research institute in Spain has built a clock whose max drift is 0.263 second in 16.8 billion years. GPS wouldn't work if relativity wasn't taken into account: max precision would then be around 15km, making the whole system essentially useless!

So yes, time is "simple" for most of us in everyday's life (as teached in kindergardens), but once you start looking closer under the hood, horrible complexity shows its face.

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites
Hi Jchd
 
I like science and I just can say how right you are ; the time is very subtle and even the time essence itself is not really known in physics
 
As far are today's software is concerned with the existing API, it is yet possible to give simple and easy ways to manage time in 99.9% of cases
 
What is disconcerting for a not-expert user is to take the date/time of 20190115_180000_test.jpg ; to put it in FileSetTime() that is the principal function provided in AutoIt, and to see 19:00 inside Windows in the file properties !?
In my case, this is 50% of errors to change date/time of files in the year for the user...
This is also not consistent now with the Windows user's experience...
 
I'm not a Windows developer, so I didn't know for example that AutoIt was giving an exact mapping of all internal Windows functions in its CORE Library
For example, with FileFindFirstFile and FileGetTime that are found in the Win32 API
And it's very interesting to go to the Windows detailed explanations when needed
I will have discovered the DST mess with : https://docs.microsoft.com/en-us/windows/win32/sysinfo/file-times
Thus, just a short note in this case in the FileGetTime AutoIt Help with a link to Win32 API File Times about DST would have been very helpful 
The AutoIt Function Reference could also refer to Win32 API with a link to propose additional information for those who could need it
Furthermore that AutoIt is already a very very well documented dev tool   😊  
 
I know now how to bypass with complex functions the "not-really-bug" I encountered with FileGetTime(), but the top of the top will be a function added to the CORE Library named FileGetTimeDST() or FileGetTime2() to propose the adjusted DST fix in a very simple way   😄  
 
What do you think ?
 
Regards
Link to post
Share on other sites

It won't work in the general case.

Pictures taken with your smartphone during your 2007 hollidays in Australia will be marked with the UTC. Same for those taken during your stay in Chile in 2013. Which filetime will you show?

To show a local time you have to decide which one: the local time of the picture at the place it was taken or the then local time of the device you use to watch them? Anyway, you have to embark a tz database and maintain it several times a year, plus know which place each picture was taken. Most Unices do that for you but Windows won't take care of all this by itself.

We human face a dilemna: suppose you take a photo with your smartphone in London at some time UTC (GMT+0). You fly to Australia and look at your photo. Your smartphone is smart enough and well connected, so it has changed timezone and knows which local time shift it has to compensate for in Australia. Which time should it show? The London time or the Aussie backward time?

In both cases there is an issue. Either you have to remember for every file where it was created to interpret time correctly (locally from the place of origin) or you can have situations where time goes "backward", where file A created before file B in UTC is displayed as more recent than B in current local timeframe where you have moved. That's a very strong argument for those who advocate dropping TZ and DST to use UTC only everywhere.

This wonderful site allows debugging and testing regular expressions (many flavors available). An absolute must have in your bookmarks.
Another excellent RegExp tutorial. Don't forget downloading your copy of up-to-date pcretest.exe and pcregrep.exe here
RegExp tutorial: enough to get started
PCRE v8.33 regexp documentation latest available release and currently implemented in AutoIt beta.

SQLitespeed is another feature-rich premier SQLite manager (includes import/export). Well worth a try.
SQLite Expert (freeware Personal Edition or payware Pro version) is a very useful SQLite database manager.
An excellent eBook covering almost every aspect of SQLite3: a must-read for anyone doing serious work.
SQL tutorial (covers "generic" SQL, but most of it applies to SQLite as well)
A work-in-progress SQLite3 tutorial. Don't miss other LxyzTHW pages!
SQLite official website with full documentation (may be newer than the SQLite library that comes standard with AutoIt)

Link to post
Share on other sites

Yes

Right

But I didn't mean to correct all date/time possible issues, even for distant travelers

I said, as far are today's software is concerned with the existing API : that means TZ in Windows for the current user in its country zone

Unless Windows is able to get EXIF geo-info from pictures to choose the traveled TZ in the API : not possible just now

That way, even in our own local time zone, the FileGetTime does not display a time in coherency with what is displayed locally in Windows

As a user in my own country, even my local time is not correct with FileGetTime...

Regards

 

PS : That's a very strong argument for those who advocate dropping TZ and DST to use UTC only everywhere = That's true, but it is not used/displayed that way in smartphones and Windows = For a future implementation   😏   by Microsoft & all...

 

Link to post
Share on other sites
Posted (edited)

  

 

On 5/25/2021 at 1:43 PM, Jos said:

The FileGetTime() retrieves the Date/Time stamp of the creation time and then convert to Local time with FileTimeToLocalFileTime().
I tested with a picture created in the USA EST and the returned time is indeed 6 hours later as I am in CET.

So in your case: the picture was taking during Daylight saving so this all makes perfect sense as the function returns the current localtime never the original time at the time the picture was taken. 

Jos

 

Hi Jos

After all the previous discussions, we must forget my sample 20190115_180000_test.jpg : just a sample and not an essay on picture geo-localization

It's not a question for photos, but for all files on my Windows !

If I want to set 2021 15 01 18:00 to my files with FileSetTime(), it will put 19:00 visible in the file properties with Windows : that's the real bad trick here...

Ask me if all my peregrinations are unclear   🤔  

Regards

Edited by LaurentC
Link to post
Share on other sites

I think it’s worth recognizing all the outstanding effort in time zone accommodations that various operating systems have put forward over the years, even if not perfect, to try to help us humans with our peculiar need to localize time based on space.

Aliens are laughing at us right now :)

Code hard, but don’t hard code...

Link to post
Share on other sites

You're not the forum master for nothing 😀 !

Difficult to say something else than well noted and I shall try !! 

I appreciate your answer to position myself for the next steps... 

Anyway, I need to finalize my own function to achieve the result 😉 

Regards 

(Sorry for the English = I'm just a froggy French...)

Link to post
Share on other sites
52 minutes ago, LaurentC said:

Sorry for the English = I'm just a froggy French...

In that case, I would just mention that the word peregrination, is not very common in modern English, and therefore its usage might leave the recipient at least fartumult if not outright farmished; or worse yet invite a retaliatory strike of comparable  inscrutables, due to the embarrassment of the recipient’s ignorance of the word peregrination.  Like in my case.

OTOH, if you were trying to be funny - good job :)

Code hard, but don’t hard code...

Link to post
Share on other sites

Too funny 

Interesting for me 

I like old school terms, that's true 

Really, I'm a little old school, exact 

And, i am funny, well I hope so, at least, I try to be funny 

Regards 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...