Daylight saving time and Timezone best practices

Question

I am hoping to make this question and the answers to it the definitive guide to dealing with daylight saving time, in particular for dealing with the actual change overs.

If you have anything to add, please do

Many systems are dependent on keeping accurate time, the problem is with changes to time due to daylight savings - moving the clock forward or backwards.

For instance, one has business rules in an order taking system that depend on the time of the order - if the clock changes, the rules might not be as clear. How should the time of the order be persisted? There is of course an endless number of scenarios - this one is simply an illustrative one.

• How have you dealt with the daylight saving issue?
• What assumptions are part of your solution? (looking for context here)

As important, if not more so:

• What did you try that did not work?
• Why did it not work?

I would be interested in programming, OS, data persistence and other pertinent aspects of the issue.

General answers are great, but I would also like to see details especially if they are only available on one platform.

---

Summary of answers and other data: (please add yours)

Do:

• Always persist time according to a unified standard that is not affected by daylight savings. GMT and UTC have been mentioned by different people, though UTC seems to be mentioned most often.
• Include the local time offset as is (including DST offset) when storing timestamps.
• Include the original timezone name, so you can reconstruct the original time at a later point and display correct offsets if needed.
• Remember that DST offsets are not always an integer number of hours (e.g. Indian Standard Time is UTC+05:30).
• If using Java, use JodaTime. - http://joda-time.sourceforge.net/
• Create a table TZOffsets with three columns: RegionClassId, StartDateTime, and OffsetMinutes (int, in minutes). See answer
• Business rules should always work on civil time (UTC/GMT).
• Internally, keep timestamps in something like civil-time-seconds-from-epoch. See answer.
• Only convert to local times at the last possible moment.
• Remember that timezones and offsets are not fixed and may change.
• Consider the type of time (actual event time, broadcast time, relative time, historical time, recurring time) what elements (timestamp, timezone offset and timezone name) you need to store for correct retrieval - see "Types of Time" in answer.
• Check if your DBMS needs to be shutdown during transition.
• Keep your OS, database and application tzdata files in sync, between themselves and the rest of the world.
• Set hardware clocks and OS clocks to UTC.
• Use NTP services on all servers.
• Store server time, not client time.
• If doing historical auditing store both UTC and local time (this allows exact pinpointing of time, as conversion tables will change).
• If using FAT32, remember that timestamps are stored in local time, not UTC.
• When dealing with recurring events (weekly TV show, for example), remember that the time changes with DST and will be different across time zones.

Don't:

• Do not use javascript date and time calculations in web apps unless you ABSOLUTELY have to.
• Never trust client datetime. It may very well be incorrect.
• Do not compare client datetimes with server datetimes.

Testing:

• When testing make sure you test countries in the Western and Eastern hemispheres, with both DST in progress and not and a country that does not use DST (6 in total).
• Test all third party libraries and applications and make sure they handle timezone data correctly.

Reference:

• Olson database, aka Tz_database - ftp://elsie.nci.nih.gov/pub
• Sources for Time Zone and DST - http://www.twinsun.com/tz/tz-link.htm
• ISO format (ISO 8601) - http://en.wikipedia.org/wiki/ISO_8601
• Mapping between Olson database and Windows TimeZone Ids, from the Unicode Consortium - http://unicode.org/repos/cldr-tmp/trunk/diff/supplemental/windows_tzid.html
• TimeZone page on WikiPedia - http://en.wikipedia.org/wiki/Tz_database
• StackOverflow questions tagged dst - http://stackoverflow.com/questions/tagged/dst
• StackOverflow questions tagged timezone - http://stackoverflow.com/questions/tagged/timezone
• Dealing with DST - microsoft DateTime best practices - http://msdn.microsoft.com/en-us/library/ms973825.aspx#datetime_topic6
• Network Time Protocol on wikipedia - http://en.wikipedia.org/wiki/Network_Time_Protocol

Other:

• Lobby your representative to end the abomination that is DST. We can always hope...

Answer 1

Business rules should always work on civil time (unless there's legislation that says otherwise). Be aware that civil time is a mess, but it's what people use so it's what is important.

Internally, keep timestamps in something like civil-time-seconds-from-epoch. The epoch doesn't matter particularly (I favour the Unix epoch) but it does make things easier than the alternative. Pretend that leap-seconds don't exist unless you're doing something that really needs them (e.g., satellite tracking). The mapping between timestamps and displayed time is the only point where DST rules should be applied; the rules change frequently (on a global level, several times a year; blame politicians) so you should make sure that you do not hard-code the mapping. Olson's TZ database is invaluable.

Answer 2

Always persist the time according to a unified standard, preferably UTC.

UTC is agnostic to daylight saving time and as such it is a good baseline. Once you have that, displaying the proper time according to time zone, DST, etc... is just a matter of applying the proper delta.

Persist globally, display locally. - Me, Now

Answer 3

I recently had a problem in a web app where on an AJAX post-back the datetime coming back to my server-side code was different from the datetime served out.

It most likely had to do with my javascript code on the client that built up the date for posting back to the client as string, because Javascript was adjusting for time zone and daylight savings, and in some browsers the calculation for when to apply daylight savings seemed to be different than in others.

In the end I opted to remove date and time calculations on the client entirely, and posted back to my server on an integer key which then got translated to date time on the server, to allow for consistent transformations.

My learning from this: Do not use javascript date and time calculations in web apps unless you ABSOLUTELY have to.

Answer 4

Keep your servers set to UTC, and make sure they all are configured for ntp or the equivalent.

UTC avoids daylight savings time issues, and out-of-sync servers can cause unpredictable results that take a while to diagnose.

Answer 5

I have hit this on two types of systems, “shift planning systems (e.g. factory workers)” and “gas depend management systems)…

23 and 25 hour long days are a pain to cope with, so are 8hr shifts that take 7hr or 9hr. The problem is you will find that each customers, or even department of the customer have different rules they have created (often without documenting) on what they do in these special cases.

Some questions are best not asked of the customer’s until after they have paid for your “off the shelf” software. It is very rare to find a customer that thinks about this type of issue up front when buying software.

I think in all cases you should record time in UTC and convert to/from local time before storing the date/time. However even know which take a given time is in can be hard with Daylight saving and time zones.

Answer 6

In general, include the local time offset (including DST offset) in stored timestamps: UTC alone is not enough if you later want to display the timestamp in its original timezone (and DST setting).

Keep in mind that the offset is not always an integer number of hours (e.g. Indian Standard Time is UTC+05:30).

For example, suitable formats are a tuple (unix time, offset in minutes) or ISO 8601.

Answer 7

While I haven't tried it, an approach to time zone adjustments I would find compelling would be as follows:

1. Store everything in UTC.

2. Create a table TZOffsets with three columns: RegionClassId, StartDateTime, and OffsetMinutes (int, in minutes).

In the table, store a list of dates and times when the local time changed, and by how much. The number of regions in the table and the number of dates would depend on what range of dates and areas of the world you need to support. Think of this as if it is "historical" date, even though the dates should include the future to some practical limit.

When you need to compute the local time of any UTC time, just do this:

SELECT DATEADD('m', SUM(OffsetMinutes), @inputdatetime) AS LocalDateTime
FROM   TZOffsets
WHERE  StartDateTime <= @inputdatetime
       AND RegionClassId = @RegionClassId;

You might want to cache this table in your app and use LINQ or some similar means to do the queries rather than hitting the database.

This data can be distilled from the public domain tz database [http://www.twinsun.com/tz/tz-link.htm].

Advantages and footnotes of this approach:

1. No rules are baked into code, you can adjust the offsets for new regions or date ranges readily.
2. You don't have to support every range of dates or regions, you can add them as needed.
3. Regions don't have to correspond directly to geopolitical boundaries, and to avoid duplication of rows (for instance, most states in the US handle DST the same way), you can have broad RegionClass entries that link in another table to more traditional lists of states, countries, etc.
4. For situations like the US where the start and end date of DST has changed over the past few years, this is pretty easy to deal with.
5. Since the StartDateTime field can store a time as well, the 2:00 AM standard change-over time is handled easily.
6. Not everywhere in the world uses a 1-hour DST. This handles those cases easily.
7. The data table is cross-platform and could be a separate open-source project that could be used by developers who use nearly any database platform or programming language.
8. This can be used for offsets that have nothing to do with time zones. For instance, the 1-second adjustments that happen from time to time to adjust for the Earth's rotation, historical adjustments to and within the Gregorian calendar, etc.
9. Since this is in a database table, standard report queries, etc. can take advantage of the data without a trip through business logic code.
10. This handles time zone offsets as well if you want it to, and can even account for special historical cases where a region is assigned to another time zone. All you need is an initial date that assigns a time zone offset to each region with a minimal start date. This would require creating at least one region for each time zone, but would allow you to ask interesting questions like: "What is the difference in local time between Yuma, Arizona and Seattle, Washington on February 2, 1989 at 5:00am?" (Just subtract one SUM() from the other).

Now, the only disadvantage of this approach or any other is that conversions from local time to GMT are not perfect, since any DST change that has a negative offset to the clock repeats a given local time. No easy way to deal with that one, I'm afraid, which is one reason storing local times is bad news in the first place.

Answer 8

This is an important and surprisingly tough issue. The fact is that is no foolproof general strategy-stardard for persisting time. For example, the SQL standard and the ISO format (ISO 8601) are clearly not enough.

From the conceptual point of view, one usually deals with two types of time-date data, and it's important to distinguish them (the above standards do not) : "physical time" and "civil time".

A "physical" instant of time is a point in the continuous universal timeline that physics deal with (ignoring relativity, of course). This concept can be adecuately coded-persisted in UTC (if you can ignore leap seconds details) or whatever.

A "civil" time is a time specification according to civil norms: a point of time here is fully specified by the Y,M,D,H,MM,S,FS fields plus a TZ (timezone specification) (and also a "calendar", actually, but lets assume we restrict the discussion to Gregorian calendar). The timezone specification and the callendar allows (in principle) to map from one representation to another. But civil and physical instantes of time are fundamentally different types of magnitudes, and they should be kept conceptually separated and treated/stored differently (i'm speaking of points of times here, roughly the same applies to intervals).

The issue is confusing because we speak of these types events interchangeably, and because the civil times are subject to political changes. The problem (and the need to distinguish these concepts) is specially evident when we consider future events. Example (taken from my discussion here

John records in his calendar a reminder for some event at datetime 2010-Jul-27, 10:30:00, with TZ "Chile/Santiago", (GMT+4 hence it corresponds to UTC time 2010-Jul-27 14:30:00). But some days afterwards, his government decides to change the country TZ to GMT+5.

Now, when the day comes... should that reminder trigger at A) 2010-Jul-27 10:30:00 "Chile/Santiago" = UTC time 2009-Jul-27 15:30:00 or B) 2010-Jul-27 9:30:00 "Chile/Santiago" = UTC time 2009-Jul-27 14:30:00 ?

There is no correct answer, unless one knows what John actually meant when he said "please ring me at "2010-Jul-27, 10:30:00 TZ=Chile/Santiago"

Did he mean a "civil date-time" ("when the clocks in my city tell 10:30")? In that case, A) is the correct answer.

Or did he mean a "physical instant of time", a point in the continuus line of time of our universe, say, "when the next solar eclipse happens". In that case, answer B) is the correct one.

A few Date/Time APIs get this distinction right: among them, Jodatime, which is the foundation of the next (third!) Java DateTime API (JSR 310).

Answer 9

Do: Lobby your representative to end the abomination that is DST.

Results: Has not worked yet. Reasons unknown.

Answer 10

If you happen to maintain database systems that are running with DST active, check carefully whether they need to be shut down during the transition in fall. Mandy DBS (or other systems as well) don't like passing the same point in (local) time twice, which is exactly what happens when you turn back the clock in fall. SAP has solved this with a (IMHO really neat) workaround - instead of turning back the clock, they just let the internal clock run at half the usual speed for two hours...

Answer 11

You need to know about the Olson database, which is available from ftp://elsie.nci.nih.gov/pub. It is updated multiple times per year to deal with the often last-minute changes in when (and whether) to switch between winter and summer (standard and daylight saving) time in different countries around the world. We're about one quarter into 2010; the current release is 2010f, the sixth of the year. Last year, the releases got up to at least 2009s (I have a copy of that). Note that there is a set of code to manage the data and the actual time zone data itself, in two separate archives (tzcode20xxy.tar.gz and tzdata20xxy.tar.gz). Both code and data are in the public domain.

This is the source of time zone names such as America/Los_Angeles (and synonyms such as US/Pacific).

If you need to keep track of different zones, then you need the Olson database. As others have advised, you also want to store the data in a fixed format - UTC is normally the one chosen - along with a record of the time zone in which the data was generated. You may want to distinguish between the offset from UTC at the time and the time zone name; that can make a difference later. Also, knowing that it is currently 2010-03-28T23:47:00-07:00 (US/Pacific) may or may not help you with interpreting the value 2010-11-15T12:30 - which is presumably specified in PST (Pacific Standard Time) rather than PDT (Pacific Daylight Saving Time).

The standard C library interfaces are not dreadfully helpful with this sort of stuff.

Answer 12

Crossing the boundary of "computer time" and "people time" is a nightmare. The main one being that there is no sort of standard for the rules governing timezones and daylight saving times. Countries are free to change their timezone and DST rules at any time, and they do.

Some countries e.g. Israel, Brazil, decide each year when to have their daylight saving times, so it is impossible to know in advance when (if) DST will be in effect. Others have fixed(ish) rules as to when DST is in effect. Other countries do not use DST as all.

Timezones do not have to be full hour differences from GMT. Nepal is +5.45. There are even timezones that are +13. That means that:

SUN 23:00 in Howland Island (-12)
MON 11:00 GMT 
TUE 00:00 in Tonga (+13)

are all the same time, yet 3 different days!

There is also no clear standard on the abbreviations for timezones, and how they change when in DST so you end up with things like this:

AST Arab Standard Time     UTC+03
AST Arabian Standard Time  UTC+04
AST Arabic Standard Time   UTC+03

The best advice is to stay away from local times as much as possible and stick to UTC where you can. Only convert to local times at the last possible moment.

When testing make sure you test countries in the Western and Eastern hemispheres, with both DST in progress and not and a country that does not use DST (6 in total).

Answer 13

I'm not sure what I can add to the answers above, but here are a few points from me:

Types of times

There are four different times you should consider:

1. Event time: eg, the time when an international sporting event happens, or a coronation/death/etc. This is dependent on the timezone of the event and not of the viewer.
2. Television time: eg, a particular TV show is broadcast at 9pm local time all around the world. Important when thinking about publishing the results (of say American Idol) on your website
3. Relative time: eg: This question has an open bounty closing in 21 hours. This is easy to display
4. Recurring time: eg: A TV show is on every Monday at 9pm, even when DST changes.

There is also Historic/alternate time. These are annoying because they may not map back to standard time. Eg: Julian dates, dates according to a Lunar calendar on Saturn, The Klingon calendar.

Storing start/end timestamps in UTC works well. For 1, you need an event timezone name + offset stored along with the event. For 2, you need a local time identifier stored with each region and a local timezone name + offset stored for every viewer (it's possible to derive this from the IP if you're in a crunch). For 3, store in UTC seconds and no need for timezones. 4 is a special case of 1 or 2 depending on whether it's a global or a local event, but you also need to store a created at timestamp so you can tell if a timezone definition changed before or after this event was created. This is necessary if you need to show historic data.

Storing times

• Always store time in UTC
• Convert to local time on display (local being defined by the user looking at the data)
• When storing a timezone, you need the name, timestamp and the offset. This is required because governments sometimes change the meanings of their timezones (eg: the US govt changed DST dates), and your application needs to handle things gracefully... eg: The exact timestamp when episodes of LOST showed both before and after DST rules changed.

Offsets and names

An example of the above would be:

The soccer world cup finals game happened in South Africa (UTC+2--SAST) on July 11, 2010 at 19:00 UTC.

With this information, we can historically determine the exact time when the 2010 WCS finals took place even if the South African timezone definition changes, and be able to display that to viewers in their local timezone at the time when they query the database.

System Time

You also need to keep your OS, database and application tzdata files in sync, both with each other, and with the rest of the world, and test extensively when you upgrade. It's not unheard of that a third party app that you depend on did not handle a TZ change correctly.

Make sure hardware clocks are set to UTC, and if you're running servers around the world, make sure they're OSes are configured to use UTC as well. This becomes apparent when you need to copy hourly rotated apache log files from servers in multiple timezones. Sorting them by filename only works if all files are named with the same timezone. It also means that you don't have to do date math in your head when you ssh from one box to another and need to compare timestamps.

Also, run ntpd on all boxes.

Clients

Never trust the timestamp you get from a client machine as valid. For example, the Date: HTTP headers, or a javascript Date.getTime() call. These are fine when used as opaque identifiers, or when doing date math during a single session on the same client, but don't try to cross-reference these values with something you have on the server. Your clients don't run NTP, and may not necessarily have a working battery for their BIOS clock.

Trivia

Finally, governments will sometimes do very weird things:

Standard time in the Netherlands was exactly 19 minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through 1937-06-30. This time zone cannot be represented exactly using the HH:MM format.

Ok, I think I'm done.

Answer 14

Make clear architectural separation of concerns - to know exactly which tier intereacts with users, and has to change date-time for/from cannonical representation (UTC). Non-UTC date-time is presentation (follows users local timezone), UTC time is model (remains unique for back-end and mid tiers).

Also, decide what's your actual audience, what you don't have to serve and where do you draw the line. Don't touch exotic callendars unless you actually have important customers there and then consider separate user-facing server(s) just for that region.

If you can acquire and maintain user's location, use location for systematic date-time conversion (say .NET culture or a SQL table) but provide a way for end-user to choose overrides if date-time is critical for your users.

If there are historical audit obligations involved (like telling exactly when Jo in AZ paid a bill 2 yrs ago in September) then keep both UTC and local time for the record (your coversion tables will change in a course of time).

Define the time referential time zone for data that comes in bulk - like files, web services etc. Say East Coast company has data center in CA - you need to ask and know what they use as a standard instead of assuming one or the other.

Don't trust time-zone offsets embedded in textual representation of the date-time and don't accept to parse and follow them. Instead always request that time zone and/or refernece zone have to be explicitly defined. You can easily receive time with PST offset but the time is actually EST since that's cleints's reference time and records were just exported at a server which is in PST.

Answer 15

One other thing, make sure the servers have the up to date daylight savings patch applied.

We had a situation last year where our times were consistently out by 1 hour for a 3 week period for north american users, even though we were using a UTC based system.

Turns out in the end it was the servers.

They just needed an up to date patch applied (server 2003)

Answer 16

Be careful when dealing with timestamps stored in the FAT32 filesystem - it is always persisted in local time coordinates (which include DST - see msdn article). Got burned on that one.

Answer 17

If your design can accommodate it, avoid local time conversion all together!

I know to some this might sound insane but think about UX: users process near, relative dates (today, yesterday, next Monday) faster than absolute dates (2010.09.17, Friday Sept 17) on glance. And when you think about it more, the accuracy of timezones (and DST) is more important the closer the date is to now(), so if you can express dates/datetimes in a relative format for +/- 1 or 2 weeks, the rest of the dates can be UTC and it wont matter too much to 95% of users.

This way you can store all dates in UTC and do the relative comparisons in UTC and simply show the user UTC dates outside of your Relative Date Threshold.

This can also apply to user input too (but generally in a more limited fashion). Selecting from a drop down that only has { Yesterday, Today, Tomorrow, Next Monday, Next Thursday } is so much simpler and easier for the user than a date picker. Date pickers are some of the most pain inducing components of form filling. Of course this will not work for all cases but you can see that it only takes a little clever design to make it very powerful.