NTP, UTC, and Working with Time on Linux
Mike Chirico (mchirico@users.sourceforge.net or mchirico@comcast.net) What time will it be 4 years from now?It is a trick question. There is NO way to calculate what time it will be (what time the clocks will read), to the nearest half second, exactly four years into the future. Are the days getting longer?Yes. There used to be 86,400 seconds in a day; but, that was back in 1820. Because the earth's rotation is slowed by ocean currents and tidal friction, the number of seconds in a day is slightly greater than 86,400. How many seconds in a minute?61, 60 and possibly 59 seconds in a minute. When a leap second is added there are 61 seconds in a minute. There have been 22 leaps seconds. The last one was 01-01-099. Can we go back in time?YES. Every time there is a leap second we go back in time. Time-Servers use a 64 bit format called the NTP timestamp. This format is divided into 2 parts. The first 32 bits are the number of seconds since January 1, 1990 (GMT), and the second 32 bits specify the fraction of a second. During a leap second, like the one that occurred between December, 31 1998 and January 1, 1999, we went back in time. In order to see this take a close look at the following: Note, starting in row 2, the number of NTP Seconds is 3,124,137,600, the fractional part, not shown, continues counting upward until it is time to rollover to the next second. But, look at row 3 in the NTP Seconds column. The number of seconds does not change. These NTP Seconds are coming from the master clock. The computer, if it is running a current version of the Linux kernel will account for this second by hitting 23:59:60, as shown in the Time column. Intelligent Alien Life?Today's cesium master clocks, and hydrogen master clocks will loose less than a second every 1.4 million years. Nothing found in nature produces such a precise signal. Nothing even comes close: certainly not the rotation of the earth. Some form of intelligent modification, at the atomic level, something beyond random chance is needed to continuously produce such precise signal intervals. Major definitions of time.Atomic Time - or TAI, International Atomic Time scale. This time scale uses the specific definition of a second: 9,192,631,770 cycles between two hyperfine levels of the ground state of cesium 133. Universal Time - or UT is counted from 0 hours at midnight, with the unit of duration the mean solar day. There are 2 subdivision of UT.
Coordinated Universal Time - or (UTC). This is time we live by. It's the time of your computer, if it is set correctly. UTC is always kept to within 0.9 seconds of UT1 by the introduction of a "leap second", which was occurring roughly every 500 days. However, it's been over a 1000 days since the last leap second. Special note, GMT (Greenwich Mean Time) is often used to mean UTC. The names are interchangeable, but UTC is the correct modern definition. However, GMT is still in heavy use.
The Date CommandYou can get the date and time with the following command.
$ date
Tue Aug 31 14:09:02 EDT 2004
And, you can modified the data format in a number of ways. This example shows a few of the options:
$ date "+%m%d%y %A,%B %d %Y %X"
083104 Tuesday,August 31 2004 02:10:04 PM
If you are creating a lot of log files or backups, where sorting by the filename may be important, then, consider the following format "+%y%m%d", (year, month, day). For example, if you executed the following command below, you would backup everyting in the ./test directory into a filename starting with the current date.
$ tar -czf `date "+%y%m%d_testbackup.tar.gz"` ./test
$ ls *_testbackup.tar.gz|sort
040829_testbackup.tar.gz
040830_testbackup.tar.gz
040831_testbackup.tar.gz
You can also dispaly the time, say, two years from now. Or use multiple combinations of plus or minus with year, month, week, day, hour, minute and second. Of course, if during that interval a leap second is introduced, or possibly subtracted, then the predicted time will be off by the number of leap seconds. There is no precise way to calculate when UTC will differ from UT1 by +-0.9 seconds.
$ date -d '+2 year +1 month -1 week +3 day -8 hour +2 min -5 seconds'
Wed Sep 27 06:28:12 EDT 2006
So how far can you use date to display times into the future? Currently, you can use the date command to calculate values 30 years into the future and a bit further. However, going forward 34 years fails, unless you are running on a 64-bit system. In fact, the last possible time that can be displayed is January 18, 2038, sometime after 22:14:07 EST, or 01-19-2038 GMT. You can calculate this number exactly by adding 2^31-1 seconds to unix epoch which is (January 1, 1970 UTC). The largest positive value that can be represented by a 32 bit number is 2^31-1, and on the Intel platform time is stored in a 32 bit field. So, you can calculate the earliest internal date by subtracting 2^31-1 seconds from epoch which would give you (12-13-1901 20:45:53 UTC). You can also use the date command to calculate dates and times for any timezone by modifying the TZ environment variable. For instance, to find the current time in New Zealand, you would execute the following:
$ export TZ=NST
$ date
Like the NTP format, dates are always stored internally as the number of seconds in UTC time, then, converted to the timezone specified in the TZ environment variable. If you unset the TZ variable, then, the timezone is obtained from the binary file /etc/localtime, which contains the timezone, offset from UTC, whether daylight saving time (dst) is in effect, and when dst begins and ends. Or, you can always get the date command to give UTC, using the -u option
$ date -u
Sun Aug 22 02:06:26 UTC 2004
The TZ environment variable can be assigned any of the standard three-letter abbreviations below. However, if you make a mistake and initialize it to something that is not on the charts, you will get GMT, and no error will be returned.
Hours From Greenwich Mean Time (GMT) Value
0 GMT Greenwich Mean Time
+1 ECT European Central Time
+2 EET European Eastern Time
+2 ART
+3 EAT Saudi Arabia
+3.5 MET Iran
+4 NET
+5 PLT West Asia
+5.5 IST India
+6 BST Central Asia
+7 VST Bangkok
+8 CTT China
+9 JST Japan
+9.5 ACT Central Australia
+10 AET Eastern Australia
+11 SST Central Pacific
+12 NST New Zealand
-11 MIT Samoa
-10 HST Hawaii
-9 AST Alaska
-8 PST Pacific Standard Time
-7 PNT Arizona
-7 MST Mountain Standard Time
-6 CST Central Standard Time
-5 EST Eastern Standard Time
-5 IET Indiana East
-4 PRT Atlantic Standard Time
-3.5 CNT Newfoundland
-3 AGT Eastern South America
-3 BET Eastern South America
-1 CAT Azores
Interesting note: take a look at the following. I am currently in in EST, Eastern Standard Time adjusted to EDT, or Eastern Daylight Time.
$ date
Tue Aug 31 14:59:54 EDT 2004
$ export TZ=HST
$ date
Tue Aug 31 09:00:16 HST 2004
$ export TZ=EST
$ date
Tue Aug 31 14:00:38 EST 2004
Note the last time is off by an hour. It should have been 15:00:38. Daylight savings time was not taken into account. This can be corrected by setting TZ as follows:
TZ=<timezone><hour offset from UTC><dst timezone>.
$ export TZ=EST05EDT
$ date
Tue Aug 31 15:01:50 EDT 2004
EST was taken from the chart below. Five hours has to be added to get UTC, hence the 05. And currently EDT, Eastern Daylight, is in effect.
DST timezones
0 BST for British Summer.
+400 ADT for Atlantic Daylight.
+500 EDT for Eastern Daylight.
+600 CDT for Central Daylight.
+700 MDT for Mountain Daylight.
+800 PDT for Pacific Daylight.
+900 YDT for Yukon Daylight.
+1000 HDT for Hawaii Daylight.
-100 MEST for Middle European Summer,
MESZ for Middle European Summer,
SST for Swedish Summer and FST for French Summer.
-700 WADT for West Australian Daylight.
-1000 EADT for Eastern Australian Daylight.
-1200 NZDT for New Zealand Daylight.
Was there an easier way to get back to my correct time? Yes, just unset the TZ environment variable.
$ unset TZ
$ date
Sun Aug 22 10:17:35 EDT 2004
Note, even though TZ was adjusted for daylight saving time, will you get the correct time 5 months from now? When does daylight saving time go into effect? The TZ value shown below adjust for dst, only during the correct dates. For instance, this entry goes into effect April, the first week, at 2am, and ends October the 5th week, at 2am. Note 10.5.0 stands for the 5th week in October, and not the 5th day.
$ export TZ=EST+5EDT,M4.1.0/2,M10.5.0/2
However, there is an easier way for you to make timezone changes, that will correctly account for dst, including when dst begins and ends. Here are a few examples.
$ export TZ=:/usr/share/zoneinfo/posix/America/Aruba
$ export TZ=:/usr/share/zoneinfo/Egypt
Need more? Take a look at filenames under /usr/share/zoneinfo
$ ls /usr/share/zoneinfo
Africa Chile Factory Iceland Mexico posix UCT
America CST6CDT GB Indian Mideast posixrules Universal
Antarctica Cuba GB-Eire Iran MST PRC US
Arctic EET GMT iso3166.tab MST7MDT PST8PDT UTC
Asia Egypt GMT0 Israel Navajo right WET
Atlantic Eire GMT-0 Jamaica NZ ROC W-SU
Australia EST GMT+0 Japan NZ-CHAT ROK zone.tab
Brazil EST5EDT Greenwich Kwajalein Pacific Singapore Zulu
Canada Etc Hongkong Libya Poland SystemV
CET Europe HST MET Portugal Turkey
Before 1904 and after 2038?
So how do you get dates and times before 1904 or after 2038?There is a popular open source package, Date-Calc by Steffen Beyer that will handle this. http://www.engelschall.com/u/sb/download/pkg/Date-Calc-5.3.tar.gz This package can be installed and used as a Perl module or, since the following 3 source files are included: DateCalc.c, DateCalc.h and ToolBox.h. It is possible to use Beyer's functions directly in a C program. First, it is helpful to get a closer look at some of the date and time commands using a simple C program. The following example will show the date and time to the microsecond, without Beyer's routines. Compiling and running the program above, is demonstrated below. The output of the time is the timezone for the current system. Or, like the data command, changing the TZ environment variable gives you the option to get the time for Any timezone. It is possible to verify EPOC: 01-01-1970 00:00:00 by making the following changes to the program, and recompiling. Now, run the changes, first setting TZ to GMT. Since 2^31-1 = 2147483647, which is the maximum positive value that can be expressed by a 32 bit number. If we set curtime to this value, and then set it to -2147483647, we will get the maximum and minimum values for time on a 32 bit system. Here is the output. Now we will take the original program and add in Beyer's routines. There are 2 includes "ToolBox.h" and "DateCalc.h". It is also necessary to define year, month, day, hour etc. as Z_int, as shown below. The above program will give the following output, which is 500 years into the future. DateCalc has many other functions, such as counting the number of days until Easter, calculating the day Easter falls on. Note Easter is always a Sunday. "DateCalc_monday_of_week" will determine when Monday occurs given the week and year. For instance, take a look at the layout of this function. In the comments, Beyer uses an "I" for required input, an "0" for derived output, and I/O for required input that may be modified. For instance, the first week in 2004 starts on a Thursday; therefore, the first Monday is in 2003. There are many functions. More examples can be run downloading the progam examples at the end of this article. Installing and Configuring NTPNTP or Network Time Protocol software can be downloaded from http://www.ntp.org/downloads.html The ntpd program runs as a daemon making continuous adjustments to your computer's System Time by sampling the time from 1 or more (preferably 3) Time-Servers. The correct time is calculated by figuring out the network delay from a series of queries to the Time-Servers. Then factoring in this delay to calculate the correct time. The NTP program will deliver accuracy to within 1-50ms, which is dependent upon the network path to the Time-Server and the Time-Server itself. For a workstation, you should use (stratum 2) Time-Servers. NOTE: If your computer clock differs by more than 1000s, then, the ntpd daemon will not start, instead, it will enter panic mode and exit. Therefore, on boot-up it is important to query a Time-Server for the time, using the ntpdate command or first run ntpd with the -g option. Red Hat and Fedora will run ntpdate against any Time-Server listed in the "/etc/ntp/step_tickers" file. So Time-server entries must be listed /etc/ntp.config, these are the Time-servers that are queried when ntpd is running; but, on startup at least one Time-Server must be list in /etc/ntp/step_tickers, for time initialization. All Time Servers give UTC time. In other words, you never have to worry about what timezone the Time Server is in; but, you want to pick the Time-Server that is has the smallest network distance. NTP on Red Hat and FedoraIf you are running Red Hat or Red Hat's Fedora it is recommended that you use their version of NTP, since its been modified to switch from the root account to the user NTP after startup. In addition, when your startup script runs it will automatically read entries in /etc/ntp/step-tickers to initialize the hardware clock. Again, on startup if your computer's time is off by more than 1000s, which would happen if the computer was powered off and the battery was removed,the ntpd daemon will enter a panic mode and exit. STEP 1: Find three or four public (stratum 2) Time-Servers near you. Stratum 2 servers are better suited for workstations, and should have better response compared to a stratum 1 server, which will not give priority to anonymous workstations. You can find a listing of servers at the following link: http://www.eecis.udel.edu/~mills/ntp/clock2b.html STEP 2: Specify the Time-Servers and restrict the access of these server. You computer can query the time from these stratum servers and set the time correctly based on the best server. However, since access is restricted, these time servers cannot initiate a time change on your compuer. For a workstation, you would configure your file as follows:
$ cat /etc/ntp.conf
# A very simple client-only NTP configuration.
server ntp-1.cede.psu.edu
restrict 146.186.218.60
server timeserver1.upenn.edu
restrict 128.91.2.13
server clock.psu.edu
restrict 128.118.25.3
driftfile /etc/ntp/drift
authenticate no
STEP 3: Create entries in the /etc/ntp/step-tickers file. Pick a server that is close to you for the initial time set on boot-up. Again, this will keep the ntpd daemon from entering panic mode on startup.
$ cat /etc/ntp/step-tickers
timeserver1.upenn.edu
clock.psu.edu
Above, two entries are shown in this file. STEP 4: Start or restart the ntpd program. You will need to do this as root.
# /etc/init.d/ntpd restart
Shutting down ntpd: [ OK ]
ntpd: Synchronizing with time server: [ OK ]
Starting ntpd: [ OK ]
Note, NTP uses UDP port 123. The ntpd script that comes with red Hat will open this port, both source port 123 and destination port 123, using the following command. iptables -D RH-Lokkit-0-50-INPUT -m udp -p udp -s $server/32 --sport 123 -d 0/0 --dport 123 -j ACCEPT Special note, the above change is not needed in Fedora Core 3.
STEP 5: Check that ntpd is operating correctly with the ntpq command as follows
$ ntpq -np
remote refid st t when poll reach delay offset jitter
==============================================================================
*146.186.218.60 204.123.2.5 2 u 613 1024 377 23.953 -5.935 2.263
+128.91.2.13 128.4.40.12 3 u 180 1024 377 11.191 -4.330 1.377
-128.182.58.100 192.5.41.41 2 u 540 1024 377 20.872 15.298 1.507
+128.118.25.3 128.118.25.12 2 u 137 1024 377 26.207 -5.419 0.561
You should be getting values for all fields. If several of the columns are zero, and jitter is very high, say 4000, then, NTP is not working correctly. But give it a few minutes. You need a few minutes on DSL or cable for enough times stamps to be sent and received. The following shows a problem.
$ ntpq -pn
remote refid st t when poll reach delay offset jitter
===================================================
tock.usno.navy 0.0.0.0 16 u - 64 0 0.000 0.000 4000.00
Setting Up and Using NTP From Source.
STEP 1: Download, compile and install the program.
$ wget http://www.eecis.udel.edu/~ntp/ntp_spool/ntp4/ntp-4.2.0.tar.gz
$ tar -xzvf ntp-4.2.0.tar.gz
$ cd ntp-4.2.0
$ ./configure
$ make
$ su -
$ make install
STEP 2: Add entries to /etc/ntp.conf using 3 or 4 of the closest stratum 2 servers near you. These servers can be found in the following file: http://www.eecis.udel.edu/~mills/ntp/clock2b.html
$ cat /etc/ntp.conf
# A very simple client-only NTP configuration.
server ntp-1.cede.psu.edu
restrict 146.186.218.60
server timeserver1.upenn.edu
restrict 128.91.2.13
server fuzz.psc.edu
restrict 128.182.58.100
server clock.psu.edu
restrict 128.118.25.3
driftfile /etc/ntp/drift
authenticate no
STEP 3: Since NTP uses UDP port 123 for both destination and source, firewall adjustments may be necessary. Here is an example of opening the 123 port for source and destination on each server above. $ iptables -A INPUT -m udp -p udp -s 146.186.218.60/32 --sport 123 -d 0/0 --dport 123 -j ACCEPT $ iptables -A INPUT -m udp -p udp -s 128.92.2.13/32 --sport 123 -d 0/0 --dport 123 -j ACCEPT $ iptables -A INPUT -m udp -p udp -s 128.182.58.100/32 --sport 123 -d 0/0 --dport 123 -j ACCEPT $ iptables -A INPUT -m udp -p udp -s 128.118.25.3/32 --sport 123 -d 0/0 --dport 123 -j ACCEPT STEP 4: Unlike the red Hat installation, the source install does not use /etc/ntp/step-tickers for the initial time set. So you will need to initialize the time. It has to be within 1000 seconds, about 16 minutes or the ntpd daemon will enter panic mode and exit. To manually set the time, enter the following command,as root, with your chosen Time Server. Note, timeserver1.upenn.edu is shown here.
$ su -
# ntpdate -s -b -p 8 timeserver1.upenn.edu
Next, start the ntp deamon.
# /usr/sbin/ntpd
Everything OK?Run the following command. You should be able to run it under any account, and running it as root is not necessary.
$ /usr/sbin/ntpq -crv
status=0654 leap_none, sync_ntp, 5 events, event_peer/strat_chg,
version="ntpd 4.1.1c-rc1@1.836 Thu Feb 13 12:17:19 EST 2003 (1)",
processor="i686", system="Linux2.6.7-ch0", leap=00, stratum=3,
precision=-17, rootdelay=24.973, rootdispersion=62.575, peer=36276,
refid=b50.cede.psu.edu,
reftime=c4df5922.cae1deac Tue, Aug 31 2004 16:08:02.792, poll=10,
clock=c4df5ca7.738194c0 Tue, Aug 31 2004 16:23:03.451, state=4,
offset=-76.829, frequency=-70.770, jitter=18.474, stability=0.293
If you look at the value for stability, which here is equal to 0.293, and frequency which is -70.770 these values tend to vary if the computer gets too hot -- the fan stops working. Or, if you are starting to get hardware problems, you may notice the values increase. Personally, I like to keep a record of these values throughout the day by putting them in to a sqlite3 www.sqlite.org database, which the values can be compared over time. A sample script ntplog can be found in the example download. Calculate and Observe Sunrise and Sunset.If UTC is not adjusted by adding or subtracting leap seconds, then, the times for a sunrise and sunset would be incorrect. If the time is correct on your computer, plus you know the longitude and latitude in degrees, then, sunset will occur at the calculated time. The following program, sunrise.c, works as follows. Note longitude is 39.95 degrees, and latitude in 75.15 degrees. If you live in the US http://www.census.gov/geo/www/tiger/latlng.txt contains a complete listing of all major cities. Note the date must be in the format Year month day, then latitude and longitude must be specified in degrees. Convert minutes and seconds of the coordinates to degrees. Below the program is put into a batch job to calculate the next 100 days. For this current location. Take a look at the following sample output. Compare 10-30-2004 with 10-31-2004. Sunrise is an hour earlier because daylight saving time has ended. The 5 in M10.5.0 is for the 5th week in October, in which a Sunday, 0 exists. If you do a man 3 tzset you will get the exact wording. Taking a look at October 2004, the 5th week of a Sunday is the 31st. And the time change occurs in the morning. Building an NTP Aware ApplicationIf you do not have any control over whether or not a local server can or is running NTP, or if there is a chance that the server does not have the correct time, then, it is still possible to get accurate time by querying an NTP Time-Server directly. First A Close Look At The NTP TimeStamp Format. The following link contains a description of the NTP timestamp format http://www.eecis.udel.edu/~mills/database/rfc/rfc2030.txt Below is a description of the NTP/SNTP Version 4 message format described in the standard. The program queryTimeServer.c prints the 4 timestamp values, by doing 3 separate queries a Time-Server. The timstamp formats must be converted from big-endian to little-endian the htonl function call. Plus seconds in the NTP time scale start in 1900. Unix and Linux define zero seconds at Jan 1st, 1970. See the references for this sample program. The download has an additional explanation. REFERENCES:
Program Examples
NTP timestamp format (RFC2030).
US Listings Of Latitude and Longitude
NTP Source
Steffen Beyer's Date-Calc. Or download Program Examples above Author's bio: Mike Chirico, a father of triplets (all girls) lives outside of Philadelphia, PA, USA. He has worked with Linux since 1996, has a Masters in Computer Science and Mathematics from Villanova University, and has worked in computer-related jobs from Wall Street to the University of Pennsylvania. His hero is Paul Erdos, a brilliant number theorist who was known for his open collaboration with others. Mike's notes page is souptonuts. Author's articles: How to Compile the 2.6 kernel for RedHat 9 and 8.0 and get Fedora Updates This is a step by step tutorial on how to compile the 2.6 kernel from source. Virtual Filesystem: Building A Linux Filesystem From An Ordinary File You can take a disk file, format it as ext2, ext3, or reiser filesystem and then mount it, just like a physical drive. Yes, it then possible to read and write files to this newly mounted device. You can also copy the complete filesystem, since it is just a file, to another computer. If security is an issue, read on. This article will show you how to encrypt the filesystem, and mount it with ACL (Access Control Lists), which give you rights beyond the traditional read (r) write (w) and execute (x) for the 3 user groups file, owner and other. Linux System Admin Tips . There are over 150 linux tips and tricks in this article. Lemon Parser Generator Tutorial . This article explains how to build grammars and programs using the lemon parser, which is faster than yacc. And, unlike yacc, it is thread safe. MySQL Tips and Tricks . A collection of essential, explicit, examples that every MySQL user should know. Getting Email on Home Linux Box. How to configure Sendmail and Fetchmail plus a simple Procmail example.
SPECIAL THANKS:Erik Sjolund (erik.sjolund@home.se) for pointing out changes in Fedora Core 3.
|