Facebook Engineers Open‑Source Precision Time Appliance for Data Centers

Facebook’s engineering team has designed and released a high‑precision time appliance, a key element of modern data‑center timing infrastructure. By creating the time card—a PCIe card that transforms any commodity server into a Stratum‑1 time source—and partnering with the Open Compute Project (OCP), we have made every component of the Open Time Server available under an open‑source license.

In March 2020, Facebook announced the migration of its data‑center and consumer‑product servers to a new, in‑house Network Time Protocol (NTP) service. The upgrade raised time‑keeping accuracy from 10 ms to 100 µs, enabling finer infrastructure control and faster distributed‑database performance.

The new NTP architecture relies on a Stratum‑1 source—directly linked to an authoritative clock such as a Global Navigation Satellite System (GNSS) or a cesium clock:

Public NTP pools (e.g., time.facebook.com) are convenient but introduce dependencies on internet connectivity and expose systems to potential outages or drift. To eliminate these risks, we built a dedicated hardware appliance that integrates a GNSS receiver with a miniaturized atomic clock (MAC). The time card encapsulates this functionality on a PCIe board, allowing any x86 server equipped with a hardware‑timestamping NIC to become a Stratum‑1 source.

Why a New Time Device?

Commercial time appliances are battle‑tested and often stable, yet they come with significant drawbacks:

• They may be outdated and vulnerable to security flaws, with patch cycles spanning months or years.
• Closed‑source firmware limits configuration and monitoring; proprietary CLIs or web UIs hinder automation.
• Proprietary hardware is non‑serviceable—component failure requires vendor repair or a new unit.
• Low‑volume production leads to high markups, making long‑term operation costly.

These limitations forced many organizations to accept trade‑offs. We decided to explore an alternative: a purpose‑built, open‑source time appliance built on x86 hardware.

Prototyping the Time Appliance

Our initial concept combined a GNSS receiver, a high‑stability oscillator, and FPGA‑based processing into a single PCIe card—see the block diagram below:

The GNSS module delivers both time‑of‑day (ToD) and a one‑pulse‑per‑second (PPS) signal. When paired with a high‑stability oscillator—such as a rubidium or oven‑controlled crystal oscillator—the system can achieve nanosecond‑level accuracy. The PPS and ToD are fed to a high‑performance NIC (e.g., NVIDIA Mellanox ConnectX‑6 Dx) that supports hardware time‑stamping and PPS in/out.

Using the ts2phc tool, we synchronized the NIC’s physical clock to the GNSS PPS within a few tens of nanoseconds, validating our approach:

To enhance reliability, we split the design into two logical blocks: the payload (the precision time engine) and the delivery (the NIC). The payload interpolates time between consecutive PPS pulses using an FPGA‑based engine and a local oscillator. In the event of GNSS loss, the engine relies on the atomic clock’s inherent stability.

The time engine implements filtering, synchronization, error checking, and PCIe subsystems. It continuously calibrates the MAC by comparing its counter to the GNSS PPS, achieving sub‑nanosecond precision over extended periods.

By leveraging a 10 MHz reference (from a rubidium clock or external SMA input), we up‑convert to 125 MHz (8‑ns periods) and associate sub‑second increments to 250‑ps steps. A proportional‑integral loop fine‑tunes the frequency via a 12.5× PLL and, if needed, direct adjustments to the oscillator.

Holdover—time‑accuracy loss when GNSS is unavailable—is a critical metric. Our prototype achieved less than 300 ns drift over 24 hours, well within the specifications of the SA.53s MAC.

The Time Appliance in Action

Once installed, the time card turns any compatible x86 server into a Stratum‑1 time source, independent of NTP, PTP, SyncE, or other protocols. The system’s precision is sufficient for most enterprise time‑synchronization needs.

Setup is straightforward: install Linux kernel 5.15+ (or build the driver from the OCP GitHub repository on kernel 5.12+). The driver exposes PHC, GNSS, PPS, and MAC devices under /sys/class/timecard/ocp0/:

$ ls -l /sys/class/timecard/ocp0/
lrwxrwxrwx. 1 root 0 Aug  3 19:49 device -> ../../../0000:04:00.0/
-r--r--r--. 1 root 4096 Aug  3 19:49 gnss_sync
lrwxrwxrwx. 1 root 0 Aug  3 19:49 i2c -> ../../xiic-i2c.1024/i2c-2/
lrwxrwxrwx. 1 root 0 Aug  3 19:49 pps -> ../../../../../virtual/pps/pps1/
lrwxrwxrwx. 1 root 0 Aug  3 19:49 ptp -> ../../ptp/ptp2/
lrwxrwxrwx. 1 root 0 Aug  3 19:49 ttyGNSS -> ../../tty/ttyS7/
lrwxrwxrwx. 1 root 0 Aug  3 19:49 ttyMAC -> ../../tty/ttyS8/

Configure chrony or a PTP daemon to use the PHC as the reference clock:

# chrony
refclock PHC /dev/ptp2 tai poll 0 trust

Validation with a Calnex Sentinel shows ±40 µs NTP stability over 48 hours and sub‑nanosecond PTP offsets. The combined PPS error between the time card and the Sentinel remains within ±200 ns.

The modular design also allows swapping the MAC for a cost‑effective OCXO or TCXO, trading holdover performance for budget.

Open‑Sourcing the Design

To broaden impact, we partnered with the Open Compute Project to launch the Time Appliance Project (TAP). All specifications, schematics, BOMs, mechanical drawings, and source code are publicly available on the TAP GitHub repository. Vendors such as Orolia and NVIDIA are already producing the time card and precision‑timing NICs, respectively.

Our Open Time Server spec details how to assemble the hardware and software stack to create a robust, fully‑controlled time appliance. By owning the entire system, engineers gain deeper visibility, tighter security, and simpler maintenance.

While the current appliance represents a major leap forward, we will continue refining accuracy, scaling synchronization across Facebook’s global infrastructure, and sharing advances with the open‑compute community.