Why did Google put atomic clocks in its data centers?

Google put atomic clocks in its data centers so that machines on opposite sides of the planet could agree on what time it is, closely enough to put transactions in the right order. Spanner is Google's globally distributed SQL database, used for its ad business and for services such as Gmail and Google Photos. Its paper calls it the first system to distribute data at global scale and support externally-consistent distributed transactions.

External consistency means the system behaves as if all transactions ran one after another, even though Spanner runs them across many servers and sometimes several data centers. Take a bank customer who deposits money and then, afterwards, pays a bill. External consistency guarantees nobody will ever see the payment without the deposit that came before it.

Spanner does this with timestamps. On a single machine that is easy: a counter hands out increasing numbers. But in a system spread across the world, many servers must assign timestamps that agree. Spanner's answer is TrueTime, an API that exposes clock uncertainty, so each machine knows how far off its clock might be, and it can compute timestamps that are guaranteed to be greater than any generated before.

To make TrueTime work, Google had to outfit its data centers with special hardware, atomic clocks, along with GPS clocks. Machines then know the global time to within a few milliseconds, and Spanner waits out the remaining uncertainty for a few milliseconds before reporting a commit.