How did Cauchy and Weierstrass finally make calculus rigorous?

Cauchy and Weierstrass made calculus rigorous by replacing the mysterious infinitely small with a careful statement about limits. Early calculus used infinitesimal quantities, and critics thought that unrigorous. Working out a rigorous foundation occupied mathematicians for much of the century following Newton and Leibniz, and the answer, due to the work of Cauchy and Weierstrass, avoided mere notions of infinitely small quantities.

The attack came first. In 1734 Bishop George Berkeley wrote The Analyst, a direct attack on the foundations of calculus, on Newton's fluxions and on Leibniz's infinitesimal change. He famously described infinitesimals as the ghosts of departed quantities. The book was prompted by a mocking remark from the free-thinking astronomer Edmond Halley, and it hit a real weak spot. A quantity that is neither zero nor anything you can write down is hard to defend.

The fix was to talk only about nearby values. Limits describe the behavior of a function at a certain input in terms of its values at nearby inputs. Infinitesimals get replaced by sequences of smaller and smaller numbers, and the infinitely small behaviour of a function is found by taking the limiting behaviour for these sequences. The letters epsilon and delta in the modern definition can be understood as error and distance: whatever error you name, however small, there must be a distance such that whenever the input is that close to the point, the output is within your error of the limit.

Cauchy's textbook Cours d'Analyse, published in 1821, gave a prototype of this definition, though a somewhat imprecise one. Weierstrass then formalized the concept of limit and eliminated infinitesimals. In the late 19th century limits replaced infinitesimals within academia, and the old puzzle of the infinitely small finally had a precise answer.