Why does stop-motion jitter, and how did Star Wars fix it?

It jitters because every frame is perfectly sharp. Stop motion physically moves objects in small increments between individually photographed frames, so they seem to move on their own when played back. Each frame is shot with the object perfectly still, so the animated object is perfectly sharp in every frame. Real moving objects have motion blur because they move while the camera's shutter is open. Without that blur, stop motion can create a distinctive and disorienting staccato effect.

Filmmakers found ways to fake the blur. They could move the model slightly during the exposure of each frame, or put a glass plate smeared with petroleum jelly in front of the lens. Ladislas Starevich was already moving the puppet or the set during the exposure in the 1920s, and Aardman moved the cameras physically during a one to two second exposure for the train chase in The Wrong Trousers.

The most sophisticated technique was go motion, developed by Phil Tippett and Industrial Light & Magic. The model was essentially a rod puppet, with rods attached to motors linked to a computer that recorded the animator's movements. The computer then replayed them with the camera rolling, so the model moved during the exposure and motion blur was created. It was used for some shots of the tauntauns and AT-AT walkers in The Empire Strikes Back (1980).

Then computers took over. Go motion was originally planned for the dinosaurs of Jurassic Park until Steven Spielberg decided to try computer graphics, and after the film Tippett Studio abandoned go motion and converted fully to CG. Stop motion itself did not disappear: it keeps a distinct look that some directors prefer.