Wash and dry your headlights using normal methods
This is a good excuse to wash your whole car ;)
Cover the headlight with a thin layer toothpaste
Use the most normal toothpaste you can find. The only reason this works is because toothpaste is a mild abrasive. If it has special whitening or other nonsense, it won't work as well.
Scrub the headlight
Using as much elbow grease as you can, scrub the headlight. Add toothpaste if it becomes too dry.
Wash and dry the headlight
Using water and a dry cloth, wash and dry your headlight.
The oxidation layer is known to come back every once in a while so you may need to repeat the process.