This is a good excuse to wash your whole car ;)
Use the most normal toothpaste you can find. The only reason this works is because toothpaste is a mild abrasive. If it has special whitening or other nonsense, it won't work as well.
Using as much elbow grease as you can, scrub the headlight. Add toothpaste if it becomes too dry.
Using water and a dry cloth, wash and dry your headlight. The oxidation layer is known to come back every once in a while so you may need to repeat the process.