Some games (ROMs) will automatically save your progress when you exit the game correctly. To exit the game correctly, press START+SELECT together to return to the RetroPie UI.
Certain games won't save your game automatically when you exit the game. You can manually save your game by pressing SELECT+R1 together. On the GPi, R1 is the shoulder button on the back of the unit.
Too manually load a game you previously saved, launch the game and press SELECT+L1 together. Again, on the GPi, L1 is the left shoulder button on the back.
For more in-depth coverage and information on managing multiple save game slots on the GPi, check out my full RetroPie game save guide.