A hostname is the name of your computer on a network. Your computer can be reached by its IP address, but those are long and hard to remember. Setting a hostname is easier to remember, more convenient to type, and generally better for managing multiple devices on a network.
By default, the Raspberry Pi hostname is raspberrypi, but if you have multiple Pi's on the same network you'll probably want to set a unique hostname for each. Otherwise, you'll have issues with SSH timing out when attempting to connect.
Certain prebuilt software installations might ship with a hostname that's already been changed. For example, the hostname for RetroPie is retropie and OctoPrint uses octopi by default. I still recommend changing these, using this guide, to avoid conflicts.