Both a private and a public key must be generated in order to use public key encryption. The public key is required for encryption while the private key is used to decrypt. The originating computer - the computer receiving the encrypted message - needs to generate the key pair. The public key will be given to the computer sending the encrypted message, but the private key will only be known to the originating computer since it is used for decryption.
The computer sending the encrypted message will use the public key. The encrypted message can be sent over the internet and cannot be decrypted without the private key.
The originating computer should be the only computer with access to the private key. Upon receiving the encrypted message the private key can be used to decrypt the message.