Not wrong at all, but wrong. (a) The private key decrypts the data encrypted by the public key and (b) the public key decrypts the data encrypted by the private key.
(a): Everybody can encrypt something, but only the owner of the private key can decrypt it.
(b): The owner "encrypt" something with his private key and everybody can decrypt it, what ensures, the it were really the owner, that encrypts the data and not somebody else.
git(hub) makes use of the second scenario: If you push something, it its signed with your private key. The receiver now validates the signature againts the public key it knows from you. If its match, everything is fine.
Update: A (maybe too) simplified description on what happens (when using github with ssh)
- Github sends you something random, that is encrypted with his _private_ key (Maybe its not that random, I dont know, but doesnt matter here)
- You receive it and decrypt it with his _public_ key. If this is possible, you are sure, that you are really talking to the official github server
- Then you send the same random stuff encrypted with your _private_ key to the github-server
- He tries to encrypt it with your _public_ key. If this is possible and its the random stuff he has sent you before, he knows for sure, that you are you.
- Then you send you stuff encrypted with his _public_ key. Now only the github server can decrypt. Also he will answer with messages/data encrypted with your _public_ key, because only you can decrypt it.
Even if its not completely correct, it should describe the idea.