The common solution I know is to agree on "branch namespaces", by prepending some string to branch names. For example, branches that start with "private/" are for private experiments. You'd then get branches like
- private/JohnDoe/refactoring-taxcalculation
- private/JohnDoe/newGUILayout
- private/JaneJones/Java8
- private/TKirk/build-spaceship
That keeps the branches separate, and makes it clear what their purpose is. However, that way the branches are still public, because anyone can see and pull them.
If you want to restrict access to these branches based on user, you'd need to have some kind of branch-based access control. There is no such thing in core git, but some git hosting servers allow this (Atlassian Stash for example). I don't know of any server that allows these kind of private branches, but maybe there is one that allows it or lets you script a solution.
Note, however, that what you are asking for is rather unusual. The common solution is the one I outlined above.