I was also struggling with this problem and didn't find automated way to do this. I had to do it manually like that:
Renaming the Solution. Easiest part.
Go to Solution Explorer (from menu View -> Solution Explorer), select the solution and then hit F4 (or from menu – View -> Properties Window), select the Name property and type the new name and then press ENTER. This will not change the name of the folder in which the entire solution exists. If you want to change that (and I did wanted to do that), you have to close Visual Studio and then rename that folder. If you open again Visual Studio and open the solution by Browsing after the new name (File -> Open -> Project/Solution ..), you will see that the new name for the solution has been changed.
This is how you can try to do it with projects:
Right-click the project in Solution Explorer, select Rename, and enter the new name
Right-click the project again and select Properties. Change the "Assembly name" and "Default namespace" on the Application tab.
When you have ReSharper installed, right-click the project again and select Refactor -> Adjust Namespaces. Accept the changes.
Change the AssemblyTitle and AssemblyProduct in Properties/AssemblyInfo.cs
Delete bin and obj directories in Windows Explorer
Open the SLN file (with a text editor such as Notepad++) and change the path to the project (there should be multiple places).
Open the Solution again. Clean and Rebuild the project.