PowerShell (the engine) runs fine under .NET 4.0. PowerShell (the console host and the ISE) do not, simply because they were compiled against older versions of .NET. There's a registry setting that will change the .NET framework loaded systemwide, which will in turn allow PowerShell to use .NET 4.0 classes:
reg add hklm\software\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1
reg add hklm\software\wow6432node\microsoft\.netframework /v OnlyUseLatestCLR /t REG_DWORD /d 1
To update just the ISE to use .NET 4.0, you can change the configuration ($psHome\powershell_ise.exe.config) file to have a chunk like this:
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<startup>
<supportedRuntime version="v4.0.30319" />
</startup>
</configuration>
You can build .NET 4.0 applications that call PowerShell using the PowerShell API (System.Management.Automation.PowerShell) just fine, but these steps will help get the in-the-box PowerShell hosts to work under .NET 4.0.
Remove the registry keys when you don't need them any more. These are machine-wide keys and forcibly migrate ALL applications to .NET 4.0, even applications using .net 2 and .net 3.5