C# rebuilds because a file has been changed, or a dependency has been changed, but also frequently for "no obvious reason". You can often build 2 or 3 times in a row without making a single change and C# will trundle off and rebuild vast amounts of the code.
Turn on Tools > Options > Projects and Solutions > Build and Run > Only build startup projects and dependencies on run. This will minimise the build to only the dependencies of the project you intend to execute, so unless everything is one massive dependency tree, this will significantly reduce the number of projects considered in the build.
The better solution, however, is to avoid asking it to compile in the first place.
Every project adds an additional overhead to the build. On my PC this is around 3 seconds. By merging the code of two projects into one .csproj files I save 3 seconds of build time. So for 300+ projects you could knock 10 minutes off the build time just by merging projects - i.e. where possible use many modules/namespaces in one assembly rather than many assemblies. You'll find your application startup time is improved too.
Many projects don't need building all the time. Break these out as library projects and just link to (reference) the binaries
Also disable "copy local" and instead point all OutputPaths to a shared folder - this will significantly reduce the overheads by avoiding making 320 copies of 320 dlls all over your hard drive.
Add new build configurations (e.g. "Debug FastBuild") that only build the assemblies you are actually working on. With the configguration manager you can untick projects you don't want to build, and presto, they are ignored by the build system. After you get code from source control, build a full "Debug" build to refresh everything, and then switch back to "fastbuild"