Some ideas to try...
As @David Paxson said in the comments, with some languauges (e.g. C#, VB) all the projects will appear to be "built", but the compiler will skip through the ones that don't need to be built much more quickly, even though they are still listed in the output. So it may simply appear to be rebuilding everything. Also, with C#, Visual Studio sometimes runs the project several times in a row, and sometimes it will decide for no obvious reason that it needs to "rebuild" the projects, but this only usually takes a few seconds extra unless you have hundreds of projects.
Restart Visual Studio (or even reboot the PC entirely).
Do a "Rebuild All" of the entire solution to clean out any old cached information and ensure all the output files are up to date.
If you use source control, then even better is to check in all of your changes, then delete your old source code folder (or safer, rename it somewhere out of the way until you know you don't need it) and force-get all the latest source code from source control again. This means you have a totally clean starting point (a clean or rebuild-all doesn't quite clean everything away, unfortunately). I do this every 2-4 weeks to keep everything running smoothly and totally in sync with the Source Control code.
If anything modifies the datestamps of any files in your build, then this could trigger a "need" to rebuild everything. Check that your system clock is set correctly, that there aren't any datestamps of source files set to times "in the future", and there are no applications running that could possibly "touch" any of your source files.
Check this option: Tools->Options > Projects and Solutions > Build and Run : "Only build startup projects and dependencies on Run". If it is unchecked, then VS will try to rebuild all projects every time, rather than only the dependencies of the startup project. If you have a large solution with a bunch of mostly unrelated projects in it, this is a very helpful option to set.
If there are projects that you don't need to build every time, then use the Configuration Manager to disable building of those projects in your current build (or create a "Debug Fast Build" configuration so you can switch quickly between a full rebuild and a partial build). Or move projects that don't need to be built often into a separate "libraries" solution and reference/link the output files (obj or dll) form your application's Solution, so the libraries only need to be rebuilt if you edit their code.
Lastly, make sure you're doing a "Build" and not a "Rebuild" :-)