JaredPar's answer is good, except he doesn't note the raison d'etre for AppDomains - which is that you can only UNLOAD an Assembly by unloading its AppDomain. If you are a long-running OS process, and you expect to have to load and then later unload assemblies for whatever reason then you need an AppDomain. The prototypical example here is ASP.NET, which loads app code assemblies on demand and then can unload them later, when the apps are no longer being actively used.
The cost you pay for the ability to unload is that independence - you need to communicate across the AppDomain boundary, Can't make a simple method call. You need to manage the AppDomain lifecycle. Etc.
If you just need to dynamically load Assemblies and don't think you'll need to unload them during the life of a single process then you probably don't need to run multiple AppDomains. A good example here might be a rich app that supports a plug-in model, where it sniffs out plug-in assemblies in a "etc" directory and loads 'em all up. However, if the plug-in model calls for unloading the plug-ins ... well.
There are outlyer scenarios. Like, suppose you want to load 2 different versions of an Assembly at the same time. You can run into pitfalls if you don't segregate them with AppDomains. But that will be fairly rare.
The core scenario that justifies the existence of AppDomains is the long running process that must be able to unload assemblies.
Of course, applications could rely on the OS process when you want to unload an assembly. In other words, you could have 3 or 4 cooperating processes running, each with its own set of Assemblies, and when you want to unload an assembly, just shut down the process that hosts that assembly. But the AppDomain offers a higher-perf mechanism to do that, without requiring process stop/start or cross-process comms, which is heavier still than the cross-AppDomain comms described previously. I mean it's still remoting but it is slower and more context switching.