Firstly, let's read about CriticalFinalizerObject in MSDN, we can read, that:
In classes derived from the CriticalFinalizerObject class, the common language runtime (CLR) guarantees that all critical finalization code will be given the opportunity to execute, provided the finalizer follows the rules for a CER, even in situations where the CLR forcibly unloads an application domain or aborts a thread.
The main word here is UNLOAD.
Secondly, let's read MSDN again, this time about Exceptions in managed threads:
If these exceptions are unhandled in the main thread, or in threads that entered the runtime from unmanaged code, they proceed normally, resulting in termination of the application.
The main word is TERMINATION.
So, when there is an unhandled exception in main thread - app terminates, but CriticalFinalizerObject helps only on unloading of Domain.
For example, CriticalFinalizerObject can helps in such situation:
// Create an Application Domain:
AppDomain newDomain = AppDomain.CreateDomain("NewApplicationDomain");
// Load and execute an assembly:
newDomain.ExecuteAssembly(@"YouNetApp.exe");
//Unload of loaded domain
AppDomain.Unload(newDomain);
This is a situation, where domain was unloaded, and CriticalFinalizerObject guarantee you, that your finalizer will be called.
In your situation with terminating of app you can try to subscribe to
AppDomain.CurrentDomain.UnhandledException
and manually finalize your objects.
UPD:
Jeffrey Richter in his book "CLR via C#" wrote about CriticalFinalizerObject, that it's for situations where you send your code for example to SQLServer, which can run C# as a procedures. In such case CriticalFinalizerObject helps you to clean your object, if SQLServer will unload your library's Domain.
Also CriticalFinalizerObject is for situations where you need in finalizer of object to call method of another object, because of CriticalFinalizerObject guarantee you, that it's finalizer will be called after finalizers of all non CriticalFinalizerObject objects.