The reason for this is that MS completely re-organised the file dialogs for Vista. Hooks are used to extend a file dialog by supplying a resource file. This gives the customiser too much power. They can all too easily modify standard elements of the dialog and indeed many apps did so. The reorganisation of the dialogs would have broken many apps that used hooks. Those would have tried to manipulate elements of the dialog that were not there, or were implemented differently. Legacy versions of the dialogs remain for such apps to "get their hooks into".
You are correct that it is impossible to get the new look when you use a hook. Instead you need to use the IFileDialogCustomize interface to customise the dialog. This is less powerful but does result in appearance and behaviour that is more consistent with the standard part of the dialog.
OFN_ENABLETEMPLATE
supplies a resource,OFN_ENABLEHOOK
is just the callback function. You can hook without adding a custom resource. – Clarion