To answer your first question, the Windows User Experience Interaction Guidelines for Windows 7 and Windows Vista specify the following order for your command buttons (p506):
Now, are you smarter than Microsoft? Well, you’d hardly be the first, but you should prove it before releasing your design. Run a usability test on a bunch of users specifically creating scenarios to check if:
Go against the Windows UX Guide only if both of the above are demonstrated to be true.
Regarding your second question, I would recommend that you do not change Cancel to Close after the user selects Apply. A Close button usually implies that any subsequent changes cannot be reverted. Users may not have noticed the button’s initial caption thus may believe the dialog never supports canceling, making the user reluctant to explore the dialog further. Leaving the caption as Cancel assures users they can discard whatever changes they make next. If some users worry that Cancel will undo whatever they Applied, then I expect they’ll just select OK. In theory. Testing would once again tell you what users really think and do, and which design makes the best trade-off.
I’m with you that these OK/Apply hybrid single-use/multi-use dialogs are klugey and confusing. One alternative that gets around the whole problem is to use “immediate commit,” where whatever changes the user makes are instantly shown in the application (this may be a “property inspector” as the Windows UX Guide calls it). Immediate commit eliminates the need for OK, Apply and Cancel. Instead you have Close, and I would also suggest you also have an Undo button that works like an Undo menu item, sequentially reverting each change the user made with each selection. In addition to avoiding the OK/Apply/Cancel/Close confusion, this design is faster (fewer clicks to try a change), makes it clear what user input has what effect, and supports incremental undo (Cancel is all or nothing).