"I'd like to know what I can expect"
I've been using C++ (mostly GCC on vxWorks) in embedded systems for more than 2 decades. I have tremendous respect for the compiler writers.
Trust your compiler: IMHO, -O3 has never broken any code ... but has, on occasion, revealed interesting coding errors.
Choose: Teams must choose whether or not to "ship what you test, and test what you ship", regardless of -O1 or -O3 choice. The teams I have worked with have always committed to ship and test with -O3.
Single Step can be un-cooperative: On a personal practice level, when using -O3 code, I typically 'abandon' gdb single step. I make much more use of breakpoints, and there are slight variations in coding choices to make auto variables (stack data) and class data more 'visible'. (You can make the gdb command 'p' your inconvenient friend).
Single Step is Necessary: Please note that even though we "test and ship"d using -O3, we debugged using -O1 code almost exclusively.
Debug is Necesary: The trade off between debugging -01 yet testing and shipping -O3 is the extra re-compiles needed to switch the two executables. The time saved in -O1 to explore, identify, and fix the code bug(s) has to make up the 2 rebuilds (to -01 and back to -O3).
Regression Test Automation: I want to say systems test (aka integration test or regression test) of -O3 has to step it up a notch, but I can't really describe it ... perhaps I should recommend that the level of test automation be higher (every one REGRESSION TESTs!). But I'm not sure. The automation level of regression test probably correlates more to team size rather than performance level.
A 'successful' embedded system does 2 things. It satisfies the requirements. And, I think more importantly, in all human visible behaviour it acts like a lightly loaded desktop. For any action (button press, cable disconnect, test equipment induced error, or even a lowly status light change), the results have no human perceptible delay. -O3 helps. A successful system can be done ... I have seen it.
-Og
. However, if your code breaks with optimisation on, you very well might also get missbehaviour with the next version of gcc or when using a different architecture, compiler, add another line of code, etc. Relying on UB is always invitation for disaster. – Juliajulian-Ofast
: Do you actually have run-time problems? If not, I strongly recommend to stick with the standard. If you have, think about local optimisations or enabling this option locally. – Juliajulian