I came over from Vue as well, the out-in is one thing I miss with Svelte. Rich Harris even acknowledged it prior to Svelte 3 but never really implemented a fix as far as I'm aware.
The problem with the single condition, delay-only, out-in transition method is that Svelte is creating the incoming element once the condition switches despite the delay on the in transition. You can slow the transitions way down and check dev tools to see this, both elements will exist the incoming transition delay does not prevent the element from having a size, just visibility.
One way around it is to do what you've done with absolute position, kinda intensive and becomes boilerplate. Another method is to set an absolute height for the container holding the elements being transitioned, pull everything else out of the container (the button in your example) and hide the overflow as seen here, very css dependent and does not always play well with certain layouts.
The last way I've used is a bit more round about but since Svelte has an outroend event that is dispatched when the animation is done you can add a variable for blue or whatever your second condition is and put in an else if block for the second condition (blue here) and wire the trigger so it's checking for the active variable and switching it off, then switch on the other variable inside the outroend event as seen here you can also remove any delay since the duration becomes the delay.
From inspecting the DOM during transitions it seems this is the only way that both elements don't exist at the same time because they depend on separate conditions, I'm sure there are even more elegant ways to achieve this but this works for me.
EDIT:
There is another option available that only works on browsers that support CSS grid spec, luckily that's nearly universal at this point. It's very similar to the absolute positioning method with an added bonus that you don't have to worry about the height of the elements at all
The idea behind this is that with CSS Grid we can force 2 elements to occupy the same space with grid-area
or grid-column
and grid-row
by giving both elements(or more than 2) the same start and end columns and rows on the implicit grid of 1 col by 1 row (grid is smart enough to not create extra columns and rows we won't be using). Since Svelte uses transforms in it's transitions we can have elements coming and going without any layout shift, nice. We no longer have to worry about absolute position affecting elements or about delays, we can fine tune the transition timing to perfection.
Here is a REPL to show a simple setup, and another REPL to show how this can be used to get some pretty sweet layering effects, woah!