From here (not about Swift, but might be the answer)
In C, static structs that are not zero-initialized or uninitialized (i.e. that are statically to something else than zero) increase the binary size (they go into the data segment, i.e. even if you only initialize one field of a struct, the binary contains a full image of the full struct)
And from here (not only about structs but might be interesting as well)
Structs can increase your binary size. If you have structs into lists they are created on the stack and they can increase your binary size.
Optionals usage will increase your binary size as well. You will be using optionals, but the thing you don't know is that the compiler has to do a lot of things; It has to do checking, it has to do unwrapping. So even though it's just a one-liner for you with a question mark, you get a lot of size in your binary.
Generic specialization is another problem that we encountered. Whenever you use generics, if you want your generics to be fast, the compiler will specialize them and give you quite a bit of binary size increase, as well.
The first quote is about C and it makes a lot of sense to me. I suppose that the same thing is happening with Swift.
If so, the reason is that in some cases your struct objects are not initialized in a runtime, but stored in a binary instead.
Hope it helps.