On the apple developer website I can download Xcode new betas separately from the simulators.
I did exactly that. After downloading Xcode (3.41GB) I downloaded the new iOS 17 beta which was 7.47 GB. After installing the new Xcode, it us prompting me to download again the iOS simulator which I had already downloaded. But when prompted from Xcode the download is also very sluggish and I don't want to wait since I have the simulator DMG in my downloads folder.
I looked here already but it did not help: Download Xcode simulator directly
I tried manually to put the content of the .DMG file into the library in Xcode as described by some older posts, like in "Cashes/com.apple.dt.Xcode/Downloads/" but this did not work. Restarting Xcode I had the same prompt asking me to redownload the simulator.