You can remove the app from the store temporarily.
- Open itunesconnect.apple.com
- Select My Apps
- Select the app you need to remove
- Select Pricing and Availability > Availability
- Remove from Sale > Save
This will remove the app temporarily from the store. It may take a bit of time to be not available on the App Store. You will have a red dot specifying the developer remove the app from the store.
How to restore to App Store
Repeat steps 1-4, select which countries you want to restore, then save.
Reviews and Ratings History
By doing the steps above, your reviews and ratings will be intact as long as you don't do anything else. I just restored an app today and everything was as I left it before.
App Review Afterwards
You do not need to go through app review again as long as you don't change the binary. It takes about 1 hour to get back on to the App Store if you're using a previously approved binary.
A new binary will trigger a new app review.