After searching I found answers that were close enough to get me to my own answer for API 24. For me it was simply a matter of removing API 24, closing Android Studio, and (after reopening) letting Android Studio detect API 24 was missing and download it for me. I assume this would also work for other API versions.
UPDATE: As others have pointed out, API 24 did not have "Sources" listed in SDK Manager at the time this question was asked/answered. However, everything was working fine before and continues to work after following this answer. So that seems to be a bit of a technicality worth mentioning.
I have the app targeting and compiling for API 24 and on Google Play in Alpha testing.
Android N APIs are now final (as of 15 June 2016).
Steps (Mac):
- Open SDK Manager: Tools > Android > SDK Manager
- Remove API 24: Deselect Checkbox > Click OK, OK again to confirm removal
- Quit Android Studio (⌘Q)
- Reopen your project in Android Studio
- Install missing platform(s) and sync project
- Click this clickable link in the Messages Gradle Sync panel
Once the Gradle Sync finishes that should take care of it.