If the app is already installed, this is an easy process, accomplished on either platform with their solutions to deep linking within apps. Google Analytics has some good documentation on how to do this with their SDK.
Essentially, in your links for your two (or more) test groups, you would embed information that would track them to the experiment group they were part of on the site:
examplepetstore://promo?utm_source=newsletter&utm_medium=email&utm_campaign=promotion
The Tough Problem:
Unfortunately, you stated that you specifically need to support the scenario where the user doesn't initially have the app installed when they click the link. This is a much tougher nut to crack. As far as I know, there isn't a way to handle this natively or easily on your own. You will likely need to rely on a third party provider of deep linking that survives the install process.
Branch.io supports "Deferred Deep Linking": https://blog.branch.io/the-ultimate-deep-linking-tutorial-on-ios-11-and-whats-coming-for-ios-12/
Firebase Dynamic Links: https://firebase.google.com/products/dynamic-links/
The big problem is that any vendor solution, like Branch's, almost certainly needs to use cookies for assisting with trying to link back to the original session through the app install process. Apple has been doing things to support privacy that is making this sort of thing difficult. Expect it to be an ongoing battle.
On Android, you should be fine with a solution like Branch or Firebase.
I wish there was a better solution, but with mobile app sandboxing / privacy, this is a challenging problem that is tough to solve without exposing users to all sorts of security / privacy implications.