Putting a 3D face filter by using the Vision API

Question

The similar question has been asked here: Add 2D or 3D Face Filters like MSQRD/SnapChat Using Google Vision API for iOS. However, it has not been answered yet.

Essentially, my work assignment is to put a 3D face filter on a person's face while the phone's front-facing camera is being used.

Given that the Mobile Vision API/the GitHub Android Vision project provide a way to detect a human face and stick some drawable images on it, but what my users want is a 3D object (cat or dog face) like what Facebook, Instagram, Snapchat, etc. have done.

I am also looking at Unity/Vuforia, but I have no idea how to integrate a Unity project to our Android app. What I want is to use a button to turn on/select this feature.

Added at 8th June 2018 Based on my reading, I believe Vuforia isn't designed for making a Face Filter on Android, and it's not that difficult to use the Android API on Unity. But, I have no idea how to do it another way around such as clicking a button to call the Unity Facial Filter plugged-in(like XZIMG) function to use the Facial Mask / Filter feature with the camera.

Cointon

Facebook & SnapChat were using DeepAR SDK to create face filter,If you get any progress on this let me know how to do it with Mobile Vision. – Braggart

News Facebook acquires video filter app Msqrd to square up to Snapchat. Unfortunately, I am not getting any further with the Mobile Vision on putting up a 3D model on a human face. However, I found a Chinese company which does a similar thing, and we are looking at the integration bit to our app. – Cointon

Good Luck @Cointon – Braggart

Just an update here. We ended up getting the solution that's provided by another Chinese based company, and we are able to integrate their SDK into our Android and IOS projects to use the Facial Recognition, and 2D/3D stickers feature to meet our requirement. They have pretty good documentation for their SDK and tutorials to teach you how to create your own stickers by using some common 2D/3D model making tools – Cointon

@Cointon did you end up using their SenseAR product? sensetime.com/internet/89 – Seel

@Seel Nah. We ended up no using any AR products as most of the companies are trying to rip you off with a lot of money (the SenseAR is good and accurate on detecting human faces and put on some 3D objects, but we don't have the budget for it). If you have the $$, go for it. Otherwise, my advice is to look deeper into OpenGL stuff. In Android, all the magic happen in the GLSurfaceView class including rendering a 3D model/sticker and video capturing. It's a huge topic, but I don't have the luxury and knowledge to dive deeper. Good luck! – Cointon

Thank u for giving us the latest updates, I'm still looking for a low-price SDK, any advice will be appreciated. – Jacquelinjacqueline