Unable to set proper orientation of 3d Model in AR augmented face

Question

I am trying to put earings on the face using ARCore Augmented Face. I am editing the 3D model in blender and adjusting the model according to canonical_face_mesh.fbx .

But my 3d model is rendering above the head.

I also tried making the pivot point of my model same as that of canonical_face_mesh.fbx, (as mentioned in this SO question) but still when running in app it's rendering

Nuggar

Answer 1

The problem was

1. I was not placing custom meshes as children of the asset node as mentioned in doc.

   .

To make the custom mesh or the object a child of the asset node:

i) Load the given canonical face mesh in blender file->import->fbx then browse for your downloaded canonical face mesh.

(Note: Every SDK has its own reference face mesh and they differ; make sure you use correct one).

ii) Then load your custom 3D model.

iii) Scale and align your model according to the canonical face mesh wherever you want it to be.

iv) In object mode press ctrl + shift +p and then click your custom model and then click the bones present in canonical face mesh.A selection menu will appear select object in that menu.

This will make your model, the child of the asset node.

2. Now you need to add Vertex Group in your model. To do so:

i) Select all vertex in your model in edit mode.

ii) Then go to context object data menu. Click the + icon and name your Vertex ( NOSE_TIP, FOREHEAD_RIGHT, or FOREHEAD_LEFT ). This you have to assign according to your model. Mine is Nose Pin so i choose NOSE_TIP. Then Click assign.

Now to cross check you can deselect your model and then click your vertex group and click on select button next to assign button you will notice that your model is selected now.

3. You need to add modifier. To add modifier

i) Click on the modifier option.

ii) Then click add modifier you will see a drop down menu in that select armature.

iii) Name it as asset as shown above select object as asset and select Nose_TIP as shown above.

Now Delete only the facemesh from the asset.

and import your model.

Nuggar

i am confused in steps, can you please elaborate more ? – Carmarthenshire

sure, in which step you are confused? – Nuggar

can you share the final 3d model that worked for you? I am facing the same problem. I tried following your steps with the same model that you used but earrings are appearing near my forehead rather than at earlobes. – Clari

drive.google.com/open?id=1BJW6uxXxFfuKCdNgcWWdIdsVVwbGt90u – Nuggar

drive.google.com/open?id=1CB0M_p3c28X1WTQbwCW70E-X0o6XSXs7 – Nuggar

These are in fbx format you can use them directly in your project – Nuggar

And the reason that they are appearing at head is that you are unable to lock them take help from 3D modeler or someone who knows Blender – Nuggar

ok so I tried using your models and it worked (thanx for that btw) but I wasn't able to replicate that so I tried different approach by using getMeshVertices() and specify the cordinates manually. But now whenever I rotate my head and when the model goes behind the face it turns black. Do you know how to circumvent that so that if earring is occluded by that face it isn't visible or only a part of it is visible? !example image – Clari

Hi, I'm trying to place a 3D Cap on head, following the steps it is appearing over the forehead but it is not getting fit on the head. I'm adding both FOREHEAD_LEFT & FOREHEAD_RIGHT in the Vertex group, but it doesn't seems to work perfectly. Since I'm a beginner in using Blender, I'd appreciate any help. Thx. – Anam

Answer 2

There are three possible problems in your case:

• A pivot point of 3D Earrings Model doesn't match Face Anchor position.

  • Face Anchor and pivot point of your model must meet each other.

• A size/proportions of 3D Model doesn't match size/proportions of canonical face mesh.

• A pivot point in your 3D model is shifted in a wrong direction of Y-axis.

P.S.

Read this post if you need more details about Augmented Faces feature and Face Anchor.

Remember that an orientation of the mesh is different for Sceneform, Unity and Unreal.

Denitadenitrate