Select the front camera in an existing ARCore session to start using Augmented Faces. Note that selecting the front camera
will cause a number ofchangesin ARCore behavior.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-10-31 UTC."],[[["\u003cp\u003eThis guide explains how to integrate the Augmented Faces feature into your Android applications for real-time face tracking and augmentation.\u003c/p\u003e\n"],["\u003cp\u003eBefore starting, ensure you have a foundational understanding of AR concepts and ARCore session configuration.\u003c/p\u003e\n"],["\u003cp\u003eTo use Augmented Faces, configure your ARCore session to utilize the front-facing camera and enable the AugmentedFaceMode.\u003c/p\u003e\n"],["\u003cp\u003eAccess and track detected faces by obtaining a Trackable object for each frame and checking its TrackingState.\u003c/p\u003e\n"],["\u003cp\u003eOnce a face is tracked, you can access its pose, mesh data (vertices, normals, UVs, indices), and use them for rendering and augmentation.\u003c/p\u003e\n"]]],["To use Augmented Faces, first configure the ARCore session to use the front camera and enable `AugmentedFaceMode` with `MESH3D`. Then, access the detected face via `getAllTrackables`, ensuring the `TrackingState` is `TRACKING`. Retrieve and cache face mesh UVs and indices. Each frame, update center/region poses, mesh vertices, and normals to render the face, using provided examples in Java and Kotlin.\n"],null,["# Augmented Faces developer guide for Android\n\nLearn how to use the Augmented Faces feature in your own apps.\n\nPrerequisites\n-------------\n\nMake sure that you understand [fundamental AR concepts](/ar/develop/fundamentals)\nand how to [configure an ARCore session](/ar/develop/java/session-config) before proceeding.\n\nUsing Augmented Faces in Android\n--------------------------------\n\n1. [Configure the ARCore session](#configure-session)\n2. [Get access to the detected face](#face-access)\n\n### Configure the ARCore session\n\nSelect the front camera in an existing ARCore session to start using Augmented Faces. Note that selecting the front camera\nwill cause a number of [changes](/ar/reference/java/com/google/ar/core/CameraConfig.FacingDirection#front)\nin ARCore behavior. \n\n### Java\n\n```java\n// Set a camera configuration that usese the front-facing camera.\nCameraConfigFilter filter =\n new CameraConfigFilter(session).setFacingDirection(CameraConfig.FacingDirection.FRONT);\nCameraConfig cameraConfig = session.getSupportedCameraConfigs(filter).get(0);\nsession.setCameraConfig(cameraConfig);\n```\n\n### Kotlin\n\n```kotlin\n// Set a camera configuration that usese the front-facing camera.\nval filter = CameraConfigFilter(session).setFacingDirection(CameraConfig.FacingDirection.FRONT)\nval cameraConfig = session.getSupportedCameraConfigs(filter)[0]\nsession.cameraConfig = cameraConfig\n```\n\nEnable [`AugmentedFaceMode`](/ar/reference/java/com/google/ar/core/Config.AugmentedFaceMode): \n\n### Java\n\n```java\nConfig config = new Config(session);\nconfig.setAugmentedFaceMode(Config.AugmentedFaceMode.MESH3D);\nsession.configure(config);\n```\n\n### Kotlin\n\n```kotlin\nval config = Config(session)\nconfig.augmentedFaceMode = Config.AugmentedFaceMode.MESH3D\nsession.configure(config)\n```\n\n### Face mesh orientation\n\nNote the orientation of the face mesh:\n\n### Access the detected face\n\nGet a [`Trackable`](/ar/reference/java/com/google/ar/core/Trackable)\nfor each frame. A `Trackable` is something that ARCore can track and that\nAnchors can be attached to. \n\n### Java\n\n```java\n// ARCore's face detection works best on upright faces, relative to gravity.\nCollection\u003cAugmentedFace\u003e faces = session.getAllTrackables(AugmentedFace.class);\n```\n\n### Kotlin\n\n```kotlin\n// ARCore's face detection works best on upright faces, relative to gravity.\nval faces = session.getAllTrackables(AugmentedFace::class.java)\n```\n\nGet the [`TrackingState`](/ar/reference/java/com/google/ar/core/TrackingState)\nfor each `Trackable`. If it is [`TRACKING`](/ar/reference/java/com/google/ar/core/TrackingState#tracking),\nthen its pose is currently known by ARCore. \n\n### Java\n\n```java\nfor (AugmentedFace face : faces) {\n if (face.getTrackingState() == TrackingState.TRACKING) {\n // UVs and indices can be cached as they do not change during the session.\n FloatBuffer uvs = face.getMeshTextureCoordinates();\n ShortBuffer indices = face.getMeshTriangleIndices();\n // Center and region poses, mesh vertices, and normals are updated each frame.\n Pose facePose = face.getCenterPose();\n FloatBuffer faceVertices = face.getMeshVertices();\n FloatBuffer faceNormals = face.getMeshNormals();\n // Render the face using these values with OpenGL.\n }\n}\n```\n\n### Kotlin\n\n```kotlin\nfaces.forEach { face -\u003e\n if (face.trackingState == TrackingState.TRACKING) {\n // UVs and indices can be cached as they do not change during the session.\n val uvs = face.meshTextureCoordinates\n val indices = face.meshTriangleIndices\n // Center and region poses, mesh vertices, and normals are updated each frame.\n val facePose = face.centerPose\n val faceVertices = face.meshVertices\n val faceNormals = face.meshNormals\n // Render the face using these values with OpenGL.\n }\n}\n```"]]
