Configure the API once per session with the appropriate mode
Configure Lighting Estimation once per session for the mode you want to use.
Java
// Configure the session with the Lighting Estimation API inENVIRONMENTAL_HDRmode.Configconfig=session.getConfig();config.setLightEstimationMode(LightEstimationMode.ENVIRONMENTAL_HDR);session.configure(config);// Configure the session with the Lighting Estimation API inAMBIENT_INTENSITYmode.Configconfig=session.getConfig();config.setLightEstimationMode(LightEstimationMode.AMBIENT_INTENSITY);session.configure(config);// Configure the session with the Lighting Estimation API turned off.Configconfig=session.getConfig();config.setLightEstimationMode(LightEstimationMode.DISABLED);session.configure(config);
Kotlin
// Configure the session with the Lighting Estimation API inENVIRONMENTAL_HDRmode.Configconfig=session.configconfig.lightEstimationMode=LightEstimationMode.ENVIRONMENTAL_HDRsession.configure(config)// Configure the session with the Lighting Estimation API inAMBIENT_INTENSITYmode.Configconfig=session.configconfig.lightEstimationMode=LightEstimationMode.AMBIENT_INTENSITYsession.configure(config)// Configure the session with the Lighting Estimation API turned off.Configconfig=session.configconfig.lightEstimationMode=LightEstimationMode.DISABLEDsession.configure(config)
ConfigureENVIRONMENTAL_HDRmode
To configureENVIRONMENTAL_HDRmode, get the light estimate for each frame,
then get the environmental HDR lighting components you want to use.
Java
voidupdate(){// Get the current frame.Frameframe=session.update();// Get the light estimate for the current frame.LightEstimatelightEstimate=frame.getLightEstimate();// Get intensity and direction of the main directional light from the current light estimate.float[]intensity=lightEstimate.getEnvironmentalHdrMainLightIntensity();// note - currently only out param.float[]direction=lightEstimate.getEnvironmentalHdrMainLightDirection();app.setDirectionalLightValues(intensity,direction);// app-specific code.// Get ambient lighting as spherical harmonics coefficients.float[]harmonics=lightEstimate.getEnvironmentalHdrAmbientSphericalHarmonics();app.setAmbientSphericalHarmonicsLightValues(harmonics);// app-specific code.// Get HDR environmental lighting as a cubemap in linear color space.Image[]lightmaps=lightEstimate.acquireEnvironmentalHdrCubeMap();for(inti=0;i<lightmaps.length/*should be 6*/;++i){app.uploadToTexture(i,lightmaps[i]);// app-specific code.}}
Kotlin
funupdate(){// Get the current frame.valframe=session.update()// Get the light estimate for the current frame.vallightEstimate=frame.lightEstimate// Get intensity and direction of the main directional light from the current light estimate.valintensity=lightEstimate.environmentalHdrMainLightIntensityvaldirection=lightEstimate.environmentalHdrMainLightDirectionapp.setDirectionalLightValues(intensity,direction)// app-specific code.// Get ambient lighting as spherical harmonics coefficients.valharmonics=lightEstimate.environmentalHdrAmbientSphericalHarmonicsapp.ambientSphericalHarmonicsLightValues=harmonics// app-specific code.// Get HDR environmental lighting as a cubemap in linear color space.vallightMaps=lightEstimate.acquireEnvironmentalHdrCubeMap();for((index,lightMap)inlightMaps.withIndex()){// 6 maps total.app.uploadToTexture(index,lightMap);// app-specific code.}}
ConfigureAMBIENT_INTENSITYmode
If you're planning to use the color correction component ofAMBIENT_INTENSITYmode, first avoid allocation of color correction on every frame by reusing a shared allocation.
Java
// Avoid allocation on every frame.float[]colorCorrection=newfloat[4];
Get the light estimate for each frame, and then get ambient intensity components
you want to use.
Java
voidupdate(){// Get the current frame.Frameframe=session.update();// Get the light estimate for the current frame.LightEstimatelightEstimate=frame.getLightEstimate();// Get the pixel intensity of AMBIENT_INTENSITY mode.floatpixelIntensity=lightEstimate.getPixelIntensity();// Read the pixel color correction of AMBIENT_INTENSITY mode into colorCorrection.lightEstimate.getColorCorrection(colorCorrection,0);}
Kotlin
funupdate(){// Get the current frame.valframe=session.update()// Get the light estimate for the current frame.vallightEstimate=frame.lightEstimate// Get the pixel intensity of AMBIENT_INTENSITY mode.valpixelIntensity=lightEstimate.pixelIntensity// Read the pixel color correction of AMBIENT_INTENSITY mode into colorCorrection.lightEstimate.getColorCorrection(colorCorrection,0)}
Energy conservationis the principle that light reflected from a surface will
never be more intense than it was before it hit the surface. This rule is
enforced in physically-based rendering, but is usually omitted from legacy
rendering pipelines used in video games and mobile apps.
If you're using a physically-based rendering pipeline with Environmental HDR
light estimation, simply ensure physically-based materials are used in your
virtual objects.
If you aren't using a physically-based pipeline, however, you have a couple of
options:
The most ideal solution for this is to migrate to a physically-based pipeline.
If that isn't possible, however, a good workaround is to multiply the
albedo value from a non-physically-based material by an energy conservation
factor. This can make sure at least theBRDF shading modelcan be converted into physically-based. Each BRDF has a different factor --
for example, for a diffuse reflection it is 1/Pi.
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-10-31 UTC."],[[["\u003cp\u003eThe Lighting Estimation API in ARCore lets you illuminate virtual objects with realistic lighting based on the real-world environment, enhancing their integration into the scene.\u003c/p\u003e\n"],["\u003cp\u003eBefore using the API, ensure familiarity with fundamental AR concepts and session configuration.\u003c/p\u003e\n"],["\u003cp\u003eChoose between \u003ccode\u003eENVIRONMENTAL_HDR\u003c/code\u003e and \u003ccode\u003eAMBIENT_INTENSITY\u003c/code\u003e modes when configuring lighting estimation, or disable it entirely using \u003ccode\u003eDISABLED\u003c/code\u003e mode.\u003c/p\u003e\n"],["\u003cp\u003e\u003ccode\u003eENVIRONMENTAL_HDR\u003c/code\u003e mode provides detailed environmental lighting information for advanced rendering, including main directional light, ambient spherical harmonics, and a HDR cubemap.\u003c/p\u003e\n"],["\u003cp\u003e\u003ccode\u003eAMBIENT_INTENSITY\u003c/code\u003e mode offers a simpler approach by providing pixel intensity and color correction values, suitable for basic lighting adjustments.\u003c/p\u003e\n"]]],["This content explains how to configure and use the Lighting Estimation API in ARCore apps. Key actions include: configuring the session with `ENVIRONMENTAL_HDR`, `AMBIENT_INTENSITY`, or `DISABLED` modes using Java or Kotlin. For `ENVIRONMENTAL_HDR`, retrieve light estimates, including intensity, direction, spherical harmonics, and cubemaps. For `AMBIENT_INTENSITY`, obtain pixel intensity and color correction. Lastly, using physically based material ensures energy conservation in the lighting model.\n"],null,["# Realistically light virtual objects in a scene\n\nLearn how to use [Lighting Estimation](/ar/develop/java/light-estimation)\nin your own apps.\n\nPrerequisites\n-------------\n\nMake sure that you understand [fundamental AR concepts](/ar/develop/fundamentals)\nand how to [configure an ARCore session](/ar/develop/java/session-config) before proceeding.\n\nConfigure the API once per session with the appropriate mode\n------------------------------------------------------------\n\nConfigure Lighting Estimation once per session for the mode you want to use. \n\n### Java\n\n // Configure the session with the Lighting Estimation API in /ar/reference/java/com/google/ar/core/Config.LightEstimationMode#ENVIRONMENTAL_HDR mode.\n Config config = session.getConfig();\n config.setLightEstimationMode(LightEstimationMode.ENVIRONMENTAL_HDR);\n session.configure(config);\n\n // Configure the session with the Lighting Estimation API in /ar/reference/java/com/google/ar/core/Config.LightEstimationMode#AMBIENT_INTENSITY mode.\n Config config = session.getConfig();\n config.setLightEstimationMode(LightEstimationMode.AMBIENT_INTENSITY);\n session.configure(config);\n\n // Configure the session with the Lighting Estimation API turned off.\n Config config = session.getConfig();\n config.setLightEstimationMode(LightEstimationMode.DISABLED);\n session.configure(config);\n\n### Kotlin\n\n // Configure the session with the Lighting Estimation API in /ar/reference/java/com/google/ar/core/Config.LightEstimationMode#ENVIRONMENTAL_HDR mode.\n Config config = session.config\n config.lightEstimationMode = LightEstimationMode.ENVIRONMENTAL_HDR\n session.configure(config)\n\n // Configure the session with the Lighting Estimation API in /ar/reference/java/com/google/ar/core/Config.LightEstimationMode#AMBIENT_INTENSITY mode.\n Config config = session.config\n config.lightEstimationMode = LightEstimationMode.AMBIENT_INTENSITY\n session.configure(config)\n\n // Configure the session with the Lighting Estimation API turned off.\n Config config = session.config\n config.lightEstimationMode = LightEstimationMode.DISABLED\n session.configure(config)\n\nConfigure `ENVIRONMENTAL_HDR` mode\n----------------------------------\n\nTo configure [ENVIRONMENTAL_HDR](/ar/reference/java/com/google/ar/core/Config.LightEstimationMode#ENVIRONMENTAL_HDR) mode, get the light estimate for each frame,\nthen get the environmental HDR lighting components you want to use. \n\n### Java\n\n void update() {\n // Get the current frame.\n Frame frame = session.update();\n\n // Get the light estimate for the current frame.\n LightEstimate lightEstimate = frame.getLightEstimate();\n\n // Get intensity and direction of the main directional light from the current light estimate.\n float[] intensity = lightEstimate.getEnvironmentalHdrMainLightIntensity(); // note - currently only out param.\n float[] direction = lightEstimate.getEnvironmentalHdrMainLightDirection();\n app.setDirectionalLightValues(intensity, direction); // app-specific code.\n\n // Get ambient lighting as spherical harmonics coefficients.\n float[] harmonics = lightEstimate.getEnvironmentalHdrAmbientSphericalHarmonics();\n app.setAmbientSphericalHarmonicsLightValues(harmonics); // app-specific code.\n\n // Get HDR environmental lighting as a cubemap in linear color space.\n Image[] lightmaps = lightEstimate.acquireEnvironmentalHdrCubeMap();\n for (int i = 0; i \u003c lightmaps.length /*should be 6*/; ++i) {\n app.uploadToTexture(i, lightmaps[i]); // app-specific code.\n }\n }\n\n### Kotlin\n\n fun update() {\n // Get the current frame.\n val frame = session.update()\n\n // Get the light estimate for the current frame.\n val lightEstimate = frame.lightEstimate\n\n // Get intensity and direction of the main directional light from the current light estimate.\n val intensity = lightEstimate.environmentalHdrMainLightIntensity\n val direction = lightEstimate.environmentalHdrMainLightDirection\n app.setDirectionalLightValues(intensity, direction) // app-specific code.\n\n // Get ambient lighting as spherical harmonics coefficients.\n val harmonics = lightEstimate.environmentalHdrAmbientSphericalHarmonics\n app.ambientSphericalHarmonicsLightValues = harmonics // app-specific code.\n\n // Get HDR environmental lighting as a cubemap in linear color space.\n val lightMaps = lightEstimate.acquireEnvironmentalHdrCubeMap();\n for ((index, lightMap) in lightMaps.withIndex()) { // 6 maps total.\n app.uploadToTexture(index, lightMap); // app-specific code.\n }\n }\n\nConfigure `AMBIENT_INTENSITY` mode\n----------------------------------\n\nIf you're planning to use the color correction component of [AMBIENT_INTENSITY](/ar/reference/java/com/google/ar/core/Config.LightEstimationMode#AMBIENT_INTENSITY)\nmode, first avoid allocation of color correction on every frame by reusing a shared allocation. \n\n### Java\n\n // Avoid allocation on every frame.\n float[] colorCorrection = new float[4];\n\n### Kotlin\n\n val colorCorrection = floatArrayOf(0.0f, 0.0f, 0.0f, 0.0f)\n\nGet the light estimate for each frame, and then get ambient intensity components\nyou want to use. \n\n### Java\n\n void update() {\n // Get the current frame.\n Frame frame = session.update();\n\n // Get the light estimate for the current frame.\n LightEstimate lightEstimate = frame.getLightEstimate();\n\n // Get the pixel intensity of AMBIENT_INTENSITY mode.\n float pixelIntensity = lightEstimate.getPixelIntensity();\n\n // Read the pixel color correction of AMBIENT_INTENSITY mode into colorCorrection.\n lightEstimate.getColorCorrection(colorCorrection, 0);\n }\n\n### Kotlin\n\n fun update() {\n // Get the current frame.\n val frame = session.update()\n\n // Get the light estimate for the current frame.\n val lightEstimate = frame.lightEstimate\n\n // Get the pixel intensity of AMBIENT_INTENSITY mode.\n val pixelIntensity = lightEstimate.pixelIntensity\n\n // Read the pixel color correction of AMBIENT_INTENSITY mode into colorCorrection.\n lightEstimate.getColorCorrection(colorCorrection, 0)\n }\n\nEnsuring *energy conservation* with Environmental HDR APIs\n----------------------------------------------------------\n\n*Energy conservation* is the principle that light reflected from a surface will\nnever be more intense than it was before it hit the surface. This rule is\nenforced in physically-based rendering, but is usually omitted from legacy\nrendering pipelines used in video games and mobile apps.\n\nIf you're using a physically-based rendering pipeline with Environmental HDR\nlight estimation, simply ensure physically-based materials are used in your\nvirtual objects.\n\nIf you aren't using a physically-based pipeline, however, you have a couple of\noptions:\n\n- The most ideal solution for this is to migrate to a physically-based pipeline.\n\n- If that isn't possible, however, a good workaround is to multiply the\n albedo value from a non-physically-based material by an energy conservation\n factor. This can make sure at least the [BRDF shading model](https://en.wikipedia.org/wiki/Bidirectional_reflectance_distribution_function)\n can be converted into physically-based. Each BRDF has a different factor --\n for example, for a diffuse reflection it is 1/Pi."]]
