The Meta 2’s movement within your environment is tracked with six degrees of freedom. In other words, as you move your head around, the Meta 2’s position and orientation are both accounted for, allowing you to place virtual objects within your environment and view them from different angles just as you would a real object.
The Meta 2 is tracked using SLAM, which stands for simultaneous localization and mapping. Specifically, it employs inside-out tracking, which means that all tracking information is derived from sensors onboard the device and no external tracking hardware is necessary. The Meta 2’s SLAM system combines data from cameras and inertial measurement units (IMUs) to detect how it moves within your environment. Its IMUs use accelerometers to track linear acceleration and gyroscopes to track rotation.
Within Unity, tracking is built into the
MetaCameraRig prefab. The
SlamLocalizer object provides tracking information (head position and rotation) for behavior updates, while the Compositor updates tracking information for rendering.
Please note that the head position which the Meta 2 reports is offset slightly up and back relative to the user’s eyes.
- Setup your scene.
- Follow the instructions presented in the headset to initialize tracking.
Limited Tracking vs. Full Tracking
In Limited Tracking mode, the Meta 2’s rotation (using the onboard IMU) continues to be tracked, but the Meta 2’s translation is no longer tracked. In other words, holograms will always appear at the same distance from the user no matter how the user’s head moves. This mode may also be referred to as “rotation-only tracking” or “three degrees of freedom (3DOF),” as opposed to Full Tracking, which may be referred to as “six degrees of freedom (6DOF).”
- A Unity application can be configured to launch in limited tracking mode. To do so, check the “Rotation Only Tracking” checkbox on the SLAM Localizer component of the MetaCameraRig prefab. If this box is checked, the SLAM initialization UI will not be shown. This option is useful for applications which do not depend on the user’s movement through their space, e.g. watching a 2D video feed.
- In Unity applications using the MetaCameraRig, limited tracking mode can be toggled on and off by pressing the F9 key. Pressing the F4 key while in Limited Tracking mode recenters the scene.
- If SLAM is unable to initialize or loses the ability to track the environment, the Meta 2 automatically switches to limited tracking mode. When SLAM is again able to map the environment, the Meta 2 will automatically return to full tracking mode. In Unity applications using the MetaCameraRig, the corresponding messages indicating either “Full Tracking” or “Limited Tracking” will temporarily appear in the headset when the tracking mode changes.
- In Limited Tracking mode, CPU usage is lower and hologram stability is improved.
SLAM Initialization Versus Relocalization
When a Meta application launches, the tracking system (SLAM) scans your environment. Instructions appear in the headset directing you to turn your head slowly from side to side in order to give the Meta 2’s cameras a good view of your surroundings. Once the tracking system has sufficient information, the instructions will fade out and the application’s content will appear.
When you launch a Meta application in the location where you last launched a Meta application, the tracking system may recognize the environment. In this case, it will “relocalize,” bypassing the normal initialization procedure and switching directly to the application’s content. The ability to relocalize is enabled by default and can be toggled on the MetaCameraRig prefab.
To Enable Relocalization:
Make sure you have the MetaCameraRig prefab in your scene.
Highlight to select the EnvironmentInitialization GameObject in the hierarchy panel.
Check the Slam Relocalization Active checkbox on the Environment Configuration component in the Inspector panel.
Save your scene.
In the event that the system can not complete the relocalization in a specified number of seconds, then the standard initialization will be triggered.
- If an application is running and you set the headset down for an extended period of time, tracking will be temporarily lost, and in certain cases it may not recover. Should you encounter this issue, press
F4to reinitialize tracking.
When launching an application on the Meta 2, it is important to note that the Meta 2’s SLAM system must be initialized before it can function. During this initialization period, the system will map your surroundings and attempt to determine the Meta 2’s position and orientation within that space. As the application runs, the SLAM system continues to process your surroundings and IMU data in order to robustly track your motion.
The best results for SLAM tracking are achieved when:
1. SLAM tracking is used in a static environment with distinct features or patterns which break up the homogeneity of the space.
Generally, this means that the more features (i.e. clutter) there are to track, the better the quality of tracking will be. An important exception to this guidance is that highly repetitive uniform patterns are difficult to track.
2. During the initialization phase, the user needs to turn their head slowly as per the instructions.
This allows the system to best capture the user’s space.
3. The user should refrain from rapidly moving to a new environment or location where their surroundings change significantly from where SLAM was initialized.
Lighting is also a factor. The SLAM system may have a difficult time locating existing features in the environment after a significant lighting change.
4. The user maintains a 10-20 inch buffer distance between the cameras and the closest objects.
The cameras are able to see more features within your environment within this range.
5. Avoid extreme lighting conditions, either too bright (direct sunlight from a window), or too dark (e.g. just light from your monitor).
Although SLAM may function in such conditions, we recommend altering your space’s lighting to a more moderate level if it does not.
6. Avoid extremely dynamic environments. SLAM will dismiss overly stimulating objects.
SLAM is capable of ignoring moving objects within your environment, but if too much is moving within your environment, SLAM may have trouble tracking the static features within your space.
Note: To re-initialize environment mapping at any time, use the F4 key.
The SLAM system exposes several event callbacks that you can use to react to changes in the state of SLAM. These can be useful for making your application more sensitive to the state of SLAM, e.g. by prevent users from interacting with your scene between the time the application launches and SLAM has initialized.
- “On Slam Sensors Ready” fires when initialization begins. Initialization happens almost immediately; just move your head a little bit and initial mapping begins.
- “On Slam Mapping in Progress” fires when scale estimation begins. This step uses the IMU to track head rotation; the gyroscope is used for smooth visualization. Following this, the SLAM system engages visual inertial mode (“VISLAM”). This step detects angular velocities as well as translational acceleration from the IMU and fuses those data streams with SLAM data from the cameras to produce low latency tracking data.
- “On Slam Mapping Complete” fires when SLAM has finished initializing.
- “On Slam Map Loading Failed” fires when SLAM tries to match the current environment to a saved map and fails to do so.
- “On Slam Tracking Lost” fires when the SLAM system is unable to locate sufficient features within the known mapping of your space. At this point, SLAM can no longer provide positional tracking. This frequently means that sensors are blocked, e.g. by covering cameras with a hand. When the SLAM system is in this state, it will attempt to recover by relocalizing.
- “On Slam Tracking Relocalized” fires when the SLAM system relocalizes after previously losing tracking.
- “On Slam Initialization Failed” fires when the SLAM system fails to create a map.
- “On Slam Localizer Reset” fires when the SLAM system resets internally to prepare for another environment mapping attempt.