These Maps Are Made For Walking: Real-Time Terrain Property Estimation for Mobile Robots (RA-L 2022)

Created on June 11, 2026

2026   ·   terrain   legged   mapping   traversability   ·   paper-review

Item Details
Authors Parker Ewen, Adam Li, Yuxin Chen, Steven Hong, Ram Vasudevan (University of Michigan, Robotics Institute)
Venue IEEE Robotics and Automation Letters (RA-L) vol. 7 no. 3, 2022 (presented at IROS 2022)
Links arXiv · Code · Dataset

One-line Summary

This is a real-time semantic mapping framework that recursively and jointly estimates the probability distributions of terrain elevation and friction coefficient from a single RGB-D camera using Bayesian inference. The class probabilities produced by a semantic segmentation network are accumulated with a Dirichlet conjugate prior, and directly measured per-class friction distributions are mixed with these weights to output the friction of each terrain tile as a multimodal Gaussian distribution. The goal is to go beyond a binary “traversable/non-traversable” label so that a legged robot can quantitatively know how slippery a surface is and adjust its gait accordingly.

The paper sits at the junction of three lines of prior work.

  • Geometric mapping: The probabilistic terrain elevation map of Fankhauser et al. (2018) is the standard starting point. It represents the terrain surface as a 2.5D triangular mesh and absorbs sensor and pose noise with a Kalman filter. The elevation map module of this paper directly continues this lineage. Its limitation is that it knows only geometry and nothing about physical properties such as friction.
  • Semantic mapping: Neural segmentation is applied to images or point clouds and terrain classes are projected onto the geometry. Brandão’s “Friction from vision” (2016) and Noh’s SMDRA dataset (2021) predicted friction from RGB. The limitation is that these are non-recursive, so they cannot refine noisy estimates with prior information.
  • Traversability estimation: This bypasses property estimation and only estimates “whether the robot can go there.” However, traversability depends on internal state such as the robot’s locomotion mode and speed, which causes over- and under-approximation (Kim 2006).

This paper simultaneously achieves geometry (elevation) + physical properties (friction) + recursive Bayesian updates + distributional outputs. Follow-up work connects to ETH Zurich’s Multi-Modal Elevation Mapping (2023) and the self-supervised traversability learning family, but the citation relationships to later work were only partially confirmed.

Problem and Motivation

The equations of motion of mobile and legged robots depend on terrain properties (friction coefficient, contact model parameters) (Neunert 2018). Knowing only “where it is high” is not enough to plan stable locomotion; one must know “how slippery that surface is.”

An ideal property map has three requirements: (1) real-time operation, (2) mitigation of sensor noise, and (3) providing properties as probability distributions rather than single values, so that risk-averse planning becomes possible. Existing methods were non-recursive, produced only point estimates, or lacked physical properties altogether, and so could not satisfy all three at once.

Key Ideas and Method

Representation: a probabilistic triangular mesh

The terrain is represented as a 2.5D triangular mesh. Each vertex carries its height and variance as \([x, y, z, \sigma^2]\), and each face has three vertices, a set of interior points, and a Dirichlet parameter vector \(\alpha\). By analogy, the terrain is a floor tiled with small triangles, where each tile holds a self-description such as “my height is this much plus or minus this error, and I am 60% likely to be grass and 30% likely to be concrete.”

Recursive elevation map: a 1D Kalman filter

Points generated from RGB-D depth are assigned to faces by a Barycentric-coordinate inside-triangle test. Vertex heights are updated with a one-dimensional Kalman filter. The measurement variance is computed by combining the depth-sensor noise covariance \(\Sigma_s\) and the camera pose covariance \(\Sigma_p\) through the law of error propagation.

\[\sigma^2 = J_s \Sigma_s J_s^T + J_p \Sigma_p J_p^T\]

The more often the same tile is observed, the sharper its height estimate becomes.

Recursive terrain class estimation: Dirichlet-Categorical conjugacy

This is the core mathematical trick. A semantic segmentation network outputs a Categorical distribution (class probabilities) per pixel. Using its conjugate prior, the Dirichlet distribution, the posterior can be updated by simple addition without any integration whenever a new observation arrives.

\[\tilde{\alpha}_j = \alpha_j + \sum_i \mathbb{1}\{z_i = j\}\]

The probability that a new measurement belongs to class \(i\) is given in closed form by \(f(z=i \mid Z, \alpha) = \tilde{\alpha}_i / \sum_j \tilde{\alpha}_j\). It is like dropping colored marbles into an urn: every time a tile is classified as grass, one more green marble goes in, and the ratio in the urn is the class probability. This closed-form update is the mathematical heart of the real-time capability.

From class to physical property: the law of total probability and a Gaussian mixture

The friction coefficient of each class is modeled not as a single value but as a conditional distribution \(f(\psi \mid z=i) = \mathcal{N}(\mu_i, \sigma_i^2)\), and these are combined with the law of total probability.

\[f(\psi \mid Z, \alpha) = \sum_i \frac{\alpha_i}{\sum_j \alpha_j} \cdot \mathcal{N}(\mu_i, \sigma_i^2)\]

That is, a multimodal Gaussian mixture weighted by class probabilities. If “this tile is 70% concrete (\(\mu \approx 0.54\)) + 30% ice (\(\mu \approx 0.19\)),” the friction distribution comes out with two peaks. This preserves the uncertainty that a single point estimate would discard.

Friction dataset: the second contribution

Existing datasets had only about three measurements per class or were not public. The authors built a device that measures pulling force with a load cell, computed the friction coefficient as \(\mu = F_{pull}/(mg)\), and collected about 10,000 samples across 10 terrain classes. Among Gaussian, Weibull, and log-normal candidates, a Kolmogorov-Smirnov test confirmed that the unimodal Gaussian generalized best, so it was adopted. Representative parameters are as follows.

Class \(\mu\) \(\sigma\)
Rubber 0.616 0.048
Grass 0.577 0.077
Concrete 0.543 0.065
Gravel 0.428 0.059
Snow 0.390 0.071
Laminate flooring 0.311 0.045
Ice 0.192 0.046

Experiments and Results

Implementation: C++ with a ROS interface. A Realsense RGB-D noise model is built in. It maps a 5 m radius around the robot at 2 cm resolution and runs at 9±5 Hz. The development hardware was a Ryzen 3600, 32 GB RAM, and an RTX 2080 Ti.

Simulation: Evaluated in CARLA, which provides ground-truth friction per terrain. An ADE20K-pretrained segmentation network was used. The baselines were (1) Unimodal Non-Recursive (a unimodal Gaussian from the most likely class at each instant, representing point-estimate CNN approaches) and (2) Multimodal Non-Recursive (a multimodal estimate from the full Categorical distribution, but without recursive updates).

Real robot: Demonstrated on a Boston Dynamics Spot with an onboard Realsense D435 across various indoor and outdoor terrains (ice, hills, low vegetation, etc.), and compared against traversability mapping (Gan 2021). The paper also mentions that it runs on Agility Robotics Digit.

The three key results are as follows.

  1. Lowest KL divergence at 2.4. Compared with 42.3 for Unimodal Non-Recursive and 3.7 for Multimodal Non-Recursive, it is closest to the ground-truth distribution. Recursive updating raises distributional accuracy.
  2. Large margin in low-friction (\(\mu \le 0.5\)) regions. High-friction regions are comparable to the baselines, but on the safety-critical identification of slippery terrain it is clearly ahead on the Precision-Recall curve. Average precision is 0.99 and average accuracy 0.95 (Multimodal Non-Recursive: 0.99/0.93).
  3. Real-time achieved. With a 1 cm mesh, of the total 527 ms the segmentation network takes 477 ms, while the elevation and property updates take only 50 ms. Swapping the segmentation network for Fast-SCNN reduces the total to about 200 ms. Memory is about 45-55 MB per 1 m × 1 m mesh.

The qualitative real-robot results are also clear. On ice, traversability mapping misjudges the surface as “safe,” whereas this method correctly predicts low friction. On hills and low vegetation, traversability under-approximates as “non-traversable,” whereas this method estimates normally. However, when a class not in Table I is estimated, no friction estimate is produced and the tile is painted gray.

Checking the claimed contributions, the dataset release and the closed-form Bayesian recursive framework do hold up. The “9±5 Hz real-time” claim depends strongly on segmentation-network performance, so taken literally it is somewhat optimistic, but it is true that the map update itself (50 ms) is fast.

Conclusion and Significance

This paper elevated legged-robot mapping from “geometry only” to “geometry + probability distributions of physical properties.” The core message is not to avoid slippery terrain, but to know the slipperiness and adjust the gait. The mathematical design that solves real-time recursive updating with a conjugate prior has high reuse value in its own right.

From a robotics practice perspective, the takeaways are as follows.

  • Where it can be used: Legged robots that need a distributional friction map as input to risk-aware footstep planning, or cases that require a vision-based property-estimation layer for driving on unstructured outdoor terrain. The MIT license, the friction dataset, and the rosbag- and CARLA-based validation paths are all public, so it can be tested without a real robot.
  • Reproduction difficulty is moderate: There are many setup steps, including ROS Noetic, CUDA 11+, PyTorch, and manual placement of external segmentation checkpoints. An NVIDIA GPU is effectively required, and CPU-only may run into memory problems.
  • Representativeness of the friction measurements: Pull-type load-cell measurements do not fully capture actual foot-ground contact dynamics (normal-force and velocity dependence, impacts). How well the per-class mean friction matches actual locomotion safety remains an open question.
  • Segmentation dependence and the open-set problem: Accuracy and speed are governed by external segmentation performance, and novel terrain outside the trained classes cannot be estimated. Even the same “grass” has different friction when wet or dry, but it is fixed as a static Gaussian, so temporal and weather variation is not reflected.
  • Planner integration not evaluated: End-to-end verification of how much the output distributions actually improve footstep planning performance is out of scope. LiDAR and multi-camera extensions are only mentioned, not implemented.

References

  • Paper: arXiv 2205.12925 (DOI 10.1109/LRA.2022.3180439) · Code: github.com/roahmlab/sel_map · Dataset: terrain_friction_dataset
  • Fankhauser, Bloesch, Hutter, Probabilistic Terrain Mapping for Mobile Robots with Uncertain Localization (2018)
  • Brandão, Hashimoto, Takanishi, Friction from Vision (2016)
  • Noh et al., SMDRA: Surface Material Dataset for Robotics Applications (2021)
  • Tu, The Dirichlet-Multinomial and Dirichlet-Categorical Models for Bayesian Inference (2014)
  • Gan et al., Multi-task Learning for Scalable and Dense Multi-layer Bayesian Map Inference (2021)