MATT-Diff
MATT-Diff is a diffusion-policy-based framework for active multi-target tracking in partially observable indoor environments. It learns multi-modal action strategies from three expert planners, using a preformer map encoder and attention-based target encoder over Gaussian beliefs. By training on demonstrations from diverse expert strategies, MATT-Diff captures the inherent multi-modality of the tracking task and generalizes to unseen environments at test time.
Network Overview
MATT-Diff learns multi-modal action strategies from multiple expert planners using a Performer map encoder and attention-based target encoder over Gaussian beliefs.
MATT-Diff conditions a denoising diffusion model on two learned encodings of the agent's current observation.
Map Encoder — processes a 4-channel ego-centric local map: (1) known free space (binary), (2) unknown space (binary), (3) visit frequency (continuous, normalized), and (4) current field of view (binary). A CNN stem downsamples the input, then dual-scale patch tokens are fed into a Performer Transformer (linear attention with Random Fourier Features + RoPE), and the CLS token is used as the map embedding.
Target Encoder — processes the Gaussian belief state (position mean and covariance) of each target slot. Undetected or lost targets are replaced by a learned null token. A Transformer encoder with self-attention processes all slots, and detected slots are aggregated via masked mean pooling into a fixed-size embedding. When no targets are detected, a global null fallback embedding is used.
Both embeddings are concatenated and passed to the diffusion policy network, which iteratively denoises a sampled action over multiple diffusion steps at inference time.
MATT-Diff is trained via imitation learning on demonstrations from three expert planners, each capturing a distinct high-level strategy:
Frontier-based explore planner
A pure exploration planner that navigates toward unexplored frontier cells, maximizing map coverage.
Uncertainty-based hybrid planner
A hybrid exploration-exploitation planner that prioritizes re-acquiring previously detected targets whose belief uncertainty has grown.
Time-based hybrid planner
A hybrid exploration-exploitation planner that switches to tracking detected targets based on elapsed time since last observation.
Quantitative comparison of MATT-Diff against baselines.
MATT-Diff acquires behaviors learned from all expert planners — it represents each distinct strategy (exploration, re-acquisition, and target tracking) that the expert planners exhibit.
Trajectory snapshots showing that MATT-Diff reproduces the behavior of each expert planner.
@inproceedings{liu2026mattdiff,
title = {MATT-Diff: Multimodal Active Target Tracking by Diffusion Policy},
author = {Liu, Saida and Atanasov, Nikolay and Koga, Shumon},
booktitle = {Proceedings of the 8th Annual Learning for Dynamics \& Control Conference (L4DC)},
year = {2026},
note = {To appear}
}