Why Detect Landslides on Mars?
Autonomous exploration of the Martian surface depends on accurate geomorphological mapping to identify hazards that threaten robotic assets. Landslides are among the most common mass-wasting events on Mars, shaping the planet’s terrain while posing direct risks to rovers and landers.
Recent datasets such as MMLSv2 have expanded the field from single-band analysis to multispectral imagery. These datasets stack multiple sensing modalities: visible RGB basemaps, digital elevation models (DEM), slope maps, and thermal inertia. Each band captures different physical properties of the surface, and their combination provides richer context for identifying landslide boundaries.
The challenge is that standard supervised models trained from scratch struggle to exploit these cross-modal relationships. Generative self-supervised learning (SSL) methods can learn rich features from unlabeled data, but their reliance on pixel reconstruction makes them computationally expensive for high-dimensional multispectral inputs. A more efficient alternative is needed.
What is JEPA?
The Joint Embedding Predictive Architecture (JEPA) is a self-supervised framework that learns representations by predicting in the latent space rather than reconstructing pixels. Given a context signal (a perturbed version of the input) and a target signal (the original), JEPA trains an encoder to extract features from both. A predictor module then predicts the target features from the context features, conditioned on the perturbation. The model is trained to minimize the L2 distance between predicted and actual target embeddings.
Unlike generative approaches such as MAE, JEPA avoids pixel-level reconstruction. This makes it more efficient for high-dimensional inputs and less prone to learning trivial features tied to low-level noise. The I-JEPA variant applies this framework to images using a shared spatial mask across all channels.
The limitation of I-JEPA for multispectral data is clear. When a spatial region is masked, the model loses information from all spectral bands at that location simultaneously. It can only rely on surrounding spatial context. This prevents the encoder from learning the physical correlations that exist between different sensing modalities at the same spatial location.
Spatio-Spectral JEPA (SS-JEPA)
Our key insight is straightforward: for a given spatial region, different spectral bands carry independent but correlated information. To learn meaningful representations from multispectral data, the model must understand these cross-modal relationships.
SS-JEPA addresses this by applying independent masks per spectral channel. Instead of masking a spatial patch across all bands simultaneously, each channel is masked separately. This means that for any given spatial location, some channels may be visible while others are masked. The encoder must then reconstruct the latent features of a masked patch in one band (e.g., thermal inertia) using visible information from other bands (e.g., DEM or RGB).
Architecture Details
SS-JEPA uses a modified Vision Transformer (ViT) as the encoder. Each patch in each channel is tokenized independently, producing a token for every (channel, spatial location) pair. The model uses combined spatial and spectral positional embeddings:
$$p_{c,j} = P_{\text{spatial}}[j] + P_{\text{spectral}}[c]$$
where $p_{c,j}$ is the positional embedding for channel $c$ at spatial location $j$. This ensures that each token has a unique identity encoding both where it is spatially and which spectral band it belongs to.
During pre-training, the masking ratio is linearly scheduled from 0.5 to 0.85 over 150 epochs. The model uses SIGReg regularization ($\lambda = 0.3$) to prevent latent space collapse, a common failure mode in embedding-based SSL where the encoder collapses to a constant output. Training uses the AdamW optimizer with cosine annealing and linear warm-up. All experiments run on a single NVIDIA RTX A4500 GPU with 20 GB VRAM.
From Encoder to Segmentation
After pre-training, the frozen encoder is integrated with a UPerNet segmentation decoder. A multiscale feature pyramid is constructed from the encoder output, providing the decoder with both high-resolution skip connections and deep semantic features. The segmentation head is trained for 70 epochs with a combined Dice loss and binary cross-entropy loss. Only the decoder parameters are updated; the encoder remains frozen to evaluate the quality of the learned representations.
Results
Backbone and Decoder Compatibility
We evaluated both I-JEPA and SS-JEPA encoders paired with eight different segmentation decoders: UPerNet, UNet, UNet++, FPN, MANet, DeepLabV3+, PAN, and SegFormer. Results are reported as mIoU mean and standard deviation over five random seeds.
| Encoder | Segmentation | mIoU (mean +/- std) | FG IoU | BG IoU |
| I-JEPA | SegFormer | 0.837 +/- 0.0066 | 0.790 | 0.884 |
| I-JEPA | UNet | 0.829 +/- 0.0017 | 0.780 | 0.878 |
| I-JEPA | UPerNet | 0.825 +/- 0.0021 | 0.774 | 0.876 |
| SS-JEPA | UPerNet | **0.865 +/- 0.0057** | **0.820** | **0.910** |
| SS-JEPA | UNet | 0.852 +/- 0.0062 | 0.824 | 0.880 |
| SS-JEPA | UNet++ | 0.840 +/- 0.0028 | 0.807 | 0.873 |
The best SS-JEPA configuration (SS-JEPA + UPerNet) achieves 0.865 mIoU, a 2.8% improvement over the best I-JEPA baseline (I-JEPA + SegFormer at 0.837 mIoU). SS-JEPA outperforms I-JEPA across every decoder architecture tested.
The reason is the nature of the predictive task during pre-training. I-JEPA with shared masking forces the model to rely on spatial context alone. SS-JEPA with independent channel masking creates an inter-modal prediction task. The encoder must learn to reconstruct features in one spectral band from visible features in other bands, capturing the underlying physical relationships between modalities.
Comparison with State-of-the-Art
| Model | mIoU | Precision | Recall | F1 |
| U-Net | 0.814 | 0.858 | 0.868 | 0.863 |
| U-Net++ | 0.823 | 0.864 | 0.879 | 0.871 |
| DeepLabV3+ | 0.829 | 0.863 | 0.889 | 0.876 |
| SegFormer | 0.812 | 0.859 | 0.863 | 0.861 |
| TinySwin+FPN | 0.832 | 0.842 | 0.892 | 0.866 |
| DualSwinV2 5-fold ensemble | **0.867** | 0.900 | 0.911 | 0.905 |
| SS-JEPA + UPerNet (Ours) | **0.865** | 0.895 | 0.907 | 0.901 |
| I-JEPA + SegFormer (Ours) | 0.837 | 0.858 | 0.876 | 0.865 |
Standard supervised architectures (U-Net, U-Net++, DeepLabV3+, SegFormer) plateau between 0.81 and 0.83 mIoU when trained from scratch. The previous state-of-the-art single-model backbone, DualSwinV2-S, reaches 0.828 mIoU. Our SS-JEPA + UPerNet surpasses it by 3.7%.
Most notably, SS-JEPA + UPerNet achieves 0.865 mIoU, within 0.2% of the 5-fold DualSwinV2 ensemble at 0.867 mIoU. The critical difference is model size: our approach uses 20M parameters compared to the ensemble’s 113M, a 5.6x reduction. For planetary exploration platforms with limited processing power, this efficiency is essential.
Channel Ablation Study
To understand the contribution of individual spectral bands, we pre-trained SS-JEPA on different channel subsets with the UPerNet decoder.
| RGB | Gray | DEM | Slope | Thermal | mIoU |
| x | x | | | | 0.812 |
| | | x | x | | 0.835 |
| x | x | x | | | 0.841 |
| x | x | x | x | | 0.854 |
| | | x | x | x | 0.846 |
| x | x | x | x | x | **0.865** |
Optical-only channels (RGB + Grayscale) yield 0.812 mIoU, limited by atmospheric dust and uniform albedo. Structural channels (DEM + Slope) improve to 0.835 mIoU, as topographic features like elevation changes at source scarps provide discriminative cues that are invariant to illumination. The full 7-channel configuration reaches 0.865 mIoU. Even though thermal inertia has lower spatial resolution, it contributes a 0.011 mIoU gain when combined with geometric features.
Efficiency Analysis
The performance-complexity trade-off places SS-JEPA + UPerNet on the Pareto frontier. With 20M parameters and 120.5 GFLOPs, it matches the accuracy of the 113M-parameter, 135 GFLOP DualSwinV2 ensemble. The transition from I-JEPA to SS-JEPA yields a 15.6% parameter reduction alongside a 2.8% mIoU gain, demonstrating that the spatio-spectral masking strategy produces more effective representations per parameter.
Latent Representation Analysis
The quality of the learned representations can be assessed through clustering metrics on the latent embeddings.
| Model | Silhouette (higher is better) | Davies-Bouldin (lower is better) | Calinski-Harabasz (higher is better) |
| I-JEPA | 0.017 | 5.251 | 521.642 |
| SS-JEPA | **0.090** | **2.699** | **3778.412** |
SS-JEPA produces substantially more compact and better-separated latent clusters. The Silhouette score improves by 5.3x, and the Calinski-Harabasz index improves by 7.2x. The singular value spectrum of SS-JEPA encodings shows broader eigenspectrum with slower decay compared to I-JEPA, indicating better utilization of latent dimensions and richer feature diversity.
Limitations and Future Work
SS-JEPA shows a bias toward broad spatial patterns. For landslides with small spatial extent, I-JEPA outperforms SS-JEPA in approximately 85% of cases, with performance differences ranging from 0.005 to 0.3 mIoU. The large number of patch tokens allows background information to overwhelm localized features, and the linear projection layer acts as a bottleneck during multimodal fusion, potentially smoothing high-frequency spatial signals.
The scarcity of labeled Martian data also remains a constraint. Further validation across broader planetary datasets would strengthen claims of generalizability. Future work will investigate adaptive weighting mechanisms to preserve localized signals during multimodal fusion.
Conclusion
SS-JEPA demonstrates that latent-space prediction is more robust to Martian sensor noise and varying lighting conditions than traditional generative SSL. By masking spectral channels independently, the model learns cross-modal correlations that supervised methods and spatial-only SSL miss. The result is a single-model architecture that matches the accuracy of heavy ensembles at a fraction of the parameter count, offering a practical pathway for automated terrain analysis in planetary exploration.