Upload folder using huggingface_hub

README.md

@@ -0,0 +1,111 @@
+---
+license: cc-by-nc-sa-4.0
+library_name: mast3r
+tags:
+  - 3d-reconstruction
+  - uncertainty-quantification
+  - evidential-deep-learning
+  - pointmap
+  - mast3r
+  - dust3r
+---
+
+# Trust3R — evidential uncertainty for feed-forward 3D reconstruction
+
+Checkpoints for **“Trust It or Not: Evidential Uncertainty for Feed-Forward 3D
+Reconstruction with Trust3R”** (ICML 2026).
+
+- Code: https://github.com/phai-lab/Trust3R
+- Project page: https://trust3r-z.github.io/
+- Paper: https://arxiv.org/abs/2605.19539
+
+Trust3R adds two lightweight heads to a **frozen MASt3R backbone**: an evidential
+uncertainty head that predicts the parameters of a Normal-Inverse-Wishart prior over each
+3D point — yielding a closed-form Student-*t* predictive distribution and a calibrated
+per-pixel uncertainty map in a **single forward pass, no ensembles and no Monte Carlo
+sampling** — and a gated residual head that applies small, gated corrections to the
+pretrained pointmap.
+
+## Files
+
+| File | Head | Backbone | Res. | Size |
+|---|---|---|---|---|
+| `trust3r_niw_mast3r_224.pth` | NIW evidential (full 3×3 covariance) + gated residual | frozen MASt3R ViT-L | 224 | 3.0 GB |
+| `trust3r_nig_mast3r_224.pth` | NIG evidential (diagonal variance) + gated residual | frozen MASt3R ViT-L | 224 | 3.0 GB |
+
+NIW is the main model. NIG is the evidential-family ablation. Each `.pth` ships a
+`.sha256` sidecar and a `.metadata.json` recording provenance, training mix and the
+evaluation protocol.
+
+## Download
+
+```bash
+pip install -U "huggingface_hub[cli]"
+mkdir -p checkpoints
+hf download phai-lab/Trust3R \
+    trust3r_niw_mast3r_224.pth trust3r_nig_mast3r_224.pth \
+    trust3r_niw_mast3r_224.pth.sha256 trust3r_nig_mast3r_224.pth.sha256 \
+    --local-dir checkpoints/
+(cd checkpoints && sha256sum -c *.sha256)
+```
+
+## Usage
+
+```python
+from mast3r.model import AsymmetricMASt3R
+
+model = AsymmetricMASt3R.from_pretrained("checkpoints/trust3r_niw_mast3r_224.pth").eval()
+```
+
+The model expression is stored inside the checkpoint, so no architecture arguments are
+needed. A minimal pair-inference example is `infer.py` in the GitHub repo; the NIW
+predictive variance is recovered from the head outputs as
+
+```python
+kappa = pred1["xyz_niw_kappa"]           # (1, 1, H, W)
+nu    = pred1["xyz_niw_nu"]              # (1, 1, H, W)
+Psi   = pred1["xyz_niw_Psi"]             # (1, 3, 3, H, W)
+trace_Psi = Psi[:, 0, 0] + Psi[:, 1, 1] + Psi[:, 2, 2]
+total_var = trace_Psi / (kappa.squeeze(1) * (nu.squeeze(1) - 4.0).clamp_min(1e-3))
+```
+
+Provenance and protocol metadata:
+
+```python
+import torch
+print(torch.load("checkpoints/trust3r_niw_mast3r_224.pth", map_location="cpu")["trust3r"])
+```
+
+## Training
+
+Initialised from `MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth`, backbone frozen,
+150k steps at 224px (batch 10, 10 epochs) on a four-dataset mix of 150k pairs per epoch:
+ScanNet++ (25k), ARKitScenes (25k), Waymo (50k) and MegaDepth (50k). AdamW, base LR
+3e-4 with cosine schedule, evidence regularisation λ_evi = 1e-3.
+
+## Evaluation
+
+`eval/reproduce_table1_table2.sh` in the GitHub repo reproduces the paper tables from
+these checkpoints — AURC, AUSE, Spearman ρ, Sim(3)-aligned MAE/RMSE and NLL over
+ScanNet++, TUM RGB-D, KITTI and ETH3D. See `eval/README.md` there for the protocol.
+
+## License and intended use
+
+**CC BY-NC-SA 4.0 — non-commercial use only**, inherited from MASt3R and DUSt3R. See
+`CHECKPOINTS_NOTICE` in the GitHub repo for the terms attached to the training datasets;
+ScanNet++, Waymo and ETH3D additionally require registration with their providers.
+
+These weights are trained at 224px for research on uncertainty-aware 3D reconstruction.
+Other resolutions are outside the trained regime.
+
+## Citation
+
+```bibtex
+@misc{zhu2026trust3r,
+  title  = {Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R},
+  author = {Zhu, Zihao and Zhao, Wenyuan and Chen, Nuo and Tian, Chao and Fan, Zhiwen},
+  year   = {2026},
+  eprint = {2605.19539},
+  archivePrefix = {arXiv}
+}
+```

trust3r_nig_mast3r_224.metadata.json

@@ -0,0 +1,42 @@
+{
+  "name": "Trust3R (NIG) \u2014 MASt3R backbone, 224px",
+  "paper": "Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R (ICML 2026)",
+  "variant": "NIG evidential head (diagonal / per-axis variance) + gated residual head, frozen MASt3R backbone",
+  "reproduces": "Table 6 (NIG ablation rows)",
+  "source_run": "gatedres_nig_xyz_mix4_150000_seed0_uq0.05_grpost_full",
+  "source_file": "checkpoint-final.pth (== checkpoint-step-150000.pth, bit-identical to checkpoint-best.pth / checkpoint-last.pth)",
+  "training_steps": 150000,
+  "epochs": 10,
+  "training_mix": "25000 @ ScanNetpp(split='train') + 25000 @ ARKitScenes(split='train') + 50000 @ Waymo(split='train') + 50000 @ MegaDepth(split='train')  [resolution=224, aug_crop=16]  -> 150k samples/epoch x 10 epochs = 150k steps @ batch_size 10",
+  "backbone_init": "MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth",
+  "load_with": "mast3r.model.AsymmetricMASt3R.from_pretrained(<this file>)",
+  "eval_protocol": {
+    "head_type": "catmlp+dpt+xyz_evi_dpt+residual_gated",
+    "resolution": 224,
+    "max_pairs_per_benchmark": 5000,
+    "scene_fraction": 1.0,
+    "subset_seed": 0,
+    "alignment": "per-image Sim(3) before computing 3D errors (--sim3_align)",
+    "gr_post_smooth_at_inference": false,
+    "gr_post_smooth_note": "IMPORTANT: the paper's Table 1/2/5/6 numbers were produced with the gated-residual post-smoothing branch DISABLED at inference (head built from the model expression above, which leaves gr_post_smooth=False). The checkpoint still carries the post-smoothing weights trained with --gr_post_smooth 1 under the keys *.residual_gated.gate_post.* and *.residual_gated.delta_post.{0,1}.*; they are not used unless you explicitly enable gr_post_smooth. Enabling it will NOT reproduce the published numbers.",
+    "delta_post_shape_note": "delta_post in this checkpoint is a depthwise-separable Sequential (.0 = depthwise 3x3, .1 = pointwise 1x1). The released model code builds delta_post as a single depthwise Conv2d, so these 8 tensors land in unexpected_keys under load_state_dict(strict=False). This is harmless for reproduction because the branch is bypassed, but it means the smoothing module cannot be reconstructed from the released code as-is."
+  },
+  "paper_numbers": {
+    "readout": "u_epi (epistemic variance-trace)",
+    "table6_evidential_family_ablation": {
+      "ScanNet++": {
+        "AURC": 0.123662,
+        "AUSE": 0.044843,
+        "SpearmanRho": 0.487491
+      },
+      "ETH3D": {
+        "AURC": 0.321256,
+        "AUSE": 0.149341,
+        "SpearmanRho": 0.322906
+      }
+    }
+  },
+  "license": "CC BY-NC-SA 4.0 (inherited from MASt3R / DUSt3R); see CHECKPOINTS_NOTICE",
+  "sha256": "95b4ff101fd427d95dcb07f9ac2f3786b5b909d40a3697a7cea86da3cef3f9c0",
+  "size_bytes": 3181008615
+}

trust3r_nig_mast3r_224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95b4ff101fd427d95dcb07f9ac2f3786b5b909d40a3697a7cea86da3cef3f9c0
+size 3181008615

trust3r_nig_mast3r_224.pth.sha256

@@ -0,0 +1 @@
+95b4ff101fd427d95dcb07f9ac2f3786b5b909d40a3697a7cea86da3cef3f9c0  trust3r_nig_mast3r_224.pth

trust3r_niw_mast3r_224.metadata.json

@@ -0,0 +1,83 @@
+{
+  "name": "Trust3R (NIW) \u2014 MASt3R backbone, 224px",
+  "paper": "Trust It or Not: Evidential Uncertainty for Feed-Forward 3D Reconstruction with Trust3R (ICML 2026)",
+  "variant": "NIW evidential head (full 3x3 covariance) + gated residual head, frozen MASt3R backbone",
+  "reproduces": "Tables 1, 2, 5, 6 (NIW rows); Figure 3",
+  "source_run": "gatedres_niw_xyz_mix4_150000_seed0_uq0.05_grpost_full",
+  "source_file": "checkpoint-final.pth (== checkpoint-step-150000.pth, bit-identical to checkpoint-best.pth / checkpoint-last.pth)",
+  "training_steps": 150000,
+  "epochs": 10,
+  "training_mix": "25000 @ ScanNetpp(split='train') + 25000 @ ARKitScenes(split='train') + 50000 @ Waymo(split='train') + 50000 @ MegaDepth(split='train')  [resolution=224, aug_crop=16]  -> 150k samples/epoch x 10 epochs = 150k steps @ batch_size 10",
+  "backbone_init": "MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth",
+  "load_with": "mast3r.model.AsymmetricMASt3R.from_pretrained(<this file>)",
+  "eval_protocol": {
+    "head_type": "catmlp+dpt+xyz_niw_dpt+residual_gated",
+    "resolution": 224,
+    "max_pairs_per_benchmark": 5000,
+    "scene_fraction": 1.0,
+    "subset_seed": 0,
+    "alignment": "per-image Sim(3) before computing 3D errors (--sim3_align)",
+    "gr_post_smooth_at_inference": false,
+    "gr_post_smooth_note": "IMPORTANT: the paper's Table 1/2/5/6 numbers were produced with the gated-residual post-smoothing branch DISABLED at inference (head built from the model expression above, which leaves gr_post_smooth=False). The checkpoint still carries the post-smoothing weights trained with --gr_post_smooth 1 under the keys *.residual_gated.gate_post.* and *.residual_gated.delta_post.{0,1}.*; they are not used unless you explicitly enable gr_post_smooth. Enabling it will NOT reproduce the published numbers.",
+    "delta_post_shape_note": "delta_post in this checkpoint is a depthwise-separable Sequential (.0 = depthwise 3x3, .1 = pointwise 1x1). The released model code builds delta_post as a single depthwise Conv2d, so these 8 tensors land in unexpected_keys under load_state_dict(strict=False). This is harmless for reproduction because the branch is bypassed, but it means the smoothing module cannot be reconstructed from the released code as-is."
+  },
+  "paper_numbers": {
+    "readout": "u_epi (epistemic covariance-trace) -- the default ranking measure",
+    "table1_uncertainty": {
+      "ScanNet++": {
+        "AURC": 0.12328,
+        "AUSE": 0.044445,
+        "SpearmanRho": 0.493043
+      },
+      "TUM RGB-D": {
+        "AURC": 0.048135,
+        "AUSE": 0.017793,
+        "SpearmanRho": 0.516943
+      },
+      "KITTI": {
+        "AURC": 0.98689,
+        "AUSE": 0.44308,
+        "SpearmanRho": 0.459576
+      },
+      "Avg": {
+        "AURC": 0.386102,
+        "AUSE": 0.168439,
+        "SpearmanRho": 0.489854
+      }
+    },
+    "table2_reconstruction": {
+      "ScanNet++": {
+        "MAE": 0.195861,
+        "RMSE": 0.284884
+      },
+      "TUM RGB-D": {
+        "MAE": 0.087329,
+        "RMSE": 0.149637
+      },
+      "KITTI": {
+        "MAE": 1.664771,
+        "RMSE": 3.077223
+      }
+    },
+    "table5_6_eth3d": {
+      "u_alea": {
+        "AURC": 0.31749,
+        "AUSE": 0.145209,
+        "SpearmanRho": 0.309347
+      },
+      "u_total": {
+        "AURC": 0.306399,
+        "AUSE": 0.134117,
+        "SpearmanRho": 0.345515
+      },
+      "u_epi": {
+        "AURC": 0.304036,
+        "AUSE": 0.131755,
+        "SpearmanRho": 0.348302
+      }
+    }
+  },
+  "license": "CC BY-NC-SA 4.0 (inherited from MASt3R / DUSt3R); see CHECKPOINTS_NOTICE",
+  "sha256": "0754dae8e29f0ac0aa6063429f1ac1da8ecb4ec2a3a9339a2038d25f47b8e442",
+  "size_bytes": 3181008103
+}

trust3r_niw_mast3r_224.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0754dae8e29f0ac0aa6063429f1ac1da8ecb4ec2a3a9339a2038d25f47b8e442
+size 3181008103

trust3r_niw_mast3r_224.pth.sha256

@@ -0,0 +1 @@
+0754dae8e29f0ac0aa6063429f1ac1da8ecb4ec2a3a9339a2038d25f47b8e442  trust3r_niw_mast3r_224.pth