tj111 commited on
Commit
55fd91c
·
verified ·
1 Parent(s): 28cadbd

Add checkpoint documentation

Browse files
Files changed (1) hide show
  1. README.md +102 -0
README.md CHANGED
@@ -1,3 +1,105 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ library_name: pytorch
4
+ tags:
5
+ - computer-vision
6
+ - feature-extraction
7
+ - swin-transformer
8
  ---
9
+
10
+ # Object Concepts from Motion
11
+
12
+ This repository contains the inference-only Motion Object Encoder checkpoints
13
+ released with [Object Concepts from Motion](https://github.com/TJ12342/object-concepts-from-motion).
14
+ The checkpoints provide dense visual representations using five Swin Transformer
15
+ backbone sizes.
16
+
17
+ The files contain the backbone, neck, and representation head weights. Optimizer,
18
+ scheduler, message-hub, and other training state have been removed. Parameters use
19
+ MMPretrain/MMEngine names and are stored as PyTorch `.pth` checkpoints.
20
+
21
+ ## Checkpoints
22
+
23
+ | File | Variant | Embedding | Stage depths | Attention heads | Size |
24
+ | --- | --- | ---: | --- | --- | ---: |
25
+ | `swin_h.pth` | Swin-H / huge | 384 | 2, 2, 18, 2 | 12, 24, 48, 96 | 3.14 GB |
26
+ | `swin_l.pth` | Swin-L / large | 192 | 2, 2, 18, 2 | 6, 12, 24, 48 | 796 MB |
27
+ | `swin_b.pth` | Swin-B / base | 128 | 2, 2, 18, 2 | 4, 8, 16, 32 | 362 MB |
28
+ | `swin_s.pth` | Swin-S / small | 96 | 2, 2, 18, 2 | 3, 6, 12, 24 | 210 MB |
29
+ | `swin_t.pth` | Swin-T / tiny | 96 | 2, 2, 6, 2 | 3, 6, 12, 24 | 124 MB |
30
+
31
+ Swin-H is the Cycle 2 checkpoint. The T, S, B, and L variants are distilled
32
+ from the Cycle 2 Swin-H model.
33
+
34
+ ## Download
35
+
36
+ Download all checkpoints into the location expected by the source repository:
37
+
38
+ ```bash
39
+ hf download tj111/object-concepts-from-motion \
40
+ --include "*.pth" \
41
+ --local-dir checkpoints
42
+ ```
43
+
44
+ Or download one checkpoint from Python:
45
+
46
+ ```python
47
+ from huggingface_hub import hf_hub_download
48
+
49
+ checkpoint_path = hf_hub_download(
50
+ repo_id="tj111/object-concepts-from-motion",
51
+ filename="swin_h.pth",
52
+ )
53
+ ```
54
+
55
+ ## Usage
56
+
57
+ Clone the source repository, install its lightweight inference dependencies,
58
+ and download the weights:
59
+
60
+ ```bash
61
+ git clone https://github.com/TJ12342/object-concepts-from-motion.git
62
+ cd object-concepts-from-motion
63
+ python -m pip install -r requirements.txt
64
+ hf download tj111/object-concepts-from-motion \
65
+ --include "*.pth" \
66
+ --local-dir checkpoints
67
+ ```
68
+
69
+ Run the feature visualization demo with an explicitly selected architecture:
70
+
71
+ ```bash
72
+ python tools/feature_visualization.py assets/pic1.png \
73
+ --arch huge \
74
+ --output assets/pic1_pca.png
75
+ ```
76
+
77
+ For direct loading, use the security-restricted checkpoint mode used by the
78
+ source repository:
79
+
80
+ ```python
81
+ import torch
82
+
83
+ checkpoint = torch.load(
84
+ checkpoint_path,
85
+ map_location="cpu",
86
+ mmap=True,
87
+ weights_only=True,
88
+ )
89
+ state_dict = checkpoint.get("state_dict", checkpoint)
90
+ ```
91
+
92
+ The repository includes a standalone PyTorch implementation and adapters for
93
+ DCDepth, BEVFormer, and SparseOcc. See the source repository for architecture
94
+ selection, checkpoint conversion, preprocessing, and downstream instructions.
95
+
96
+ ## Limitations
97
+
98
+ These are representation checkpoints, not complete task-specific models.
99
+ DCDepth, BEVFormer, and SparseOcc evaluation can require separately trained
100
+ decoders, prediction heads, or full task checkpoints. The files are not packaged
101
+ for `transformers.AutoModel` or the hosted Hugging Face Inference API.
102
+
103
+ ## Integrity
104
+
105
+ SHA-256 checksums are provided in `SHA256SUMS`.