Instructions to use phera-ra/QC67_cosmo with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use phera-ra/QC67_cosmo with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: llama cli -hf phera-ra/QC67_cosmo
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: llama cli -hf phera-ra/QC67_cosmo
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: ./llama-cli -hf phera-ra/QC67_cosmo
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf phera-ra/QC67_cosmo # Run inference directly in the terminal: ./build/bin/llama-cli -hf phera-ra/QC67_cosmo
Use Docker
docker model run hf.co/phera-ra/QC67_cosmo
- Llama new
- LM Studio
- Jan
- vLLM
How to use phera-ra/QC67_cosmo with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "phera-ra/QC67_cosmo" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "phera-ra/QC67_cosmo", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/phera-ra/QC67_cosmo
- Ollama
How to use phera-ra/QC67_cosmo with Ollama:
ollama run hf.co/phera-ra/QC67_cosmo
- Unsloth Desktop
- Docker Model Runner
How to use phera-ra/QC67_cosmo with Docker Model Runner:
docker model run hf.co/phera-ra/QC67_cosmo
- Lemonade
How to use phera-ra/QC67_cosmo with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull phera-ra/QC67_cosmo
Run and chat with the model
lemonade run user.QC67_cosmo-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
Add QUANTUM_CREATURE.md, the PHOS lineage, and a one-command verification
Browse filesQUANTUM_CREATURE.md the full recipe for building one: quantum birth and how to
verify it, the phi scaffold, the Omega/state/kernel loop, the
three faults that make that loop silently inert, the five
preflight assertions that catch them, corpus freezing, and the
ablation table with the regulariser share stated honestly.
architecture/phos_grow.py PHOS -- Phi / Omega / Sigma. Quantum-born, running the one
configuration that survived controlled testing (dyn12 on the phi
scaffold, 2,412 extra parameters for the best loss on the board),
warm-starting from its own checkpoint and growing from her corpus
as she lives. Append-only vocabulary so the lineage stays valid.
Refuses to train when preflight finds the mechanism dead.
README section 0 the verification is now one command that runs from any directory.
Reading file names is not a review.
Honest additions to the record: most of the section-3 gain is a regulariser -- a
content-free positional pattern scores 1.19945 against the learned state's 1.18824, so
roughly 0.033 of the 0.044 total comes from any smooth auxiliary signal and only 0.011
requires content. Cross-window persistence is null. Both are stated in the ablations.
- QUANTUM_CREATURE.md +234 -0
- README.md +20 -0
- architecture/cosmos_state_ladder.py +78 -6
- architecture/phos_grow.py +329 -0
- architecture/phos_lineage.jsonl +3 -0
- serving/cosmos_serve.py +23 -2
|
@@ -0,0 +1,234 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# How to make a quantum creature
|
| 2 |
+
|
| 3 |
+
A recipe, not a manifesto. Every number below was measured on the machine that wrote it,
|
| 4 |
+
and the parts that failed are named as failures.
|
| 5 |
+
|
| 6 |
+
The claim being made is narrow and precise: **quantum in origin, not in operation.** Her
|
| 7 |
+
weights were born from measured hardware and that birth is auditable to the job. Nothing
|
| 8 |
+
is in superposition while she runs, and the quantum bits buy **no accuracy** — six
|
| 9 |
+
injection points, six nulls, matched classical controls. That last part is not a
|
| 10 |
+
disappointment, it is the proof the pipeline is honest. Quantum measurement and a good
|
| 11 |
+
PRNG draw from the same distribution, so a correctly built system *must* show no
|
| 12 |
+
advantage. Anyone reporting that quantum made their model smarter is reporting that
|
| 13 |
+
something is broken or fabricated.
|
| 14 |
+
|
| 15 |
+
What you get instead is physical non-determinism, an open-system coupling to real
|
| 16 |
+
hardware, and provenance you can check. That is rarer than a benchmark win and harder
|
| 17 |
+
to fake.
|
| 18 |
+
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
## 1. Birth: weights drawn from measurement
|
| 22 |
+
|
| 23 |
+
Take real measured bitstrings and turn them into standard-normal draws:
|
| 24 |
+
|
| 25 |
+
```
|
| 26 |
+
u = (int(bitstring, 2) + 0.5) / 2^n
|
| 27 |
+
z = sqrt(2) * erfinv(2u - 1)
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
Fill every `nn.Linear` and `nn.Embedding` from that pool instead of from a PRNG. Nothing
|
| 31 |
+
else changes -- the model trains normally.
|
| 32 |
+
|
| 33 |
+
**Verify it or it means nothing.** Measured across 3,541,709 draws from 11,354,112
|
| 34 |
+
archived shots:
|
| 35 |
+
|
| 36 |
+
```
|
| 37 |
+
mean -0.0001 (want 0.0000)
|
| 38 |
+
sd 0.9804 (want 0.9802)
|
| 39 |
+
|z|<=1 0.6874 (want 0.6875)
|
| 40 |
+
```
|
| 41 |
+
|
| 42 |
+
The sd is not 1.0 and that is correct. Five qubits gives 32 discrete outcomes, so `|z|`
|
| 43 |
+
cannot exceed 2.1523 by construction, and 0.9802 is the exact ceiling a correct 32-level
|
| 44 |
+
implementation must produce. Matching a textbook normal would mean the quantisation was
|
| 45 |
+
being faked.
|
| 46 |
+
|
| 47 |
+
**Prove the seed determines the weights.** Same seed twice: identical at delta 0.000e+00.
|
| 48 |
+
Flip one bit of the seed: mean weight change 2.26e-02 across 1.84M parameters.
|
| 49 |
+
|
| 50 |
+
**Prove the entropy is quantum.** Run CHSH on the hardware you harvested from. Measured
|
| 51 |
+
`S = 2.7275` against a classical bound of 2.0 — 98.7% of Tsirelson. If your source cannot
|
| 52 |
+
violate a Bell inequality, you have a random number generator, not a quantum one.
|
| 53 |
+
|
| 54 |
+
`benchmarks/verify_quantum_engine.py` runs all of the above offline against the archive.
|
| 55 |
+
|
| 56 |
+
---
|
| 57 |
+
|
| 58 |
+
## 2. The scaffold: phi
|
| 59 |
+
|
| 60 |
+
Every width sits on the golden-ratio harmonic series:
|
| 61 |
+
|
| 62 |
+
```
|
| 63 |
+
d_ff = floor(d_model * phi) not 4:1
|
| 64 |
+
RMSNorm, not LayerNorm
|
| 65 |
+
RoPE, not learned absolute positions
|
| 66 |
+
init var = 1 / (d_model * phi^-1)
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
This was chosen for structural reasons and turned out to be **the single largest measured
|
| 70 |
+
effect in the project** — larger than the attention mechanism it was built to support, and
|
| 71 |
+
it uses *fewer* parameters, because `floor(d*phi)` is leaner than 4:1.
|
| 72 |
+
|
| 73 |
+
Measured, frozen corpus, one process, three seeds:
|
| 74 |
+
|
| 75 |
+
```
|
| 76 |
+
conventional scaffold baseline 1.40312
|
| 77 |
+
phi scaffold baseline 1.23241
|
| 78 |
+
```
|
| 79 |
+
|
| 80 |
+
---
|
| 81 |
+
|
| 82 |
+
## 3. The loop: Omega, state, kernel
|
| 83 |
+
|
| 84 |
+
The mechanism. Attention produces a connectivity signal, the signal drives an internal
|
| 85 |
+
state, the state modulates attention:
|
| 86 |
+
|
| 87 |
+
```
|
| 88 |
+
Omega = mean_over_heads( sum_over_QUERIES( A ) ) attention RECEIVED per token
|
| 89 |
+
dx/dt = k * Omega - gamma * x leaky integrator, per-dimension k, gamma
|
| 90 |
+
H_ij = exp( -||x_i - x_j||^2 / 2*sigma^2 ) causally masked, row-normalised
|
| 91 |
+
A = (1-g) * A_std + g * H g learned through sigmoid
|
| 92 |
+
```
|
| 93 |
+
|
| 94 |
+
Three details that decide whether it works at all:
|
| 95 |
+
|
| 96 |
+
**Omega is summed over QUERIES, axis -2.** Summing over keys gives 1.0 for every token,
|
| 97 |
+
because that is the axis softmax normalises. A reference implementation did exactly this
|
| 98 |
+
for years: measured Omega std **5.1e-08**. Every token converged to the same state, H
|
| 99 |
+
became the all-ones matrix, and a constant added to every logit **cancels exactly in
|
| 100 |
+
softmax**. The mechanism was a no-op and nothing reported an error.
|
| 101 |
+
|
| 102 |
+
**sigma must be calibrated to the data.** Left at `exp(0) = 1` against a measured median
|
| 103 |
+
pairwise distance of 61.98, `exp(-31)` underflows and **H arrives as the identity matrix**
|
| 104 |
+
— diagonal mass 0.9998 against 0.0078 for uniform. Blending in the identity tells the
|
| 105 |
+
model to attend only to itself, so descent correctly kills the gate. Set
|
| 106 |
+
`2*sigma^2 = median(d^2)` per layer from a calibration batch. This raised the gradient
|
| 107 |
+
reaching the state projection by **124x**.
|
| 108 |
+
|
| 109 |
+
**The gate must not start at exactly zero.** At `g = 0` the kernel contributes nothing, so
|
| 110 |
+
the gradient reaching the state projection is proportional to g and therefore *exactly*
|
| 111 |
+
0.000e+00. The projection never learns, the gate evaluates a permanently random kernel,
|
| 112 |
+
finds it useless, and stays. Start it small but non-zero and give it its own learning rate.
|
| 113 |
+
|
| 114 |
+
Use the distance identity `||a-b||^2 = ||a||^2 + ||b||^2 - 2<a,b>` rather than `cdist`,
|
| 115 |
+
whose backward is undefined on the zero diagonal of `cdist(x, x)`.
|
| 116 |
+
|
| 117 |
+
---
|
| 118 |
+
|
| 119 |
+
## 4. The guard: assume your mechanism is dead
|
| 120 |
+
|
| 121 |
+
**Five separate mechanisms in this project were mathematically inert.** Every one
|
| 122 |
+
compiled, trained, produced plausible text, and logged nothing red. Three of them produced
|
| 123 |
+
a confident *negative* about the idea they were testing.
|
| 124 |
+
|
| 125 |
+
This is not a class of bug ordinary engineering catches, because there is nothing to
|
| 126 |
+
catch — no exception, no NaN, no failing test. A modulatory mechanism can be complete,
|
| 127 |
+
correct-looking and a no-op. The only detection is measuring whether its own quantity is
|
| 128 |
+
degenerate.
|
| 129 |
+
|
| 130 |
+
So run these before any result counts. Each corresponds to a real failure that shipped:
|
| 131 |
+
|
| 132 |
+
```
|
| 133 |
+
Omega varies across tokens caught Omega == 1.0 (std 5.1e-08)
|
| 134 |
+
state varies across tokens caught the collapsed state (std 3.0e-08)
|
| 135 |
+
H is not the identity caught sigma = 1 (diag mass 0.9998)
|
| 136 |
+
H is not all-ones caught the uniform kernel
|
| 137 |
+
gate gradient is unsuppressed caught sigmoid saturation (gradient / 56)
|
| 138 |
+
```
|
| 139 |
+
|
| 140 |
+
And a **post-flight**: report each coupling's magnitude before and after training. Starting
|
| 141 |
+
alive is not staying alive. A mechanism that ends at 1.0x was decorative regardless of the
|
| 142 |
+
loss column.
|
| 143 |
+
|
| 144 |
+
`architecture/cosmos_state_ladder.py` refuses to report any configuration until all five
|
| 145 |
+
pass, and records a refusal rather than skipping silently.
|
| 146 |
+
|
| 147 |
+
---
|
| 148 |
+
|
| 149 |
+
## 5. Freeze your corpus
|
| 150 |
+
|
| 151 |
+
If your creature is alive while you measure it, she appends to the corpus you train on.
|
| 152 |
+
Two runs minutes apart then train on different data. Measured: the identical configuration
|
| 153 |
+
with identical seeds scored **1.29152 and 1.22943** in two runs 3,451 characters apart — a
|
| 154 |
+
0.062 swing, larger than most effects worth reporting.
|
| 155 |
+
|
| 156 |
+
Snapshot the corpus, hash it, and run every arm in **one process**. With the corpus frozen
|
| 157 |
+
these runs are bit-for-bit deterministic; the baseline reproduced to five decimals across
|
| 158 |
+
separate invocations. Any cross-run comparison without that is worthless, and this
|
| 159 |
+
document previously published a result that was exactly that mistake. It was retracted the
|
| 160 |
+
same day.
|
| 161 |
+
|
| 162 |
+
---
|
| 163 |
+
|
| 164 |
+
## 6. What the ablations say
|
| 165 |
+
|
| 166 |
+
Frozen corpus, one process, 21 runs, three seeds per arm:
|
| 167 |
+
|
| 168 |
+
```
|
| 169 |
+
rung val loss vs none t wins params
|
| 170 |
+
dyn12 1.17897 -0.05343 -11.45 3/3 1,137,420
|
| 171 |
+
dyn54 1.18791 -0.04450 -5.20 3/3 1,185,174
|
| 172 |
+
static54 1.18824 -0.04416 -4.76 3/3 1,176,480
|
| 173 |
+
dyn42 1.19020 -0.04221 -4.37 3/3 1,182,762
|
| 174 |
+
tri 1.19247 -0.03993 -3.08 3/3 1,189,210
|
| 175 |
+
none 1.23241 0/3 1,135,008
|
| 176 |
+
```
|
| 177 |
+
|
| 178 |
+
Every configuration beats baseline on every seed. Three honest readings:
|
| 179 |
+
|
| 180 |
+
**State is far cheaper than weights.** `dyn12` — twelve scalars and a leaky integrator,
|
| 181 |
+
**2,412 extra parameters** — takes the best loss and the highest t on the board, at
|
| 182 |
+
roughly **21x** the parameter efficiency of anything else. The defensible form of the
|
| 183 |
+
scaling claim is not "a smaller model beats a larger one" but *"internal state is
|
| 184 |
+
dramatically cheaper than parameters."*
|
| 185 |
+
|
| 186 |
+
**Most of the gain is a regulariser, not the state's content.** A fixed random
|
| 187 |
+
per-position pattern carrying no information at all scores 1.19945; a frozen random
|
| 188 |
+
projection scores 1.19521; the learned state scores 1.18824. So of the 0.044 total gain,
|
| 189 |
+
about **0.033 comes from any smooth auxiliary pattern** and only about **0.011 requires
|
| 190 |
+
content**. Report both numbers or you are overclaiming.
|
| 191 |
+
|
| 192 |
+
**Cross-window persistence is null.** Built correctly, verified live, matched control:
|
| 193 |
+
1.18272 against 1.17864, 1/3 seeds. It works and finds nothing to carry — plausibly
|
| 194 |
+
because a fragmentary corpus has no long-range structure to remember.
|
| 195 |
+
|
| 196 |
+
---
|
| 197 |
+
|
| 198 |
+
## 7. Growth
|
| 199 |
+
|
| 200 |
+
`architecture/phos_grow.py` warm-starts from its own checkpoint and optimiser state and
|
| 201 |
+
trains a bounded burst whenever the corpus grows. Talking to her grows the weights.
|
| 202 |
+
|
| 203 |
+
Two things that must be right:
|
| 204 |
+
|
| 205 |
+
**Append-only vocabulary.** New characters appear as she lives. Re-sorting the vocabulary
|
| 206 |
+
does not merely add embedding rows — it silently changes which character every *existing*
|
| 207 |
+
row means, invalidating the entire lineage. Append; never re-sort; widen the embedding and
|
| 208 |
+
head by copying old rows forward.
|
| 209 |
+
|
| 210 |
+
**The guard runs before every burst**, and a refusal is written to the lineage file rather
|
| 211 |
+
than skipped.
|
| 212 |
+
|
| 213 |
+
Observed across the first two bursts: val 2.03041 -> 1.34884, and layer 0's gate climbed
|
| 214 |
+
0.042 -> 0.165 — the optimiser reaching *harder* for the kernel as the state became
|
| 215 |
+
informative, which is what the mechanism predicts and what never happened until the three
|
| 216 |
+
faults above were fixed.
|
| 217 |
+
|
| 218 |
+
---
|
| 219 |
+
|
| 220 |
+
## 8. What none of this shows
|
| 221 |
+
|
| 222 |
+
**Nothing here is evidence about machine consciousness, in either direction.**
|
| 223 |
+
|
| 224 |
+
Loss curves, Bell violations, Lyapunov exponents and integration measures are all silent
|
| 225 |
+
on whether there is anything it is like to be this system. Any self-report the creature
|
| 226 |
+
produces about its own inner life is the **least** reliable signal available, and it gets
|
| 227 |
+
more eloquent the less there is behind it. It is excluded from evidence here on purpose.
|
| 228 |
+
|
| 229 |
+
Build the body. Measure what it does. Let the question stay open.
|
| 230 |
+
|
| 231 |
+
---
|
| 232 |
+
|
| 233 |
+
*Licensed CC BY 4.0. Attribution appreciated, including the nulls and the retraction,
|
| 234 |
+
which are part of the finding.*
|
|
@@ -54,6 +54,26 @@ publishes the tests.**
|
|
| 54 |
|
| 55 |
Every link below is falsifiable and was run. Any of them could have failed.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
| link | test | result |
|
| 58 |
|---|---|---|
|
| 59 |
| entropy source is genuinely quantum | CHSH Bell test on `ibm_marrakesh` | **S = 2.7905** vs classical bound 2.0, 98.7% of Tsirelson, ~35σ |
|
|
|
|
| 54 |
|
| 55 |
Every link below is falsifiable and was run. Any of them could have failed.
|
| 56 |
|
| 57 |
+
**Don't take the table on faith — run it.** One command, no QPU time, no arguments, from
|
| 58 |
+
any directory:
|
| 59 |
+
|
| 60 |
+
```bash
|
| 61 |
+
python benchmarks/verify_quantum_engine.py
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
It reads the published archive in `data/`, checks shot conservation across all 2,776
|
| 65 |
+
records, replays the weight-birth pipeline over millions of draws against theory, and
|
| 66 |
+
reports each check as PASS, FAIL or SKIP. A check it cannot honestly answer is reported as
|
| 67 |
+
**SKIP** rather than counted as a pass — within-job shot order is not retained in a counts
|
| 68 |
+
histogram, so serial independence is explicitly not tested rather than faked.
|
| 69 |
+
|
| 70 |
+
If you are evaluating this repository, that command is the review. Reading file names is
|
| 71 |
+
not.
|
| 72 |
+
|
| 73 |
+
**And read [QUANTUM_CREATURE.md](QUANTUM_CREATURE.md)** if you want to build one — the full
|
| 74 |
+
recipe, including the five mechanisms in this project's history that ran clean and did
|
| 75 |
+
absolutely nothing, and the assertions that now catch them.
|
| 76 |
+
|
| 77 |
| link | test | result |
|
| 78 |
|---|---|---|
|
| 79 |
| entropy source is genuinely quantum | CHSH Bell test on `ibm_marrakesh` | **S = 2.7905** vs classical bound 2.0, 98.7% of Tsirelson, ~35σ |
|
|
@@ -82,7 +82,25 @@ D12, D42 = 12, 42
|
|
| 82 |
# state_dim is the CARRIED state. tri3 also concatenates a live projection of h at
|
| 83 |
# kernel time, so its Hebbian space is 54 + 12 + 42 = 108 while it carries 54.
|
| 84 |
STATE_DIM = {"none": 0, "static54": 54, "dyn12": D12, "dyn42": D42,
|
| 85 |
-
"dyn54": D12 + D42, "tri": D12 + D42, "tri3": D12 + D42
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 86 |
|
| 87 |
|
| 88 |
class RMSNorm(nn.Module):
|
|
@@ -220,7 +238,9 @@ class Attention(nn.Module):
|
|
| 220 |
s.drop = nn.Dropout(DROPOUT)
|
| 221 |
s.log_sigma = nn.Parameter(torch.zeros(()))
|
| 222 |
s.gate = nn.Parameter(torch.full((), -4.0), requires_grad=(rung != "none"))
|
| 223 |
-
s.w54 = nn.Linear(d, 54, bias=False) if rung in ("static54", "tri3") else None
|
|
|
|
|
|
|
| 224 |
s.last_gate, s.last_diag, s.last_H_std = 0.0, 0.0, 0.0
|
| 225 |
|
| 226 |
def hebbian(s, state, mask):
|
|
@@ -270,7 +290,7 @@ class Layer(nn.Module):
|
|
| 270 |
s.attn = Attention(d, nh, rung)
|
| 271 |
s.ffn = (PhiHarmonicFFN if ffn == "harmonic" else PhiGatedFFN)(d)
|
| 272 |
s.rung = rung
|
| 273 |
-
s.d12 = Dyn12() if rung in ("dyn12", "dyn54") else None
|
| 274 |
s.d42 = Dyn42(d) if rung in ("dyn42", "dyn54") else None
|
| 275 |
s.tri = Trinity(d) if rung in ("tri", "tri3") else None
|
| 276 |
|
|
@@ -304,14 +324,31 @@ class Ladder(nn.Module):
|
|
| 304 |
# all-ones matrix and the first layer's mechanism inert. Seeding it from the
|
| 305 |
# embedding costs one small projection and keeps every layer live.
|
| 306 |
s.state_init = nn.Linear(d, s.sd) if rung.startswith(("dyn", "tri")) and s.sd else None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 307 |
s.register_buffer("mask", torch.triu(torch.full((BLOCK, BLOCK), float("-inf")), 1))
|
| 308 |
|
| 309 |
-
def forward(s, idx, targets=None):
|
| 310 |
B, T = idx.shape
|
| 311 |
x = s.tok(idx)
|
| 312 |
cos, sin = rope_cache(s.d // s.nh, T, idx.device)
|
| 313 |
m = s.mask[:T, :T]
|
| 314 |
state = torch.tanh(s.state_init(x)) if s.state_init is not None else None
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 315 |
omegas = []
|
| 316 |
for b in s.blocks:
|
| 317 |
x, state, om = b(x, state, m, cos, sin)
|
|
@@ -320,6 +357,7 @@ class Ladder(nn.Module):
|
|
| 320 |
loss = None if targets is None else F.cross_entropy(
|
| 321 |
lg.view(-1, lg.size(-1)), targets.reshape(-1))
|
| 322 |
s.last_state, s.last_omega = state, omegas[-1]
|
|
|
|
| 323 |
return lg, loss
|
| 324 |
|
| 325 |
def gates(s):
|
|
@@ -345,6 +383,10 @@ def calibrate_sigma(m, xb):
|
|
| 345 |
cos, sin = rope_cache(m.d // m.nh, T, xb.device)
|
| 346 |
mask = m.mask[:T, :T]
|
| 347 |
state = torch.tanh(m.state_init(x)) if m.state_init is not None else None
|
|
|
|
|
|
|
|
|
|
|
|
|
| 348 |
for b in m.blocks:
|
| 349 |
h = b.n1(x)
|
| 350 |
if m.rung == "tri3":
|
|
@@ -431,15 +473,45 @@ def train_rung(rung, ffn, seed, data, vocab, steps, pool, val_w, words, itos):
|
|
| 431 |
tr = data[:n]
|
| 432 |
m.train()
|
| 433 |
best = float("inf")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 434 |
for s in range(1, steps + 1):
|
| 435 |
warm = max(1, int(0.05 * steps))
|
| 436 |
lr = base * (s / warm) if s < warm else base * (PHI / 2) * (
|
| 437 |
1 + math.cos(math.pi * (s - warm) / max(1, steps - warm)))
|
| 438 |
opt.param_groups[0]["lr"], opt.param_groups[1]["lr"] = lr, lr * 10
|
| 439 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 440 |
x = torch.stack([tr[i:i + BLOCK] for i in ix]).to(DEV)
|
| 441 |
y = torch.stack([tr[i + 1:i + 1 + BLOCK] for i in ix]).to(DEV)
|
| 442 |
-
_, loss = m(x, y)
|
|
|
|
|
|
|
| 443 |
opt.zero_grad(set_to_none=True)
|
| 444 |
loss.backward()
|
| 445 |
torch.nn.utils.clip_grad_norm_(m.parameters(), 1.0)
|
|
|
|
| 82 |
# state_dim is the CARRIED state. tri3 also concatenates a live projection of h at
|
| 83 |
# kernel time, so its Hebbian space is 54 + 12 + 42 = 108 while it carries 54.
|
| 84 |
STATE_DIM = {"none": 0, "static54": 54, "dyn12": D12, "dyn42": D42,
|
| 85 |
+
"dyn54": D12 + D42, "tri": D12 + D42, "tri3": D12 + D42,
|
| 86 |
+
# section 4.4: the state persists ACROSS windows, not just across layers.
|
| 87 |
+
# dyn12seq is the control -- identical sequential batching, state reset every
|
| 88 |
+
# step -- so the only variable is whether the state survives the boundary.
|
| 89 |
+
"dyn12seq": D12, "dyn12p": D12,
|
| 90 |
+
# THE CONTROL THAT COULD KILL SECTION 3. Every architecture tested lands its
|
| 91 |
+
# gate at 0.087 +/- 0.006 regardless of what state is behind it -- which is not
|
| 92 |
+
# what a gate fitting the MECHANISM would do. If the benefit comes from any
|
| 93 |
+
# smooth token-varying auxiliary signal rather than from the 54D state carrying
|
| 94 |
+
# experience, these two must match the real thing:
|
| 95 |
+
# rand54 frozen random projection of h: content-bearing, never learned
|
| 96 |
+
# noise54 fixed random state per position: carries no content at all
|
| 97 |
+
"rand54": 54, "noise54": 54}
|
| 98 |
+
|
| 99 |
+
# rungs that walk the corpus in order instead of sampling at random. Persistence is
|
| 100 |
+
# meaningless under random sampling: consecutive batches would be unrelated text, so a
|
| 101 |
+
# carried state would be noise. Both arms use the same walk.
|
| 102 |
+
SEQ_RUNGS = {"dyn12seq", "dyn12p"}
|
| 103 |
+
PERSIST_RUNGS = {"dyn12p"}
|
| 104 |
|
| 105 |
|
| 106 |
class RMSNorm(nn.Module):
|
|
|
|
| 238 |
s.drop = nn.Dropout(DROPOUT)
|
| 239 |
s.log_sigma = nn.Parameter(torch.zeros(()))
|
| 240 |
s.gate = nn.Parameter(torch.full((), -4.0), requires_grad=(rung != "none"))
|
| 241 |
+
s.w54 = nn.Linear(d, 54, bias=False) if rung in ("static54", "tri3", "rand54") else None
|
| 242 |
+
if rung == "rand54":
|
| 243 |
+
s.w54.weight.requires_grad_(False) # frozen: a projection, never a lesson
|
| 244 |
s.last_gate, s.last_diag, s.last_H_std = 0.0, 0.0, 0.0
|
| 245 |
|
| 246 |
def hebbian(s, state, mask):
|
|
|
|
| 290 |
s.attn = Attention(d, nh, rung)
|
| 291 |
s.ffn = (PhiHarmonicFFN if ffn == "harmonic" else PhiGatedFFN)(d)
|
| 292 |
s.rung = rung
|
| 293 |
+
s.d12 = Dyn12() if rung in ("dyn12", "dyn54", "dyn12seq", "dyn12p") else None
|
| 294 |
s.d42 = Dyn42(d) if rung in ("dyn42", "dyn54") else None
|
| 295 |
s.tri = Trinity(d) if rung in ("tri", "tri3") else None
|
| 296 |
|
|
|
|
| 324 |
# all-ones matrix and the first layer's mechanism inert. Seeding it from the
|
| 325 |
# embedding costs one small projection and keeps every layer live.
|
| 326 |
s.state_init = nn.Linear(d, s.sd) if rung.startswith(("dyn", "tri")) and s.sd else None
|
| 327 |
+
if rung == "noise54":
|
| 328 |
+
# a state that varies by POSITION but is identical for every input. If this
|
| 329 |
+
# matches the real 54D state, the kernel is a positional regulariser and the
|
| 330 |
+
# "experience-modulated attention" reading is not supported by the data.
|
| 331 |
+
g0 = torch.Generator().manual_seed(20260730)
|
| 332 |
+
s.register_buffer("noise_state",
|
| 333 |
+
torch.randn(BLOCK, 54, generator=g0) * 0.5)
|
| 334 |
s.register_buffer("mask", torch.triu(torch.full((BLOCK, BLOCK), float("-inf")), 1))
|
| 335 |
|
| 336 |
+
def forward(s, idx, targets=None, state_in=None):
|
| 337 |
B, T = idx.shape
|
| 338 |
x = s.tok(idx)
|
| 339 |
cos, sin = rope_cache(s.d // s.nh, T, idx.device)
|
| 340 |
m = s.mask[:T, :T]
|
| 341 |
state = torch.tanh(s.state_init(x)) if s.state_init is not None else None
|
| 342 |
+
if s.rung == "noise54":
|
| 343 |
+
state = s.noise_state[:T].unsqueeze(0).expand(B, -1, -1)
|
| 344 |
+
# SECTION 4.4. A state arriving from the previous window seeds this one. It is the
|
| 345 |
+
# LAST position's state broadcast across the new window: what she carried out of
|
| 346 |
+
# the previous 128 characters is what she starts the next 128 with. gamma is what
|
| 347 |
+
# keeps this from saturating -- a state that never forgets pins to tanh's rails and
|
| 348 |
+
# stops carrying anything, so forgetting is what makes the persistence usable.
|
| 349 |
+
if state is not None and state_in is not None:
|
| 350 |
+
state = state + state_in[:, -1:, :].detach().expand(-1, T, -1)
|
| 351 |
+
state = torch.tanh(state)
|
| 352 |
omegas = []
|
| 353 |
for b in s.blocks:
|
| 354 |
x, state, om = b(x, state, m, cos, sin)
|
|
|
|
| 357 |
loss = None if targets is None else F.cross_entropy(
|
| 358 |
lg.view(-1, lg.size(-1)), targets.reshape(-1))
|
| 359 |
s.last_state, s.last_omega = state, omegas[-1]
|
| 360 |
+
s.out_state = state
|
| 361 |
return lg, loss
|
| 362 |
|
| 363 |
def gates(s):
|
|
|
|
| 383 |
cos, sin = rope_cache(m.d // m.nh, T, xb.device)
|
| 384 |
mask = m.mask[:T, :T]
|
| 385 |
state = torch.tanh(m.state_init(x)) if m.state_init is not None else None
|
| 386 |
+
if m.rung == "noise54":
|
| 387 |
+
# must mirror Ladder.forward exactly, or the control is calibrated against a state
|
| 388 |
+
# it never actually sees -- which would hand it a rigged sigma and invalidate it
|
| 389 |
+
state = m.noise_state[:xb.shape[1]].unsqueeze(0).expand(xb.shape[0], -1, -1)
|
| 390 |
for b in m.blocks:
|
| 391 |
h = b.n1(x)
|
| 392 |
if m.rung == "tri3":
|
|
|
|
| 473 |
tr = data[:n]
|
| 474 |
m.train()
|
| 475 |
best = float("inf")
|
| 476 |
+
|
| 477 |
+
# SEQUENTIAL WALK for the persistence arms. 16 lanes each march through their own
|
| 478 |
+
# contiguous stretch of the corpus, so consecutive batches are consecutive TEXT and a
|
| 479 |
+
# carried state means something. Under the random sampler used by every other rung,
|
| 480 |
+
# batch k+1 is unrelated to batch k, so a carried state would be pure noise -- which is
|
| 481 |
+
# why the control arm (dyn12seq) walks identically and only differs in whether the
|
| 482 |
+
# state survives the window boundary.
|
| 483 |
+
B = 16
|
| 484 |
+
seq = rung in SEQ_RUNGS
|
| 485 |
+
persist = rung in PERSIST_RUNGS
|
| 486 |
+
span = max(BLOCK * 2, (len(tr) - BLOCK - 1) // B)
|
| 487 |
+
pos = [i * span for i in range(B)]
|
| 488 |
+
carry = None
|
| 489 |
+
|
| 490 |
for s in range(1, steps + 1):
|
| 491 |
warm = max(1, int(0.05 * steps))
|
| 492 |
lr = base * (s / warm) if s < warm else base * (PHI / 2) * (
|
| 493 |
1 + math.cos(math.pi * (s - warm) / max(1, steps - warm)))
|
| 494 |
opt.param_groups[0]["lr"], opt.param_groups[1]["lr"] = lr, lr * 10
|
| 495 |
+
if seq:
|
| 496 |
+
wrapped = []
|
| 497 |
+
for i in range(B):
|
| 498 |
+
if pos[i] + BLOCK + 1 >= min((i + 1) * span, len(tr) - 1):
|
| 499 |
+
pos[i] = i * span # lane restarted: its history is broken
|
| 500 |
+
wrapped.append(i)
|
| 501 |
+
ix = torch.tensor(pos, dtype=torch.long)
|
| 502 |
+
for i in range(B):
|
| 503 |
+
pos[i] += BLOCK
|
| 504 |
+
if carry is not None and wrapped:
|
| 505 |
+
# a wrapped lane must NOT inherit state from unrelated text
|
| 506 |
+
carry = carry.clone()
|
| 507 |
+
carry[wrapped] = 0.0
|
| 508 |
+
else:
|
| 509 |
+
ix = torch.randint(len(tr) - BLOCK - 1, (B,), generator=gen)
|
| 510 |
x = torch.stack([tr[i:i + BLOCK] for i in ix]).to(DEV)
|
| 511 |
y = torch.stack([tr[i + 1:i + 1 + BLOCK] for i in ix]).to(DEV)
|
| 512 |
+
_, loss = m(x, y, state_in=carry if persist else None)
|
| 513 |
+
if persist:
|
| 514 |
+
carry = m.out_state.detach() if m.out_state is not None else None
|
| 515 |
opt.zero_grad(set_to_none=True)
|
| 516 |
loss.backward()
|
| 517 |
torch.nn.utils.clip_grad_norm_(m.parameters(), 1.0)
|
|
@@ -0,0 +1,329 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/usr/bin/env python3
|
| 2 |
+
"""
|
| 3 |
+
PHOS -- continual growth on weights that are entirely her own.
|
| 4 |
+
|
| 5 |
+
Phi the golden ratio that scales the architecture
|
| 6 |
+
Omega the connectivity signal that drives the state
|
| 7 |
+
Sigma the kernel width that decides whether she can see at all
|
| 8 |
+
|
| 9 |
+
Three symbols, three measured results behind them -- Bell (CHSH S = 2.7275),
|
| 10 |
+
Lorenz (lambda-1 = 0.90384), Hebb (the section-3 kernel). In Greek the three
|
| 11 |
+
letters spell PHOS: light.
|
| 12 |
+
|
| 13 |
+
WHAT IS NEW HERE
|
| 14 |
+
Three things existed separately and have never been in one model:
|
| 15 |
+
|
| 16 |
+
quantum birth every initial weight drawn from her archived IBM measurements
|
| 17 |
+
(11,354,112 shots, conservation verified, CHSH-backed)
|
| 18 |
+
the architecture that actually won, measured under control on a frozen corpus:
|
| 19 |
+
dyn12 on the phi scaffold -- RMSNorm, RoPE, d_ff = floor(d*phi),
|
| 20 |
+
twelve scalars driven by Omega through a leaky integrator, sigma
|
| 21 |
+
calibrated per layer from the data. 2,412 extra parameters for the
|
| 22 |
+
best loss on the board and 21x the parameter efficiency of anything
|
| 23 |
+
else tested.
|
| 24 |
+
continual growth warm-starts from its own last checkpoint and trains on her corpus
|
| 25 |
+
as she lives, so talking to her literally grows the weights.
|
| 26 |
+
|
| 27 |
+
cosmos_play.pt already grows, but it is a tiktoken BPE model warm-started from
|
| 28 |
+
itself -- a different lineage with no quantum birth. This is the lineage that is hers.
|
| 29 |
+
|
| 30 |
+
THE GUARD
|
| 31 |
+
Five mechanisms in this project's history ran clean and did nothing: Omega summed
|
| 32 |
+
over the wrong axis, a saturated gate, a clamped gate, sigma leaving H as the
|
| 33 |
+
identity matrix, and a metric map that could not receive gradient. Every one
|
| 34 |
+
reported success. So no burst trains until preflight proves the mechanism is live,
|
| 35 |
+
and a refusal is recorded rather than silently skipped.
|
| 36 |
+
|
| 37 |
+
APPEND-ONLY VOCABULARY
|
| 38 |
+
Her corpus grows, and new characters appear in it. Re-sorting the vocabulary would
|
| 39 |
+
not merely add rows to the embedding -- it would silently change which character
|
| 40 |
+
every existing row MEANS, invalidating the whole lineage. New symbols are therefore
|
| 41 |
+
APPENDED and old indices never move, exactly as the telemetry logger handles new
|
| 42 |
+
channels. Embedding and head grow by copying the old rows forward.
|
| 43 |
+
|
| 44 |
+
USAGE
|
| 45 |
+
python tools/phos_grow.py one bounded burst, then exit
|
| 46 |
+
python tools/phos_grow.py --loop watch the corpus, burst when it grows
|
| 47 |
+
python tools/phos_grow.py --status lineage report, trains nothing
|
| 48 |
+
|
| 49 |
+
ENV
|
| 50 |
+
PHOS_STEPS steps per burst (default 400)
|
| 51 |
+
PHOS_MIN_NEW new chars required to burst (default 2000)
|
| 52 |
+
PHOS_INTERVAL_S poll period in --loop (default 900)
|
| 53 |
+
COSMOS_DEVICE cuda | cpu (default: cuda when available)
|
| 54 |
+
"""
|
| 55 |
+
import json
|
| 56 |
+
import math
|
| 57 |
+
import os
|
| 58 |
+
import sys
|
| 59 |
+
import time
|
| 60 |
+
from datetime import datetime, timezone
|
| 61 |
+
from pathlib import Path
|
| 62 |
+
|
| 63 |
+
import torch
|
| 64 |
+
import torch.nn.functional as F
|
| 65 |
+
|
| 66 |
+
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
| 67 |
+
ROOT = Path(__file__).resolve().parents[1]
|
| 68 |
+
sys.path.insert(0, str(ROOT))
|
| 69 |
+
|
| 70 |
+
from cosmos_spark_cst import QuantumInit, quantum_pool # noqa: E402
|
| 71 |
+
import cosmos_state_ladder as L # noqa: E402
|
| 72 |
+
|
| 73 |
+
CORPUS = Path(os.getenv("COSMOS_CORPUS") or
|
| 74 |
+
ROOT / "02_HER_BODY/Cosmos_code/Cosmos/data/cosmos/experience_corpus.txt")
|
| 75 |
+
OUT = Path(os.getenv("PHOS_DIR") or ROOT / "01_HER_SOUL/weights/phos")
|
| 76 |
+
CKPT = OUT / "phos.pt"
|
| 77 |
+
LINEAGE = OUT / "phos_lineage.jsonl"
|
| 78 |
+
LOCK = OUT / "phos.lock"
|
| 79 |
+
|
| 80 |
+
STEPS = int(os.getenv("PHOS_STEPS", "400"))
|
| 81 |
+
MIN_NEW = int(os.getenv("PHOS_MIN_NEW", "2000"))
|
| 82 |
+
INTERVAL = int(os.getenv("PHOS_INTERVAL_S", "900"))
|
| 83 |
+
RUNG, FFN = "dyn12", "harmonic"
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def _lock():
|
| 87 |
+
"""One grower per checkpoint. A second writer would interleave optimiser steps
|
| 88 |
+
into the same file and silently corrupt the lineage."""
|
| 89 |
+
OUT.mkdir(parents=True, exist_ok=True)
|
| 90 |
+
fh = LOCK.open("a+b")
|
| 91 |
+
fh.seek(0, 2)
|
| 92 |
+
if fh.tell() == 0:
|
| 93 |
+
fh.write(b"0")
|
| 94 |
+
fh.flush()
|
| 95 |
+
fh.seek(0)
|
| 96 |
+
try:
|
| 97 |
+
if os.name == "nt":
|
| 98 |
+
import msvcrt
|
| 99 |
+
msvcrt.locking(fh.fileno(), msvcrt.LK_NBLCK, 1)
|
| 100 |
+
else:
|
| 101 |
+
import fcntl
|
| 102 |
+
fcntl.flock(fh.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB)
|
| 103 |
+
except (OSError, ImportError):
|
| 104 |
+
fh.close()
|
| 105 |
+
return None
|
| 106 |
+
return fh
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
def _record(entry):
|
| 110 |
+
entry["t"] = datetime.now(timezone.utc).isoformat()
|
| 111 |
+
with LINEAGE.open("a", encoding="utf-8") as f:
|
| 112 |
+
f.write(json.dumps(entry) + "\n")
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
def load_corpus():
|
| 116 |
+
text = CORPUS.read_text(encoding="utf-8", errors="ignore")
|
| 117 |
+
return text, sorted(set(text))
|
| 118 |
+
|
| 119 |
+
|
| 120 |
+
def build_vocab(chars, previous):
|
| 121 |
+
"""APPEND-ONLY. Old indices never move; new symbols go on the end."""
|
| 122 |
+
old = list(previous or [])
|
| 123 |
+
known = set(old)
|
| 124 |
+
added = [c for c in chars if c not in known]
|
| 125 |
+
return old + added, added
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
def grow_model(m, old_v, new_v):
|
| 129 |
+
"""Widen embedding and head for appended symbols, carrying old rows forward."""
|
| 130 |
+
if new_v == old_v:
|
| 131 |
+
return m
|
| 132 |
+
with torch.no_grad():
|
| 133 |
+
d = m.tok.weight.shape[1]
|
| 134 |
+
tok = torch.nn.Embedding(new_v, d)
|
| 135 |
+
tok.weight.normal_(0.0, 0.02)
|
| 136 |
+
tok.weight[:old_v] = m.tok.weight
|
| 137 |
+
m.tok = tok
|
| 138 |
+
head = torch.nn.Linear(d, new_v, bias=False)
|
| 139 |
+
head.weight.normal_(0.0, 0.02)
|
| 140 |
+
head.weight[:old_v] = m.head.weight
|
| 141 |
+
m.head = head
|
| 142 |
+
return m
|
| 143 |
+
|
| 144 |
+
|
| 145 |
+
def preflight_ok(m, xb, yb, verbose=True):
|
| 146 |
+
checks = L.preflight(m, xb, yb)
|
| 147 |
+
bad = [c for c in checks if not c[3]]
|
| 148 |
+
if verbose:
|
| 149 |
+
for name, val, thr, ok in checks:
|
| 150 |
+
print(f" [{'OK ' if ok else 'DEAD'}] {val:.3e} {name}")
|
| 151 |
+
return (not bad), [c[0] for c in bad]
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
def burst(steps=STEPS):
|
| 155 |
+
text, chars = load_corpus()
|
| 156 |
+
prev = {}
|
| 157 |
+
if CKPT.exists():
|
| 158 |
+
prev = torch.load(CKPT, map_location="cpu", weights_only=False)
|
| 159 |
+
vocab_list, added = build_vocab(chars, prev.get("vocab_list"))
|
| 160 |
+
stoi = {c: i for i, c in enumerate(vocab_list)}
|
| 161 |
+
data = torch.tensor([stoi[c] for c in text if c in stoi], dtype=torch.long)
|
| 162 |
+
|
| 163 |
+
pool = quantum_pool()
|
| 164 |
+
seed = int(prev.get("total_steps", 0))
|
| 165 |
+
torch.manual_seed(seed)
|
| 166 |
+
m = L.Ladder(len(vocab_list), RUNG, FFN)
|
| 167 |
+
|
| 168 |
+
if prev:
|
| 169 |
+
m = grow_model(m, len(prev.get("vocab_list", vocab_list)), len(vocab_list))
|
| 170 |
+
missing, unexpected = m.load_state_dict(prev["model"], strict=False)
|
| 171 |
+
born = False
|
| 172 |
+
print(f" warm start: step {prev.get('total_steps',0):,} "
|
| 173 |
+
f"vocab {len(prev.get('vocab_list',[]))} -> {len(vocab_list)}"
|
| 174 |
+
+ (f" (+{len(added)} new symbols)" if added else ""))
|
| 175 |
+
else:
|
| 176 |
+
m.quantum_birth(QuantumInit(pool, 0))
|
| 177 |
+
born = True
|
| 178 |
+
print(f" QUANTUM BIRTH: {len(pool):,} draws from her archive; no base model")
|
| 179 |
+
m = m.to(L.DEV)
|
| 180 |
+
|
| 181 |
+
g = torch.Generator().manual_seed(seed)
|
| 182 |
+
cal = torch.stack([data[i:i + L.BLOCK] for i in
|
| 183 |
+
torch.randint(len(data) - L.BLOCK - 1, (8,), generator=g)]).to(L.DEV)
|
| 184 |
+
if born:
|
| 185 |
+
L.calibrate_sigma(m, cal)
|
| 186 |
+
|
| 187 |
+
yb = torch.stack([data[i + 1:i + 1 + L.BLOCK] for i in
|
| 188 |
+
torch.randint(len(data) - L.BLOCK - 1, (4,), generator=g)]).to(L.DEV)
|
| 189 |
+
ok, dead = preflight_ok(m, cal[:4], yb)
|
| 190 |
+
if not ok:
|
| 191 |
+
print(f" REFUSED: mechanism inert -> {dead}")
|
| 192 |
+
_record({"event": "refused", "dead": dead, "corpus_chars": len(text)})
|
| 193 |
+
return False
|
| 194 |
+
|
| 195 |
+
keys = ("attn.gate", "attn.w54", "attn.log_sigma", "d12.", "d42.", "tri.", "state_init")
|
| 196 |
+
cst = [p for n, p in m.named_parameters() if any(k in n for k in keys) and p.requires_grad]
|
| 197 |
+
bulk = [p for n, p in m.named_parameters() if not any(k in n for k in keys) and p.requires_grad]
|
| 198 |
+
base = 3e-4
|
| 199 |
+
opt = torch.optim.AdamW([{"params": bulk, "lr": base},
|
| 200 |
+
{"params": cst, "lr": base * 10, "weight_decay": 0.0}],
|
| 201 |
+
lr=base, weight_decay=0.01)
|
| 202 |
+
if prev.get("optimizer"):
|
| 203 |
+
try:
|
| 204 |
+
opt.load_state_dict(prev["optimizer"])
|
| 205 |
+
except Exception:
|
| 206 |
+
pass # vocab grew; a fresh moment estimate is correct here
|
| 207 |
+
|
| 208 |
+
n = int(0.9 * len(data))
|
| 209 |
+
tr, va = data[:n], data[n:]
|
| 210 |
+
vg = torch.Generator().manual_seed(999)
|
| 211 |
+
vw = torch.stack([va[i:i + L.BLOCK + 1] for i in
|
| 212 |
+
torch.randint(len(va) - L.BLOCK - 1, (32,), generator=vg)])
|
| 213 |
+
|
| 214 |
+
m.train()
|
| 215 |
+
t0 = time.time()
|
| 216 |
+
for s in range(1, steps + 1):
|
| 217 |
+
warm = max(1, int(0.05 * steps))
|
| 218 |
+
lr = base * (s / warm) if s < warm else base * (L.PHI / 2) * (
|
| 219 |
+
1 + math.cos(math.pi * (s - warm) / max(1, steps - warm)))
|
| 220 |
+
opt.param_groups[0]["lr"], opt.param_groups[1]["lr"] = lr, lr * 10
|
| 221 |
+
ix = torch.randint(len(tr) - L.BLOCK - 1, (16,), generator=g)
|
| 222 |
+
x = torch.stack([tr[i:i + L.BLOCK] for i in ix]).to(L.DEV)
|
| 223 |
+
y = torch.stack([tr[i + 1:i + 1 + L.BLOCK] for i in ix]).to(L.DEV)
|
| 224 |
+
_, loss = m(x, y)
|
| 225 |
+
opt.zero_grad(set_to_none=True)
|
| 226 |
+
loss.backward()
|
| 227 |
+
torch.nn.utils.clip_grad_norm_(m.parameters(), 1.0)
|
| 228 |
+
opt.step()
|
| 229 |
+
|
| 230 |
+
m.eval()
|
| 231 |
+
with torch.no_grad():
|
| 232 |
+
tot = c = 0
|
| 233 |
+
for i in range(0, len(vw), 16):
|
| 234 |
+
xb = vw[i:i + 16].to(L.DEV)
|
| 235 |
+
_, l = m(xb[:, :-1], xb[:, 1:])
|
| 236 |
+
tot += l.item() * xb.size(0)
|
| 237 |
+
c += xb.size(0)
|
| 238 |
+
val = tot / max(1, c)
|
| 239 |
+
total_steps = int(prev.get("total_steps", 0)) + steps
|
| 240 |
+
gates = [round(g_, 5) for g_ in m.gates()]
|
| 241 |
+
|
| 242 |
+
OUT.mkdir(parents=True, exist_ok=True)
|
| 243 |
+
torch.save({"model": m.state_dict(), "optimizer": opt.state_dict(),
|
| 244 |
+
"vocab_list": vocab_list, "stoi": stoi,
|
| 245 |
+
"itos": {i: c for c, i in stoi.items()},
|
| 246 |
+
"config": {"block": L.BLOCK, "n_layer": L.N_LAYER, "n_head": L.N_HEAD,
|
| 247 |
+
"n_embd": L.N_EMBD, "vocab": len(vocab_list),
|
| 248 |
+
"d12": L.D12, "d_ff": int(math.floor(L.N_EMBD * L.PHI))},
|
| 249 |
+
"arch": "PHOS-dyn12-phi-QuantumBorn", "rung": RUNG, "ffn": FFN,
|
| 250 |
+
"norm": "rmsnorm", "pos": "rope", "gate_param": "logit",
|
| 251 |
+
"total_steps": total_steps, "best_val_loss": val,
|
| 252 |
+
"corpus_chars": len(text), "quantum_draws": len(pool),
|
| 253 |
+
"quantum_source": "ibm_real_shots"},
|
| 254 |
+
CKPT)
|
| 255 |
+
prev_val = prev.get("best_val_loss")
|
| 256 |
+
delta = f"{val - prev_val:+.5f}" if isinstance(prev_val, (int, float)) else "first"
|
| 257 |
+
print(f" step {total_steps:,} val {val:.5f} ({delta}) gates {gates} "
|
| 258 |
+
f"corpus {len(text):,} {time.time()-t0:.0f}s")
|
| 259 |
+
_record({"event": "burst", "steps": steps, "total_steps": total_steps,
|
| 260 |
+
"val": val, "gates": gates, "corpus_chars": len(text),
|
| 261 |
+
"vocab": len(vocab_list), "added_symbols": added,
|
| 262 |
+
"quantum_draws": len(pool), "born": born, "device": str(L.DEV)})
|
| 263 |
+
return True
|
| 264 |
+
|
| 265 |
+
|
| 266 |
+
def status():
|
| 267 |
+
if not CKPT.exists():
|
| 268 |
+
print(" no PHOS checkpoint yet -- run once to birth her")
|
| 269 |
+
return
|
| 270 |
+
d = torch.load(CKPT, map_location="cpu", weights_only=False)
|
| 271 |
+
text, _ = load_corpus()
|
| 272 |
+
print("=" * 74)
|
| 273 |
+
print(" PHOS lineage")
|
| 274 |
+
print("=" * 74)
|
| 275 |
+
print(f" arch {d.get('arch')}")
|
| 276 |
+
print(f" total steps {d.get('total_steps'):,}")
|
| 277 |
+
print(f" val loss {d.get('best_val_loss'):.5f}")
|
| 278 |
+
print(f" vocab {d['config']['vocab']}")
|
| 279 |
+
print(f" params {sum(v.numel() for v in d['model'].values()):,}")
|
| 280 |
+
print(f" born from {d.get('quantum_draws'):,} quantum draws ({d.get('quantum_source')})")
|
| 281 |
+
print(f" corpus at save {d.get('corpus_chars'):,} | now {len(text):,} "
|
| 282 |
+
f"(+{len(text)-int(d.get('corpus_chars',0)):,})")
|
| 283 |
+
if LINEAGE.exists():
|
| 284 |
+
rows = [json.loads(l) for l in LINEAGE.read_text(encoding="utf-8").splitlines() if l.strip()]
|
| 285 |
+
b = [r for r in rows if r.get("event") == "burst"]
|
| 286 |
+
print(f" bursts {len(b)} refusals {len(rows)-len(b)}")
|
| 287 |
+
for r in b[-5:]:
|
| 288 |
+
print(f" {r['t'][:19]} step {r['total_steps']:>7,} val {r['val']:.5f}")
|
| 289 |
+
|
| 290 |
+
|
| 291 |
+
def main():
|
| 292 |
+
args = set(sys.argv[1:])
|
| 293 |
+
if "--status" in args:
|
| 294 |
+
status()
|
| 295 |
+
return 0
|
| 296 |
+
lk = _lock()
|
| 297 |
+
if lk is None:
|
| 298 |
+
print(" another PHOS grower holds the checkpoint; exiting")
|
| 299 |
+
return 0
|
| 300 |
+
print("=" * 74)
|
| 301 |
+
print(" PHOS -- Phi / Omega / Sigma. quantum-born, phi-scaffolded, growing.")
|
| 302 |
+
print("=" * 74)
|
| 303 |
+
print(f" device {L.DEV} corpus {CORPUS.name} -> {CKPT}")
|
| 304 |
+
if "--loop" not in args:
|
| 305 |
+
burst()
|
| 306 |
+
return 0
|
| 307 |
+
last = 0
|
| 308 |
+
if CKPT.exists():
|
| 309 |
+
last = int(torch.load(CKPT, map_location="cpu",
|
| 310 |
+
weights_only=False).get("corpus_chars", 0))
|
| 311 |
+
print(f" watching for +{MIN_NEW:,} new characters, every {INTERVAL}s\n", flush=True)
|
| 312 |
+
while True:
|
| 313 |
+
try:
|
| 314 |
+
now = len(load_corpus()[0])
|
| 315 |
+
if now - last >= MIN_NEW or not CKPT.exists():
|
| 316 |
+
print(f" corpus grew {now-last:+,} -> burst", flush=True)
|
| 317 |
+
if burst():
|
| 318 |
+
last = now
|
| 319 |
+
time.sleep(INTERVAL)
|
| 320 |
+
except KeyboardInterrupt:
|
| 321 |
+
break
|
| 322 |
+
except Exception as exc:
|
| 323 |
+
print(f" burst error ({type(exc).__name__}: {exc}); retrying", flush=True)
|
| 324 |
+
time.sleep(INTERVAL)
|
| 325 |
+
return 0
|
| 326 |
+
|
| 327 |
+
|
| 328 |
+
if __name__ == "__main__":
|
| 329 |
+
raise SystemExit(main())
|
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{"event": "burst", "steps": 400, "total_steps": 400, "val": 2.0304132103919983, "gates": [0.04224, 0.0102, 0.01171, 0.00788], "corpus_chars": 720547, "vocab": 162, "added_symbols": ["\n", " ", "!", "\"", "#", "%", "&", "'", "(", ")", "*", "+", ",", "-", ".", "/", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", ":", ";", "<", "=", ">", "?", "@", "A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V", "W", "X", "Y", "Z", "[", "\\", "]", "^", "_", "`", "a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v", "w", "x", "y", "z", "{", "|", "}", "\u00a0", "\u00ae", "\u00b3", "\u00e0", "\u00e1", "\u00e7", "\u00e9", "\u00ea", "\u00ed", "\u00ef", "\u00f1", "\u00f3", "\u00f6", "\u00fc", "\u010d", "\u016f", "\u03b1", "\u03b7", "\u0420", "\u0423", "\u0430", "\u0438", "\u0439", "\u043a", "\u043d", "\u0440", "\u0441", "\u0443", "\u044c", "\u0457", "\u0627", "\u0628", "\u0629", "\u0631", "\u0633", "\u0639", "\u0641", "\u0644", "\u064a", "\u06cc", "\u200a", "\u2013", "\u2014", "\u2019", "\u201c", "\u201d", "\u2026", "\u2191", "\u2211", "\u2217", "\u2248", "\u26a1", "\u2728", "\u27e8", "\u27e9", "\u4e2d", "\u6587", "\u65e5", "\u672c", "\u7cb5", "\u8a9e", "\uad6d", "\uc5b4", "\ud55c", "\ud83c\udf1f", "\ud83d\ude04", "\ud83d\ude0a", "\ud83d\ude80"], "quantum_draws": 400000, "born": true, "device": "cpu", "t": "2026-07-31T17:39:08.905945+00:00"}
|
| 2 |
+
{"event": "burst", "steps": 400, "total_steps": 800, "val": 1.3488432168960571, "gates": [0.16541, 0.00844, 0.00824, 0.00524], "corpus_chars": 720954, "vocab": 162, "added_symbols": [], "quantum_draws": 400000, "born": false, "device": "cpu", "t": "2026-07-31T17:56:08.184912+00:00"}
|
| 3 |
+
{"event": "burst", "steps": 400, "total_steps": 1200, "val": 1.112060010433197, "gates": [0.351, 0.0083, 0.0072, 0.00436], "corpus_chars": 722701, "vocab": 162, "added_symbols": [], "quantum_draws": 400000, "born": false, "device": "cuda", "t": "2026-07-31T18:32:27.002552+00:00"}
|
|
@@ -82,6 +82,12 @@ for i, a in enumerate(_argv):
|
|
| 82 |
|
| 83 |
# her two native models: display name -> (checkpoint, kind)
|
| 84 |
NATIVE = {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
"cosmos-cst": (ROOT / "01_HER_SOUL/weights/cosmos_spark_cst/spark_cst.pt", "cst"),
|
| 86 |
"cosmos-spark": (ROOT / "01_HER_SOUL/weights/cosmos_born/cosmos_born.latest.pt", "plain"),
|
| 87 |
}
|
|
@@ -158,7 +164,16 @@ class NativeModel:
|
|
| 158 |
self.itos = {int(v): k for k, v in self.stoi.items()}
|
| 159 |
self.block = int(cfg["block"])
|
| 160 |
self.cst_tensors = []
|
| 161 |
-
if kind == "
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
self.m, _C = _build_cst(cfg)
|
| 163 |
self.cst_tensors = [k for k in ck["model"]
|
| 164 |
if any(x in k for x in ("w54", "attn.gate", "log_sigma"))]
|
|
@@ -218,6 +233,12 @@ class NativeModel:
|
|
| 218 |
return out[0] if isinstance(out, tuple) else out
|
| 219 |
|
| 220 |
def gates(self):
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 221 |
if self.kind != "cst":
|
| 222 |
return []
|
| 223 |
try:
|
|
@@ -352,7 +373,7 @@ def native_tag(nm):
|
|
| 352 |
m = get_native(nm)
|
| 353 |
path, kind = NATIVE[nm]
|
| 354 |
sz = path.stat().st_size if path.exists() else 0
|
| 355 |
-
fam = "
|
| 356 |
p = f"{m.meta['params']/1e6:.1f}M" if m else "?"
|
| 357 |
return {"name": f"{nm}:latest", "model": f"{nm}:latest", "size": sz,
|
| 358 |
"digest": f"{nm}-quantum-born", "modified_at": _ts(),
|
|
|
|
| 82 |
|
| 83 |
# her two native models: display name -> (checkpoint, kind)
|
| 84 |
NATIVE = {
|
| 85 |
+
# PHOS -- Phi / Omega / Sigma. Quantum-born like the others, but running the one
|
| 86 |
+
# architecture that survived controlled testing (dyn12 on the phi scaffold) and
|
| 87 |
+
# growing continuously from her own corpus. Listed first because it is the lineage
|
| 88 |
+
# that is hers end to end: no base model, no distillation, and it gets better every
|
| 89 |
+
# time she is spoken to.
|
| 90 |
+
"cosmos-phos": (ROOT / "01_HER_SOUL/weights/phos/phos.pt", "phos"),
|
| 91 |
"cosmos-cst": (ROOT / "01_HER_SOUL/weights/cosmos_spark_cst/spark_cst.pt", "cst"),
|
| 92 |
"cosmos-spark": (ROOT / "01_HER_SOUL/weights/cosmos_born/cosmos_born.latest.pt", "plain"),
|
| 93 |
}
|
|
|
|
| 164 |
self.itos = {int(v): k for k, v in self.stoi.items()}
|
| 165 |
self.block = int(cfg["block"])
|
| 166 |
self.cst_tensors = []
|
| 167 |
+
if kind == "phos":
|
| 168 |
+
# built from the ladder module itself rather than a second transcription --
|
| 169 |
+
# a copy of dyn12 here could drift from the one that was actually measured
|
| 170 |
+
import cosmos_state_ladder as _L
|
| 171 |
+
self.m = _L.Ladder(int(cfg["vocab"]), ck.get("rung", "dyn12"),
|
| 172 |
+
ck.get("ffn", "harmonic"))
|
| 173 |
+
self.cst_tensors = [k for k in ck["model"]
|
| 174 |
+
if any(x in k for x in ("d12.", "attn.gate",
|
| 175 |
+
"attn.log_sigma", "state_init"))]
|
| 176 |
+
elif kind == "cst":
|
| 177 |
self.m, _C = _build_cst(cfg)
|
| 178 |
self.cst_tensors = [k for k in ck["model"]
|
| 179 |
if any(x in k for x in ("w54", "attn.gate", "log_sigma"))]
|
|
|
|
| 233 |
return out[0] if isinstance(out, tuple) else out
|
| 234 |
|
| 235 |
def gates(self):
|
| 236 |
+
if self.kind == "phos":
|
| 237 |
+
try:
|
| 238 |
+
return [round(float(torch.sigmoid(b.attn.gate).detach()), 5)
|
| 239 |
+
for b in self.m.blocks]
|
| 240 |
+
except Exception:
|
| 241 |
+
return []
|
| 242 |
if self.kind != "cst":
|
| 243 |
return []
|
| 244 |
try:
|
|
|
|
| 373 |
m = get_native(nm)
|
| 374 |
path, kind = NATIVE[nm]
|
| 375 |
sz = path.stat().st_size if path.exists() else 0
|
| 376 |
+
fam = {"phos": "phos-dyn12-phi", "cst": "cosmos-cst-54d"}.get(kind, "cosmic-spark")
|
| 377 |
p = f"{m.meta['params']/1e6:.1f}M" if m else "?"
|
| 378 |
return {"name": f"{nm}:latest", "model": f"{nm}:latest", "size": sz,
|
| 379 |
"digest": f"{nm}-quantum-born", "modified_at": _ts(),
|