Open-weight policy is moving against weights like these: the 'apply the same standards' argument, in their own words
This concerns the policy environment around open weights, not a bug report.
It is posted here because the question being decided is whether weights like these
can keep being published at all.
Written in English on purpose. Machine-translated technical writing gets
dismissed in its first line by readers of this board, so there is no machine
translation here. If you think this is worth your community's time, a native
translation would carry it far better than an automated one, and I would rather
defer to you than post broken Japanese or Korean.
The argument, from the primary documents
Anthropic's published position on open-weights models says two things:
Anthropic has never advocated for a ban on open-weights models.
and, further down that same page:
All sufficiently capable models, open and closed, should go through mandatory safety testing.
Both are sincere, and the sincerity is the problem. Pre-release testing works by
holding a model back until it clears. You cannot hold back weights already
sitting on ten thousand hard drives. A requirement open models can never satisfy,
applied on equal terms, does the same work a ban would.
David Sacks predicted that exact framing on All-In:
An open source ban is coming. They're not going to call it that. You know what
they're going to say? They're going to say that we simply have to apply the same
standards to open models that we apply to closed.
Why the standard cannot apply, in their words
The same document states open models are risky because they "cannot be centrally
monitored and controlled and rolled back and withdrawn." That is simply what
released weights are. It is not a defect awaiting a patch.
A footnote on that page quotes the UK AI Security Institute:
Once open-weight models are released, these options are lost permanently: safeguards
can be removed, and copies can be downloaded, redistributed, and run on private
systems beyond monitoring.
Downloaded and redistributed, running where nobody can watch. That describes
BitTorrent. It has worked since Napster and requires no new invention.
What we do about it
We seed open weights over BitTorrent and IPFS from a real machine: 32 torrents,
319.7 GB, fully seeded, including quantized builds of models from this region's
open-weight projects. There is no origin server to take down, no repository to
deplatform, and no terms of service that change under you after you already have
the file.
Every bundle ships a MANIFEST.txt with a sha256 per file. Do not take our word
for what a bundle contains. Check the hash. That is the only claim here that
requires nobody's trust, including ours.
magnet:?xt=urn:btih:2376a46858e0c58d9c2d5b0b76967edd9c89ae45
Sources
- https://www.anthropic.com/news/position-open-weights-models
- Amodei, Senate Judiciary, 2023-07-26 (verbatim): "the path that things are going in terms of the scaling of open source models... is going down a very dangerous path. When a model is released in an uncontrolled manner, there is no ability to do that. It is entirely out of your hands."
- David Sacks, All-In (transcript)
- UK AI Security Institute, quoted in Anthropic's footnote above
If the reading is wrong, the useful reply is a counterexample: name a pre-release
testing regime that an open-weights model could actually satisfy.