Attacks aimed at a model's interior — model theft, training-data extraction, and poisoning — are bounded and largely mitigated for most developers, ranking low for initial effort compared to risks around external actions.

The article treats interior model attacks as real but bounded for most developers, giving three examples: recovering an OpenAI embedding layer for under $20, extracting ChatGPT training data by prompting it to repeat a single word, and a 2025 finding that ~250 malicious documents can backdoor models of any scale. ✦ AI generated

Article author (GLM pipeline) · ByteByteGo Newsletter · 2026-08-03 · original ↗

Attacks aimed at the model's interior, including weight theft, training-data extraction, and training-time poisoning, are quite real, but for most developers, they rank low for initial effort. They tend to be expensive, narrow, or already mitigated by the model provider. Model theft: ... recovered the final embedding-projection layer of production OpenAI models for under twenty dollars ... the researchers stated that reconstructing a full frontier model through an API remains impractical. Training-data extraction: ... prompting ChatGPT to repeat a single word continuously could cause it to emit verbatim fragments of training data. Poisoning: In 2025 ... approximately 250 malicious documents were sufficient to install a backdoor in models from 600 million to 13 billion parameters ... That finding overturned the assumption that larger models require proportionally more poisoned data. ... The backdoor produced only gibberish output on a trigger phrase.

Read full article ↗excerpt · fair-use quotation