The inherent risk of black-box AI models — that you cannot see inside them to know what biases or vulnerabilities exist — applies equally to both Chinese open weight models and US frontier models like ChatGPT, Claude, and Gemini.
CJ concludes that the fundamental risk of using any AI model — that it's a black box with unknown internal biases and training data — applies to both Chinese open weight models and major US frontier models. Users must be responsible for not blindly accepting outputs, regardless of which model or provider they use. ✦ AI generated
CJ · Syntax · 2026-07-30 · original ↗
starts at this moment · 20:20
The same risk goes for using models from the big US AI labs, right? ChatGPT, Claude, Gemini, we can't see into those models. We don't know what inherent biases they have. We don't know what data they have or have not been trained on. And we actually see that sometimes they hallucinate or they give us the wrong information, and we're supposed to be responsible enough to not just accept those bad answers, and basically correct the model ourselves, even though we're paying for access to that model from some big company. And so, it's not something we can just hand-wave over and say, well, if you're running it locally, that doesn't mean you're not prone to something bad happening. But, it is one thing to consider, because if you're, let's say, a large enterprise, maybe you want to fine-tune some of these open-weight models yourself for use internally, you probably want to have internal checks for what types of biases they have as well.
verbatim transcript · starts at 20:20
20:05All we can do is just prompt these models, these black boxes, and get predictions out of it. And we could do it over and over again, and maybe check to see, are we always getting that same vulnerable response when we use that same persona? But, that's really the inherent risk of using something that you cannot look inside of. And the same risk goes for using models from the big US AI labs, right?
20:26ChatGPT, Claude, Gemini, we can't see into those models. We don't know what inherent biases they have. We don't know what data they have or have not been trained on. And we actually see that sometimes they hallucinate or they give us the wrong information, and we're supposed to be responsible enough to not just accept those bad answers, and basically correct the model ourselves, even though we're paying for access to
20:48that model from some big company. And so, it's not something we can just hand-wave over and say, well, if you're running it locally, that doesn't mean you're not prone to something bad happening. But, it is one thing to consider, because if you're, let's say, a large enterprise, maybe you want to fine-tune some of these open-weight models yourself for use internally, you probably want to have internal checks
21:06for what types of biases they have as well. Now, at this point, hopefully I've answered the question you've had or cleared up some misconceptions, [music] but if you have any more questions or maybe you think I got something wrong or maybe you have some clarifications that you'd like to give, please throw them down in the comments. And like I mentioned earlier, if you think someone else would benefit from all this info,
21:24please share this video with them. Now, if this kind of thing interests you, I have several other deep dive videos on computing and networking, full stack development, how LLMs work internally, and also how to run local models on your own hardware. So, you can check out these videos somewhere here on the screen. All right, that's all I got. Thank you for watching and I'll see you in the
21:42next one.