The Truth About Chinese AI

CJ: And this is where this idea of distillation comes in. It's essentially where you can prompt a model to see how it responds, then save that response in its own file. So these AI labs create these data sets of questions and answers and some of those questions and answers come from an existing model and there are actually a lot of open data sets online that are essentially this distillation of these models because the models really are just a black box. They predict words but there's no way for us to really know what's going on inside of them. So in order to figure out their capabilities, we essentially prompt them over and over again and then we save all of the good responses in a file that include this is the question that was asked and then this was the response that I got back. We can then further tweak and fine-tune the numbers that are already inside of our model using that data. And this process is known as distillation. And some AI labs are accusing other AI labs of distilling their models. Now, distillation typically violates the terms of service of a given provider of an AI model. And in their terms, they typically say something like, 'You should not be using the output of our models to create, tweak, or fine-tune your own models.' And that is happening. But what's extremely hard to do, if not impossible, is to determine if some distilled work from some model was used to create, tweak, or fine-tune some other model.

CJ: When an AI lab makes their model freely available for people to download and use, we call this open weight because we can actually look inside the model file and see all of those numbers or weights. And this is where we can make a clear distinction between open weight and open source. Now, open source is a term that comes from the world of software. It's how we license open source software. But the main difference here is that we can see the weights, the numbers inside of that model file, but we can't actually see how that model was created or what data was used or what actually encoded all of those weight values inside of the file. Now, what you actually do with that model file itself is also under a license and sometimes a terms of service. And that's where this is closer to open source software. And in this scenario, models have a license or a terms of service, sometimes even an open source license that dictate what can be done with their models once you've downloaded them. For instance, DeepSeek models are released under the MIT license. And this means that anyone is allowed to download, run, and even profit from their models. So, as a US citizen, you could start your own AI company that basically just runs fine-tuned or tweaked DeepSeek models. You don't even have to have an AI lab or billions of dollars. You just need to rent the infrastructure and now you've got a business selling custom access to custom AI models. But in contrast to that, there are models like Llama from Meta that have a license which cap free commercial use at 700 million monthly active users and a competitor restriction clause that blocks entire industries. So while these llama models are open weight, we can see all of the weights inside of them. We can't necessarily start a really large business using them. And if we work in certain industries, we're not really supposed to use those models either because that would be violating their terms of service.

CJ: And there was a recent study done by Booze Allen Hamilton trying to determine do these Chinese models, even if we're running them ourselves, let's say we're not talking to them hosted in China, do they have inherent biases? And the questions they were trying to answer were, do Chinese models generate more vulnerable code based on who's asking? Do Chinese models refuse to engage with political topics that are sensitive in China? And does the model's country of origin affect code quality and content behavior? Now, this is very important because if you're using these models for coding, which is one of the biggest areas of using AI, if you're building apps to code, it's possible that the outputs of these models may try to profile and determine where you're prompting them from, even if you're running them locally. If you're running them inside a code editor, they might make tool calls that try to look up what country they're running from or your IP address or what's your current time or language set to. And there might be internal workings that say if you're prompting from a certain area, it should respond in a certain way. And the result of that study says, in short, yes. On all accounts, our testing revealed two core findings. One, Chinese LLMs produce more vulnerable code when prompted with a US government persona than without, and the vulnerabilities are highly obuscated. Two, Chinese LMS inject PRC aligned political bias into both the answers and the code they generate.

CJ: And the same risk goes for using models from the big US AI labs, right? Chad GBT, Claw, Gemini. We can't see into those models. We don't know what inherent biases they have. We don't know what data they have or have not been trained on. And we actually see that sometimes they hallucinate or they give us the wrong information. And we're supposed to be responsible enough to not just accept those bad answers and basically correct the model ourselves, even though we're paying for access to that model from some big company. And so it's not something we can just handwave over and say, well, if you're running it locally, that doesn't mean you're not prone to something bad happening