LLMs trained on human behavioral data can be probed to extract realistic human behaviors, enabling simulation of entire lived experiences.
Jun Park explains how LLMs trained on human behavioral data can be probed to extract realistic human behaviors, leading to the 'Smallville' simulation where 25 NPCs autonomously woke up, did routines, formed relationships, and self-organized a Valentine's Day party. ✦ AI generated
Jun Park · 20VC · 2026-08-01 · original ↗
starts at this moment · 1:58
So, this was 2023. we had this idea that large language models are often used for simpler tasks like classification, simple generation but we thought that these models actually had a lot more potential. One of the early observations that we made was that these models are trained on so much of human behavioral data, sentiment data that were expressed on the web. So if you poke at them sort of at the right angle, you could actually extract a lot of realistic human behaviors out of them. I thought that was really interesting and it was also particularly interesting in that it was domain agnostic. So if you look at the literature in computer science for many decades we've always had the vision of creating agents that are meant to be generalizable that are meant to really be able to act like human in any environment. And my mind went to well maybe we have that opportunity here. So what we ended up doing was well if we were to fast forward many years into doing this what would be the most ambitious vision that we might have and that was creating entire lived experience of a town. So the idea here was we would make a game town and we would populate it with 25 NPCs. So non-playable characters except these characters would actually wake up in the morning, do their routines, go to work, have relationships and do all that. They would actually remember their interactions. They would actually plan their days. And some of the surprising things you end up seeing was the simulation itself was set the day before Valentine's Day and you actually see these agents come together, have parties like self-organized. So they would actually plan parties, they would decorate the cafe and so forth.
verbatim transcript · starts at 1:58
1:58simple generation but we thought that these models actually had a lot more potential. One of the early observations that we made was that these models are trained on so much of human behavioral data, sentiment data that were expressed on the web. So if you poke at them sort of at the right angle, you could actually extract a lot of realistic human behaviors out of them. I thought
2:19that was really interesting and it was also particularly interesting in that it was domain agnostic. So if you look at the literature in computer science for many decades we've always had the vision of creating agents that are meant to be generalizable that are meant to really be able to act like human in any environment. And my mind went to well maybe we have that opportunity here. So
2:42what we ended up doing was well if we were to fast forward many years into doing this what would be the most ambitious vision that we might have and that was creating entire lived experience of a town. So the idea here was we would make a game town and we would populate it with 25 NPCs. So non-playable characters except these characters would actually wake up in the
3:03morning, do their routines, go to work, have relationships and do all that. They would actually remember their interactions. They would actually plan their days. And some of the surprising things you end up seeing was the simulation itself was set the day before Valentine's Day and you actually see these agents come together, have parties like self-organized. So they would actually plan parties, they would decorate the cafe and so forth. We
3:26thought that was really interesting. Now two fundamental contribution from that work. One was it was one of the earliest example of creating agents. So this particular uh set of agents were paired with back in back in the day GPT3.5 text. So we didn't quite have chat GPT back then. Uh and then was paired with memory planning and reflection really the first times that those concepts came out to be uh an explicit part of the
3:53architecture in quote unquote agentic workflows. The reason why we actually got that inspiration was if you had more than one agent side by side you want them to remember each other. Back in the day, lynching mortals didn't really have the concept of memory. So I thought, okay, you have to give them me the memory so that they don't say, "Hey, nice meeting you every time they meet
4:13their roommate." So we had them give have this concept of memory and planning and reflection to make sense of very long-term landscape. >> How do you solve that memory problem? Cuz everyone says, "Oh, we have a memory problem today." How do you solve the memory problem of agents to prevent that from happening? So back in the day was actually the initial idea was fairly simple which was that these language