Volt's CSS-inspired query selector language is a powerful innovation that allows expressive, composable selection of packages by metadata, relationships, and state across projects.

(00:00:01) Package management sits at the foundation of modern software development, quietly powering nearly every software project in the world. (00:00:09) Tools like NPM and Yarn have long been the core of the JavaScript ecosystem, enabling developers to install, update, and share code with ease. (00:00:18) But as projects grow larger and the ecosystem more complex, this older infrastructure is beginning to show its limits with performance bottlenecks, dependency conflicts, and growing concerns around supply chain security. (00:00:30) Darcy Clark and Rui Adorno are veterans of this ecosystem. (00:00:34) Both spent years maintaining the NPM CLI and helping guide the Node.js project, where they saw firsthand the technical debt and design trade-offs that define modern JavaScript tooling. (00:00:46) Now they're building Volt, a new package manager and registry that rethinks performance, security, and developer experience from the ground up. (00:00:55) In this episode, Darcy and Rui join Josh Goldberg to discuss how Volt works. (00:01:00) why they believe package management needs a server-side reboot, what lessons they've drawn from NPM's evolution, and how features like declarative querying, self-hosted registries, and real-time security scanning could reshape how developers build and share JavaScript in the years ahead. (00:01:19) This episode is hosted by Josh Goldberg, an independent full-time open-source developer. (00:01:24) Josh works on projects in the TypeScript ecosystem, most notably TypeScript ESLint, a powerful static analysis toolset for JavaScript and TypeScript. (00:01:34) He is also the author of the O'Reilly Learning TypeScript book, a Microsoft MVP for developer technologies, and a co-founder of SquiggleConf, a conference for excellent web developer tooling. (00:01:46) Find Josh on Bluesky, (00:01:49) posted on and.com as Joshua K. (00:01:51) Goldberg. (00:02:04) Darcy Clark and Voria Dorner, The Bolt Company. (00:02:07) Welcome to Software Engineering Daily. (00:02:08) How's it going? (00:02:09) Good. (00:02:10) Thanks for having us, Josh. (00:02:11) Yeah, thank you. (00:02:12) I'm very excited. (00:02:13) Just to start off, let's go in order, alphabetically, your first name. (00:02:17) Darcy, who are you and how did you come to work with Volt and package managers? (00:02:21) So who I am, I'm a software engineer who has been developing, I would say, in JavaScript for at least about 20 years now. (00:02:31) And I got into package management by jumping headfirst into the NPM. (00:02:37) Inc., the NPM company in 2019, was hired, and then shortly after had brought on Roy and worked with him very closely. (00:02:46) We were actually a part of the NPM or the acquisition of NPM by GitHub in 2020. (00:02:53) And that was a pretty exciting time for us. (00:02:56) We got to see what it's like to work at both a fast moving and rocket ship of a venture backed startup, and then we also got to see what it was like to go into (00:03:07) a very large enterprise company. (00:03:09) And that was also being acquired by the largest enterprise company, i.e. (00:03:14) Microsoft. (00:03:15) And yeah, we got to support the world's largest package registry. (00:03:18) And I really enjoyed the space and I really care deeply about community and open source. (00:03:24) So yeah, just fell in love many, many years ago with JavaScript and fell in love with also building software and making my own software. (00:03:32) Got the opportunity laid into sort of the NPM company's life. (00:03:37) Span to actually do it full-time, and we're back at it again here with our new company, Volt. (00:03:42) And you, Roy? (00:03:44) Hello, my name is Roy Adorno, and also a software developer, and... (00:03:50) Very similar to Darcy, about 20 years of experience in this field, and my story with the package management ecosystem is very good to end with Darcy. (00:03:59) I joined his team back at NPM in 2019, and we live together through the GitHub acquisition, and we have... (00:04:09) foster the NPM CLI and participating in the Node ecosystem for over three years together in both these companies. (00:04:17) And yeah, these days I'm a member of the technical steering committee in the Node.js project and currently serving as the vice chair there, which means I'm kind of help steering, fostering the project. (00:04:32) Most of that is kind of helping running the weekly meetings, but also try to (00:04:37) be helpful to the other collaborators in the project. (00:04:40) And yeah, I do like to mention a little bit of my background. (00:04:45) I've been in Canada for half my life here, but I'm where is indeed from Brazil, where I grew up. (00:04:50) So (00:04:51) So, that's kind of a little bit about me and my Mac. (00:04:54) Great. (00:04:55) Before we dive into all the interesting stuff that Volt is doing, I'd like to walk our listeners through what it means to be a package manager. (00:05:02) So, what is the delineation between the core language or framework base, such as Node, and the package manager on top of it, something like NPM? (00:05:10) How does that all work together? (00:05:12) So that's a great question. (00:05:13) These things obviously have to have some mutual understanding about where things live in order for the one to be helpful to the other. (00:05:21) And I think there's, in many ecosystems now, actually a dedicated package manager for the language or runtimes associated. (00:05:30) And Node really, I think, has we sort of trailblazed the path in the early aughts in terms of the relationship between NPM and Node. (00:05:39) having distributed essentially a package manager with the runtime for so long. (00:05:43) In terms of the delineation of where the packages actually live, the runtime needs to be aware of where to be looking at these things. (00:05:50) And the package manager needs to be aware of where it should be placing these things to be helpful, right? (00:05:55) And sometimes, yeah. (00:05:57) Folks think that these things should be one and the same, or that's a more common approach that Bun and Dino are starting to take in terms of being both a package manager and a runtime. (00:06:07) And especially with the approach Dino has taken, leaning into sort of like the ESM spec and sort of browser standards. (00:06:15) We are seeing a blurring of the lines between what it means to be a package manager and a runtime. (00:06:21) But yeah, traditionally the (00:06:23) handoff has been where's the location of my modules or where does the runtime look for modules and how can the package manager place those modules in a consistent place. (00:06:34) And the value add really for the package manager is consistency, security, great developer experience around ensuring you get access to updates, you understand that there's a net new version of a piece of software and that's huge in terms of the value add traditionally if you're from (00:06:52) developing 20 years ago, you remember consuming software directly from CDNs and you would have no clue when there was a new release of a piece of software and you were hot linking and there's a lot of problems with that strategy and/or you were bundling the software yourself and you would be shipping legacy code all the time. (00:07:10) And so package managers really add a lot of value in trying to keep your code evergreen and ensure that you are safer that way. (00:07:18) And I think there's a lot of value in the space that is for package management or package tooling. (00:07:24) But they definitely work closely with runtimes and there has to be some sort of condom in understanding about where these things should live for them to actually be operationalized by the runtime. (00:07:34) That makes sense. (00:07:36) I've seen online quite a lot of discourse about things like Core Pack, about how much effort the core runtime should even put into allowing users to define which package manager they're using, or those API hooks between the two of them. (00:07:48) I guess, Roy, what's the current state of things? (00:07:50) And what do you see as the role of something like Node integrating with hooks for something like a package manager? (00:07:56) Yeah, it's a good question. (00:07:58) The Node runtime ships a packet manager, right? (00:08:01) There is that canonical story of NPM being the package manager for the runtime. (00:08:06) And yeah, it does take energy from the project, just managing that relationship. (00:08:12) It's already won. (00:08:14) And I think there are further complications in trying to manage that directly, the relationship with all the possible package managers you could possibly use at the Node runtime. (00:08:24) There is that. (00:08:25) But it was the interesting part of the story is that CorePAC, the whole discussion did lingered for a while within the Node project, right? (00:08:33) And it was not too long ago. (00:08:35) I think it was early this year that the Node TSC finally sat down and had a vote to decide because it was a very polarizing topic, subject within the project. (00:08:48) On the one side, you have a group of people who really love their user experience. (00:08:53) and the capabilities that Core Pack were enabling. (00:08:57) And on the other side, there were a group of people that were very concerned about some of the details of the implementation. (00:09:06) And eventually, we had a vote, I think it was early in the summer. (00:09:10) We decided that the project itself wants to move past the Core Pack story. (00:09:15) So the idea here is that we're kind of doing a (00:09:19) We're doing a form in which it's still being shipped for a while. (00:09:23) If I'm not mistaken, it's still included in Node 25, we just shipped. (00:09:28) It's the current version at the moment we're speaking. (00:09:30) And the idea is to start announcing to the user base that, okay, this is going away, this is being deprecated, but really give users some time before flipping off that switch. (00:09:46) so that they can adapt the ecosystem as how can prepare and adapt to new strategies to serve the other package managers, right? (00:09:52) And I think I'm pretty sure Darcy can also build up a little bit more on the Core Pack side of things. (00:10:00) Before we dive too deep, can you just define what is Core Pack? (00:10:03) So Core Pack was traditionally, or when it was first introduced, was actually under the name PMM. (00:10:11) which was package manager manager, which is funny because that basically tells you what CorePak is. (00:10:16) It's a package manager manager. (00:10:18) And so it was very much tailored to managing a subset of the ecosystem that we consider to be packages. (00:10:28) Funny enough, when you think about package manager, what actually differentiates a package manager from another package? (00:10:34) right? (00:10:34) Is a package manager a package itself? (00:10:36) Like is a runtime a package? (00:10:38) So you get into these sort of like circular thoughts and actually definitions. (00:10:43) And if you do think about... (00:10:45) package manager as a piece of software similar to any other package, then you start to see that, hey, it needs all the same kinds of assurances we have with all of our traditional dependencies or traditional packages that we want our package managers to take care of. (00:11:01) And that includes ensuring that there's integrity and there's consistency for people's projects and what was happening in the ecosystem at the time and why I think Mayal from Yarn who have pushed forward PMM and which became (00:11:15) Core Pack was pushing forward was because he was seeing a lot of issues with consistency in terms of folks installing their node modules, and so he was seeing that certain package managers, his... (00:11:29) or PMPM or MPMs were innovating in different ways, and they would conflict when someone would use one package manager or another. (00:11:38) And so the idea was to try to, at the project level, isolate this sort of use case of type of software that you're using within your project by creating a net new definition for this. (00:11:48) this type of software that's being used or that is a dependency of your project, which in and of itself is the package manager. (00:11:56) And so by creating a net new key, top level key in the package JSON, the hope was that there could be this less understanding about what tool you're actually using to install your dependencies. (00:12:08) He reached out to us at the time. (00:12:10) So back when we were at the NPM team, he definitely reached out to us and we talked a bit about the future of PMM and CorePak before it landed in Node. (00:12:18) There's a lot of feedback that actually was given to the Node project at the time before it even landed. (00:12:24) And today, you can still use CorePak if you'd like, if you still think it's a good solution for making sure that your teams are not potentially running into errors or issues with using different package managers in your projects. (00:12:36) I think generally the (00:12:38) help with Corepac was that we would see tooling carve-outs across projects, which I think is good and bad in some ways. (00:12:48) I feel like it's a bit of a regulatory capture move to say, hey, let's sort of lock in. (00:12:55) your tooling ecosystem at this level, and anything that comes down the pipeline needs to be blessed into this new subset registry or subset ledger that we own and manage. (00:13:07) And so that was a bit, at least from my perspective, was always like a bit of the problem I had with that, creating a net new definition of tooling that needed to be managed. (00:13:17) When I traditionally have looked at a package manager as just one of my packages, and we already have had mechanisms in place (00:13:24) at the package manager level to actually enforce good practices around making sure those things were not conflated with each other. (00:13:32) That's a good transition point then, because you've gone from working on a package manager or a package manager manager. (00:13:40) You've gone to working on both the package manager and the registry itself. (00:13:45) So we have on the traditional side of things, NPM now also Yarn PNPM. (00:13:50) Then we have these kind of all-in-one bundlers or toolkits like Deno and Bun. (00:13:54) And separately, you have VLT.sh or Volt. (00:13:57) What are you doing differently? (00:13:58) Can you introduce to us what is the point of Volt in this multifaceted world? (00:14:02) Totally. (00:14:03) There's a massive opportunity, I think, here that exists for investment in infrastructure. (00:14:10) What you would have seen in terms of innovation and package management for JavaScript in the last decade and a half was (00:14:17) only focused on the client side behaviors. (00:14:21) And so when Yarn came out and PMPM came out around the same time, it was actually previously IED. (00:14:27) I think you actually had Sultan on to talk about that in the history of PMPM. (00:14:31) Around 2016, these were just net new clients that still interacted with the same APIs that the NPM client did, first class NPM client, which is (00:14:42) the public NPM registry, right? (00:14:44) And those APIs and those endpoints really haven't changed in 15 plus years. (00:14:50) So the clients that are coming out even now, Deno and Bun, are still looking at those old APIs and still having to try to squeeze out performance gains and try to do things that are interesting purely at the client level. (00:15:04) And so we think that there's a huge opportunity here to unlock the server side aspect of this, like to create net new endpoints, to (00:15:12) innovate and invest in the registry side of things. (00:15:14) And so we've actually started a project called VSR, which is complementary to the Volt client. (00:15:21) And VSR is an acronym for the Volt Serverless Registry. (00:15:24) It works kind of like a lightweight proxy to upstreams like NPM, but also introduces a private package registry for you to self-host or publish into. (00:15:35) So we think this is a huge opportunity for us to innovate on those endpoints. (00:15:39) and create more modern ways of interacting with your packages. (00:15:43) Part of the big problem that we see and have historically seen is a lot of wasted and redundant compute happen in all of our machines. (00:15:53) So if you, Josh, and me, and Roy all install the same package, we all have to resolve the fancy graph (00:16:00) at that edge, which seems really wasteful in terms of opportunities to cash and have a centralized understanding of the dependency graph. (00:16:07) And so the two products that we have today that are out, the Volk client, which is both backwards compatible with those legacy understandings of what a registry is, as well as (00:16:18) communicates with VSR, what we're complementing to that is an understanding of an indexing and crawling of the pancy graph, which is what we're doing today behind the scenes. (00:16:29) And so I think there's a huge opportunity for us to innovate on the tooling we can offer the ecosystem by sort of looking at there being an understanding of the resolved pancy graph out there that we can all share and essentially have like a global cache of versus having to do all this like redundant compute. (00:16:48) And I think in the modern AI agentic wave that's coming, we're only going to see more and more of these machines doing essentially this resolution at the edge. (00:16:58) And it feels like a huge opportunity to obviously save some folks some time and money and cycles and also secure the folks as well. (00:17:08) So we think that's a different way of looking at the problem. (00:17:12) And I think that that's sort of the innovative part of what we're doing with Volt. (00:17:16) So let's say that I'm an arbitrary team working on an arbitrary project, and I have experienced pain that my installs are slow. (00:17:24) Let's say I've got a lot of node modules in my dependency graph. (00:17:26) Are you saying that one of the benefits, say, of using Volt with VSR as the backing registry is that you'll have much faster installs that I can use? (00:17:35) Correct. (00:17:35) We actually give you the registry proxy that you can run locally. (00:17:39) So this is kind of a beautiful conceptually that you can self-host a registry. (00:17:43) MPM never did that. (00:17:45) Or well, it started at open source and it went closed source in 2013 or 14. (00:17:49) So unfortunately, all the innovation from then on or any kind of improvements that were made to the registry code was privatized. (00:17:58) So we think that there's a great opportunity here for us to give you a registry that can be co-located with your client. (00:18:05) and let more of the power go into that infrastructure and server-side API endpoints. (00:18:11) And because you're running essentially a local instance of a registry, you get like round-tripping benefits. (00:18:18) You get a lot of other benefits of sort of having this proxy lightweight instance sitting alongside it. (00:18:24) Traditionally, the only other option folks would have had would be a Verdaccio instance, if you know that project. (00:18:31) So Verdaccio is like the only other project out there that (00:18:35) folks really have available to them today to mimic a MPM registry. (00:18:40) And so we are providing an alternative to that. (00:18:43) Roy, anything you want to add? (00:18:44) Not really. (00:18:45) I think Darcy is very thorough in his answers, and I appreciate that. (00:18:50) What I should say is Roy has spent a lot of time in making an efficient graph resolution algorithm, and we've spent a lot of time (00:18:59) on our lock file format, which we imagine is going to be the exchange format that we'll use between the server and the client. (00:19:07) And so if you look actually at our benchmarks that we've just recently published on our website, we actually are the fastest package manager that isn't named Bun. (00:19:16) And we think that we can get there, to be quite honest, since we aren't even compiled and we're dealing with the cold start that all the other JavaScript-based package managers face. (00:19:28) the cold start problem that we all face by having to rely on a different runtime. (00:19:33) Yeah, let's talk about performance and the lock file. (00:19:36) I guess a two-parted question for either of both of you. (00:19:38) One, what is a lock file for those who haven't had the joyous pleasure? (00:19:42) And two, how does that impact performance, or how does the dependency graph impact performance of installs? (00:19:47) I think this one's for you, Roy. (00:19:49) I think you can explain this quite well. (00:19:52) Yeah, it is a fun topic to talk about because (00:19:56) Yeah, the lock files serve multiple purposes, right? (00:19:59) It is there to help you lock a given install, right? (00:20:04) It's right there in the name. (00:20:06) In the sense of more, you wanna make sure you're not being surprised installing some, you know, maybe this new package just got a new version and it's shipping malware. (00:20:16) So one of the ways you try to lock your current install is by using a lock file, like, so the rest of the team, they just want to reproduce a given install. (00:20:25) They can just be installed from that source. (00:20:29) But it also serves other purposes, like speeding up and stuff, because you already have a realized, a fully realized graph of how all these practices should look like at the end in the user machine in order for this project to work. (00:20:43) So there is also that component that plays a big part on why LogFile exists. (00:20:50) So it really serves multiple purposes like that. (00:20:54) And at the end of the day, it is also serving the humans that are managing these projects because they can keep track on how the dependencies are evolving, right? (00:21:04) And it's something you can audit after or during each install in order to make sure you're really getting the artifacts you're supposedly getting at the end, right? (00:21:16) So it has multiple purposes. (00:21:18) And (00:21:19) It is a fine palace in act, trying to serve all these different personas, these difference, and use of log file in a way that it works great for not everyone, but it's a step of the way, right? (00:21:36) Yeah. (00:21:37) I'm going to give you what sounds like a softball question, but I suspect it's not. (00:21:41) Let's say that I have package A that requires B at 1.1 or higher, and then package C that requires B at 1.1.1 or higher. (00:21:50) Why is it so difficult just to figure out what versions of packages A, B, and C I want? (00:21:56) Or why is dependency resolution with versioning so difficult in the package space? (00:22:01) So in general, (00:22:03) I believe you just used some grammar, right? (00:22:05) Like some non-standard Semver grammar. (00:22:08) Is that right? (00:22:09) Yeah, I gave you an imprecise Semver resolution specifiers for these packages. (00:22:14) So this is something that a lot of folks like take for granted in our ecosystem. (00:22:18) And in fact, they take it quite for granted in most packaging ecosystems. (00:22:22) There is no actual specification. (00:22:25) There's like a quasi specification or standard for ranges or essentially like (00:22:31) groups to be associated together of software versions. (00:22:35) So every time we say SEMVR, we're kind of actually, we're stealing a term that actually doesn't have any basis for the ranges. (00:22:44) Like ranges have not been codified in the SEMVR spec. (00:22:48) Just so everybody knows, if you want to go check out the SEMVR spec, you're going to find that it only codifies what a version is. (00:22:55) It doesn't codify ranges, it doesn't codify to group things together. (00:23:00) Like how you do that is all (00:23:02) based around how the package manager interprets that spec. (00:23:06) But definitely in terms of like, how do we group, let's say A and B or two versions of C, when you get into sort of what we call like the diamond dependency problem, where you have two different disparate dependencies that both have a transit of what we would call like a transitive dependency, and you have them at two different explicit versions. (00:23:27) In some ecosystems, let's take Java, you can only ever have one version of a library. (00:23:32) So you have to get rid of that like conflict, what we would say is like a conflict between those. (00:23:38) packages that are these dependencies that have different transitive dependencies. (00:23:42) And you have to choose one, right? (00:23:44) You have to basically say, oh, I'm going to choose one. (00:23:46) I'll take maybe the greater one. (00:23:48) The beautiful thing about the JavaScript ecosystem is many years ago, Isaac Schlitter, who created NPM, and he was benevolent BDFL for a while taking care of Node, was able to figure out that, hey, we can actually get around this problem by allowing multiple versions of the same thing to exist in a project. (00:24:08) nesting these dependencies, these architectures. (00:24:10) And that's a beautiful way to kind of like resolve that problem. (00:24:14) And I would say that's, I like to call that like the first layer dependency hell, where you have this issue of, I have these two direct dependencies that then have a shared transit of dependency that we associate being the same thing, but need to resolve to only have one thing. (00:24:31) And the way that you can do that if they are truly in conflict is to just (00:24:37) have two of them, have two versions of those things. (00:24:39) And we would say that's like the nested strategy for a package manager will opt for what's the easiest thing to do is like, we just don't ever try to share these two versions. (00:24:50) Like you don't ever try to do what we call is like de-duplication of your depancy graph. (00:24:55) But when you do want to do de-duplication, because there's many reasons you want to do that, you have to then... (00:25:01) come up with a grammar, which is like that non-standard grammar that we have for greedily or having sort of some slipperiness between your dependencies that allows you to opt into versions that may not exist today, but could exist in the future that allow for us to optimize for getting the best, most evergreen version we can without breaking software along the way. (00:25:26) So that small little example, I think really, and I'm not sure (00:25:31) if I'm doing a great job at summarizing this, but it really, I think, unlocks many of the details and nuances of what it means to do package resolution and also the problem of dependency hell in general that we were facing back in the 2010s. (00:25:46) Really sounds like it puts the hell in dependency hell, yeah. (00:25:50) Resolve multiple versions of packages with different strategies per package manager. (00:25:54) It's one of those things that we all have to understand, that there's this language the package managers (00:26:01) have in their brains this sort of parsing that happens of the specs that we put in our package JSON files and these dependency specs a lot of folks take for granted. (00:26:13) They sort of are like, That's just December value. (00:26:16) Well, actually, no, it's not. (00:26:17) That's going to do something different potentially. (00:26:20) And we spend a lot of time talking about grammar and definitions and specs and schemas in the package manager world, and I would say like one of the things that I constantly (00:26:31) ask other people is like, what do you think a package is? (00:26:35) Like, what is a package? (00:26:37) And I get many different answers to that. (00:26:39) Or what is the most minimal thing that could represent a package? (00:26:44) Like, what does that string look like? (00:26:47) And I get many different answers to that. (00:26:49) So. (00:26:49) All right, we've talked about the base of what a package manager is, what a registry is. (00:26:53) I want to focus a little more on Volt and what's coming up next for y'all for the next bit of the interview. (00:26:59) You've re-implemented the core parts of the manager and your registry, and you've added in some niceties, like working on the edge and you duplicated for better performance. (00:27:06) What are the other selling features or parts of your product that you're excited about? (00:27:11) So we are starting to roll out net new features to the client and the registry. (00:27:16) One is obviously being safe by default has become standard. (00:27:20) And what I mean by that is not running arbitrary scripts or install scripts. (00:27:26) has now become cool, which I'm very happy about. (00:27:29) Our package manager obviously takes care of this by default. (00:27:33) If you run vault install, we're not going to run any install scripts. (00:27:36) And we're going to print a nice message, which we just shipped recently. (00:27:40) Roy was actually the one to implement this feature. (00:27:43) And we utilize our... (00:27:46) amazing query language, which is the other sort of core feature of the client, as well as we're putting this into the registry to allow you to do sort of a nuanced allow list if you do want to opt into running those install scripts and you do want to create some kind of configured codified list of dependencies that should be running. (00:28:09) in, let's say, install scripts, because there are legitimate reasons for that. (00:28:13) Mounting native binaries, free runtimes, or just generally building a package that you think is secure or you trust it because you've seen it before. (00:28:22) And so that's a huge unlock. (00:28:24) We call that, we've created a net new command in the Vault client called build. (00:28:29) which takes one of our query selectors, which we think is super powerful compared to sort of what the other package managers have done, which is just used package names and package versions. (00:28:40) language itself is like a huge unlock in terms of being able to expressively write both with relationships or with metadata, something that selects within your fancy graph, very nuanced node lists, essentially. (00:28:56) This language is very much inspired by CSS, which I have a (00:29:01) soft spot in my heart for. (00:29:03) Not everybody does, especially the Tailwind folks out there, but it's honestly, I think, the coolest thing that we have, and we are seeing the benefits of using it in our core graph logic by unlocking something like this, which was super easy for us to implement. (00:29:20) And we're super happy that, you know, we can say as well, along with PMPM now and BUN, that, you know, we're safe by default, we're not running install scripts. (00:29:28) So that's a recent unlock. (00:29:31) Another recent ship that we had was just the ability to actually select in that query selector syntax projects across your whole system. (00:29:39) So Roy also shipped this. (00:29:41) It's called the host selector. (00:29:43) You can literally write a selector that will query for any project that's being configured or installed with Volt, and we can essentially do cross projects. (00:29:55) So if traditionally you're using monorepos, you'll know that it's totally easy to obviously select projects or list dependencies in your monorepo projects, but actually sharing dependencies, configuration, or doing querying across, let's say, many monorepos or many repos hasn't been possible before. (00:30:15) So the selector essentially runs essentially in the global context of your system and can essentially go and query anything (00:30:24) that you can query for across all the projects we can find on your system that are configured to the vault plan. (00:30:31) And Roy, I'm not sure if you want to dive into more on that, but yeah, that's the high level of what the host selector does. (00:30:38) There's so many directions we could go from here. (00:30:40) We could dive a little bit more maybe on the query language itself, what it is, how it works, or maybe the other. (00:30:47) I think another pillar of that is having a browser-based UI, and I think that's also fundamental (00:30:54) for a lot of folks out there to actually be able to visualize. (00:30:58) Like when you're applying a query, what packages are you actually selecting? (00:31:02) Because then you have a real-time experience that allows, you know, to traverse and navigate your install using the query selector in a more user-friendly way. (00:31:12) What's an example use case of this that hasn't been brought up yet? (00:31:16) Like let's say I'm managing a monorepo. (00:31:17) What are some things I might use this query selector syntax for? (00:31:21) So a lot of the time it's useful for auditing. (00:31:24) our observability, but also it's great for transposing configuration or setting and getting values from different, you know, package JSONs, let's say. (00:31:35) We actually introduced, you know, the PKG command into the NPM when we were there. (00:31:39) We've obviously carried it through to the Volt client, which we think is great. (00:31:43) Bonn itself just introduced it in a recent minor version. (00:31:47) They now have PKG as well. (00:31:49) It's kind of like the fun JQ built right into your package manager. (00:31:54) And so the great thing here is you can get and set and remove or unset values in your package.json file. (00:32:01) And utilizing a query selector syntax, you can imagine you can start to share configs, you can share dependency definitions, you can share all types of things and actually get and set those across your projects very easily. (00:32:17) So you can sort of transpose and automate, you know, the config definitions that live within package.json across your (00:32:24) repo. (00:32:24) So there's a lot of automations that this sort of unlocks. (00:32:27) There's a lot of things that we think even we haven't conceptualized yet, but the idea here is you can quickly go from having sort of like a template for your workspaces that may have a whole bunch of test scripts that you define and you can go and then action set, a package set utilizing the query selector syntax to go and pick up that definition and then apply it across all the (00:32:53) workspaces. (00:32:54) So we look at it as like being very composable, right? (00:32:57) So the PKG is a great example of what we think is a great composable sub-command within the client. (00:33:03) And with the powerful query selector, which we've added to almost all, I think all of our commands now support essentially a flag called the scope. (00:33:12) And the scope is essentially just a flag for defining the query selector that you'd like to apply that to. (00:33:19) So you can even do publish, pack, and publish. (00:33:21) We have those traditional (00:33:23) sub-commands for creating and distributing packages. (00:33:26) We also have install, obviously. (00:33:28) We have version, all those commands. (00:33:30) They have this special flag called scope, which you can define and it's a query selector. (00:33:34) And it means that that operation will be ran against that selector, like those dependencies that match that selector. (00:33:44) So it's kind of like PMPM, if you love PMPM and their filtering syntax, it's kind of like the filtering syntax on steroids. (00:33:51) which I know I really loved what I saw when Sultan brought out that filter syntax. (00:33:56) It's just a little bit bespoke or baroque, maybe. (00:33:59) I don't know if that term, but it's a little bit foreign for, I think, a lot of developers to know what specific little dots or slashes you have to put. (00:34:09) I really don't want to have to remember XPath type selectors. (00:34:14) CSS to me was just the perfect kind of... (00:34:17) in between, between something that's super expressive, but also super powerful. (00:34:21) And it only gets better with the more metadata that you put into your packages and your fancies. (00:34:28) We can select by that and we can select by other kinds of states for your packages. (00:34:32) So yeah, huge unlock for, I think, workspace management is the selector syntax. (00:34:37) And then we are trying to create more and more of those primitive kinds of operations like PKG or run or even exec all support this. (00:34:48) Expanding a little on your example, or just to give a concrete example here that I find is pretty fun. (00:34:55) Let's say you're Cindrator, right? (00:34:57) And you have over 1000 packages. (00:34:59) out there, right? (00:35:00) Let's say you configure all of them using code line, right? (00:35:04) So now you could go ahead, let's say you're just migrated, you moved from Twitter and you're now on BlueSky, and you want to update all your package JSON to update that metadata everywhere. (00:35:15) But let's say you have some, you know, some other maybe conditions you want to apply. (00:35:20) Maybe you just want to apply to the package where you were actually mentioning OpenCollective for some reason, right? (00:35:27) So (00:35:27) This language just gives you a very powerful way to express all that. (00:35:33) So you could use the command like vote PKG and go ahead and set. (00:35:37) your social media handler to now point to Blue Sky instead, and you could be pointing, okay, let me use the host local selector so that I'm selecting across all of the projects computed on my machine, and let's apply some other conditions like, okay, let's just select the ones that I'm pointing the funding to OpenCollective. (00:35:56) So this is the kind of very expressive and advanced scenarios you could work and the query language enables you. (00:36:04) to actually kind of have that workflow available now. (00:36:07) It's interesting that we've seen a similar pattern in many areas of tooling where at first the core base one package at a time problem gets solved. (00:36:16) How to dedupe packages, how to talk to a registry. (00:36:19) And then, of course, products grow bigger, needs are larger, more complex. (00:36:23) And now we're solving problems for workspaces, monorepos, microservices, and microfrontends. (00:36:29) And one area that (00:36:30) continuously always comes up as security, which I'd like to talk a little bit about with you too. (00:36:36) Just looking at your blog post on the host context that you've been talking about, one of the example queries is the colon malware query, which sounds like a horrifying thing to have results at all in your project. (00:36:46) So can you talk about the malware query selector, what it means to have malware detection at the registry and or CLI level? (00:36:54) Totally. (00:36:54) So it does seem super scary to run that query and maybe get results. (00:37:00) Unfortunately, it's very possible that you could have installed some piece of malware along the way, and you don't realize when you've actually done it historically. (00:37:10) That selector is actually a mutable selector, which is updating all the time. (00:37:14) So the way that works is actually we have partnerships with security insights providers, like Socket in our case is (00:37:22) is one of those key vendors that we're working with. (00:37:24) And we look at those folks, those partners like metadata providers that we're hoping to enrich your depancy graph with more and more metadata. (00:37:32) So that makes that selector language even more and more powerful for you to codify heuristics into or for your teams, or in this case, to find potentially vulnerable or insecure packages. (00:37:44) So security, we take like super seriously. (00:37:47) We have default queries that we're trying to prevent malware from getting (00:37:52) into, but sometimes these security companies can be slow. (00:37:57) MPM has historically been slow to realize that there is malware being distributed. (00:38:02) And so it means that there's a active and passive scanning happening and you can actually opt in to the selector that we have actually called scanned. (00:38:14) And so you can choose to refuse to even install stuff before it's essentially scanned, right? (00:38:20) So this is a beautiful thing where you can say, hey, I'm not going to trust any package. (00:38:25) This heuristic to me is important that I actually want to only install packages that have been scanned and we're providing which (00:38:32) hasn't come out yet, but we're going to be providing a specific vendor, specific scanning selectors. (00:38:39) So I want to make sure that Socket was actually the one to scan it and I'm going to wait until they have before I actually consume. (00:38:46) So today we sort of mask the underlying vendor by sort of giving these nice, again, primitive selectors like malware or CVE, and then we give you types, especially for CVEs, the risk. (00:38:59) type. (00:38:59) So in the cases of SRCWE specifically are the types of CVEs you can actually put in there something like a REDOS, like a regular expression denial of service. (00:39:11) that type is like 1, I think the number is like 1333. (00:39:15) So if you don't care about regular expression denial of service, because maybe your developer tool and a package has been maybe flagged for this like inefficient regular expression, which you're like, okay, thank you, SNIC, not to throw shade or purple. (00:39:30) purple company, old school purple company that is trying to obviously protect the ecosystem, but at the same time has kind of spammed our ecosystem with maybe not so efficient advisories, you can sort of contextually apply or contextually filter out that noise with these selectors. (00:39:49) And so we look at as both us providing more and more metadata is going to create, I think, a more secure ecosystem. (00:39:56) We don't know how everybody will apply (00:39:59) the heuristics to their own projects, whether or not they even consider some vendors to be suspect or not. (00:40:06) If they want to opt in to some or not, we want to give everybody the option to sort of choose. (00:40:11) But there are certain things that we consider to be default. (00:40:14) We don't want you to install malware, but it might get on your machine. (00:40:18) prior to it having been flagged. (00:40:20) So that's just like, how could that have happened? (00:40:23) You still wanna be able to find it, right? (00:40:25) Like, let me easily find it if it did happen. (00:40:28) And so we think it is still super powerful for you to be able to do a scan across all your configured projects, like we show in the blog post for potentially malware that might've been previously installed that is now being flagged by any one of our vendors that says, Hey, this is malware. (00:40:45) So that's kind of how that selector works. (00:40:47) And there's more nuances there for sure. (00:40:49) Like Socket provides us with some great insights, like whether or not a package has file system access, whether it wants network access. (00:40:57) And we created some nice quick selectors there, like FS and HTTP. (00:41:03) And so you can quickly, and also environment secrets or environment access. (00:41:08) So you can quickly say, (00:41:10) If I do or don't trust a package to have file system access, I can essentially gate that by using CSS's good old not selector, right? (00:41:19) So there's, I don't want anything that has file system access, you know, like find me those packages. (00:41:25) And we have the override mechanism that you could essentially gate, or we have modifiers as well, which are like our version of overrides, which can get the installation of those packages. (00:41:38) I wonder if you could adopt that system for other concerns beyond just malware. (00:41:43) For example, there are community initiatives such as E18E around swapping packages that are older and or slower for, say, modern, faster packages or just native functionality. (00:41:52) Could you add like a colon slow or colon just use node for this now you silly goose selector like that? (00:41:58) Yes. (00:41:59) I think there's like a couple of things here where we've also flagged like the type of a package. (00:42:04) So like if it's CJS, ESM, like we have that insight. (00:42:09) There are definitely options for us to help with creating variants or distributions that are easily swappable from one to another. (00:42:17) We would love to work with the existing maintainers though, so that they can be the ones to kind of control that flow. (00:42:23) Many years ago, we actually proposed an RFC for what we call distributions. (00:42:28) which would allow for you to have sort of conditional variants of a package. (00:42:33) So you could have that full source with all your test files and stuff that a lot of people complain about, but maybe you think is still important to have all the reading docs and everything else. (00:42:44) put right into your package. (00:42:46) But then you can have that optimized production version that's compiled and has very minimal files inside of the package as well, living alongside each other under the same scope, or have some sort of definition that's respected by the package manager to essentially make these two things well understood by the package manager. (00:43:06) And so I think distributions and variants are the way forward in regards to that kind of (00:43:12) bit flipping or switching between one condition or another. (00:43:16) So whether or not it's your environment or the configuration that would tell the package manager that, hey, I can opt into a polyfill because I'm on an older version of node and I need that polyfill, or I can completely drop that and use the version that basically just stubs out the (00:43:35) native API now. (00:43:37) Those are options as well. (00:43:38) And we're very close with some of the folks that maintain those legacy libraries. (00:43:42) So we're very mindful that they also want the good things to happen. (00:43:46) I won't name names, but certain maintainers that we know of do want folks to modernize and do want folks to get off of maybe, you know, having two or three or 400 dependencies for stuff that is now built into the runtimes. (00:44:00) They want to help with that transition as well. (00:44:02) So the ecosystem will, I think, (00:44:05) evolve over time. (00:44:06) But our hope is that we can provide, again, those like primitives that would actually help us do innovative things, not just for like getting rid of polyfills or getting rid of those types of things, but also provide innovative opportunities for like native specific conditional or natively built variants to kind of coexist in a blessed way versus the ecosystem we have today. (00:44:30) Yeah, and from the end user point of view, a way to integrate the query language and also be doing these replacements, it's using this graph modifier feature that we shipped a while ago into the client, so it gives you a way to actually point. (00:44:46) add a package or a group of package and replace that definition with something else that is actually, it actually happens during the graph build phase of the package manager. (00:44:59) So we do have one already out, which is equivalent to the more classic overrides. (00:45:05) And we do have more already planned to add on that graph modifier support in terms of supporting things like package extensions. (00:45:13) I think this is how some of the other (00:45:15) clients that call it, which basically allows you to define maybe new dependencies at a given package right within the graph and all sorts of crazy stuff like that. (00:45:28) Also granting some extra power to the end user in this entire supply chain. (00:45:35) That all sounds lovely. (00:45:36) What are some other interesting aspects of Volt that you think users would be pleasantly surprised to discover? (00:45:43) Oh my goodness. (00:45:44) Well, (00:45:45) We've talked a little bit about the UI, which Roy mentioned before. (00:45:50) And I think that our NPM interop story is quite good. (00:45:56) I would say that something that you would be very happy to know if you're a DevTools author or if you're someone that likes this space. (00:46:04) We have the best NPM docs that exist out there. (00:46:07) I will put my foot down and say we have (00:46:10) the only NPM docs that exist out there. (00:46:13) If you go looking, you will not find very many NPM registry docs specific. (00:46:18) I should have been a little bit more specific. (00:46:20) The NPM CLI was well documented from our time there. (00:46:24) I would say the NPM registry was never well documented. (00:46:27) And so if you go to docs.vlt.sh, you'll find pretty comprehensive documentation, which gives you obviously endpoints that exist. (00:46:38) in our registry implementation, but it also provides a quite fairly good comprehensive documentation about what the API surface looks like for the MPM registry itself. (00:46:49) The beautiful thing here is we actually give you an interactive docs experience as well. (00:46:54) created by our facilitated by our friends at Scalar. (00:46:58) If you know those folks, we basically distribute with the registry an interactive docs experience. (00:47:04) So when you spin up the VSR instance, you get a slash dash slash docs endpoint. (00:47:11) And there it's a Scalar experience that has (00:47:15) mechanisms for you to actually go and interact and actually play with the APIs in real time, which I think is great. (00:47:22) So that I think will help with folks understanding what's available to them. (00:47:27) And over time, it's only going to improve and get better. (00:47:30) And so the hope there is that other dev tools authors or folks that traditionally would have wanted to build something for the NPM registry can now be like, hey, (00:47:41) I know exactly what the request and response should look like. (00:47:44) I know what endpoints should exist. (00:47:46) And here, Volt's helping me out. (00:47:48) And we're going to maintain, you know, as best we can, backwards compatibility with the MPM registry now and going forward. (00:47:55) So that's another key, I think, highlight that should be exciting for folks, sort of generally about what we're doing and what we're trying to maintain, especially because we care very deeply about compatibility. (00:48:06) We care deeply about bringing forward the ecosystem. (00:48:10) So. (00:48:11) I'm not sure if Roy has any key highlights. (00:48:12) Go ahead. (00:48:14) Yeah, of course, mine are going to be a little bit more to the client side of things, but it really is the small things that I really love when I'm using it. (00:48:25) I mean, of course, the speed, it's very good, but it's kind of like, for us, it's more of a baseline. (00:48:32) We need to be fast. (00:48:33) We know people are not going to be using the tool if it's not fast enough. (00:48:37) That's the little details. (00:48:38) Like, for example, on the query language, when you're using it from the terminal, the vault query command, by default, it prints the same tree structure that users are used with from NTMLS, kind of the classic reference. (00:48:55) For me, personally, that's so much more usable than having the JSON blob be the default output. (00:49:02) But also, something we added to vault-ls and vault-query is (00:49:07) different outputs, not only the classic JSON output, which is improved, it has all these security insights, metadata added to it, but we do have also mermaid output, which I find super useful, super helpful, and kind of plugs well with all, you know, plugs well with HackMD. (00:49:25) I'm a huge fan of HackMD, like we're using all the time. (00:49:28) I can just, you know, copy and paste basically an output and visualize my graph in the render. (00:49:34) rendered view, but it could also be an ocean, right? (00:49:36) It can be a GitHub issue. (00:49:38) You could just be porting this mermaid output, whatever platform you want to take it to. (00:49:43) And yeah. (00:49:45) And I think the other thing I really love is really spawning the browser-based UI, right? (00:49:50) I'm trying to understand what's going on. (00:49:52) I want to find some information about a specific package and just being able, it's like for me, and I'm a big terminal user, but I do love, like, sometimes I just want the product to take my hand. (00:50:05) Guide me to that experience. (00:50:07) Just take me to the place. (00:50:08) Let me find the information I want. (00:50:10) I don't want to be typing and finding about 20 different commands to type right to get there. (00:50:15) So for me, those are kind of like top of mind, the things I love the most about the client. (00:50:20) That's really exciting stuff. (00:50:21) Is there anything else on the technical side or about Volt that we should cover before we wrap up for the day? (00:50:27) I would say we are actively working on the large version of (00:50:34) crawling your defancy graph and realizing before you do, which I think, you know, walks that tight rope between the opportunity to both get performance benefits that have never been seen before, as well as security guarantees that are theoretical in a lot of other tools. (00:50:53) I won't mention SBOMs, but I just mentioned SBOMs, which are highly, highly theoretical, and we could have a whole episode on that. (00:51:03) So I think there's a huge opportunity here and we're super excited about being at the forefront of it. (00:51:08) And we have a lot we're going to be shipping and sharing very soon, both in terms of better performance, better tooling, but also more of a destination for folks to hit going forward. (00:51:21) So look out for that, I guess, very soon. (00:51:24) And yeah, we're excited about that. (00:51:26) Well, in that case, I just have a couple more questions for each of you. (00:51:31) What learnings could I, as a developer, attain from studying and understanding Brazilian jiu-jitsu? (00:51:36) Good question you may ask. (00:51:40) I do practice it. (00:51:42) It's kind of a recent thing, so I'm definitely not a big, strong Brazilian jiu-jitsu guy. (00:51:50) I'm just there to learn and being humbled every time. (00:51:54) But it is a great workout. (00:51:56) It is a great way for us developers to spend all (00:52:01) calories, it takes a lot of your muscular strength and flexibility. (00:52:08) So it's a very healthy workout for us to try and just, if you're able to get it a couple of times during the week, just getting us out of the desk or chair we're seeing all day long and getting in shape. (00:52:23) I've been told that Brazilian Jiu-Jitsu or BJJ involves a lot of strategy and kind of centering yourself. (00:52:28) And there's three people who (00:52:31) actively work on open source projects around the internet on GitHub. (00:52:34) I imagine it's probably quite useful for us to get practiced at centering ourselves and regulating our emotions, even in quick response times. (00:52:43) Darcy, the last question is for you. (00:52:45) Can you tell us about Kurt Cobain's journal? (00:52:47) Oh, no. (00:52:50) Yeah. (00:52:51) I mean, as a teenager, having been a little bit of a punk myself, I think I got as a present (00:52:59) a mass-produced copy of Kurt Cobain's journal, having loved Nirvana and being a bunch of punk bands. (00:53:07) It's probably the coolest thing, but also felt very inappropriate to own and seemed, in retrospect, completely unlike something he would have wanted to have exist. (00:53:20) So I think we should probably all burn our copies if you have one and let it be. (00:53:27) So that's my, that's my thoughts on that. (00:53:29) Having been, a 14 or 15 year old, getting that I'm sure as I think it was maybe a Christmas gift. (00:53:36) It was, is both cool and insightful and probably shaped a lot of my early, you know, garage band days. (00:53:44) But looking back in retrospect, I should have never had access to that thing. (00:53:48) I don't think we, I don't think anybody should have had access to that thing. (00:53:52) But it was nice to see, (00:53:55) just how creative, but also how much, unfortunately, it was very pained. (00:54:00) I'm also, around the same time, I'm sure I watched Dying Darko for the first time, which became my favorite film. (00:54:08) So, have that on VHS, that also feels like a precious memento of my time as a teenager up in the cold Sudbury, Ontario, small town, cold Sudbury, Ontario. (00:54:21) So, yeah, (00:54:23) Kirk Komain's journal, let it rest, let's all burn our copies. (00:54:26) Hold on to your VHSs, though, of Donnie Darko. (00:54:29) That's a classic. (00:54:30) That's a great film. (00:54:31) Could you just quickly explain the full plot and back in context with that film for us? (00:54:37) I feel like, unfortunately, the director's cut ruined it for a lot of folks. (00:54:41) So if you see the original, I hope you do. (00:54:43) There was an original copy without all these added things. (00:54:47) The plot is... (00:54:49) There's a bunny in it. (00:54:50) There you go. (00:54:50) There's a bunny in it. (00:54:51) So that's all you need to know. (00:54:53) It's a beautiful, and especially around Halloween, great thriller to watch. (00:54:57) And it's one of my favorite movies for sure. (00:54:59) And I will not ruin any of it. (00:55:02) You can come up to your own conclusions about what did or didn't happen throughout the movie. (00:55:08) And that's why I think it's so beautiful about it. (00:55:11) So yeah. (00:55:12) With all that being said, destruction is a form of creation. (00:55:15) It's been lovely to talk to you too about Vault and package registries and clients. (00:55:19) I'll start with you, Darcy. (00:55:21) If folks want to learn more about you or Vault on the internet, where would they go? (00:55:25) So you can go check out our website of VLT.sh, is where the home of all of our information about our company, the clients, products, and we'll put announcements there as well. (00:55:36) We put some blog posts out there. (00:55:39) You can also follow us on Bluesky, and it's VLT.sh, or @vlt.sh, just our domain. (00:55:45) You can also check us out on x.com, used to be Twitter, still is Twitter, slash at (00:55:53) slash or at, I guess, VLTPKG. (00:55:56) And you can also find our GitHub, all our projects, along with a couple extra projects there at github.com slash VLTPKG as well. (00:56:05) For myself, you can follow me again, x.com, our Twitter, at Darcy, just my first name, which I think is pretty cool. (00:56:13) I'm old. (00:56:13) I'm very old. (00:56:15) That's how I got that. (00:56:17) And as well, I'm Bluesky for myself. (00:56:19) It's my full name at darcyclark.me, my personal website. (00:56:24) So that's how you can find us. (00:56:26) Great. (00:56:27) And you, Ry, how about you and the work you're doing on Node? (00:56:30) I guess the place I'm more active these days is Bluesky. (00:56:33) So if you want, you should definitely follow me there. (00:56:37) It's R-U-Y Adorno. (00:56:40) Yeah. (00:56:41) And I think if that's too hard to find, just, like Darcy said, go to VLT.sh, go to the company page that has, or names in it, and there are links to social media, and then you'll be able to find me there. (00:56:57) Excellent. (00:56:57) Well, for Software Engineering Daily, this has been Darcy Clark, Roy Adono, and Josh Goldberg. (00:57:02) Thanks for listening, everyone. (00:57:03) Have a great day. (00:57:04) Cheers.