I know this may be too much to ask for, but dynamic requires would be really great. I know tink is working on a similar technique, installing as-you-go. However, that’s on the users client and not on a server, and it probably doesn’t scale well. Either way, this is a bit misleading now:
- all of npm’s 400,000 packages are already pre-installed, just
require()them
Hi larsgw,
If we could get a bit more information as to how you want to use this, it would help us implement it for you, since its actually a UI issue that prevents us from doing this now, not a technical one. Unlike tink, we do not “install as you go”, all the packages really are there. Not just all 400,000, but much larger since every version is there (if you manually inspect the folders, you’ll find them all). The reason we only allow you “access” to ones that are string-literally required is that we can’t figure out which one you want otherwise. Allow me to explain:
When you type require(“x”), it first is unclear what version of x you want. This can be immediately resolved by assuming you want the latest version of the package at the time of typing it. So, if you open a document today and type require(“x”), you’ll get latest as of that time, and then if you return to that document tomorrow, and type require(“y”), you’ll get latest as of the time you typed that. It’s identical to the behavior of typing “npm install x” on your computer, it is resolved as a function of the current state of the package database.
However, if you do require(variable), we don’t know what version you want. You could argue that this could be resolved by only supporting variable of the form “x@semver”, but that actually isn’t sufficient, because that package’s dependencies are semvers themselves, and we have to resolve those semvers as a function of original require time too (if not, every time you ran the document, the internal dependencies could change!).
Now, there are certainly a number of strategies we could use: just use the time the document was last edited, the latest time a literal require happened, etc., but we simply haven’t had enough insight into how people are wanting to use a dynamic import to know the best way to do it.
So, the good news is that this is totally technically feasible, and again, since they’re all already there (really! that statement isn’t misleading at all!), they’ll all “require” instantly, you won’t suffer from an install-time even in this theoretically dynamic world.
The bad news is we simply haven’t figured out a non-confusing way to expose it yet. If you could tell us your use case, you could definitely help us get there!
I’ve gone ahead and put together a little sample RunKit notebook to show you that these packages are actually there (and a hack to be able to get to them - WARNING though: this is using internal RunKit stuff that is not promised to be supported forever, these file paths may changes and thus it is not recommended to use this for production, just for understanding purposes):
https://runkit.com/tolmasky/variable-require-example
In this example, you’ll see that I first require ramda just to be able to get its filepath. Once I have that, I am able to use this knowledge to require any other package on npm from that same time period.
Note though the implication of this: I am always going to get a package from that date. So if a new package gets added later, it won’t see it. Again, this can be remedied by changing this date, but its not the best interface in the world.
Cool! That makes sense. I was trying to get this to work: RunKit. Basically a demo on how to create browser bundles for my project, which has plugins now. However, I kept getting errors, which makes sense now. I personally wouldn’t mind the dependency resolving to the latest version, within a version range perhaps, but that probably differs per project.
Sorry!
No need to apologize, that wasn’t meant to come of mean, apologies if it did. Ultimately it’s on us to make these features easy to use.
Your use case is very helpful in helping us think about this. Out of curiosity, if there was a way to take a shrinkwrap/lockfile and use that to guide the specific versions for this bundle feature, would that be desirable over simply choosing the latest? In other words, imagine that people could POST you the shrinkwrap/lockfiles of the packages they want bundled (that way there is no question as to how the dependencies should resolve), and you could do something like require_explicit(shrinkwrap).
Well generating that shrinkwrap wasn’t really what I had in mind, I ended up trying it on Glitch which didn’t work either, so I just hardcoded the currently possible values: https://juniper-coat.glitch.me/. I don’t know if there’s an npm package that can generate shrinkwraps given a number of dependencies, and having users do that doesn’t really fit with the UI I had in mind.