Though it’s hard to keep all 1.7m+ packages updated at once, it would be good to know how often they actually update. Simple statistic measurement. Like “50 recently updated packages were fully updated across the system after an average of 32 minutes”
Why do I need this?
I published a scoped package two hours ago. After about an hour, it appeared in “add library” search. But it still couldn’t be required.
The problem was solved by deleting the notebook that needed the package and creating it again. I also don’t understand why exactly does it work this way. Now interested in the inner structure of how the notebooks are stored and executed, and how are the packages synced (or not synced).
Thank you for the suggestion. It’s a good idea, and we’d like to give package maintainers a lot more insight into the state of their packages on RunKit.
Right now we’re currently reprocessing those 1.7M+ packages to make available on Node 17. When we do this it can take a little more time than normal for a package to become available. On a normal day I’d estimate that a package is available on RunKit within a couple minutes of being published to NPM.
I’m not entirely sure why your package showed up in the “add library” search, but I have a theory which I’ll attempt to verify outside this thread and hopefully fix.
As for your question for how things work under the hood, I can answer that for you. When you require a package in a RunKit notebook we track that require and assign it a timestamp. We use that timestamp to resolve your package version and it’s various dependencies. The client manages the timestamp and our backend (module-fs) resolves the require when it’s evaluated. What we do is say “resolve to the largest (non-alpha/beta) semver version of that package which is available at that timestamp”. So when you come back to your notebook and start editing, you’re not going to randomly and unknowingly get a new version of that package. And this keeps all the packages dependencies and subdependencies fixed too.
If a new version of that package is available, we show a UI that lets the notebook author update the package if they want. And we’re actively working on a better way to expose even more control over it… e.g. updating a single deep subdependency.
In your case, I think what happened was that you required the package before it was fully available (again, still not sure why it showed up in the “add package” search before it was fully processed) but it locked in a timestamp for the package before it was actually available. That UI for updating packages in notebooks that I mentioned earlier doesn’t work if the package was required before any version was available (this is an issue we’re aware of and it’s kind of an ugly edge case). When you remove the require it cleared the previously stored timestamp and when you re-added it, the timestamp was after when the package was made available on RunKit, thus making it accessible. We don’t clear that timestamp until you evaluate because we don’t want you cut and pasting code to change which package version your Notebook was using.
Hope that clears things up, and my apologies for it taking a bit longer than normal for your package to be made available. If you have any more questions feel free to ask them. We always love getting into the technical details when we can!