New packages and versions not showing up

This issue is being tracked across a lot of threads, but I can provide some new context here.

RunKit uses npm’s public registry to download and process packages as they are published. Prior to this downtime, npm had a two ways to access the registry: skimdb and replicate. That changed when npm shut down skimdb. This was done without notice and we were caught completely off guard. On top of that most of the team was traveling to attend JSConf EU and then some TC39 delegates meetings. It really couldn’t have happened at a worse time for us. :frowning_face:

Some more technical details:
For legacy reasons scoped packages were already being pulled from replicate, But we believe npm made changes to that registry as well, which caused RunKit to not detect when new packages were published. We’re still investigating what exactly caused that issue.

Over the last couple days we’ve been working on migrating our system over to the new replicate registry. That said, it’s not as simple as changing a URL, so we’re having to rewrite some code and run some database migrations. We believe that phase of the task is done, and now we’re attempting to process the backlog of the tens of thousands of packages that we missed.

Since RunKit is not affiliated with npm, we’re at their mercy if they choose to change how things work under the hood.

I really wish I could give you an update as to when things will be back to normal, but I’m afraid processing npm packages can be a little… unpredictable. That said, it looks like @angular/core (one of the packages you mentioned earlier) received an update 18 hours ago and it’s currently live on RunKit, so things are moving along.

Again, I apologize for the troubles. I will update folks as soon as we believe things are back to normal.

- Randy

Edit: Small update here, we’ve still got a backlog of a few hundred thousand packages to process, we estimate it will take about 5 more days to complete. So some packages will start showing up over the next few days. I’ll update this thread when that process is complete.

Edit 2 (Monday June 17th): Quick update. We’re about halfway through the package backlog with about 200,000 more to go.

2 Likes