Need to manually require peer dep?

Bug Reports
1

Hello,

Are you supposed to have to manually require peer dependencies of a package? Example, I want to use request-promise (which has a peer dep on request) in a notebook, so I do:

var request = require("request-promise");

And get:

Error: Cannot find module ‘request’

But if I do this it works:

require("request");
var request = require("request-promise");

See:

Thanks!

2

Hi jmm,

Currently you do have to require them. In the future, we are hoping to do the following:

  1. Have the stack trace when it is missing explicitly tell you that it is a peer dependency issue, and have a fix-me button that just adds the require.
  2. If you bring in the require from the search bar (vs typing it yourself), just have it auto-include it (explicitly, like actually add it to the code).

Any form of “pretend its a normal dependency” runs into difficult side effects (for example, two packages that list the same peer dependency SHOULD have the identical objects, such that one modifying it affects the other, whereas normal dependencies don’t).

3

Hi francisco,

Thanks for the reply.

Any form of “pretend its a normal dependency” runs into difficult side effects (for example, two packages that list the same peer dependency SHOULD have the identical objects, such that one modifying it affects the other, whereas normal dependencies don’t).

Won’t two packages with semver-compatible normal dependencies on the same package result in them having the identical object? I may have misunderstood you.

Instead of handling it by inserting a require() for the peer dep in the code have you considered having a separate field to list them? Kind of like the External Resources feature on jsfiddle. It could be very tailored, or perhaps just a package.json field where certain properties (like dependencies) are used.

I’m thinking it’d be a positive to enable RunKit code to be as similar as practical to non-RunKit code (but I don’t have a lot of experience with it). My first reaction was that it seems odd to insert a require() to represent a peer dep when that’s not how it would normally be. Then I thought of the require("pkg@1.2.3") feature that’s also unique to RunKit and had the idea about the package.json field where you could enter packages that are peer dependencies of dependencies, and could enter versions of dependencies as an alternative to embedding them in the require() calls. So, using my example, something like this:

package.json

{
  "dependencies": {
    "request": "^2.34",
    "request-promise": "4.0.2"
  }
}

var request = require("request-promise");

For the time being you could consider just mentioning the possibility that this is the issue in the Whoops, should we have {package}? error message.

Thanks!

4

Won’t two packages with semver-compatible normal dependencies on the same package result in them having the identical object? I may have misunderstood you.

No (at least not the way we’ve implemented it, which represents the npm 2 non-deduping behavior). If you have package A that requires B, and package C that requires B, both will have unique copies for B in their node_modules folder as such:

A/
  node_modules/
     B/
C/
  node_modules/
    B/

Because of this, modifications to B from A will not affect B in C. This favors in-package consistency, and is part of the reason that peer dependencies exist. With a peer dependency, the package is basically saying “I expect this package to exist now, but I am not taking the responsibility of providing it myself, thus it will be potentially shared with my peers”. With deduping (npm3-style), you would get essentially the same effect of a peer-dependency with normal dependencies, in that you tree is flattened to this:

A/
B/
C/

Here both A and C do in fact share “identical” B’s (as you expected), where one modifying it affects the other. Unfortunately, this type of package resolution is kind of a mess, as explained in npm’s documentation here: https://docs.npmjs.com/how-npm-works/npm3-nondet (and as I further explain in this bug: npm v3 non-determinism does actually result in different code (not just tree structure) · Issue #10999 · npm/npm · GitHub ). Basically the order you require things can change the code you get with this style of resolution (or, to keep it deterministic, we’d actually have to dirty past code if you require a new package the is earlier in the alphabet that would add a semver “compatible” package that increases the constraints).

So for example, you require package C which wants B >1.0.0&& <1.5.0, so we give you 1.4.9, and you hit run. Then you require package A which wants B > 1.0.0 && <1.4.0. Since you required them in THIS order, C will get 1.4.9 and A goes to 1.3.9. BUT, had you down the reverse, and required A before C, then A’s 1.3.9 would have happily satisfied B’s semver range, and you’d share the package. This gets more complex because then npm will install things in alphabetical order so typing npm install will yield a different result (this is how npm 3 works today).

Anyways, there’s more information in the links above, but suffice it to say that npm-2 style package installation is always deterministic and gives you a nice escape hatch when you want to actually share dependencies across package boundaries in the form of peer dependencies (which of course also give you the ability to choose the version you want). But again, since the way packages “make it in” to a notebook is by you typing require, they need to appear somewhere in the code. We could implement a system like you’ve described though where you just say "I want React x for all my notebooks’, and then we just make a secret note of require(“react”) at that version. We’re also working on a way to select the version of the package visually in the notebook (require(“x”) would show a menu next to it with all the version).

I hope this brings some context into all this, sorry for the rambling.

5

sorry for the rambling.

Not at all, thanks for taking the time to provide this detailed breakdown!

No (at least not the way we’ve implemented it, which represents the npm 2 non-deduping behavior).

I have been primarily using npm@3, but I also had the mistaken idea that in some cases npm@2 automatically did limited deduping, e.g. when installing from a package.json with deps that have semver-compatible deps. Thanks for straightening me out about that.

Thanks for those links, I hadn’t seen either of those before. (P.S. I had a bit of trouble at first understanding the scenario in the bug report without the contents of index.js or looking up test-a|test-b – was it meant to include a link to a repro or the contents of index.js?)

Not to sidetrack this too much, but from reading through that issue, it seems to me that maybe the problem is that the semver range syntax isn’t expressive enough. mhils suggesting making something like ^1.0.0 automatically resolve to the latest compatible version. At the moment I’m not sure if that would be a good idea, but maybe it would be a good idea to introduce some kind of operator to the syntax that allows opting in to that. E.g. “^^1.0.0” means the highest compatible version that’s published.

But again, since the way packages “make it in” to a notebook is by you typing require, they need to appear somewhere in the code. We could implement a system like you’ve described though where you just say "I want React x for all my notebooks’, and then we just make a secret note of require(“react”) at that version. We’re also working on a way to select the version of the package visually in the notebook (require(“x”) would show a menu next to it with all the version).

So, obviously I don’t know much about how your system works under the hood beyond what you’ve explained to me, but re: this part…

since the way packages “make it in” to a notebook is by you typing require, they need to appear somewhere in the code

…is that a current implementation detail or an essential invariant? In other words, is there any reason it has to work that way, or it’s theoretically possible to do it another way, like read from a package.json field that’s part of the notebook to get the top-level dependency and peer dependency packages into the notebook? I guess I’m wondering if that design is based on technical reasons or UX. Just curious.

(FWIW, my thought right now is that I wouldn’t mind entering the top level deps and peer deps into a package.json field (especially when specifying version), but again I admittedly have only a little experience with the platform.)

I was also wondering, how is that meant to interact with the download feature? I’m reading this from the home page:

Download
All notebooks are just node modules, so you can download them and run them on your own setup with no changes.

Won’t having the version in require() calls like require("some-pkg@1.2.3") interfere with that or at least make it more complicated? I tried a quick experiment with downloading and running a notebook with calls like that. (I looked around and couldn’t find instructions for using a download, I just poked around and experimented.) So I did npm i and stuff installs and I get this message (not sure what it means):

Not installing as notebook: whatever

Then I do node ./ and I get:

Error: Cannot find module ‘request@2’

Thanks!!