npm install and build of forked github repo

Question

I'm using a module for my angular app called angular-translate. However, I've had to make a few small modifications to the source code to get everything working the way I'd like, and now I want to persist those changes on npm install. A colleague suggested that I fork the repo of the source code and point to my forked repo as a dependency, which I've tried in these ways, e.g.

npm install https://github.com/myRepo/angular-translate
npm install https://github.com/myRepo/angular-translate/archive/master.tar.gz

The first gives me a directory like this with no build. Just a package.json, .npmignore, and some markdown files

-angular-translate
   .npmignore 
   .nvmrc
    CHANGELOG.md 
    package.json
    etc

The second npm install gives me the full repo, but again I don't get a build like when I use the command npm install angular-translate. I've seen some dicussion of running the prepublish script, but I'm not sure how to do this when installing all the modules. I've also tried publishing the fork as my own module to the npm registry, but again I get no build, and I'm not sure that's the right thing to do...

I apologise for my ignorance on the topic. I don't have a huge amount of experience with npm. Would love to get some feedback on this issue. It seems like it could be a common enough issue when modifications need to be made to a package's source code? Maybe there's a better solution?

Eskilstuna

This is caused by a wrongly configured repo on Github. It is missing a .npmignore or package.json#files. See my answer for more details. – Alkmaar

Answer 1

Try npm install <ghusername>/<repoName>, where <ghUsername> is your GitHub username (without the @) and <repoName> is the name of the repository. That should correctly install it. You will most likely want to use the --save or --save-dev flag with the install command to save dependency in your package.json.

If that isn't working correctly, check the contents of your .npmignore file.

Don't panic if the install command takes a long time; installing from a git repository is slower than installing from the npm registry.

---

Edit:

Your problem is that in your case, dist/ is not committed to the repo (since it is in the .gitignore). That is where the actual code lives. dist/ is built from the files in src/ before the package is published to the npm registry, but dist/ is never committed to the repo.

It's ugly, but in this case you will have to remove dist/ from the .gitignore and then run:

npm run build
git add .
git commit
git push

(Ensure that you have run npm install first)

You then should be able to install from github.

There might be another way to do this using a prepare script, but I'm not sure if that's possible; I've never tried it. Edit: Cameron Tacklind has written an excellent answer detailing how to do this: https://mcmap.net/q/47823/-npm-install-and-build-of-forked-github-repo

Cutcherry

Thanks for the reply, Ryan. Gave that a try, but I got the same result as the 1st npm install. I then tried removing the .npmignore file and I get the whole repo, same as the second result. The problem still remains that the build isn't happening. – Eskilstuna

Is there really no other way to do with other than unignore dist? I want to make a PR to the original repo, but with the dist, is he not gonna like it? – Men

@KaMok I think another option is to, in the package.json file of your fork, under scripts, rename prepublish to prepare. It seems when npm install or npm install github:user_name/fork_name --save (and same for yarn) is executed it also runs what's in the prepare script. This is assuming the prepublish script of this package creates the build files, which is usually the case. – Aesthesia

and if you need a specific branch, npm install <ghusername>/<repoName>#branchName – Ferdinandferdinanda

This is a very hacky solution. With rare exceptions, built files should never be included in repositories. See my answer for a better way with prepare. – Alkmaar

@CameronTacklind In general, with rare exceptions, you should never install from git. The problem with prepare is that it requires listing build-time deps in dependencies, which is also a bad practice. It's just bad any way you look at it. – Cutcherry

@Cutcherry You are incorrect. Installing from git works very well as long as it is setup properly. docs.npmjs.com/cli/install "If the package being installed contains a prepare script, its dependencies and devDependencies will be installed, and the prepare script will be run, before the package is packaged and installed." – Alkmaar

Someone made a post about how we can install using post install script. Please look into this for reference, worked well for me. – Airship

There is no /dist in my .gitignore – Factious

Answer 2

TL;DR use a prepare script

and don't forget package.json#files or .npmignore

Code published to npmjs.com is often not what's in the repository for the package. It is common to "compile" JavaScript source files into versions meant for general consumption in libraries. That's what's usually published to npmjs.com.

It is so common that it's a feature of npm to automatically run a "build" step before publishing (npm publish). This was originally called prepublish. It seems that Npm thought it would be handy to also run the prepublish script on an npm install since that was the standard way to initialize a development environment.

This ended up leading to some major confusion in the community. There are very long issues on Github about this.

In the end, in an effort to not change old behavior, they decided to add two more automatic scripts: prepublishOnly and prepare.

prepublishOnly does what you expect. It does not run on npm install. Many package maintainers just blindly switched to this.

But there was also this problem that people wanted to not depend on npmjs.com to distribute versions of packages. Git repositories were the natural choice. However it's common practice to not commit "compiled" files to git. That's what prepare was added to handle...

---

prepare is the correct way

If you have a repository with source files but a "build" step is necessary to use it,
prepare does exactly what you want in all cases (as of npm 4).

prepare: Run both BEFORE the package is packed and published, on local npm install without any arguments, and when installing git dependencies.

You can even put your build dependencies into devDependencies and they will be installed before prepare is executed.

Here is an example of a package of mine that uses this method.

---

Problems with .gitignore

There is one issue with this option that gets many people. When preparing a dependency, Npm and Yarn will keep only the files that are listed in the files section of package.json.

One might see that files defaults to all files being included and think they're done. What is easily missed is that .npmignore mostly overrides the files directive and, if .npmignore does not exist, .gitignore is used instead.

So, if you have your built files listed in .gitignore, like a sane person, and don't do anything else, prepare will seem broken.

If you fix files to only include the built files or add an empty .npmignore, you're all set.

My recommendation

Set files (or, by inversion, .npmignore) such that the only files actually published are those needed by users of the published package. Imho, there is no need to include uncompiled sources in published packages.

---

Original answer: https://mcmap.net/q/46699/-automatically-build-npm-module-on-install-from-github

Alkmaar

PREPARE worked for me instead of POSTINSTALL.i just ran npm run build in prepare. – Illfounded

Answer 3

Update for those using npm 5:

As of npm@5, prepublish scripts are deprecated.

Use prepare for build steps and prepublishOnly for upload-only.

I found adding a "prepare": "npm run build" to scripts fixed all my problems.

Junket

That worked for me, too (in a fork I just made) — thanks! I wonder... why don't all packages just include this script config by default? Is it because package authors only consider the use case of installing their package from npm and not the case of installing it from a git repo? So they're just used to manually running npm run build and npm run publish and don't run into any problems or pain unless they someday try to install the repo via git? – Overzealous

There is one issue that gets many people when using prepare. See my answer for more details – Alkmaar

Answer 4

Just use the command npm install git+https://[email protected]/myRepo/angular-translate.git. Thanks.

Selection

Answer 5

To piggyback off of @RyanZim's excellent answer, postinstall is definitely a valid option for this.

Either do one of the following:

1. Update the package.json in your forked repo to add a postinstall element to scripts. In here, run whatever you need to get the compiled output (Preferred).
2. Update your package.json, and add a postinstall that updates the necessary directory in node_modules.

If you've forked another persons repository, then it might be worth raising an issue to illustrate the issue that installing their package through GitHub does not work as it does not provide the necessary means to build the script. From there, they can either accept a PR to resolve this with a postinstall, or they can reject it and you can do #2.

Hirz

Can you explain or have any resource on how to do #2? Which commands do I need to run after npm install when the gitrepo only contains src not dist – Woad

@Woad Apologies, I'm trying to remember the context of where I did this in the past. I think that by #2 I meant to add a postinstall step to your main packages.json that would run the steps for the top answer, so something along the lines of "postinstall": "cd node_modules/scrape-twitter/ && npm install && npm run build" to ensure that the package causing you trouble is built before your application runs. As said, though, I prefer the approach of forking the repo to my own user, adding the postinstall to the package itself, and using that in my application. – Hirz

I think adding to prepare should be preferred (@Simon's answer), because if installing from npm, you wouldn't need to do any additional building because it should already include a dist/ dir. See docs.npmjs.com/misc/scripts: "prepare: Run both BEFORE the package is packed and published, ... and when installing git dependencies" – Overzealous

postinstall is one step short of the right solution. Just use prepare. It's 2020. – Alkmaar

Answer 6

If you are using yarn like me. Imagine that you want to use a package like this: yarn add ghasemikasra39/gridfs-easy --save where the ghasemikasra39 is the username and gridfs-easy is the name of the repo

Voltcoulomb