git submodule vs npm package?

Question

I'm using git submodules to build and share components between projects. The projects are not in production yet, so at this point submodules are serving them well.

But I'm concerned about maintenance and deployment. Would it be a good idea to transform the submodules into npm packages?

Orphism

Answer 1

An npm package will allow fragmentation across different package versions. On the other hand, git submodules have a bit of a learning curve, and the tooling is really not that good. With git submodules, you have all the source in one folder.

If it's at all possible, I'd recommend using a plain monorepo for all projects. You may need to create build time variables (via babel plugin/s), you may need some sort of "live config" get served from the backend. I worked with git submodules for a year and I've recently worked with a project that uses npm to share code.

I would recommend using only one git submodule, for all shared code, instead of several submodules. I would strongly consider using lerna, and use your one git submodule to track lerna's packages directory. And if the team decides they don't like git submodules, you can easily make this repo a (../sibling) git repo, instead of a submodule. However, above all this, I'd recommend using a plain monorepo.

Here's a great talk on monorepo's from Netflix: https://www.youtube.com/watch?v=VNqmHJtItCs (strong focus on discouraging npm-style packages)

Here's google's infamous monorepo talk: https://www.youtube.com/watch?v=W71BTkUbdqE

This is a great site to read to help you think about good development flows: https://trunkbaseddevelopment.com/ (it primarily advocates for the monorepo approach)

If you are developing software for different clients(different people/companies paying you for similar projects), and have some agreement that they should be at least ~80% the same, you may really enjoy using build flags to help get started on splitting functionality, but I'm sure you should very proactively keep the code around the build flags clean, and refactor into re-usable components/packages. Give each client some sort of build-flags.json. Build flags should be named for features only, which in theory can all be individually toggled. Some code may be totally custom for each project, in this case, you may want to consider using dynamic imports, but generally this is a pain point I have yet to fully cross, although I have plenty of unrefined ideas around this.

If a monorepo is just not happening, I would actually recommend using npm packages+separate repos over git submodules, assuming you can do good semantic versioning of the package. (And, yalc seems to be a good tool for linking together packages, as opposed to the standard npm/yarn link)

Gearalt

It's one of those questions "we" (many devs, imo) love to ask, but might not fit StackOverflow's guidelines, answers are equivalent to mini blog posts, or large comments, but are given a feel of authority by just being on StackOverflow in SO's Q/A format. – Gearalt

@Devin Can you explain more about the build flags? – Scalariform

I have seen at least 2 babel plugins out there, but I don't remember what they were. Not a great google search, but here's a starting point: google.com/… – Gearalt

I have 3 repos: app-back , app-front , app-core... I mounted app-core/src to src/core folder of the other 2 repos. Is there anything wrong with this implementation? – Syncarpous

@Syncarpous Sorry I am just seeing your question now. IMO it sounds like a monorepo would serve you best here. By having a monorepo, your frontend/backend will always be in sync, and if there's a breaking change in core, you only have to create one PR which updates all 3 areas of code (front, back, core) – Gearalt

@DevinRhode Hi Devin, I actually code in golang now, so that's not an issue for me anymore. and yes we're using monorepos for some of our projects. Thank you. – Syncarpous

Answer 2

My findings after trying both lerna, npm workspaces and git submodules. I find it is not a case of the one vs the other.

The reason why I say this is because one can have submodules that are part of the monorepo. Doing exactly this made my development experience better as I could clone an existing project and actively develop it within the bigger project (monorepo). I could then contribute back to the cloned project once satisfied with the changes. This is something that you cannot do with npm workspaces alone. Hence my argument that it is not a case of one vs the other. They solve different problems and can therefore complement each other.

Before using npm workspaces I would use npm link all the time. npm workspaces makes this use-case of developing with multiple packages more convenient. Even when the team you work with does not use a monorepo; you could use one to develop multiple packages and test them in conjunction. Once satisfied, you can push the individual repos. This is something you cannot do with git alone.

Maybe you can think of more novel ways of combining the features of npm and git.

Airlie

I've used both submodules and npm workspaces in the past, was just deciding which one to use for a new project, and your comment helped me realize I can use both together in a nice way. Thanks! – Tooley