Complexity of Multi-Monorepo Approach

“From a Single Repo, to Multi-Repos, to Monorepo, to Multi-Monorepo Nice write-up. I have explored different repo strategies quite a bit myself in the course of a few efforts that I&#x27;ve been involved with. On one, we originally had a monolythic framework and everything the article said about cons is pretty spot on. However, I&#x27;ll qualify by saying that I think the problems come less because of the nature of monolyths in general and more because of lack of experience with modular design.<p>We wrote a new framework from scratch using a monorepo approach, with separate packages via Lerna. The problem here was tooling. Dependent builds were not supported and I&#x27;ve had to delete node_modules more times than I&#x27;d ever cared to count. The article talks about some github specific problems (namely, the issues list being a hodge-podge of every disparate package). We tried zenhub, it works ok, but it&#x27;s a hack and it kinda shows. I&#x27;ve seen other projects organize things via tags. Ultimately it comes down to what the team is willing to put up with.<p>We eventually broke the monorepo out into multi-repos, and while that solved the problem of managing issues, now the problem was that publishing packages + cross-package dependencies meant that development was slower (especially with code reviews, blocking CI tests, etc).<p>Back to a monorepo using Rush.js (and later Bazel). Rush had similar limitations as Lerna (in particular, no support for dependent tests) and we ditched it soon afterwards. Bazel has a lot of features, but it takes some investment to get the most out of it. I wrote a tool to wrap over it[0] and setup things to meet our requirements.<p>We tried the &quot;multi-monorepo&quot; approach at one point (really, this is just git submodules), and didn&#x27;t get very good results. The commands that you need to run are draconian and having to remember to sync things manually all the time is prone to errors. What&#x27;s worse is that since you&#x27;re dealing with physically separate repos, you&#x27;re back to not having good ways to do atomic integration tests across package boundaries. To be fair, I&#x27;ve seen projects use the submodules approach[1] and it <i>could</i> work depending on how stable your APIs are, but for corporate requirements, where things are always in flux, it didn&#x27;t work out well.<p>Which brings me to another effort I was involved with more recently: moving all our multi-repo services into a monorepo. The main rationale here is somewhat related to another reason submodules don&#x27;t really fly: there&#x27;s a ton of packages being used, a lot of stakeholders with various degrees of commit frequency, and reconciling security updates with version drift is a b*tch.<p>For this effort we also invested into using Bazel. One of the strengths of this tool is how you can specify dependent tasks, for example &quot;if I touch X file, only run the tests that are relevant&quot;. This is a big deal, because at 600+ packages, a full CI run consumes dozens of hours worth of compute time and we see several dozens commits a day. The problem with monorepos comes largely from the sheer scale: bumping something to the next major version requires codemods, and there&#x27;s always someone doing some crazy thing you never anticipated.<p>With that said, monorepos are not a panacea. A project from a sibling team is a components library and it uses a single repo approach. This means a single version to manage for the entire set of components. You may object that things are getting bumped even when they don&#x27;t need to, but it turns out this is actually very well received by consumers, because it&#x27;s far easier to upgrade than having to figure out the changelog of dozens of separate packages.<p>I used a single repo monolyth-but-actually-modular setup for my OSS project[2] and that has worked well for me, for similar reasons: people appreciate curation, and since we want to avoid willy-nilly breaking c”