this post was submitted on 18 Jul 2024
7 points (73.3% liked)

Programming

17349 readers
357 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities [email protected]



founded 1 year ago
MODERATORS
 

I just finished watching Why Google Stores Billions of Lines of Code in a Single Repository and honestly, while it looks intriguing, it also looks horrible.

Have you run into issues? Did you love it? How was it/

all 16 comments
sorted by: hot top controversial new old
[–] [email protected] 8 points 3 months ago* (last edited 3 months ago) (1 children)

We use them at Meta. It's easier to interact with other parts of the codebase, but it doesn't play well with libraries so you end up redoing a lot of stuff in-house.

I would only recommend a monorepo if you're a company with at least 5,000+ engineers and can dedicate significant time to internal infra.

[–] [email protected] 0 points 3 months ago (1 children)

it doesn’t play well with libraries

What do you mean by that? Is it the versioning of libraries that isn't possible meaning an update to the interface requires updating all dependent apps/libs?

Anti Commercial-AI license

[–] [email protected] 2 points 3 months ago

Updating a library in a monorepo means copying it all over and hoping the lib update didn't break someone else's code. Whereas updating a library normally would never break anything, and you can let people update on their own cadence

[–] [email protected] 3 points 3 months ago* (last edited 3 months ago)

They work great when you have many teams working alongside each other within the same product.

It helps immensely with having consistent quality, structure, shared code, review practices, CI/CD....etc

The downside is that you essentially need an entire platform engineering team just to set up and maintain the monorepo, tooling, custom scripts, custom workflows....etc that support all the additional needs a monorepo and it's users have. Something that would never be a problem on a single repository like the list of pull requests maybe something that needs custom processes and workflows for in a monorepo due to the volume of changes.

(Ofc small mono repos don't require you to have a full team doing maintenance and platform engineering. But often you'll still find yourself dedicating an entire FTE worth of time towards it)

It's similar to microservices in that monorepo is a solution to scaling an organizational problem, not a solution to scaling a technology problem. It will create new problems that you have to solve that you would not have had to solve before. And that solution requires additional work to be effective and ergonomic. If those ergonomic and consistency issues aren't being solved then it will just devolve over time into a mess.

[–] [email protected] 2 points 3 months ago* (last edited 3 months ago)

Most companies will never have a monorepo at the level of these bigger companies. So I personally don't think most people need to worry about the limitations of github/lab as platforms.

However if you happen to be having those kinds of issues, I think looking at what the big companies are doing and/or starting to split things up makes sense.

There's also alternatives with custom ci jobs within non GitHub/lab within the git universe that may help out with those sorts of operations. I know actions still feel very beta in some toolsets so it may be easier/more useful to run your own arch. I've been enjoying forgeo/gitea for example, but it's not like you can't do the same with girlab runners or GitHub enterprise. Depends on use case.

[–] [email protected] 1 points 3 months ago

I have no experience using them across an entire company, let alone team. So far, it always seemed good enough to keep just each project in a repo. But yeah, for larger projects consisting out of multiple applications, I would not want to work without a monorepo for that.

Many of the benefits from the video still apply, mainly the consistency in code changes was always useful. You can check out any commit and you've got the exact state how everything worked together at the time. No wrangling different versions, no inconsistencies between APIs.
In our build process, we include the Git commit into the applications to have it logged on start-up, so when we get an error report+logs, we can always easily look at the respective code.

But it does depend on your build tooling, if this works well. JVM languages with e.g. Gradle's multi-project builds are great. Rust's workspaces are a treat. Python is fucking atrocious with everything we've tried (pipenv, poetry, lots of custom scripts+symlinks).

[–] [email protected] 1 points 3 months ago (1 children)

I've been a big fan of monorepos because it leads to more consistent style and coding across the whole company. It makes the code more transparent so you can see what's going on with the rest of the company, too, which helps reduce code islands and duplicated work. It enables me to build everything from source, which helps catch bugs that would only show up in prod due to version drift. It also means that I can do massive refactorings across the company without breaking anything.

That said, tooling is slowly improving for decentralized repos, so some of these may be doable on git now/soon.

[–] [email protected] 0 points 3 months ago* (last edited 3 months ago) (1 children)

(...) you can see what’s going on with the rest of the company, too.

That's a huge security problem.

Edit for those who are down voting this post, please explain why you believe that granting anyone in the organization full access to all the projects used across all organizations does not represent a security problem.

[–] [email protected] -1 points 3 months ago

Because security through obscurity is not security at all.

[–] [email protected] 0 points 3 months ago (1 children)

We use a mono repo for a new cloud based solution. So far it's been really great.

The shared projects are all in one place so we don't have to kick things out to a package manager just to pull them back in.

We use filters in azure pipelines so things only get built if they or dependent projects get changed.

It makes big changes that span multiple projects effortless to implement.

Also running a local deployment is as easy as hitting run in the ide.

So far no problems at all.

[–] [email protected] -2 points 3 months ago (1 children)

We use filters in azure pipelines so things only get built if they or dependent projects get changed.

Any guides on how to do this? I know about filtering triggers by where changes happens, but how do dependent projects get triggered? Is that a manually maintained list or is that something automatic? I mostly use Gitlab, but am curious how Azure Pipelines would do it.

Anti Commercial-AI license

[–] [email protected] 0 points 3 months ago* (last edited 3 months ago) (1 children)

You have a list of filters like "src/libs/whatever/*" if there is a change the pipeline runs.

I wrote a tool that automatically updates these based on recursive project references (c#)

So if any project referenced by the service (or recursively referenced by dependencies) changes the service is rebuilt.

[–] [email protected] -2 points 3 months ago

I see. OK. I thought that was built into Azure pipelines.

Pretty cool tool you built 👍 Is it language agnostic?

Anti Commercial-AI license