• 0 Posts
  • 9 Comments
Joined 1 year ago
cake
Cake day: July 2nd, 2023

help-circle
  • I recommend wrapping the git cli commands using subprocess, using porcelain output modes etc, and parsing the output.

    We have had stability problems with GitPython (which wraps gitdb). On Linux gitdb does clever things with sliding mmap, which caused some crashes (in a multi threaded environment), and I found simple race conditions in the code for writing loose objects, which is about as simple an operation as can be, so I lost faith with it. I do use gitdb in one read-only single-threaded system; it’s undoubtedly fast.

    The biggest issues with git libraries are around the complexity of git configurations. Any independent reimplementation is probably going to support the most common 99% of features but that 1% always comes back to bite you! We use a lot of git features in service of a gigantic monorepo, like alternates and partial clones and config tricks.

    If we use command-line git we get 100% compatibility with all git configuration and ODB features, and it’s hard to ensure that with an independent git implementation (even libgit2).

    When you say “that solution doesn’t scale well” - we have made it scale. git itself scales well for operations it can perform natively, you just have to use the features effectively, often the high-level operations but sometimes lower-level commands like git cat-file --batch, git mktree --batch, etc. It’s not as fast as gitdb but fast enough, and I can have high confidence that I can write something once and it won’t break or cause problems later.