3

I have three repositories.

  • RepoCommon
  • RepoTn
  • RepoTh

RepoCommon is added as submodule to both RepoTn and RepoTh. Both RepoTn and RepoTh are two different repositories because earlier in the development it was recognized that those two are independent application. But now later in the development part we realized RepoTn is highly dependent on RepoTh.

As a solution I'm thinking of adding RepoTh as a submodule to RepoTn. But the one thing that bothers me is the "extra" RepoCommon inside RepoTh. If I do that it will essentially have the following structure:

  • RepoTn
    • RepoCommon
    • RepoTh
      • RepoCommon -> this is the part I'm trying to remove

I'm wondering if it's even possible to remove RepoCommon from RepoTh when I want to add it as a submodule to RepoTn.

Note: RepoTh needs to live on as it's own separate repository as well. Also, another reason I'm asking this because if I can remove/ignore the RepoCommon I can modify the project file to make sure system uses the correct RepoCommon folder. I am using Visual Studio 2019 if that makes any difference. Any other suggestion for a better solution is also welcome.

Improve this question
1
  • 2
    This has nothing to do with c#.. and barely something to do with visual studio. If you remove those tags, this might help more people in the future solve this problem on other platforms. Commented Sep 14, 2020 at 17:05

2 Answers 2

Reset to default
6

The short answer is no. This creates a problem, which you can solve by using the long answer, but it might be wise to avoid this setup. See VonC's answer.

The long answer

You have to remember here what a submodule is—but before we get there, let's define exactly what a Git repository is. There are a lot of little caveats to this, but in short, a Git repository is a collection of commits.1 Each commit has a unique number: a big ugly hash ID, reserved to that commit, across every Git everywhere. That commit serves to hold a snapshot of every file that Git knew about at the time whoever made the commit, made it.2

Along with these stored files, which are internally kept as what Git calls blob objects, Git can keep what Git internally calls a gitlink. We'll look at the exact form of a gitlink in a moment, when we get to submodules.

Once you have selected some commit—by its hash ID, which you might have Git look up by branch name for instance—in a repository, you can have Git check out that commit (using git checkout or, since Git 2.23, git switch). This extracts that commit into a series of usable (and modifiable!) ordinary files. Git copies3 these files into Git's index, which is a per-repository data structure that Git uses to make new commits.4 If the commit contains any gitlinks, Git also copies those gitlinks into its index.

1The biggest caveat is that the .git directory, in which the repository resides, is actually two main databases, plus some ancillary data. One maps from hash IDs to internal Git objects, which include commits but also include three other object types. The second maps names—including branch and tag names, which are the ones that humans mostly use here—to hash IDs so that the hash IDs in the first database can be used by humans, for instance.

When you clone a repository, your Git copies the object database as-is, and uses the other Git's name-to-hash-ID database to construct your own (different) name-to-hash-ID database. Nothing else—none of the ancillary data, such as hooks, the index, or special pseudorefs like CHERRY_PICK_HEAD—gets copied. So the object database matters a lot, and since humans use the name database to define their concept of branches—which isn't quite the same as Git's—that second database matters too. Here, however, we're mostly concerned with the object database.

2Besides the snapshot, commits store metadata, but here we're only concerned with the snapshot itself.

3The stored files are kept in a special, read-only, Git-only format in which files are compressed and (importantly) de-duplicated. What goes into the index is actually the blob hash ID, so that the index "copy" isn't really a copy at all; but the index also holds the file's name, and data that make Git go fast.

4The index has additional roles, but by holding a "copy" of every file along with the file's path name, the index acts as a template for the next commit. When you use git add, what you are actually doing is updating Git's index, so that the next commit won't be the same as the current commit.

How gitlinks become submodules

A submodule in Git is just another—separate—Git repository, with one special bit of relationship: the submodule Git repository is being controlled by some other Git repository. The controlling Git repository is the superproject and the controlled Git repository is the submodule.

Given that a repository works by having a commit checked out—so that you have files that you can use, instead of files that are in a special compressed format that only Git itself can use—all that the superproject Git itself needs to do, once the submodule Git repository is cloned into place, is to run git checkout in that submodule. So that's what it does:

cd $path
git checkout $hash

To git checkout some commit within the submodule, the superproject needs to know two things:

  • What's the path? What value goes into $path above?
  • What's the hash ID? What value goes into $hash above?

These two things are precisely what's in the gitlink. Each commit stores files. A commit can also store gitlinks. The files, or gitlinks, get copied into the superproject Git's index. Each entry in the index consists of a path name, a hash ID, and some other internal Git data. The path-name-and-hash-ID are exactly what the superproject Git needs to do the above commands.

So that's all there is to it ... well, almost all. There's one more thing required: the superproject Git may need to run git clone, to actually create the submodule repository. The information the superproject Git needs here is stored in the .gitmodules file in the superproject.5 Once the superproject Git has run git clone, however, the .gitmodules data is no longer required.6

5Note that you can create a sort of half-assed submodule, in which there is no .gitmodules file entry for the submodule, but there are gitlinks stored in the superproject's commits.

6Some git submodule commands will still update .gitmodules for you, and some git submodule commands use data that get copied from .gitmodules to .git/config. But none of this is required for everyday use of the submodule, once it's cloned.

Submodule recursion and git clone

If Git repository A lists Git repository B as a submodule (in A's .gitmodules and gitlinks-in-commits), and Git repository B lists Git repository A as a submodule (in B's .gitmodules and gitlinks-in-commits), any automated operation that clones and checks out a commit and directs the submodule to clone-and-checkout its commit could result in infinite recursion:

  • A clones B
  • A tells B: check out commit CB, where CB has a gitlink for CA
    • so B clones A
    • and B tells A: check out commit CA
      • so A clones B
      • and A tells B: check out commit CB
        • ...

The real key(s) here, then, are those git clone and/or git checkout steps. When does the superproject run git clone? When does a superproject direct a submodule to git checkout some commit?

If you keep the submodule recursive modes turned off, the answer is that a superproject never runs git clone itself and never runs git checkout itself. This allows you to:

  • clone A
  • tell A to clone B now and check out one commit without recursion

and then stop.

If you turn recursion on, A will clone B and tell it to check out the commit, which may tell B to clone A, and since recursion is on, that will keep going forever (or until you run out of disk space anyway). So just leave recursion off and you'll be OK—but everyone who uses these repositories must know to do this.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

5

If possible, you should add an additional repository, which will record those dependencies:

RepoTnParent
  RepoTn
  RepoTh
  RepoCommon

That way, if you need to build RepoTn, you would load the RepoTnParent which will in turn load the right version RepoTn, RepoTh and RepoCommon.

Improve this answer

2 Comments

That's the route I will probably have to take to reduce redundancy. What I don't like about this approach it doesn't describe the dependency that well enough. i.e it seems like everyone can be dependent on each other or not at all. For example: in my case RepoTh isn't dependent on RepoTn, but it's not that evident from this structure. On the other hand it becomes "too much" loosely coupled, which can be a good thing.
@SZT I agree: the dependencies are recorded only in RepoTnParent here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.