How do I adapt my svn:externals strategy to git submodules?

Question

I'm having trouble figuring out how to change my mindset to git and have run in to the following problem. I have the situation where we have a shared engine and multiple projects that use the engine. Internal development teams and second party teams may be working on projects that use the shared engine, and want to be using HEAD of the shared engine as much as possible during development, until just a few weeks before ship, where the shared engine will be tagged and branched, and the project will then use that branch. The project teams typically only work on one project at a time, but may make changes to the shared engine during debugging or to add features. When they commit those changes, our build system runs to find any problems they may have introduced with the commit.

I (think I) want to use this same model with a new project/new company. In svn, the structure was something like this: shared_engine

project_in_dev-+
               +- svn:external shared_engine:head
project_about_to_ship-+
                      +-svn:external shared_engine_rev1_branch

This worked very well:

• Project developers could do one command to check out all the dependencies they would need
• Project developers could do engine work and commit in to the shared engine easily
• We could easily rev or change the shared engine the project was using with externals and revisions
• Engine updates were easy to get with your daily "update from the root project"

OK, now I've moved to git, and submodules SEEM to be the new way to deal with external code, but it seems like I lose some features.

• It's a multiple step process to actually get all the dependencies of the project. Project developers have to do a git clone then a git submodule init/git submodule update --recursive
• It's a multiple step process to update the root project and the submodule, so if changes are made to the root project by another developer that match changes to the submodule, you don't get the matching code immediately and could get very confused
• The submodule is locked to a particular commit, and if you make changes to the submodule you will have trouble getting it to work with the head of the shared engine
• I have no control over what revision of the shared engine the project developer has checked out without giving instructions on what to update to

So my questions are as follows:

• First and foremost, are the above assumptions about submodules correct? It seems to be based on what I've read, but I'm not 100% certain as I'm still figuring out git
• If my assumptions are correct, am I approaching the problem with the correct process? Do I need to readjust my thinking when using git? In other words, is there another way to do what I'm trying to do and need to think about the process differently?
• Assuming I haven't blown the first two, and submodules won't do what I want, what will? I read about subtree merges but those don't seem exactly right either as it looks like I can't get changes made to the shared code back in to the repository.

Thanks so much for your help and patience. If it's not obvious, I'm very new to git, and I like it and want to embrace it, but I'm still having some conceptual misunderstandings because I've probably been brain damaged by years of using central repos. I want to learn! Also, I've been rtfm'ing all day, and looking at various blogs posts, stackoverflow questions, etc, and I still don't get it, I obviously need it spelled out step by step for my situation. I have no coworkers to ask about this, any user groups in the Seattle area where there might be some git gurus? :)

Answer 1

I'm pretty new to GIT, but we recently did a trial migration from SVN to GIT.

Turns out GIT is very fast. You can make a giant repository and branch it for a subrelease of a sub-sub-project, and you'll never notice a delay.

In SVN, branching is so expensive you can't split an entire repository for a small release of a subproject. This kind of forces the externals approach.

So the way I'm looking at it now, SVN externals are a workaround, and you can omit them when migrating to GIT.

Answer 2

You are right that a submodule always references a specific revision, which is fixed when you git add the submodule directory (and therefore you can control exactly what is checked out on the developer box). But I see this as a feature, since you can always request the HEAD of a submodule when you need it. On the other side this means that you always get the same state when you checkout an old state of you project, regardless whatever changed in the submodules. You can think about them as svn externals which are pinned to a specific revision.

As for changes in the submodule, they are just normal git repos, where you can work with your normal workflow, as if they where cloned into an own working copy. There is one difference to a regular clone, that the checkout of a submodule is very likely a detached head, so you must create a branch on your own when you do changes in there.

For the many commands part, yes, there is a need to do more work, that's the price to pay for this feature. You may add a script which performs the submodule checkout if there are many of them.

EDIT

I found a detailed explanation about submodules: http://longair.net/blog/2010/06/02/git-submodules-explained/.