Making mercurial subrepositories behave like subversion externals

Question

Hi guys,

The FAQ, and hginit.com have been really useful for helping me make the transition from svn to hg.

However, when it comes to using Hg's subrepository feature in the manner of subversion's externals, I've tried everythign and cannot replicate the nice behavior of svn externals.

Here's the simplest example of what I want to do:

1. Init "lib" repository This repository is never to be used as a standalone; it's always included by main repositories, as a sub-repository.

2. Init one or more including repositories To keep the example simple, I'll "init" a repository called "main"

3. Have "main" include "lib" as a subrepository

4. Importantly -- AND HERE'S WHAT I CAN'T GET TO WORK: When I modify a file inside of "main/lib", and I push the modification, then that change gets pushed to the "lib" repository -- NOT to a copy inside of "main".

Command lines speak louder than words. I've tried so many variations on this theme, but here's the gist. If someone can reply, in command lines, I'll be forever grateful!

1. Init "lib" repository

$ cd /home/moi/hgrepos ## Where I'm storing my hg repositories, on my main server

$ hg init lib

$ echo "foo" > lib/lib.txt

$ hg add lib

$ hg ci -A -m "Init lib" lib

2. Init "main" repository, and include "lib" as a subrepos

$ cd /home/moi/hgrepos

$ hg init main

$ echo "foo" > main/main.txt

$ hg add main

$ cd main

$ hg clone ../lib lib

$ echo "lib=lib" > .hgsub

$ hg ci -A -m "Init main" .

This all works fine, but when I make a clone of the "main" repository, and make local modifications to files in "main/lib", and push them, the changes get pushed to "main/lib", NOT to "lib".

IN COMMAND-LINE-ESE, THIS IS THE PROBLEM:

$ /home/moi/hg-test

$ hg clone ssh://[email protected]/hgrepos/lib lib

$ hg clone ssh://[email protected]/hgrepos/main main

$ cd main

$ echo foo >> lib/lib.txt

$ hg st

M lib.txt

$ hg com -m "Modified lib.txt, from inside the main repos" lib.txt

$ hg push

pushing to ssh://[email protected]/hgrepos/main/lib

That last line of output from hg shows the problem.

It shows that I've made a modification to a COPY of a file in lib, NOT to a file in the lib repository. If this were working as I'd like it to work, the push would be to hgrepos/lib, NOT to hgrepos/main/lib. I.e., I would see:

$ hg push

pushing to ssh://[email protected]/hgrepos/lib

IF YOU CAN ANSWER THIS IN TERMS

OF COMMAND LINES RATHER THAN IN ENGLISH,

I WILL BE ETERNALLY GRATEFUL!

Thank you in advance!

Emily in Portland

Answer 1

The problem is with your .hgsub file. It points to where the lib repo is, so if lib is a sibling to main it should be:

lib=../lib

Also your hg add lib and hg add main lines don't make sense. To what repo outside of main and lib are those being added? You're running them while in in /home/moi/hgrepos.

Here's your script with some tweaks:

+ cd /home/ry4an/hgtest
+ hg init lib
+ echo foo
+ cd lib
+ hg commit -A -m Init lib
adding lib.txt
+ cd /home/ry4an/hgtest
+ hg init main
+ echo foo
+ cd main
+ echo lib=../lib
+ hg clone ../lib
destination directory: lib
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ hg add .hgsub main.txt
+ hg commit -m Init main: initial file and a .hgsub
committing subrepository lib
+ cd /home/ry4an/hgtest
+ hg clone main main-clone
updating to branch default
pulling subrepo lib
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ cd main-clone
+ echo foo
+ hg commit -m Modified lib.txt, from inside the main repos
committing subrepository lib
+ hg push
pushing to /home/ry4an/hgtest/main
pushing subrepo lib
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
searching for changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files

To do that over ssh:// you need only make a single change. When cloning the main repo change hg clone main main-clone to hg clone ssh://host/hgtest/main main-clone -- cloning the main automatically clones the lib -- that's the subrepo benefit.

Here's a log of that working:

+ cd /home/ry4an/hgtest
+ hg init lib
+ echo foo
+ cd lib
+ hg commit -A -m Init lib
adding lib.txt
+ cd /home/ry4an/hgtest
+ hg init main
+ echo foo
+ cd main
+ echo lib=../lib
+ hg clone ../lib
destination directory: lib
updating to branch default
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+ hg add .hgsub main.txt
+ hg commit -m Init main: initial file and a .hgsub
committing subrepository lib
+ cd /home/ry4an/hgtest
+ hg clone ssh://localhost/hgtest/main main-clone
The authenticity of host 'localhost (::1)' can't be established.
RSA key fingerprint is 0c:58:d6:d3:d3:16:14:ee:3b:be:01:bc:c7:3c:92:0b.
Are you sure you want to continue connecting (yes/no)? yes
ry4an@localhost's password: 
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 3 changes to 3 files
updating to branch default
pulling subrepo lib
ry4an@localhost's password: 
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
remote: Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
+ cd main-clone
+ echo foo
+ hg commit -m Modified lib.txt, from inside the main repos
committing subrepository lib
+ hg push
ry4an@localhost's password: 
pushing to ssh://localhost/hgtest/main
pushing subrepo lib
ry4an@localhost's password: 
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
remote: added 1 changesets with 1 changes to 1 files

Answer 2

Ryan,

Worked!!! Thank you so much!

As for the extraneous use of "hg add", I got fuzzy-headed reading this article about subrepositories: http://mercurial.selenic.com/wiki/subrepos

For anyone interested, I also worked out how to make this same example work, substituting a 'ssh' URL for actual file paths.

So, here are the two scenarios most folks will be faced with:

A) Using subrepositories in a completely local situation. This is Ryan's solution, essentially. I imagine only developers working solo will be in this boat.

cd /home/moi/hgrepos hg init lib cd lib echo foo > lib.txt hg ci -A -m Init

cd /home/moi/hgrepos hg init main cd main echo foo > main.txt echo lib = ../lib > .hgsub hg clone ../lib hg add .hgsub main.txt hg ci -m Init

cd /home/moi/hgrepos hg clone main main-clone cd main-clone/lib echo "Modified while on main trunk" >>lib.txt hg commit -m "Modified lib.txt, while on main trunk" hg push

cd /home/moi/hgrepos/lib hg update 1 files updated, 0 files merged, 0 files removed, 0 files unresolved cat lib.txt foo Modified while on main trunk

-----------------------------------------------------------------

B) Using subrepositories over ssh.
I imagine most developers working on teams will be in this boat.

1) Set up lib cd /home/moi/hgrepos hg init lib cd lib echo foo > lib.txt hg ci -A -m Init

2) Set up main cd /home/moi/hgrepos hg init main cd main echo foo > main.txt echo lib=ssh://[email protected]/hgrepos/lib > .hgsub hg clone ssh://[email protected]/hgrepos/lib lib hg add .hgsub main.txt hg ci -m Init

3) Clone lib to hgtest dir cd /home/moi/hgtest hg clone ssh://[email protected]/hgrepos/lib lib

4) Clone main to hgtest dir cd /home/moi/hgtest hg clone ssh://[email protected]/hgrepos/main main

5) Modify lib.txt while on main trunk cd /home/moi/hgtest/main/lib echo "Modified while on main trunk" >>lib.txt hg commit -m "Modified lib.txt, while on main trunk" hg push

6) Verify that lib.txt got changed in the lib repository cd /home/moi/hgtest/lib hg pull hg update 1 files updated, 0 files merged, 0 files removed, 0 files unresolved cat lib.txt foo Modified while on main trunk

Thanks again Ryan, what a relief to have this solved!

Best, Emily

Answer 3

Oops, sorry for the formatting in previous answer. Here it is again, formatted!

So, here are the two scenarios most folks will be faced with:

A) Using subrepositories in a completely local situation. This is Ryan's solution, essentially. I imagine only developers working solo will be in this boat.

    cd /home/moi/hgrepos
    hg init lib
    cd lib
    echo foo > lib.txt
    hg ci -A -m Init

    cd /home/moi/hgrepos
    hg init main
    cd main
    echo foo > main.txt
    echo lib = ../lib > .hgsub
    hg clone ../lib
    hg add .hgsub main.txt
    hg ci -m Init

    cd /home/moi/hgrepos
    hg clone main main-clone
    cd main-clone/lib
    echo "Modified while on main trunk" >>lib.txt
    hg commit -m "Modified lib.txt, while on main trunk"
    hg push
    cd /home/moi/hgrepos/lib
    hg update
    1 files updated, 0 files merged, 0 files removed, 0 files unresolved

    cat lib.txt
    foo
    Modified while on main trunk

# -----------------------------------------------------------------

B) Using subrepositories over ssh.  
   I imagine most developers working on teams will be in this boat.

    1) Set up lib
    cd /home/moi/hgrepos
    hg init lib
    cd lib
    echo foo > lib.txt
    hg ci -A -m Init

2) Set up main
cd /home/moi/hgrepos
hg init main
cd main
echo foo > main.txt
echo lib=ssh://[email protected]/hgrepos/lib > .hgsub
hg clone ssh://[email protected]/hgrepos/lib lib
hg add .hgsub main.txt
hg ci -m Init

3) Clone lib to hgtest dir
cd /home/moi/hgtest
hg clone ssh://[email protected]/hgrepos/lib lib

4) Clone main to hgtest dir
cd /home/moi/hgtest
hg clone ssh://[email protected]/hgrepos/main main

5) Modify lib.txt while on main trunk
cd /home/moi/hgtest/main/lib
echo "Modified while on main trunk" >>lib.txt
hg commit -m "Modified lib.txt, while on main trunk"
hg push

6) Verify that lib.txt got changed in the lib repository
cd /home/moi/hgtest/lib
hg pull
hg update
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
cat lib.txt
foo
Modified while on main trunk