cvs to mercurial conversion gets tags wrong

Question

I've tried all the recommended conversion techniques

Mostly they manage to get the latest version of the files right, but every one of them trashes my history. Many (most?) of the tags from my cvs project have at least one file in error when I run "hg up $tag"

My cvs repo is not all that complicated. Why can't anything convert it? I'd like to dump cvs and convert to mercurial, but not without history.

To recap my frustration:

I tried hg convert (tried --branchsort,--timesort, fuzz=0)

I tried cvs2svn and then hg convert.

tailor does not work with recent versions of mercurial

fromcvs disappeared from the face of the earth

hg-cvs-import has been abandoned for 4 years and doesn't work with recent versions of hg

I have tried using the two most recent versions of mercurial ( 1.5 and 1.5.1 ).

Answer 1

Mark, it's a sub-optimal solution, but when a company I was with did a CVS->Mercurial migration we decided that all we cared about were tag snapshots, so we build a little for loop like:

for thetag in $(cat LIST_OF_RELEASE_TAGS) ; do
   cvs update -r $thetag
   hg commit --addremove -m "snapshot $thetag" -u "import"
   hg tag $thetag
done

That assumed a linear chain of tags, but we only pulled in the main/production branch. A more sophisticated loop would call 'hg update' before each commit to get parentage that reflects CVS branching.

It's definitely not "full history" but it was enough to make us feel good about continuing in Mercurial without loosing our ability to say "What the hell was in version 1.1.11?!" and we could always go back to cvs is cvs blame level history was needed.

Answer 2

I found a solution of sorts. I'm not thrilled with it, but it will have to do for now. I was able to detect the tags that were causing trouble and omit those tags from the conversion. Missing tags are much better than wrong tags (assuming the original cvs repo is kept for backup)

WARNING: The following assumes you have made a copy of CVSROOT and are working on that. Do not muck with your original.

This is a bash solution that works for me on my linux box. It will probably burn your house down and invite your grade school bully to move next door to you. You've been warned.

It uses cvsps to identify the problem tags, rcs to delete them and then removes the tags from the CVSROOT/history. After removing the cvsps cache, the hg conversion works as expected.

CVSROOT=/path/to/your/copy
MODULE=cvsmodule
rm -rf ~/.cvsps ~/.hg.cvsps # this cache is EVIL!

BADTAGS="`cvsps -q -x $MODULE |grep Tag: |grep -e FUNKY -e INVALID | awk '{print $2}' `"
while [ ! -z "$BADTAGS" ];do
    cd $CVSROOT/$MODULE
    for badtag in $BADTAGS;do
      echo removing tag $badtag
      grep -lr $badtag . | xargs --no-run-if-empty -l1 rcs -q -n$badtag
      grep -v "$badtag|$MODULE" < $CVSROOT/CVSROOT/history > $CVSROOT/CVSROOT/history_
      mv $CVSROOT/CVSROOT/history_  $CVSROOT/CVSROOT/history
    done
    BADTAGS="`cvsps -q -x $MODULE |grep Tag: |grep -e FUNKY -e INVALID | awk '{print $2}' `"
done
rm -rf ~/.cvsps ~/.hg.cvsps # this cache is EVIL!
mkdir ~/hgcvt
cd ~/hgcvt
cvs co $MODULE
hg convert $MODULE

Answer 3

I realize now that there are certain fundamental incompatibilities between cvs tags and hg tags.

In cvs, a version of a file have tags associated with its different versions.

In hg, a version is an alias for a changeset . In other words the state of the working files at some snapshot in time

The distinction is subtle, but important.

It is possible to make a tagged release in cvs of a version that does not represent a snapshot in time. This is not possible in hg.

Of course one could apply patches to get replicas. However, this would create a lot of new heads on the repository with arguably little benefit (assuming the cvs repo is still kept around for posterity).

I'm afraid a perfect conversion from cvs to mercurial is not practical. Ry4an's solution would work for those who care only about recreating the versions. I am more interested in the history and evolution of the source files.

I wrote the following script to simply munge all the cvs tags in the $CVSROOT prior to the conversion. e.g tag "v321" becomes "v321_prehg". That way developers will know those tags are not-authoritative and they must go back to the read-only cvs tree.

#!/usr/bin/python
import os
import sys
import stat

def die(msg):
    sys.stderr.write(msg)
    sys.exit(1)

cvsroot =os.getenv("CVSROOT")
if cvsroot is None:
    die("CVSROOT not defined" )

print "CVSROOT=%s" % cvsroot

for rcsfile in os.popen("find %s -name '*,v'" % cvsroot).xreadlines():
    rcsfile = rcsfile.replace('\n','')
    print "rcsfile:%s" % rcsfile
    st=os.stat(rcsfile)
    if  st.st_mode & stat.S_IWUSR == 0:
        os.chmod(rcsfile,st.st_mode | stat.S_IWUSR)

    f = open(rcsfile,"r")
    inlines=f.readlines()
    f.close()

    outlines=[]
    insymbols=False
    symbolsDone=False
    for l in inlines:
        if insymbols and not symbolsDone:
            if l.find('\t') == 0:#tag line
                l= l.replace(":","_prehg:",1)
            else:
                symbolsDone=True
        else:
            if l == "symbols\n":
                insymbols=True
        outlines.append(l)

    f = open(rcsfile,"w")
    f.writelines( outlines )
    f.close()

Answer 4

fromcvs is back. I'm testing it out on a very large repo of ours, and it's extremely fast and handles incremental conversion.