Automatically deleting pyc files when corresponding py is moved (Mercurial)

Question

(I foresaw this problem might happen 3 months ago, and was told to be diligent to avoid it. Yesterday, I was bitten by it, hard, and now that it has cost me real money, I am keen to fix it.)

If I move one of my Python source files into another directory, I need to remember to tell Mercurial that it moved (hg move).

When I deploy the new software to my server with Mercurial, it carefully deletes the old Python file and creates it in the new directory.

However, Mercurial is unaware of the pyc file in the same directory, and leaves it behind. The old pyc is used preferentially over new python file by other modules in the same directory.

What ensues is NOT hilarity.

How can I persuade Mercurial to automatically delete my old pyc file when I move the python file? Is there another better practice? Trying to remember to delete the pyc file from all the Mercurial repositories isn't working.

Answer 1

You need:

1) A real deployment infrastructure, even if it's just a shell script, which does everything. Cloning/checking out an updated copy from source control is not a deployment strategy.

2) Any deployment system should completely clean the directory structure. My usual preference is that each deployment happens to a new directory named with a date+timestamp, and a symlink (with a name like "current") is updated to point to the new directory. This gives you breadcrumbs on each server should something go wrong.

3) To fix whatever is running the Python code. New .py source files should always take precedence over cached .pyc files. If that is not the behavior you are seeing, it is a bug, and you need to figure out why it is happening.

Answer 2

How about using an update hook on the server side? Put this in the repository's .hg directory's hgrc file:

[hooks]
update = find . -name '*.pyc' | xargs rm

That will delete all .pyc files whenever you update on the server. If you're worried about the cost of rebuilding all the .pyc files you could always get just a little more clever in the hook and delete only the .pyc's for which there is no .py, but that's probably overkill.

Answer 3

1. Do not store .pyc files in the repository.
2. Automatize .pyc delete with: find . -name '*.pyc' -delete
3. While develop use -B argument in Python.

Answer 4

I use the .hgignore file to skip versionning of all my .pyc and .py~ (editor's temp files). For example, this is my version :

# use glob syntax.
syntax: glob

.directory
*.pyc
*~
*.o
*.tgz
*.tbz2
*.gz
*.bz2

Also adding a hook on update to remove them is also a interesting trick if you want to not only ignore noise but remove it from your local workspace area.

Answer 5

What I have actually done:

1) I am considering Nicholas Knight's suggestion about using a proper deployment strategy. I have been reading about Buildout and Collective.hostout to learn more. I need to decide whether such heavy-weight strategies are worthwhile for my project's relatively simple requirements.

2) I have adopted Ry4an's update hook concept, in the short-term, until I decide.

3) I ignored Ry4an's warning about overkill, and wrote a Python script to only delete stray .pyc files.

#!/usr/bin/env python
""" Searches subdirectories of the current directory looking for .pyc files which
    do not have matching .py files, and deletes them.

    This is useful as a hook for version control when Python files are moved.
    It is dangerous for projects that deliberately include Python 
    binaries without source.
"""
import os
import os.path
for root, dirs, files in os.walk("."):
    pyc_files = filter(lambda filename: filename.endswith(".pyc"), files)
    py_files = set(filter(lambda filename: filename.endswith(".py"), files))
    excess_pyc_files = filter(lambda pyc_filename: pyc_filename[:-1] not in py_files, pyc_files)
    for excess_pyc_file in excess_pyc_files:
        full_path = os.path.join(root, excess_pyc_file)
        print "Removing old PYC file:", full_path
        os.remove(full_path)

My update hooks now call this rather than the "find" commands suggested by others.