I'm also disappointed with PeerCodeReview. It ties comments to a specific revision, and when you "Resubmit" a newer revision for review, you are closing the old review and opening a new one - but all comments stay on the old review only! So there is no easy way to see a comment together with the newer source that addresses it. Together with the total lack of support for reviewing commits/diffs, this makes PeerCodeReview unsuitable for me.
The CodeReview plugin seems nice (didn't actually try it), but I still miss the ability to comment on specific lines.
I should not that my use case was not classic "code review" but reviewing a single LaTeX document. This has different requirements:
Reviewing before a commit is not needed; on the contrary, a "pipelined" workflow
is critical: review comments are created when the reviewer has spare time,
and addressed when the writer gets to them, frequently in much later commits.
It'd be nice to track which parts of the text have been reviewed at what revisions.
This is not critical, as reviewing is usually done a section at a time and it's easy
enough to track manually.
There are many small independent comments. Each comment should be tracked
independently, a single "approve/reject" resolution for a commit.
Most comments are well-localized in the file, so I want a flow that allows attaching
them to specific places in the file, and they should keep their place when the file is
edited.
The Right flow for this would be to open a ticket for each comment, closing them with commit hooks when addressed. The only problem is that there is no easy way to tie a ticket to specific lines in the file, making it pretty cumbersome. I'm tempted to write a minimalistic plugin that ties tickets to source/diff lines. Would anybody else like such a beast?
What I'll probably do in practice is put TODO comments inside the source itself and not use fancy Trac interface at all for this. Version control will make sure the comments stay where they belong between edits.
[For LaTeX specifically, I'll probably use the todonotes
and/or fixme
packages to display the comments nicely, and maybe latexdiff
for visual diffs.] I'll edit this later to report how it went...
BTW, this approach is not limited to documents - I've worked with a team that used it a lot for code reviewing during intensive agile development and it worked well enough. They had a similar desire to "pipeline" the reviewing process - developement must go on, but all changes must be reviewed before doing a release. The hardest part was tracking what has or has not been reviewed, which was done by tagging "clean" revisions and diffing to them; in retrospect, having a "reviewed" branch and cherry-picking into it would work better. (Of course, non-localized, architectural questions would go into Trac tickets instead, or would start out as TODO comments and would be migrated into tickets if they proved non-trivial to address.)