Preamble
Ok so first up cards on the table I am a huge fan of CodeRush.
My suggestions of what to evaluate are based on my own experiences with CodeRush and also the very small amount of experience I have with Resharper. I have not looked at JustCode as at the time I was looking at it, it refused to coexist with any of the other tools on the market.
If someone from Telerik should read this and this has changed, please feel free to contact me and I will look again(RoryBecker @ gmail)
So what topics do I think should be compared?
My own personal preferences are:
Short answer: Interface & Intuitiveness, Navigability, CodeGeneration, Refactoring, Performance, Customization, Extensibility, Support and Community
Long Answer: (As above but with more detail.)
Interface & Intuitiveness:
Basic UI is a factor. You need to be at home in your main work tool without being distracted too much by things you see as unnecessary. Most tools will default to showing you everything you can do. This approach, whilst great for advertising can, be a little
overwhelming. Tools should not be looked upon poorly for this, but they should provide a way of toning things down. (If you require help with this in CodeRush, just ask)
I have noted that there are some tools, whose UI simply refuses to get out of the way, and I have found that this makes those tools so much of an annoyance, that I am forced to look elsewhere.
Navigability:
There are several ways of navigating around your code. Such ways should be simple upon first glance, with scope for the power-user to learn more than just the surface detail.
(Example : CodeRush has TabToNextReference [Tab], The References Toolwindow [Shift-F12], QuickNav [Ctrl+Shift+Q] and QuickFileNav[Ctrl+Alt+F]. CodeRush also provides ways to customize beyond the basic functionality. See my blogpost http://rorybecker.blogspot.com/2010/02/enhance-quicknav-coderush-or-coderush.html)
Refactoring:
Common Refactorings are ones you use all the time. (Rename, Introduce Local, Extract method).
All refactorings should save you time. It should take far less time to complete their function compared with how long it would manually take to execute same.
However they should do much more that that.
They should be insanely quick to use, and they should not require any deviation / context switching from what you were already doing. When you have to think about "I have to rename here and here and here" or you are dragged away from the editor to fill out a form, you are context switching away from the problem at hand, and you are going to have to context-switch back.
You should not have to fill out a form of any kind in order to use them. After all, imagine if you had to hit OK after every 3rd or 5th [enter] keystroke. How annoying would that get?
When you're in "the zone" doing these kind of basic refactorings, they become every bit as important as the simpler things like saving a file or changing the document you're looking at.
There are many areas that can take advantage of refactorings... How many of these do you use day to day? How many are covered by each tool you are evaluating? VB, C#, Javascript, Html, XML, SQL
CodeGeneration
Code is generated in many ways.... I'm not talking about massive globules of code that are generated to form your data access layer. Those are certainly possible, but generally there are specialist tools for that sort of thing. Instead I'm talking about 2-3 other areas of code generation.
Simple Code Generation
The first sort is kind of like a refactoring but, unlike a refactoring, this sort of code generation most certainly does have side effects. For example... Adding a setter to a property is very useful, but it certainly is not considered a refactoring.
Consume-First Declaration
A specialization of the above. The CodeGeneration is activated whilst the caret is positioned within to a reference to a class, method, enum etc that does not already exist. The CodeGeneration then analyses the context (meta data already present in the code) and uses this to declare the actual class, method, enum etc.
ie Creating a method, from a call to said method. Deriving, in some cases, everything about the signature from the meta data. Method Name, Param Names, Param Types and Return Type can all be calculated, given enough contextual guidance. (This is present in many cases).
- Templates
The second kind of CodeGeneration is that of Templates. These are like Visual studio snippets but with more intelligence. In CodeRush many are implemented as mnemonic key sequences.
For example vs[space] creates a 'variable' of type 'string' and pre-highlights the name of the variable so you can over type it with what you really want to call it.
This template is intelligent. It will expand differently depending on context. (In VB) If expanded inside a method the expansion will be "Dim VarName As String" with VarName being highlighted. However if expanded within a class the expansion will, by default, be "Private VarName As String". The default scope for fields can of course be altered.
[Note: Generally speaking Templates appear cross language... Templates learned in C# are duplicated and will be found useful in VB and vice-versa. This can make it far easier to transfer between languages, making far fewer mistakes along the way.]
Clearly in the above example, 's' maps to 'string'. In CodeRush there are > 60 different predefined mappings of this sort (in just the standard list and not including generic variations) and you can add as many more as you like.
This mnemonic style of naming really comes into it's own with templates like ms[space] ('method' returning a 'String'), md8[space] ('Method' returning a 'date'), ml.d8[space] ('method' returning 'list' of 'dates')
The mnemonics system aids you by giving you a simple set of rules to access a seemingly infinite set of combinations. These associations are all logical and are also on hand via a companion training tool. (The CodeRush Training Toolwindow)
Performance
Good performance is clearly a requirement. Any tool will, by it's very presence, have an impact on performance. Any one who says different, is wrong. The trick is to find the balance between having as little impact on performance as possible, whilst providing as much functionality as can be squeezed out of said performance.
Here are a couple of links to blogposts (one mine, the other by Mark Miller - Chief architect of CodeRush) detailing performance and resource usage improvements that have been made in CodeRush.
http://rorybecker.blogspot.com/2009/06/coderush-920-speed-and-memory.html
http://community.devexpress.com/blogs/markmiller/archive/2009/06/17/performance-and-memory-milestones-in-coderush-and-refactor-pro-preview-of-9-2.aspx
Take special note of the sizes of projects used in testing, and see how your own projects compare.
You might also choose to compare things like memory usage, VS startup time and Time to completely load a Project (give each tool 2 attempts minimum at this as they may perform special analysis the first time a project is opened). All of these can be a factor in your daily usage of such a tool.
Customization
Ideally there should be a wide range of options available for you to tweak to your liking. Standards exist, but they should not be "the be all and end all". You should be able to adjust things to your liking.
So the question becomes, how easily can the settings of your chosen tool be configured to work the way you like to. (This is one area in which you should again seek the help of the respective communities)
Extensibility
No tool gets everything right. They are constantly producing new versions with new features and improvements to existing ones.
For me a large part of what makes CodeRush as good as I feel it is, is the extensibility provided. Sometimes I might require functionality which simply doesn't make commercial sense for a tool maker to support directly. Perhaps it is too specialist for enough people to care about.
In these cases I find it useful to be able to add to the existing tool in order to create the functionality I need. CodeRush provides the DXCore, (the framework upon which itself and RefactorPro were built) upon which to build such 3rd party plugins.
I have built a number of these plugins and they are available (free and with source code) here -> http://code.google.com/p/dxcorecommunityplugins/wiki/RoryBecker
Support
I doubt there is anyone who would argue that support is critical to any product that hopes to have any sort of longevity in the market. Look for Email Support, Forums, Issue Tracking, Blogs and the ability to access the people behind the products, should the need arise.
Community
As critical as official support is, it can rarely compare to the support you can get from
other developers in the same, or similar position to you. Both of the main tools on the market have large numbers of users who work with the tools themselves every day. I would argue that CodeRush has a better 3rd party plugin community, but this is simply my opinion... You may decide that this area does not matter as much to you.
As I sit here writing this, I realize that I have already spent way too much time on it. I will stop for now, and If I think of anything else I will come back and add relevant content.
Conclusion
As stated in the summary, you should (IMHO) look to compare:
Interface & Intuitiveness, Navigability, CodeGeneration, Refactoring, Performance, Customization, Extensibility, Support and Community.
However you should also look to the community for advice and help with a tool. So if something is bothering you... See if the company or the community has a solution for you.
Crowd sourcing FTW ! :D
If you have any CodeRush related questions, feel free to contact me using any of the method specified at the bottom of my author page on the community plugin site. -> http://code.google.com/p/dxcorecommunityplugins/wiki/RoryBecker