What criteria should be used to judge and compare Java applications?

Question

I have to support a Java Programming Challenge for students and we have to come up with some technical criteria to judge their java application. What is the best way to compare and judge an application, based on the code and the documentation?

Edit to give more details about the challenge: The students are supposed to develop an application of their choice, based on JavaME for one of our devices (Multi-Function-Printer). The app will run on the device and should make use of the functions provided by our SDK/J and will be judged from marketing and support (my department). They will use an emulator for the first step and will be provided with a real device if they qualify for the finals.

Answer 1

If the challenge has a goal, such as writing a game playing program of some sort, then clearly the first criteria should be how well the goal was accomplished by the program under review. The second criteria (certainly for student projects) should be the readability and structure of the code. Is there any evidence of copy/paste? Then why wasn't that section turned into a function/subroutine/procedure?

Extra credit points are for code that uses particularly novel methods (but if they're non-obvious, they must be documented sufficiently for the author's peers to maintain it!) to improve efficiency or functionality.

Answer 2

You can judge them on many different aspects, I would particularly look for:

Using methods where needed

Students shouldn't be copying and pasting functionality; they should know when to create a method, what to include in it.

Usage of data structures

I've seen people use an array to store what should be member variables. For example,

foo[0] = 'icewolf'; 
foo[1] = '119'

Where icewolf is your username and 119 is your reputation score. Student should instead use

String username; 
int reputation;

or something similar.

Interfaces, Abstract classes, Concrete classes

Students should understand when to use which. If this is a web application and they find themselves implementing the doPost method same as doGet in each class, then they should probably create a superclass in which doPost just calls doGet by default.

Readability of code

Discourage students from writing code to explain every trivial statement. Encourage students to document non-trivial statements.

I'd pay a great deal of attention to their design and completion. I personally would give more points to an application that has a cleaner, easy to understand design than one that was full of features but all over the place.

Answer 3

Well many of this depends on what are they studying and what can you expect them to deliver.

If they were studying to become programmers I'd have them produce a set of requirements and a design document where the address the chosen ways to cover those requirements. Also make them provide a set of tests with the code.

After that,

• Does the functionality satisfy the requirements?
• Is it well designed (OO-wise)?
• Are the algorithms well chosen?
• Are they well implemented?
• Is variable/method/class naming done correctly?
• Do the tests have good coverage?
• Does it use properly Java's features? (generics, and so on)
• Is the code well commented (non obvious and descriptive)?
• Does it pass the test suite?
• Is the code readable?
• Is it maintainable?
• Does it solve a relevant problem?
• Is the application interesting?

Mix and match to your liking

Answer 4

I think the answer depends upon what Java Programming Challenge was.

If all the projects are trying to solve a similar problem, can you automate testing, and do some sort of comparison based upon performance?

If the projects have to conform to an API, are there some automated tests you could run against each project, whoever passes most wins.

If it was an open ended challenge, then you should be able to just pick whichever is coolest. Get all the projects put on a website, and see which gets the most hits.... Sounds a bit like a geek-factor without the 0898 numbers.

I would guess a fair, but time consuming test would be for you to sit and code review the projects.

Whether it was an open or closed challenge, there are certain tools you could run against the codebase to measure code quality. eg JDepends, Clover (unit test coverage) etc. however these are quite subjective.

Answer 5

You could use a lint-checker like CheckStyle, PMD, or FindBugs. Create a profile that you feel their code should match and see how many violations you get.

I think any objective measure has to go with some subjective ones, too, as many of the other posters have suggested.