Who is doing investigations into measurement of functionality and usability?

Question

I have developed some ideas about system construction, that are a little different to mainstream development. I am developing a demonstration/proof of concept project.

So far, I have found that useful information about the completeness of a system can be gathered by taking an indirect view of the development. This indirect view is based on the description of the problem, rather than the software solution. Sample questions (about the PROBLEM description) that are useful in this assessment are:

Can all errors be classified as type errors; context errors; or lies?
Does every class type have attributes?
Does every attribute have a data type?
How many conditions are complete; how many are incomplete; how many are overlapping?

I intuit that similar assistance can be provided with regard to usability (specifically the user interface), functionality, and coding style. I am seeking details of work/research being done in this area.

Continuing work on, comments on and answers to my original question have refined my views as follows:

The references I am seeking may come from a variety of disciplines: psychology; aesthetics; neuro-linguistics; neuro-aesthetics; and ....?
Possible avenues to pursue are the IEEE/ACM and Richard Pawson's work on Naked Objects.
The aspects I am investigating may be impossible to achieve by direct means, and so indirect means need to be sought. These indirect means could well be heuristic; relative; must be based on artefacts arising from the development process (massive metric gathering projects are NOT required).

These criticism's have some incidental value; not least forcing me to clarify and further specify my questioning. Negative responses are more useful if they point me towards negative work (ie that supports the assertion being made). I am trying to provide help and assistance in some aspects of system development in some circumstances. I am not trying to be a theoretical back seat driver; nor do I pretend to have any silver bullets.

Finally, a reformulation of my question is:

Can you provide references to work being done on the measurement of functionality, usability, or coding style? Who (person/organisation) is doing work in this area? Where can I find such references.

Answer 1

All of those concepts cannot be measured. They can't even be objectively agreed upon.

I dare to say they have no clear interpretation in the physical world. They only exist in human mind. Every other person will perceive and measure those in accordance with their life experience, knowledge, experience and attitude to the problem field, development of engineering, artistic and interpersonal abilities. Evey if you can force some individuals to "measure" it, it will be highly subjective.

How do you define beauty and love, joy and sadness? Usability and efficiency will have much to do with those.

Some ideas can come from psychological studies. But only a few ideas. At best you can apply this knowledge to try to evoke some specific response in the users' minds. But it might work or it might not.

When you can't understand precisely the model of response of the users, you can't plan for specific response. Consequently, you can't measure the degree of a particular characteristic of your program. So you can't define the scale in the first place.

Answer 2

I think some of these features would have to be measured after the product has been in use for quite some time, as compared to another, similar implementation.

Consider the example of multiple GUI implementations for a piece of software. You could measure things like how long it took a user to accomplish a certain task using one specific implementation relative to accomplishing the same task (for the nth time) on the (almost) same piece of software using different GUI implementation. This would provide for some sort of a useful metric in relative terms.

Going along this path might help you clarify these ideas in terms of getting (publishable?) results. From reading your original description, it sounds like you are looking for absolute rather than relative metrics. However, from quickly looking over this question and trying to come up with an interesting, useful, reply during a five minute break from my work, the relative metric was the best I could come up with.

I hope this helps,

Brian J. Stinar

Answer 3

I am answering my own question to give an indication of how far I have got in obtaining an answer to my original question. It is apparent that any metrication of concepts related to elegance will share the following characteristics:

1. they are likely to be heuristic
2. they are likely to have to be based on multiple iterations of the development process
3. they are likely to be relative rather than absolute
4. they are likely to be based on areas of study other than software/system development, in addition to needing information from the system development itself. Examples of possible domains are aesthetics, psychology, neurolinguistics and neuroaesthetics;
5. If the work I am doing is to be of any value to software developers, it must not be based on complex, mathematical or statistical models, but must offer guidelines based on their work, and work by other developers.
6. Any guidelines are likely to be based on the answers to questions like:
   • What is the frequency of objects, attributes, data types?
   • How does this frequency compare with other projects?
   • Is this measure of any value in aiding the developer to evaluate his project/see room for improvement?

I am still seeking references to work being done in this arena.