How verbose should validation output be?

Question

I have an application that reads a database and outputs alerts to any dependencies that are not being met. My thinking on this issue is "Give the minimum information that points the user to the issue." I have been told by a co-worker that I should be as verbose as possible, printing out the values of the database fields for each field I mention verses giving the minimum message that "field one needs to be less then field two".

I know that there must be some convention or standard for this issue as it reminds me of compiler errors and warnings. Does anyone know how a compiler messages are are chosen?

What suggestion does the community have for this issue?

Answer 1

I think the key is to be concise. Put as much detail as is required for the reason for the warning to be communicated and nothing more.

Answer 2

I would suggest that you should implement both modes. During normal operation you need a useful but short message. But sometimes things could go wrong and in this case a 'dump' mode which gives the user all possible information is a life saver.

Answer 3

When writing, know your audience.

If you're logging warning/error messages for your own consumption, then it's fairly easy: what do you need to know when something goes wrong?

If you're logging warning/error messages for someone else, then things get tricky. What do they know? What does their mental model of the system look like? What sorts of problems can they solve, and what information do they need to solve them?

Pushing every last scrap of data into a message is punting - at best, the reader will have to wade through irrelevant information in order to find what they need; at worst, they'll become confused and end up making decisions based on the wrong data.

The compiler analogy is apt: think how annoying it would be if the entire symbol table was dumped along with every warning...

Answer 4

Dealing with errors Vs. warnings first: An error should be for something which violates the standard. A warning should be for something which is allowed, but quite likely isn't what the author intended.

For example, the W3C Markup Validator will warn about the use of the syntax <br /> in an HTML document. In XHTML this means "A line break", but in an HTML document, while being allowed, actually means "A line break followed by a greater than sign" (even if most browsers don't respect this).

As for verbosity, what is best does depend on who is using the system. Some users would be better with brief messages that they can skim through, while other users (perhaps those less advanced) would find the additional information useful. Without knowing more about who they are, I'd tend towards using a flag (-v is traditional) to let the user select which version they prefer.

Answer 5

I think there are 3 levels of the details of an error message for the 3 typical user groups:

1. The end user. This is a surfer on a web site or an user of a desktop application. He should receive an error message if the problem can not be compensate. It should include the minimum of information. The end user should not receive any information over the system like current configuration and file paths. The end user should contact the administrator. A continuous error id can be helpful that the administrator can find more informations.
2. The administrator need more helpful information to solve the problem self. It can include information like table xy not fount or login to database failed.
3. The developer: If the administrator can not solve the problem then it will contact the software vendor. In this case the administrator should be able to send a log file that the developer can solve it also if he can not reproduce the problem.

Answer 6

For normal, day-to-day operation, I give a data validation message that gives enough information that the user can fix the problem, so that the data validates. For example, if I have two fields (fieldA and fieldB) and one of them have to be greater than the other, then I would state that on the validation output, specifying which field is the offending field.

For example, if A has to be greater than B, and they supply an answer less than B, then the message would be "fieldA needs to be higher than fieldB"

That said, I also program a debug mode into my applications (especially the web-applications) which has a verbose mode, telling exactly what's happening with everything. If that's turned on you would see two messages, the user-friendly error, and then "FieldA=XX and FieldB=YY: XX is not greater than YY".

That's simplified, but it's the general idea.

Answer 7

The specifics of the content of a log can be discussed, but it is my experience that the level of verbosity will quickly determined during stress test.
If the system can not function properly, it is because you just:

• get either too verbose with your logs, or
• did log too often (actually, I believe Jeff himself had a similar problem)

Atwood: We were logging in such a way that the log.... during the log call was triggering another log call. Which is normally okay, but with the load that we have, eventually they would happen so close together that there's also a lock. So, there's two locks going on there.

Spolsky: [...] you have a tendency to wanna log everything. But then you just get logs that are, you know, a hundred megabyte per user and you get thirty of them a minute and it can't possibly be analyzed or stored in any reasonable way. So the next thing you have to do is to start culling your logs or just have different levels of debugging, where it's like in high debug mode everything is logged and in low debug mode nothing is logged. And... it's kind of hard to figure out what you really want in a log.

Atwood: I mean that, ironically, to troubleshoot this hang, which turned out to be because of logging, we were adding more logging.

Spolsky: [laughs]

Atwood: The joke just writes itself! The joke just writes itself, right...

So my point is, when you will run your system in a production-like environment, you should quickly be able to determine if the level of verbosity you choose is sustainable.

Answer 8

I've blogged about my personal opinion on how to log. I hope it can be of some help to you.

Thorsten