Compromising with non-programmers

Question

The data for the SQL db used in my PHP code comes from an excel file made by non-programmers. When I can't manage to explain an issue that I am having with their excel file, I usually just try to code around it. This has lead to some pretty patchy code on my part. Does anyone else have experience with this? Is it generally better to try to code around others or is it better to be firm and demand a more robust table structure?

Answer 1

You should demand that the data is correct. Do not code around something that is ultimately wrong.

Answer 2

As a SQL professional, I can assure you this is completely normal. I regularly have to deal with C# developers who do a poor job of database design and querying. Heck - I have to deal with SQL professionals who do a poor job too. (And can admit to having done a poor job in the past too).

I would try to encourage a culture of learning and self-improvement. Look for opportunities to refactor, and try to schedule a regular time (a couple of days per month) for refactoring, assuming you have decent testing in place to make sure that functionality doesn't actually change.

Answer 3

That depends, who's time is more valuable? This is not as straightforward as you may think. In the end, this is a business decision, not a technical one.

If you have 10,000 users, the user's time is probably more valuable in aggregate and you should code to accept as much as possible, since that will save them time.

• If your application is publicly available, this is a competitive advantage: being user friendly.
• If your application is internal, this is cost containment: total cost of ownership.

If you have 5 users, and they are not particularly important, go ahead and demand whatever you like, it'll take them less time in the aggregate to fix their data than it will for you to code around their issues.

If you have 1 user, and it's the CEO, you best be accommodating because their time is way more valuable than yours.

Answer 4

You could all agree a standard format and agree what should happen if data is not in that standard format (i.e. it doesn't get imported into the db until it conforms to the format).

How many users provide data to you? Would a tool that validated the format of the excel files be a quick win? Users could validate their files and be informed if any rows or columns don't comply with a standard format.

Answer 5

If you decide to be firm and demand a more robust table structure, you will need to provide the non-programmers with clear, concise, and unambiguous documentation, as well as good error reporting tools to allow them to more quickly and easily figure out when they have made an error and what it is. There is nothing worse than having someone come yelling at you that "you need to do things this way, or the program won't work" (or even saying it nicely but firmly), and then three months later having to do the task again and having no way to figure out what the criteria are without going back to ask the programmer.

So, sometimes it is easier to just code around the bad data. It also helps make your program more flexible and robust. But if you're starting to bend over backwards to do it, then make sure there are easy ways for the non-programmers to check for errors and that the messages are clear and unambiguous, and there is good documentation on what is expected of them for the input.

Answer 6

So far, there was once a client I asked him to follow a set structure and explained him that that structure specifically is needed because the program has been set some rules to follow. (It was a CSV file after all which he managed from excel).

Coding your way around it just going to be a nightmare for you. Just agree an standard like Russ said.

Answer 7

You should provide a clear specification or interface, and only accept data according to that specification.

In the case of some excel template, you should try and make it as foolproof as possible for those on the data entry side. This can involve locking rows, providing macros with bespoke GUIs for easy entry etc.

You might want to demand something a little simpler like a form of CSV.

If you can, you should try and validate the data as best you can before importing into the DB.

Certainly, it's not a good idea to work around these issues - and sounds like a waste of valuable coding time.

By the way - this seems to be a dupe of http://stackoverflow.com/questions/1996752/tips-for-communicating-with-non-programmers (although perhaps phrased a little differently).

Answer 8

Yes you should demand correct data.

But...you can help them. Create a tool that will validate the data that they can use before it gets to you.