Working with a totally messy DB?!

Question

You work for a company for a few months. You are working with a new application by yourself, application which is nicely built, that has a nice database, everything tied up together with constraints, foreign keys, etc.

Suddenly you are asked to integrate your application to a very messy database. Messy in the level of:

• Not having a single Primary Key;
• Not having a single Foreign Key;
• Having duplicate records in all tables that have N-N relationship with each other.

In plain words: a total crappy mess.

The messy database is widely used by the company which inserts many records everyday without any data integrity enforcement, meaning they mess up every single day with the data not knowing they do so. It simply works for them, then no bother!

You are a small fish in the company and don't want to try causing a revolution making the efforts be directed to fix the database first... Your politics skills are 0!

Then you finally do some analysis and find out you'll not only have to try and rely on the messy database for a few things but insert maybe even more records that you can't know for sure isn't redundant, and even if it is, in the future someone can just insert it once again... I mean, the solution is possibly accomplish-able but you are unsure you are entering in a eternal maintenance required machine...

How would you position yourself if you can kind of avoid having to handle with these deep mess?

Would you simply DO what they ask? Would you try to quietly walk out of it? Or would you try to make it clear the database is f'ed up and WILL have to be addressed in some point?

Answer 1

If I was in the situation I would do this:

1. Collect data showing what the issue is and how it is negatively impacting you and the company. I am assuming this is something that will have a negative impact on reports at the very least.
2. Figure out how you would solve it and roughly how long you think it would take.
3. Define what the company will receive as a benefit from you doing this update.
4. Put it all together in a report and present to your manager.
5. If they decide it is a good idea to follow your suggestions then you are in the clear!
6. If they do not decide to follow your suggestions don't get upset. Instead tell them thanks their time and end the meeting.
7. Now you have to make a choice. You can either stay or leave. Leave if it is to much to deal with and you can find a new job. Stay if that isn't the case. Remember if you stay and you have to do extra work due to the decision of your boss then let them know how much extra work they are causing.

There really is no easy answer. Just remember don't get emotional with them about it and let the facts speak for themselves. It is tough for a sane person to refute good logic. Bosses and companies want to save money and time. If they don't then maybe it is time to move on.

Answer 2

You might get closed for asking a very subjective question, but my suggestions:

1) Work on your politics skills. You should at least be able to make the business case as to the problems their current implementation poses and (just as importantly) what it would take to fix it. Even if the answer is "no" to addressing it, laying out the problems and possible solutions (and ideally, a guess as to level of effort) puts it on the company to make the decision.

2) In the meantime... you have to try and work around it/abstract your way through it. There's no silver bullet and no single correct answer not knowing the exact interaction between apps, but it could be any combination of -- finding distinct/max records, making some small changes to the "bad" database that reduce the amount of duplication etc. of data.

Answer 3

I would say, mention that the database is a problem, but try not to complain about too much. Everyone probably knows, and they are sick of hearing about it.

Plow through as best you can, but the busted up state of the database puts you behind schedule or introduces other problems, honestly point to that as an impediment. Don't jump at the chance to blame the bad database, but if it genueinely is a problem, say so.

I guess the short answer is, be honest and up front, with both your managers and yourself. If something is a problem, say so without bias. If something can be dealt with, deal with it, and don't blow it out of proportion.

Then one of two things will happen. Either the management will appreciate your level-headedness and honestly, and you're golden. Or they will resent it, in which case you really don't want to work there anyway, and you're doing yourself a favor finding that out as soon as possible.

Just my two cents.

Answer 4

I'll comment solely upon this:

"but insert maybe even more records that you can't know for sure isn't redundant, and even if it is, in the future someone can just insert it once again... "

You're going to basically need to make sure the data you put into that database is clean. Wrap your interactions in transactions and treat everything as an idempotent upsert--as in, when you are done, it will be there and up-to-date.

Slowly, the percentage of "good" data will start to climb.

The only problem is when your "good" data has a problem due to someone not telling you something about how the data schema works. Make sure every row you insert or update is tagged in such a way that you know it's yours.

And good luck. You'll need it.

Answer 5

Keep in mind that lots of real world data doesn't have integrity. If someone gave you a list of names just because two or more are the same doesn't make them the same person. How would you handle that situation? You can't force people to change there names, but you can require them to give you an email address.