ETL for small business

Question

My day job is programming on "connector" (windows console apps) to extract data from source such as excel, csv...etc. then upload it to database and do a lot of queries to make reports for clients. Is my job qualified as ETL?

I'll like to start a service company that is involved with ETL for small business. Is ETL mostly involved with custom coding for client or can you buy retail software that can extract from different data sources, transform it then upload it to the database?

Is database schema going to be different for each customers or can you have a generic/universal? Do you have to code some type of dash board software for reporting or can you buy retail software for that?

sorry if my questions doesn't seem to make sense. I'm just trying to understand what is involved in ETL. Can someone also point me to a good beginner ETL book?

Answer 1

ETL is the part where you convert the Excel/CSV and import it in to the database. That's the ETL phase of the process.

After that, it becomes an analysis problem, and that's very domain specific.

For example, a very common ETL problem is web site statistics. Loading logs in to a form that can then be reported upon.

Turns out to be a generally "generic" problem, since web stats are pretty much the same wherever you go, but tool distinguish themselves by the kinds of reports and data mining they can perform on the imported data.

But, on the whole, ETL is pretty specific. Typically down to the business level, but sometimes theres a larger market at the industry level (EDI, Healthcare) where there's common formats that are "standard" (and, boy, do I use that term loosely).

So, you'll see tools that are industry specific in the formats they support, but then stop at the reporting level since that is almost always business specific.

Answer 2

I recently started to read a couple of books from Ralph Kimball.

Here is one you should buy

The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleanin, ...