Is this a proper idea of BI workflow ?

Question

Hi all,

I am new to Business Intelligence.

I just got hired by a company in order to complete their websolution, implementing a BI Module. After lot of reading, I think I could get an idea of what a BI Process looks like, you'll find enclose my idea of a BI process.

Can you please tell me if this is a correct vision of the all workflow ? If not please correct me. Another question, I can't see the place of data mining in the schema, where should I use it if needed ?

Thanks a lot,

Answer 1

Below are a couple points you may want to consider:

1. Is it correct that the presentation layer is either Excel or a web service and nothing else? What if you wanted to deploy a web reporting platform such as SSRS, Cognos, Business Objects, Microstrategy, etc.? These platforms generally access the data directly without web services.

2. You may need to output additional data to flat files and downstream databases. The data may be based off either the OLAP cube or the raw source data. Therefore, you may want to include these as destinations at the main data warehouse level and the OLAP level.

3. If you implement a web presentation layer, it may not only be over an OLAP cube. It is likely you will receive a request to report on data that isn't contained in an OLAP cube and it may not be worth the effort to add it just to support a few reports. If this is the case, then you will probably want to note that the main data warehouse could be a source for a web reporting platform -- not just OLAP.

4. Data marts don't generally contain aggregated data -- they generally contain detail-level data. The OLAP cube performs aggregations over the detailed data, but the underlying data mart itself contains transaction-level facts.

5. You could have additional data sources that are imported by the ETL process such as web services. You may also want to stress that there are multiple databases and flat files just as you stressed there can be multiple data marts and OLAP cubes.

6. You don't necessarily need separate data marts for the OLAP cubes. If you are building one central data warehouse, you may be able to create one central OLAP cube and limit access to dimensions and slices of data based on cube security roles.

7. You may want to indicate that you need to load a staging database prior to loading the data warehouse. Perhaps that isn't the case, but generally I recommend having a separate staging database so that you have a place to load before you merge into the data warehouse. Some ETL processes can go directly into the data warehouse since they can merge on the way in, but there are plenty of instances where it probably makes sense to load the data in a staging database before merging it into the final destination. This also means ETL / Stored Procedures probably should be its own entity in the diagram since it is a method for moving data more than it is a destination or object. If you look at your diagram, all your entities are actual objects and not processes like MDX.

8. You may want to indicate data is loaded into data marts via an ETL process. ETL isn't just about loading the data warehouse -- it is about moving data at any stage. Even processing cubes requires an ETL process.

9. Your OLAP server doesn't necessarily have to be on another physical server -- it could be a service on the database server. It just depends on whether you tackle performance issues by scaling up or scaling out or if you intend to publish cubes to multiple OLAP servers because of geographic performance issues.