There are projects that become industry-wide references and then there are those that are dusted away beneath the carpet. The bigger solution providers have a higher rate of success than smaller solution providers simply because they have a very well defined program on how to deliver certain projects. They have a gold mine of internal documents/trainings/mentoring and have a better rate of success.
Being part of a firm that is regarded as one of the biggest and most prestigious consulting firms – the big blue, I have had a lot of experience in managing EDW/BI implementations. I have learned first hand on what are the key points-of-failure and I share them with my readers.
Ownership by Client/Insufficient User Involvement
The business user/project sponsor needs to take full ownership of the project. This is not an IT-driven exercise and cannot be done in isolation. The project sponsor needs to drive the process, involving the key business users from the concerned departments to define the requirements that will lead to an implementation that is actually useful for them.
Consultants are outsiders and they cannot ensure proper involvement and acceptance of the product within the client organisation. However, it is the responsibility of the consultants to maintain the project sponsor’s involvement and to ensure that key users are involved from the start. The project sponsor is only there to pull the strings that will make the process work, it is the consultants role to ensure that they get the maximum output from the project sponsor’s interest.
Once the EDW/BI offering goes Live! the business users need to feel comfortable with the application. Sometimes, the figures don’t tie up a 100% with the existing systems. This could be based on the redefinition of business rules, or data integrity issues and the users need to understand that the figures presented by the new system may be different but not incorrect. They need to start trusting the new system in order to base their decisions on it and then only will be truly be a successful implementation.
When organizations are presented with the idea that their enterprise-wide data will be consolidated and linked to each other and they can do analysis on that data in ways that were inconceivable with the older reporting systems, the wish list has a tendency to explode. Taking on a large scope from the beginning will only make an already challenging task more challenging. Although I do not maintain that it is not possible, but it will make things a bit too difficult, increases the delivery time and hence there is a greater possibility that interest in the new system would’ve fizzled before it goes live.
Data integrity is a core issue with EDW/BI implementations and it cannot be taken lightly. There need to be rules for processing the data as it comes in from the source system. What sort of mistakes/integrity faults can be tolerated and what gets discarded needs to be addressed. As more source systems are added (larger scope), the more difficult this exercise will become.
The volume of source systems present their own risk, however the overall quality of the data (even if there are fewer data sources) present another challenge. An exercise must be taken in order to judge the overall quality and integrity of the data. If this exercise is done initially it will assist in defining business rules and filters to the data. This is to ensure that the EDW is populated with data that meets a certain level or standard. If the data is simply pushed into the EDW without any cleansing or filtering, there are bound to be problems when it comes to the validation of the results of the BI reports with the existing reports. If the user loses confidence in the authenticity of the data, it makes it more difficult for the system to be adopted.
This issue isn’t restricted to EDW/BI implementations. All poorly funded projects have a higher probability of failure. If there is a strict budget to work with, it would be wiser to lessen the scope than make false promises and fail to deliver. Data quality can easily devour a huge chunk of the project budget if the quality is specially poor. A lot of effort will need to be put in to ensure there are comprehensive business rules, the data will then be cleansed based on these rules and the final output validated. If it doesn’t add up, the discrepancies will need to be investigated and the offending data items identified. Then this discrepancy will either need to be rectified (more business rules) or accounted for (the difference can remain in the final result). So be sure to plan really carefully when it comes to assigning the budget.
Even though the consultants may have secured a project sponsor, but sometimes there are rivalries or rough edges between the different businesses of an organisation. This can often lead to deadlocks that will be sure to bring your budget to the straining point and often may be the singular cause of failure. The consultants need to ensure that all rivalries are put aside by selling the ownership or the benefits to the business head. Once a business head realises that he/she will benefit from a successful implementation, they are more likely to put aside any differences and work cohesively.
Poorly defined or incomplete KPI’s
The business user is the person who is able to define what KPI’s he/she will need in order to perform their job. The business user must be the driving force behind the definition of the KPI’s. It is often hard to get them to think out of the conventional model, but since the EDW/BI implementation is a fresh system, they should have a key role in defining how they want to perform their analysis. Most of the legacy systems were built to identify certain needs and there was no consideration of analysis regarding the business indicators.
Complex Data Model/Cubes
KISS – Keep It Simple, Stupid!
The more complexity there is to a system, the harder it will become to maintain in the longrun. In some cases, the KPI’s will need to be readjusted based on a better understanding of the requirement (and a simpler system is easier to rework than a highly complex one). Also consider the Large Scope point – keeping a limited scope will naturally tend to limit the data and hence its complexity.
Strong coupling between EDW and BI environment
Do not try to make the EDW structure in order to support the current choice of BI or Reporting or Analysis tool. The user may want the data to be available in excel. The BI tool may be scrapped for another one in the future. Make sure the EDW is designed independantly of the choice of the BI tool. Different business users may have different needs for analysis and there may be several ways that certain data may be required. Keep the data in structures that are generic and relevant to the business definition across a broad range of business users instead of limiting the scope to a specific tool or requirement.