Building the Enterprise Data Warehouse (EDW)
The EDW Can Be Implemented by a Top Down or Bottom Up Process
An EDW is composed of multiple subject areas -- finance, human resources, marketing, sales, and manufacturing. In a top down scenario, the entire EDW is architected, and then a small slice of a subject area is chosen for construction.
Subsequent slices are constructed, until the entire EDW is complete. From the perspectives of time and cost, a typical EDW project will take three to four years to complete, from initial concept to final launch date, at a cost in the $3-4 million range, for a mid-size organization. Larger, global organizations should expect to spend $10-50 million to build the initial EDW.
EDW Phased Subject Area Development
The top down approach was the first method to be used in the development of data warehouses, and remains the best and most sophisticated design process for a data warehouse, although there are several challenges in the successful completion of such a large project, including:
- Competing internal business requirements
- Tight deadlines
- Out-of-date source systems
- Varying and conflicting user demands
- Ineffective or faulty business analysis tools
- Instilling changes in corporate culture
These challenges need to be addressed and managed in the decision-making stages, where the future of an organization’s data warehousing and ultimate CRM strategy is decided. Proven technologies are available to ensure its success, if reasonable and rational planning, evaluation, selection, and development processes are followed.
The EDW project is enterprise-wide, and by its nature requires that the EDW team cross many corporate boundaries: departmental, political, functional, departmental, process, ownership, organizational, and geographic. As well, meeting these challenges successfully requires a development team with a significant level of technical, business, and political acumen.
Top Down Approach to EDW
A top down approach provides two major advantages from an implementation perspective:
- Coordinated environment
- Single point of control & development.
The disadvantages of a top down approach are:
- Time required to complete project
- Too much time spent analyzing problems
- Difficult to control the project
- Enterprise-wide nature of the project
- Too much risk
Bottom Up Approach to EDW
Some corporate data warehousing teams developed a bottom up approach to achieve the objectives of an EDW. In the bottom up approach, a smaller enterprise data mart architecture (EDMA) is developed to provide a context and foundation for further development.
Once the EDMA is complete, an initial pilot project is undertaken with one business department or group, for the first incremental data mart, to validate the process. The EDMA is expanded in this area to include the full range of detail required for the design and development of the incremental data mart. Subsequent phases fill in the EDMA, until the team and the organization is ready to construct the EDW.
This bottom up approach has an important benefit to the organization, because its allows the project team to develop the skills and techniques required for data warehousing in a much lower risk and exposure environment than a full scale EDW project.
Incremental ADMs greatly assist in keeping the team focused, whereas EDW teams tend to attempt to solve tomorrow’s problems today, building generalist systems which may lead to many project delays and slipped deadlines. In contrast, incremental data marts are built to solve specific problems for the business, focusing on one business problem at a time.
Advantages of a bottom up approach are:
- Offers low risk and low exposure
- Yields incremental design
- Requires lower level, shorter term political will
- Can be delivered quickly
- Enables a focused approach
- Provides faster ROI
The disadvantages of a bottom up approach are:
- EDMA required to integrate incremental data marts
- Success may swamp allocated resources
- Needs multiple team coordination
Both EDW development processes have their pros and cons and should be evaluated in terms of the overall corporate customer strategy.
Building the Data Warehouse
|
What do you think about this article?
More in Technology
Latest Articles