Building a data warehouse is a process with no exact completion date. Unlike conventional databases, which tend to be stable for a long time after deployment, a data warehouse is changing to meet the changing requirements of the business for which it was created. There are 7 main stages to build a data warehouse.
Step 1: Define Requirements – Discover
It’s safe to say that any database project of tangible size and complexity will fail if you skip a step or make a mistake during the Discover phase. At this stage of building a data warehouse, is carried out the analysis and definition of requirement. And also are taken into account the business tasks. It is especially important for the projects, when are created data warehouses. At this stage, a comprehensive study is carried out, within the framework of which provides answers to six main questions are formulated: What? Where? Who! When? and for what, and how to build data warehouse?
Stage 2. Design – Design
The process of development of the semantic and schematic models of the data warehouse is at the forefront of the phase of design. These models should be consistent with MIS objectives, important for business users and reflect the analytical needs of BI. In case of a data warehouse project, the users can build conceptual and logical data models. Also there are spatial models to represent multidimensional cubes. Decision matrices can be used to better define the requirements that should be included in each spatial model.
Stage 3. Development – Develop
The Data Track Develop phase consists of two main parts. In the first part, the mapping of data models to the corresponding physical constituent parts of the project (relational data warehouse models and OLAP cubes) is built, if necessary, the sizes of databases in data warehouse example are set and tables are split, naming conventions for data warehouse objects of business users and technology users are determined. In addition, the data indexing strategy is developed and the indexing order is determined.
Stage 4. Implementation – Deploy
Usually, transactional databases from Dataart.com are deployed immediately: on Friday night, users were in the old system, and on Monday they can already register in the new database. Unlike transactional databases, data warehouses are being deployed , reaching different user groups in the enterprise one at a time. The rate at which coverage grows and the order in which individual user groups are accessed is part of the deployment plan during the implementation phase.
The deployment of a data warehouse occurs in a rapid, cascading process across tiers. First, the technological part is put in place – servers, storage, communication channels, etc.
Stage 5. Daily Operations – Day to Day
Day-to-day management is essential, and yet it is often overlooked in planning and deployment. Not only is it necessary to ensure that regular maintenance (daily, weekly, etc.) of the software and hardware is performed, but it is also necessary to monitor the performance, operation and growth of all systems. As noted at the beginning of this article, work on a data warehouse is never finished.
Stage 6. Defense – Defend
The process of securing a data warehouse includes more than backing up and verifying the protection of SQL queries. It’s in the software used against SQL injection attacks. You need to plan for holistic and comprehensive protection because the data warehouse contains the most valuable information about the enterprise in a compiled, clean format.
Stage 7. Decommissioning – Decommission
There comes a time when the data warehouse. Or some of its components (it can be a staging database, a data mart, a reporting database, a cube) no longer meets the operational requirements. From this point on, Phase 7 begins, Decommission. Not every database can be reworked or upgraded to the new changed conditions. In some cases, it is easier to abandon such a database and build it all over again. Especially if the database was created to solve urgent problems, that is, without proper design and taking into account the goals and level of development of the enterprise. In this case, the project manager is responsible for the consistency of this process.
Effective cycle
In the process of working with the components of the data warehouse, new cycles of determining the requirements can begin, associated with the emergence of new tasks and requirements for processing the accumulated data in the warehouse. These new requirements can lead to changes and expansion of design tasks and solutions at one or more levels. It is necessary to integrate all changes in the existing data warehouse to deploy new solutions that meet user needs, while taking into account the merits of the existing data warehouse. New requirements can change day-to-day operations, so the storage is like a well-oiled machine.