Citizendia
Your Ad Here

A data warehouse is a repository of an organization's electronically stored data. Data warehouses are designed to facilitate reporting and analysis. [1] This classic definition of the data warehouse focuses on data storage. However, the means to retrieve and analyze data, to extract, transform and load data, and to manage the dictionary data are also considered essential components of a data warehousing system. Many references to data warehousing use this broader context. An expanded definition for data warehousing includes business intelligence tools, tools to extract, transform, and load data into the repository, and tools to manage and retrieve metadata. Business intelligence tools are a type of Application software designed to report analyze and present data Extract Transform and Load ( ETL) is a process in data warehousing that involves extracting data from outside sources Metadata ( meta data, or sometimes metainformation) is "data about data" of any sort in any media

Contents

Benefits of data warehousing

Some of the benefits that a data warehouse provides are as follows: [2][3]

Data warehouse architecture

Data Warehouse Architecture (DWA) is a way of representing the overall structure of data, communication, processing and presentation that exists for end user computing within the enterprise.

Conceptualization of a data warehouse architecture consists of the following interconnected layers:

Operational database layer
The source data for the data warehouse
Informational access layer
The data accessed for reporting and analyzing and the tools for reporting and analyzing data
Data access layer
The interface between the operational and informational access layer
Metadata layer
The data directory (which is often much more detailed than an operational system data directory).

Normalized versus dimensional approach to storage of data

There are two leading approaches to storing data in a data warehouse - the dimensional approach and the normalized approach.

In the dimensional approach, transaction data are partitioned into either "facts", which are generally numeric transaction data, and "dimensions", which are the reference information that gives context to the facts. For example, a sales transaction can be broken up into facts such as the number of products ordered and the price paid for the products, and into dimensions such as order date, customer name, product number, order ship-to and bill-to locations, and salesperson responsible for receiving the order. A key advantage of a dimensional approach is that the data warehouse is easier for the user to understand and to use. Also, the retrieval of data from the data warehouse tends to operate very quickly. The main disadvantages of the dimensional approach are: 1) In order to maintain the integrity of facts and dimensions, loading the data warehouse with data from different operational systems is complicated, and 2) It is difficult to modify the data warehouse structure if the organization adopting the dimensional approach changes the way in which it does business.

In the normalized approach, the data in the data warehouse are stored following, to a degree, the Codd normalization rule. Tables are grouped together by subject areas that reflect general data categories (e. g. , data on customers, products, finance, etc. ) The main advantage of this approach is that it is straightforward to add information into the database. A disadvantage of this approach is that because of the number of tables involved, it can be difficult for both users to join data from different sources into meaningful information and then access the information without a precise understanding of the sources of data and of the data structure of the data warehouse. A data structure in Computer science is a way of storing Data in a computer so that it can be used efficiently

These approaches are not exact opposites of each other. Dimensional approaches can involve normalizing data to a degree.

Another important decision in designing a data warehouse is which data to conform and how to conform the data. For example, one operational system feeding data into the data warehouse may use "M" and "F" to denote sex of an employee while another operational system may use "Male" and "Female". Though this is a simple example, much of the work in implementing a data warehouse is devoted to making similar meaning data consistent when they are stored in the data warehouse. Typically, extract, transform, load tools are used in this work. Extract Transform and Load ( ETL) is a process in data warehousing that involves extracting data from outside sources

Top-down versus bottom-up design methodologies

Top-down design

Bill Inmon, one of the first authors on the subject of data warehousing, has defined a data warehouse as a centralized repository for the entire enterprise. William Harvey Inmon (born July 20, 1945, in San Diego California) is recognized by many as the "father of data warehousing" [4] Inmon is one of the leading proponents of the top-down approach to data warehouse design, in which the data warehouse is designed using a normalized enterprise data model. "Atomic" data, that is, data at the lowest level of detail, are stored in the data warehouse. Dimensional data marts containing data needed for specific business processes or specific departments are created from the data warehouse. In the Inmon vision the data warehouse is at the center of the "Corporate Information Factory" (CIF), which provides a logical framework for delivering business intelligence (BI) and business management capabilities. The CIF is driven by data provided from business operations

Data warehouses versus operational systems

Operational systems are optimized for preservation of data integrity and speed of recording of business transactions through use of database normalization and an entity-relationship model. Data integrity is a term used in Computer science and Telecommunications that can mean ensuring data is "whole" or complete the condition in which data are Database normalization, sometimes referred to as canonical synthesis, is a technique for designing Relational database tables to minimize duplication of An entity-relationship model (ERM is an abstract conceptual representation of structured data Operational system designers generally follow the Codd rules of data normalization in order to ensure data integrity. Edgar Frank "Ted" Codd ( August 23, 1923 – April 18, 2003) was a British computer scientist who while working Database normalization, sometimes referred to as canonical synthesis, is a technique for designing Relational database tables to minimize duplication of Codd defined five increasingly stringent rules of normalization. Fully normalized database designs (that is, those satisfying all five Codd rules) often result in information from a business transaction being stored in dozens to hundreds of tables. Relational databases are efficient at managing the relationships between these tables. A relational database is a Database that groups data using common attributes found in the data set The databases have very fast insert/update performance because only a small amount of data in those tables is affected each time a transaction is processed. Finally, in order to improve performance, older data are usually periodically purged from operational systems.

Data warehouses are optimized for speed of data retrieval. Frequently data in data warehouses are denormalised via a dimension-based model. Denormalization is the process of attempting to optimize the performance of a Database by adding redundant data or by grouping data The star schema (sometimes referenced as star join schema is the simplest style of Data warehouse schema. Also, to speed data retrieval, data warehouse data are often stored multiple times - in their most granular form and in summarized forms called aggregates. Data warehouse data are gathered from the operational systems and held in the data warehouse even after the data has been purged from the operational systems.

History

The concept of data warehousing dates back to the late-1980s [2] when IBM researchers Barry Devlin and Paul Murphy developed the "business data warehouse". In essence, the data warehousing concept was intended to provide an architectural model for the flow of data from operational systems to decision support environments. Decision support systems constitute a class of computer-based Information systems including knowledge-based systems that support Decision-making activities The concept attempted to address the various problems associated with this flow - mainly, the high costs associated with it. In the absence of a data warehousing architecture, an enormous amount of redundancy of information was required to support the multiple decision support environment that usually existed. In larger corporations it was typical for multiple decision support environments to operate independently. Each environment served different users but often required much of the same data. The process of gathering, cleaning and integrating data from various sources, usually long existing operational systems (usually referred to as legacy systems), was typically in part replicated for each environment. A legacy system is an old Computer system or Application program that continues to be used because the user (typically an organization does not want to replace or Moreover, the operational systems were frequently reexamined as new decision support requirements emerged. Often new requirements necessitated gathering, cleaning and integrating new data from the operational systems that were logically related to prior gathered data.

Based on analogies with real-life warehouses, data warehouses were intended as large-scale collection/storage/staging areas for corporate data. Data could be retrieved from one central point or data could be distributed to "retail stores" or "data marts" which were tailored for ready access by users. A data mart is a subset of an organizational data store usually oriented to a specific purpose or major data subject that may be distributed to support business needs

Key developments in early years of data warehousing were:

Evolution in organization use of data warehouses

Organizations generally start off with relatively simple use of data warehousing. Over time, more sophisticated use of data warehousing evolves. The following general stages of use of the data warehouse can be distinguished:

Off line Operational Databases 
Data warehouses in this initial stage are developed by simply copying the data of an operational system to another server where the processing load of reporting against the copied data does not impact the operational system's performance.
Off line Data Warehouse 
Data warehouses at this stage are updated from data in the operational systems on a regular basis and the data warehouse data is stored in a data structure designed to facilitate reporting.
Real Time Data Warehouse 
Data warehouses at this stage are updated every time an operational system performs a transaction (e. g. , an order or a delivery or a booking. )
Integrated Data Warehouse 
Data warehouses at this stage are updated every time an operational system performs a transaction. The data warehouses then generate transactions that are passed back into the operational systems.

Disadvantages of data warehouses

There are also disadvantages to using a data warehouse. Some of them are:

The future of data warehousing

Data warehousing, like any technology niche, has a history of innovations that did not receive market acceptance. [5]

A 2007 Gartner Group paper predicted the following technologies could be disruptive to the business intelligence market . A disruptive technology or disruptive innovation is a term describing a technological innovation product or service that uses a "disruptive" strategy rather than [6]

Another prediction is that data warehouse performance will continue to be improved by use of data warehouse appliances, many of which incorporate the developments in the aforementioned Gartner Group report. Service-oriented architecture ( SOA) is a method for Systems development and integration where functionality is grouped around Business processes Software as a service ( SaaS, typically pronounced 'sass' is a model of Software deployment where an application is hosted as a service provided to customers across An in-memory database ( IMDB; also main memory database system or MMDB) is a Database management system that primarily relies on Main memory See also Visualization and Information graphics Visualization is any technique for creating Images Diagrams or A data warehouse appliance is an integrated set of servers storage OS DBMS and software specifically pre-installed and pre-optimized for data warehousing.

Finally, management consultant Thomas Davenport, among others, predicts that more organizations will seek to differentiate themselves by using analytics enabled by data warehouses. Thomas H Davenport (born October 17, 1954) is an American academic and author specializing in business process innovation and knowledge management Business analytics is how organizations gather and interpret Data in order to make better business decisions and to optimize Business processes. [7]

See also

References

  1. ^ Inmon, W. Business intelligence ( BI) refers to technologies applications and practices for the collection integration analysis and presentation of business Information and Business intelligence tools are a type of Application software designed to report analyze and present data Data integration is the process of combining Data residing at different sources and providing the user with a unified view of these data. A data mart is a subset of an organizational data store usually oriented to a specific purpose or major data subject that may be distributed to support business needs Data mining is the process of Sorting through large amounts of data and picking out relevant information A data mining agent is a software program built for the primary purpose of finding information efficiently A data warehouse appliance is an integrated set of servers storage OS DBMS and software specifically pre-installed and pre-optimized for data warehousing. A database management system ( DBMS) is Computer software designed for the purpose of managing Databases DBMSes may use any of a variety of Data models Decision support systems constitute a class of computer-based Information systems including knowledge-based systems that support Decision-making activities An Executive Information System (EIS is a type of Management information system intended to facilitate and support the information and Decision making needs Extract Transform and Load ( ETL) is a process in data warehousing that involves extracting data from outside sources In Computing, master data management (MDM comprises a set of processes and tools that consistently defines and manages the non-transactional data entities of an Online Analytical Processing or OLAP (ˈoʊlæp is an approach to quickly provide answers to analytical queries that are multi-dimensional in nature Online transaction processing, or OLTP, refers to a class of systems that facilitate and manage transaction-oriented applications typically for data entry and retrieval An operational data store (or " ODS " is a Database designed to integrate data from multiple sources to make analysis and reporting easier A snowflake schema is a logical arrangement of tables in a Relational database such that the entity relationship diagram resembles a snowflake The star schema (sometimes referenced as star join schema is the simplest style of Data warehouse schema. H. Tech Topic: What is a Data Warehouse? Prism Solutions. Volume 1. 1995.
  2. ^ Yang, Jun. WareHouse Information Prototype at Stanford (WHIPS). [1]. Stanford University. July 7, 1998. Events 1456 - A retrial verdict acquits Joan of Arc of heresy 25 years after her death Year 1998 ( MCMXCVIII) was a Common year starting on Thursday (link will display full 1998 Gregorian calendar)
  3. ^ Caldeira, C. "Data Warehousing - Conceitos e Modelos". Edições Sílabo. 2008. ISBN 978-972-618-479-9
  4. ^ Ericsson, R. "Building Business Intelligence Applications with . NET". 1st Ed. Charles River Media. February 2004. pp. 28-29.
  5. ^ Pendse, Nigel and Bange, Carsten "The Missing Next Big Things", http://www.olapreport.com/Faileddozen.htm
  6. ^ Schlegel, Kurt "Emerging Technologies Could Prove Disruptive to the Business Intelligence Market", Gartner Group. July 6, 2007
  7. ^ Davenport, Thomas and Harris, Jeanne "Competing on Analytics: The New Science of Winning". Events 1044 - The Battle of Ménfő takes place 1189 - Richard the Lionheart is crowned King of England Year 2007 ( MMVII) was a Common year starting on Monday of the Gregorian calendar in the 21st century. Harvard Business School Press. 2007. ISBN 1-422-10332-3

External links


Dictionary

data warehouse

-noun

  1. (computing) A collection of data, from a variety of sources, organized to provide useful guidance to an organization's decision makers.
© 2009 citizendia.org; parts available under the terms of GNU Free Documentation License, from http://en.wikipedia.org
 Dapyx Software network: MP3 Explorer | Ebook Manager | Zenithic