Tech

What is a data warehouse?Source of business intelligence

Databases are usually relational (SQL) Also NoSQL, And transaction (OLTP), analysis (OLAP), or hybrid (HTAP). Departmental and special purpose databases were initially considered a significant improvement in business practices, but were later ridiculed as “islands.” Attempts to create an integrated database for all data across the enterprise are categorized as follows: Data lake A data warehouse if the data remains in native format and if the data is in a common format and schema. A subset of data warehouses are called data marts.

Defined data warehouse

Basically, a data warehouse is an analytic database, usually relational, created from two or more data sources, and typically stores petabytes of historical data. Data warehouses often have a large amount of computing and memory resources to execute complex queries and generate reports. Often, these are business intelligence (BI) systems and machine learning data sources.

Why use a data warehouse?

One of the main motivations for using an enterprise data warehouse (EDW) is that it limits the number and types of indexes that an operational (OLTP) database can create, which slows down analytic queries. By copying the data to the data warehouse, you can index all the important things in the data warehouse to improve the performance of your analytic queries without impacting the write performance of the OLTP database.

Another reason to use an enterprise data warehouse is to allow you to combine data from multiple sources for analysis. For example, a sales OLTP application probably doesn’t need to know the weather at the point of sale, but sales forecasts can use that data. Adding historical weather data to your data warehouse makes it easy to incorporate it into your model of historical sales data.

Data warehouse and data lake

A data lake that stores files of data in native format is essentially a “schema at read time”. This means that applications that read data from the lake must impose their own types and relationships on the data. The data warehouse, on the other hand, is a “write schema”. That is, data types, indexes, and relationships are imposed on the data when it is stored in the EDW.

“Schema on read” is suitable for data that may be used in some contexts, and while there is a risk that the data will not be used at all, there is little risk of losing the data. (((QuboleVendors of cloud data warehouse tools for data lakes estimate that 90% of the data in most data lakes is inactive. ) “Schema on write” is suitable for data that has a specific purpose and needs to be properly associated. To data from other sources. There is a risk that misformatted data will not be properly converted to the desired data type and may be discarded during import.

Copyright © 2021 IDG Communications, Inc.

What is a data warehouse?Source of business intelligence

Source link What is a data warehouse?Source of business intelligence

Show More

Related Articles

Back to top button