Data Mining Designing Data Warehousing

Abstract

Data mining is a combination of database and artificial intelligence technologies.
Although the AI field has taken a major dive in the last decade; this new emerging field has
shown that AI can add major contributions to existing fields in computer science. In fact,
many experts believe that data mining is the third hottest field in the industry behind the
Internet, and data warehousing. Data mining is really just the next step in the process of
analyzing data. Instead of getting queries on standard or user-specified relationships, data
mining goes a step farther by finding meaningful relationships in data. Relationships that
were thought to have not existed, or ones that give a more insightful view of the data. For
example, a computer-generated graph may not give the user any insight, however data
mining can find trends in the same data that shows the user more precisely what is going
on. Using trends that the end-user would have never thought to query the computer about.
Without adding any more data, data mining gives a huge increase in the value added by the
database. It allows both technical and non-technical users get better answers, allowing
them to make a much more informed decision, saving their companies millions of dollars.
Data warehousing and on-line analytical processing (OLAP) are essential elements of
decision support, which has increasingly become a focus of the database industry. Many
commercial products and services are now available, and all of the principal database
management system vendors now have offerings in these areas. Decision support places
some rather different requirements on database technology compared to traditional online transaction processing applications. This paper provides an overview of data
warehousing and OLAP technologies, with an emphasis on their new requirements. We
describe back end tools for extracting, cleaning and loading data into a data warehouse;
multidimensional data models typical of OLAP; front end client tools for querying and data
analysis; server extensions for efficient query processing; and tools for metadata
management and for managing the warehouse. In addition to surveying the state of the art,
this paper also identifies some promising research issues, some of which are related to
problems that the database research community has worked on for years, but others are
only just beginning to be addressed. This overview is based on a tutorial that the authors
presented at the VLDB Conference, 1996