Data virtualization

Data virtualization is an approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted at source, or where it is physically located, and can provide a single customer view (or single view of any other entity) of the overall data. Unlike the traditional extract, transform, load ("ETL") process, the data remains in place, and real-time access is given to the source system for the data. This reduces the risk of data errors, of the workload moving data around that may never be used, and it does not attempt to impose a single data model on the data (an example of heterogeneous data is a federated database system). The technology also supports the writing of transaction data updates back to the source systems. To resolve differences in source and consumer formats and semantics, various abstraction and transformation techniques are used. This concept and software is a subset of data integration and is commonly used within business intelligence, service-oriented architecture data services, cloud computing, enterprise search, and master data management.

Data virtualization is an approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted at source, or where it is physically located, and can provide a single customer view (or single view of any other entity) of the overall data. Unlike the traditional extract, transform, load ("ETL") process, the data remains in place, and real-time access is given to the source system for the data. This reduces the risk of data errors, of the workload moving data around that may never be used, and it does not attempt to impose a single data model on the data (an example of heterogeneous data is a federated database system). The technology also supports the writing of transaction data updates back to the source systems. To resolve differences in source and consumer formats and semantics, various abstraction and transformation techniques are used. This concept and software is a subset of data integration and is commonly used within business intelligence, service-oriented architecture data services, cloud computing, enterprise search, and master data management. ==Functionality== Data Virtualization software provides some or all of the following capabilities: Abstraction – Abstract the technical aspects of stored data, such as location, storage structure, API, access language, and storage technology. Virtualized Data Access – Connect to different data sources and make them accessible from a common logical data access point. Data transformation – Transform, improve quality, reformat, aggregate etc. source data for consumer use. Data federation – Combine result sets from across multiple source systems. Data delivery – Publish result sets as views and/or data services executed by client application or users when requested. Data virtualization software may include functions for development, operation, and/or management. A metadata engine collects, stores and analyzes information about data and metadata (data about data) in use within a domain. Benefits include: Reduce risk of data errors Reduce systems workload through not moving data around Increase speed of access to data on a real-time basis Allows for query processing pushed down to data source instead of in middle tier Most systems enable self-service creation of virtual databases by end users with access to source systems Increase governance and reduce risk through the use of policies Reduce data storage required Accelerate processes up to five times through the declarative approach Drawbacks include: May impact Operational systems response time, particularly if under-scaled to cope with unanticipated user queries or not tuned early on. Does not impose a heterogeneous data model, meaning the user has to interpret the data, unless combined with Data Federation and business understanding of the data Requires a defined Governance approach to avoid budgeting issues with the shared services Not suitable for recording the historic snapshots of data. A data warehouse is better for this Change management "is a huge overhead, as any changes need to be accepted by all applications and users sharing the same virtualization kit" Designers should always keep performance considerations in mind Avoid usage: For accessing Operational Data Systems (Performance and Operational Integrity issues) For federating or centralizing all data of the organization (Security and hacking issues) For building very large virtual Data warehouse (Performance issues) As an ETL process (Governance and performance issues) If you have only one or two data sources to virtualize ==History== Enterprise information integration (EII) (first coined by Metamatrix), now known as Red Hat JBoss Data Virtualization, and federated database systems are terms used by some vendors to describe a core element of data virtualization: the capability to create relational JOINs in a federated VIEW. ==Technology== Some data virtualization solutions and vendors: AnalyticsCreator IBM data Virtualization Actifio Copy Data Virtualization Capsenta Ultrawrap, acquired by data.world 2019 Data Virtuality DataWerks Delphix Data Virtualization Platform Microsoft Gluent Data Platform Querona Red Hat JBoss Enterprise Application Platform Data Virtualization (discontinued) Stone Bond Technologies Enterprise Enabler Data Virtualization Platform Stratio Generative AI Data Fabric Teiid, part of JBoss Developer Studio TIBCO Data Virtualization Veritas Provisioning File System / Data Virtualization Veritas Technologies Virtual Data Platform XAware Another more up-to-date list with user rankings is compiled by Gartner. ==See also== (EII) ==References== ==Further reading==

Text derived from https://en.wikipedia.org/w/index.php?oldid=1355405156 under CC BY-SA 4.0; modifications have been made.