With the ever-growing availability of so-called complex data, especially on the Web, decision-support systems such as data warehouses must store and process data that are not only numerical or symbolic. Warehousing and analyzing such data requires the joint exploitation of metadata and domain-related knowledge, which must thereby be integrated. In this paper, we survey the types of knowledge and metadata that are needed for managing complex data, discuss the issue of knowledge and metadata integration, and propose a CWM-compliant integration solution that we incorporate into an XML complex data warehousing framework we previously designed.
Nowadays, many decision support applications need to exploit data that are not only numerical or symbolic, but also multimedia, multistructure, multisource, multimodal, and/or multiversion. We term such data complex data. Managing and analyzing complex data involves a lot of different issues regarding their structure, storage and processing, and metadata are a key element in all these processes. Such problems have been addressed by classical data warehousing (i.e., applied to "simple" data). However, data warehousing approaches need to be adapted for complex data. In this paper, we first propo…
In a data warehousing process, mastering the data preparation phase allows substantial gains in terms of time and performance when performing multidimensional analysis or using data mining algorithms. Furthermore, a data warehouse can require external data. The web is a prevalent data source in this context. In this paper, we propose a modeling process for integrating diverse and heterogeneous (so-called multiform) data into a unified format. Furthermore, the very schema definition provides first-rate metadata in our data warehousing context. At the conceptual level, a complex object is repres…
The data warehousing and OLAP technologies are now moving onto handling complex data that mostly originate from the Web. However, intagrating such data into a decision-support process requires their representation under a form processable by OLAP and/or data mining techniques. We present in this paper a complex data warehousing methodology that exploits XML as a pivot language. Our approach includes the integration of complex data in an ODS, under the form of XML documents; their dimensional modeling and storage in an XML data warehouse; and their analysis with combined OLAP and data mining te…
Data warehousing is continuously gaining importance as organizations are realizing the benefits of decision oriented data bases. However, the stumbling block to this rapid development is data quality issues at various stages of data warehousing. Quality can be defined as a measure of excellence or a state free from defects. Users appreciate quality products and available literature suggests that many organization`s have significant data quality problems that have substantial social and economic impacts. A metadata based quality system is introduced to manage quality of data in data warehouse. …
In a data warehousing process, the phase of data integration is crucial. Many methods for data integration have been published in the literature. However, with the development of the Internet, the availability of various types of data (images, texts, sounds, videos, databases...) has increased, and structuring such data is a difficult task. We name these data, which may be structured or unstructured, "complex data". In this paper, we propose a new approach for complex data integration, based on a Multi-Agent System (MAS), in association to a data warehousing approach. Our objective is to take …
Large organizations today are being served by different types of data processing and informations systems, ranging from the operational (OLTP) systems, data warehouse systems, to data mining and business intelligence applications. It is important to create an integrated repository of what these systems contain and do in order to use them collectively and effectively. The repository contains metadata of source systems, data warehouse, and also the business metadata. Decision support and business analysis require extensive and in-depth understanding of business entities, tasks, rules and the env…
Research in data warehousing and OLAP has produced important technologies for the design, management and use of information systems for decision support. With the development of Internet, the availability of various types of data has increased. Thus, users require applications to help them obtaining knowledge from the Web. One possible solution to facilitate this task is to extract information from the Web, transform and load it to a Web Warehouse, which provides uniform access methods for automatic processing of the data. In this chapter, we present three innovative researches recently introd…
This article presents the implementation process of a Data Warehouse and a multidimensional analysis of business data for a holding company in the financial sector. The goal is to create a business intelligence system that, in a simple, quick but also versatile way, allows the access to updated, aggregated, real and/or projected information, regarding bank account balances. The established system extracts and processes the operational database information which supports cash management information by using Integration Services and Analysis Services tools from Microsoft SQL Server. The end-user…
In a data warehousing process, the data preparation phase is crucial. Mastering this phase allows substantial gains in terms of time and performance when performing a multidimensional analysis or using data mining algorithms. Furthermore, a data warehouse can require external data. The web is a prevalent data source in this context, but the data broadcasted on this medium are very heterogeneous. We propose in this paper a UML conceptual model for a complex object representing a superclass of any useful data source (databases, plain texts, HTML and XML documents, images, sounds, video clips...)…
Big data integration could involve a large number of sources with unpredictable redundancy information between them. The approach of building a central warehousing to integrate big data from all sources then becomes infeasible because of so large number of sources and continuous updates happening. A practical approach is to apply online query scheduling that inquires data from sources at runtime upon receiving a query. In this paper, we address the Time-Cost Minimization Problem for online query scheduling, and tackle the challenges of source permutation and statistics estimation to minimize t…
Scientific research requires access, analysis, and sharing of data that is distributed across various heterogeneous data sources at the scale of the Internet. An eager ETL process constructs an integrated data repository as its first step, integrating and loading data in its entirety from the data sources. The bootstrapping of this process is not efficient for scientific research that requires access to data from very large and typically numerous distributed data sources. a lazy ETL process loads only the metadata, but still eagerly. Lazy ETL is faster in bootstrapping. However, queries on the…
Metadata represents the information about data to be stored in Data Warehouses.It is a mandatory element of Data Warehouse to build an efficient Data Warehouse.Metadata helps in data integration,lineage,data quality and populating transformed data into data warehouse.Spatial data warehouses are based on spatial data mostly collected from Geographical Information Systems(GIS)and the transactional systems that are specific to an application or enterprise.Metadata design and deployment is the most critical phase in building of data warehouse where it is mandatory to bring the spatial information …
Data warehouses are overwhelmingly built through a bottom-up process, which starts with the identification of sources, continues with the extraction and transformation of data from these sources, and then loads the data into a set of data marts according to desired multidimensional relational schemas. End user business intelligence tools are added on top of the materialized multidimensional schemas to drive decision making in an organization. Unfortunately, this bottom-up approach is costly both in terms of the skilled users needed and the sheer size of the warehouses. This paper proposes a to…
Big Data technology is described. Big data is a popular term used to describe the exponential growth and availability of data, both structured and unstructured. There is constructed dataspace architecture. Dataspace has focused solely - and passionately - on providing unparalleled expertise in business intelligence and data warehousing strategy and implementation. Dataspaces are an abstraction in data management that aims to overcome some of the problems encountered in data integration system. In our case it is block vector for heterogeneous data representation. Traditionally, data integration…
The data needed for machine learning (ML) model training, can reside in different separate sites often termed data silos. For data-intensive ML applications, data silos pose a major challenge: the integration and transformation of data demand a lot of manual work and computational resources. With data privacy and security constraints, data often cannot leave the local sites, and a model has to be trained in a decentralized manner. In this work, we present a vision on how to bridge the traditional data integration (DI) techniques with the requirements of modern machine learning. We explore the …
SSBD Ontology: A Two-Tier Approach for Interoperable Bioimaging Metadata
RIKEN Information R&D and Strategy Headquarters（日本理化学研究所信息研发与战略本部）；RIKEN Bioresource Research Center（日本理化学研究所生物资源研究中心）；RIKEN Center for Biosystems Dynamics Research（日本理化学研究所生物系统动力学研究中心）
RENSA: Rich Environment Metadata to Navigate Shared and Distributed Endpoints for Automated Federated SPARQL Query Generation
Graduate University for Advanced Studies, SOKENDAI（综合研究大学院大学（SOKENDAI））；National Institute of Informatics（信息学研究所）；Database Center for Life Science（生命科学数据库中心）
MetaMP: Seamless Metadata Enrichment and AI Application Framework for Enhanced Membrane Protein Visualization and Analysis
Center for Artificial Intelligence in Public Health Research (ZKI-PH)（公共健康研究人工智能中心）；Robert Koch Institute（罗伯特·科赫研究所）；University of Marburg（马尔堡大学）；Institute of Medical Informatics（医学信息学研究所）；University of Münster（穆尔斯特大学）；Université Paris Cité（巴黎城市大学）；Centre National de la Recherche Scientifique (CNRS)（国家科学研究中心）；Department of Mathematics and Computer Science（数学与计算机科学系）；Freie Universität Berlin（柏林自由大学）
MetaboKG: An Analysis-centric Knowledge Graph Framework for Untargeted Metabolomics
Univ. Côte d’Azur（蔚蓝海岸大学）；CNRS（法国国家科学研究中心）；ICN（信息学与认知神经研究所）；Interdisciplinary Institute for Artificial Intelligence (3iA) Côte d’Azur（蔚蓝海岸跨学科人工智能研究所）；Inria（法国国家数字与信息技术研究所）；I3S（信息与系统研究所）
Hochschule Niederrhein University of Applied Sciences（下莱茵应用科学大学）；RWTH Aachen University（亚琛工业大学）；Fraunhofer Institute for Applied Information Technology FIT（弗劳恩霍夫应用信息技术研究所）
A Categorical Approach to Semantic Interoperability across Building Lifecycle
Eindhoven University of Technology（埃因霍温理工大学）；Conexus AI, Inc.（Conexus AI 公司）；Topos Institute（Topos 研究所）；Carnegie Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）
