Analytics corresponds to a relevant and challenging phase of Big Data. The generation of knowledge from extensive data sets (petabyte era) of varying types, occurring at a speed able to serve decision makers, is practiced using multiple areas of knowledge, such as computing, statistics, data mining, among others. In the Big Data domain, Analytics is also considered as a process capable of adding value to the organizations. Besides the demonstration of value, Analytics should also consider operational tools and models to support decision making. To adding value, Analytics is also presented as part of some Big Data value chains, such the Information Value Chain presented by NIST among others, which are detailed in this article. As well, some maturity models are presented, since they represent important structures to favor continuous implementation of Analytics for Big Data, using specific technologies, techniques and methods. Hence, through an in-depth research, using specific literature references and use cases, we seeks to outline an approach to determine the Analytical Engineering for Big Data Analytics considering four pillars: Data, Models, Tools and People; and three process groups: Acquisition, Retention and Revision; in order to make feasible and to define an organization, possibly designated as an Analytics Organization, responsible for generating knowledge from the data in the field of Big Data Analytics.
Requirements engineering (RE), as a part of the project development life cycle, has increasingly been recognized as the key to ensuring on-time, on-budget, and goal-based delivery of software projects;compromising this vital phase is nothing but project failures. RE of big data projects is even more crucial because of the main characteristics of big data, including high volume, velocity, and variety. As the traditional RE methods and tools are user-centric rather than data-centric, employing these methodologies is insufficient to fulfill the RE processes for big data projects. Because of the i…
Models are fundamentally crucial to many scientific fields, including software engineering, systems engineering, enterprise modeling, and business modeling. This paper focuses on diagrammatic conceptual modeling, as opposed to mathematical or computational models, wherein a conceptual model is a translation of reality processes into an abstract mechanism that has similar structure and parallel events of the external processes. Although various modeling approaches exist, including UML (Unified Modeling Language) in software engineering and its dialect, SysML (System Modeling Language), in syste…
This paper describes the use of the Levels of Conceptual Interoperability Model (LCIM) as a framework for conceptual modeling and its descriptive and prescriptive uses. LCIM is applied to show its potential and shortcomings in the current simulation interoperability approaches, in particular the High Level Architecture (HLA) and Base Object Models (BOM). It emphasizes the need to apply rigorous engineering methods and principles and replace ad-hoc approaches.
The increasing importance of such fields as embedded systems, pervasive computing, and hybrid systems control is increasing attention to the time-dependent aspects of system modeling. In this paper, we focus on modeling conceptual time. Conceptual time is time represented in conceptual modeling, where the notion of time does not always play a major role. Time modeling in computing is far from exhibiting a unified and comprehensive framework, and is often handled in an ad hoc manner. This paper contributes to the establishment of a broader understanding of time in conceptual modeling based on a…
This paper studies the differences and similarities between domain ontologies and conceptual data models and the role that ontologies can play in establishing conceptual data models during the process of information systems development. A mapping algorithm has been proposed and embedded in a special purpose Transformation Engine to generate a conceptual data model from a given domain ontology. Both quantitative and qualitative methods have been adopted to critically evaluate this new approach. In addition, this paper focuses on evaluating the quality of the generated conceptual data model elem…
Servitization is one of the most significant trends that reshapes the information world and society in recent years. The requirement of collecting,storing, processing, and sharing of the Big Data has led to massive software resources being developed and made accessible as web-based services to facilitate such process. These services that handle the Big Data come from various domains and heterogeneous networks, and converge into a huge complicated service network (or ecosystem), called the Big Service.The key issue facing the big data and big service ecosystem is how to optimally configure and …
This study introduces another application of software engineering tools, conceptual modeling, which can be applied to other fields of research. One way to strengthen the relationship between software engineering and other fields is to develop a good way to perform conceptual modeling that is capable of addressing the peculiarities of these fields of study. This study concentrates on humanities and social sciences, which are usually considered softer and further away from abstractions and (abstract) machines. Specifically, we focus on conceptual modeling as a software engineering tool (e.g., UM…
This paper aims to contribute to further understanding of dynamism (the dynamic behavior of system models) in the mathematical and conceptual modeling of systems. This study is conducted in the context of the claim that software engineering lacks underpinning scientific theories, both for the software it produces and the processes by which it does so. The research literature proposes that information theory can provide such a benefit for software engineering. We explore the dynamism expressive power of conceptual modeling as a software engineering tool that can represent physical systems in th…
The use of conceptual models to foster requirements engineering has been proposed and evaluated as beneficial for several decades. For instance, goal-oriented requirements engineering or the specification of scenarios are commonly done using conceptual models. Bringing such model-based requirements engineering approaches into industrial practice typically requires industrial training. In this paper, we report lessons learned from a training program for teaching industry professionals model-based requirements engineering. Particularly, we as educators and learners report experiences from design…
This paper has a dual character, combining a philosophical ontological exploration with a conceptual modeling approach in systems and software engineering. Such duality is already practiced in software engineering, in which the current dominant modeling thesis is object orientation. This work embraces an anti-thesis that centers solely on the process rather than emphasizing the object. The approach is called occurrence-only modeling, in which an occurrence means an event or process where a process is defined as an orchestrated net of events that form a semantical whole. In contrast to object o…
With recent advances in the development of more powerful quantum computers, the re-search area of quantum software engineering is emerging. Quantum software plays a critical role in exploiting the full potential of quantum computing systems. As a result, it has been drawing increasing attention recently to provide concepts, principles, and guidelines to address the ongoing challenges of quantum software development. The importance of the topic motivated us to voice out a call for action to develop a readiness model that will help an organization assess its capability of migration from classic …
In open-source software development environments; textual, numerical and relationship-based data generated are of interest to researchers. Various data sets are available for this data, which is frequently used in areas such as software engineering and natural language processing. However, since these data sets contain all the data in the environment, the problem arises in the terabytes of data processing. For this reason, almost all of the studies using GitHub data use filtered data according to certain criteria. In this context, using a different data set in each study makes a comparison of …
Even though there are more development to improving the Quality of Service and requirement engineering in web services yet there is a big scarcity for its related standardization in day to day progress leading to vast needs in its area. Also in web service environment it always has been a big challenge to raise the standard of Quality of Service in requirement engineering analysis.
Abstraction applied in computer networking hides network details behind a well-defined representation by building a model that captures an essential aspect of the network system. Two current methods of representation are available, one based on graph theory, where a network node is reduced to a point in a graph, and the other the use of non-methodological iconic depictions such as human heads, walls, towers or computer racks. In this paper, we adopt an abstract representation methodology, the thinging machine (TM), proposed in software engineering to model computer networks. TM defines a singl…
Over recent decades, scenarios and scenario-based software/system engineering have been actively employed as essential tools to handle intricate problems, validate requirements, and support stakeholders' communication. However, despite the widespread use of scenarios, there have been several challenges for engineers to more willingly utilize scenario-based engineering approaches (i.e., scenario methods) in their projects. First, the term scenario has numerous published definitions, thus lacking in a well-established shared understanding of scenarios and scenario methods. Second, the conceptual…
One of the great challenges the information society faces is dealing with the huge amount of information generated and handled daily on the Internet. Today, progress in Big data proposals attempts to solve this problem, but there are certain limitations to information search and retrieval due basically to the large volumes handled the heterogeneity of the information, and its dispersion among a multitude of sources. In this article, a formal framework is defined to facilitate the design and development of an environmental management information system, which works with a heterogeneous and larg…
In previous works, we proposed a one-category (entitled thimac) conceptual model called a thinging machine (TM), which integrates staticity (e.g., objects) and dynamism (e.g., events) without losing valuable aspects of diagrammatic intuition in conceptual modeling. We proposed applying TM to conceptual modeling in software engineering (e.g., on or above the level of UML as a conceptual modeling language). In this paper, to show such an application in software engineering, we first present a complete high-level description of a library service system to demonstrate the TM s applicability. Furth…
This paper reviews recent advances in big data optimization, providing the state-of-art of this emerging field. The main focus in this review are optimization techniques being applied in big data analysis environments. Integer linear programming, coordinate descent methods, alternating direction method of multipliers, simulation optimization and metaheuristics like evolutionary and genetic algorithms, particle swarm optimization, differential evolution, fireworks, bat, firefly and cuckoo search algorithms implementations are reviewed and discussed. The relation between big data optimization an…
Modern spreadsheet systems can be used to implement complex spreadsheet applications including data sheets, customized user forms and executable procedures written in a scripting language. These applications are often developed by practitioners that do not follow any software engineering practice and do not produce any design documentation. Thus, spreadsheet applications may be very difficult to be maintained or restructured. In this position paper we present in a nutshell two reverse engineering techniques and a tool that we are currently realizing for the abstraction of conceptual data model…
Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization
School of Information, Computer, and Communication Technology（信息、计算机与通信技术学院）；Sirindhorn International Institute of Technology, Thammasat University（素金国际技术研究所，泰国 Thammasat 大学）
A growing number of empirical software engineering researchers suggest that a complementary focus on theory is required if the discipline is to mature. A first step in theory-building involves the establishment of suitable theoretical constructs. For researchers studying software projects, the lack of a theoretical construct for context is problematic for both experimentation and effort estimation. For experiments, insufficiently understood contextual factors confound results, and for estimation, unstated contextual factors affect estimation reliability. We have earlier proposed a framework th…
An important factor in guaranteeing the quality of a system is developing a conceptual model that reflects the knowledge about its domain as well as knowledge about the functions it has to perform. In software engineering, conceptual modeling has gained importance as a discipline that offers languages, methods, and methodologies to address the complexity of software development. The key to understanding such complexity is using tools such as diagrams at various levels of representation. A conceptual model must include all relevant static and behavioral aspects of its domain. In UML, the static…
Researchers have been highly active to investigate the classical machine learning workflow and integrate best practices from the software engineering lifecycle. However, deep learning exhibits deviations that are not yet covered in this conceptual development process. This includes the requirement of dedicated hardware, dispensable feature engineering, extensive hyperparameter optimization, large-scale data management, and model compression to reduce size and inference latency. Individual problems of deep learning are under thorough examination, and numerous concepts and implementations have g…
This report describes de design of an experiment that intends to compare two variants of a modeldriven system development method, so as to assess the impact of requirements engineering practice in the quality of the conceptual models. The conceptual modelling method being assessed is the OO-Method [Pastor and Molina 2007]. One of its variants includes Communication Analysis, a communication-oriented requirements engineering method [España, González et al. 2009] and a set of guidelines to derive conceptual models from requirements models [España, Ruiz et al. 2011; González, España et al. 2011].…
The Hellenic Open University has embarked on a large-scale effort to enhance its textbook-based material with content that demonstrably supports the basic tenets of distance learning. The challenge is to set up a framework that allows for the production-level creation, distribution and consumption of content, and at the same time, evaluate the effort in terms of technological, educational and organizational knowledge gained. This paper presents a model of the educational process that is used as a development backbone and argues about its conceptual and technical practicality at large.
User eXperience (UX) is becoming increasingly important for success of software products. Yet, many companies still face various challenges in their work with UX. Part of these challenges relate to inadequate knowledge and awareness of UX and that current UX models are commonly not practical nor well integrated into existing Software Engineering (SE) models and concepts. Therefore, we present a conceptual UX-aware model of requirements for software development practitioners. This layered model shows the interrelation between UX and functional and quality requirements. The model is developed ba…
Deep Learning and Machine Learning: Advancing Big Data Analytics and Management with Design Patterns
Georgia Institute of Technology（佐治亚理工学院）；Indiana University（印第安纳大学）；Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学）；University of Hawaii（夏威夷大学）；National Taiwan Normal University（台湾师范大学）；Kyoto University（京都大学）；Rutgers University（罗格斯大学）；AppCubic；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Purdue University（普渡大学）；Emory University（埃默里大学）
Software and hardware co-design and optimization of HPC systems has become intolerably complex, ad-hoc, time consuming and error prone due to enormous number of available design and optimization choices, complex interactions between all software and hardware components, and multiple strict requirements placed on performance, power consumption, size, reliability and cost. We present our novel long-term holistic and practical solution to this problem based on customizable, plugin-based, schema-free, heterogeneous, open-source Collective Mind repository and infrastructure with unified web interfa…
