The development of ICTs (Information and Communication Technologies) and the usage of personal data for both research and commercial purposes over the last years have brought the question of the protection of personal data. The GDPR (General Data Protection Regulation) has defined the ways how personal data should be treated, but the application of these requirements still remains an open issue.

The last decades witnessed a significant evolution in terms of data generation, management, and maintenance, especially in the RDF format. Moreover, in the energy domain, semantic data is finding its way and can be used for various data

Big data plays a relevant role in promoting both manufacturing and scientific development through industrial digitization and emerging interdisciplinary research. Semantic web technologies have also experienced great progress, and scientific communities and practitioners have contributed to the problem of big data management with ontological models, controlled vocabularies, linked datasets, data models, query languages, as well as tools for transforming big data into knowledge from which decisions can be made.

The “processing frameworks” are one of the most essential components of a Big Data systems. There are three categories of such frameworks namely: Batch-only frameworks (Hadoop), Stream-only frameworks (Storm, Samza), and Hybrid frameworks (Spark, Hive and Flink). In this lecture, we will introduce them and cover one of the major Big Data frameworks, Apache Spark. We will cover Spark fundamentals and the model of “Resilient Distributed Datasets (RDDs)” that are used in Spark to implement in-memory batch computation.

This lecture focuses on architecting Big Data solution. We will discuss the role and importance of the components in realizing system architectures. The participants will be introduced to unique problem characteristics that drive Big Data and the unending technology options to solve them. The application of the introduced concepts and components will be discussed in real-world example of practical use-cases.

This lecture will cover the existing advanced Big Data architectures following a bottom-up approach. In this lecture, the important knowledge to design and architect scalable solutions for challenging problems will be introduced. The primary components in the architecture of such systems and their architectures will be presented and discussed including “inter alia distributed kernels” and cluster managers, distributed file systems and storage systems.