Data Management for Data Science

Contents:

• Data management processes in the context of the data life cycle
• Tools for data management
• Provision of data for data science processes
• Data quality and data security
• Data handling in the context of IoT
• ETL/ELT processes
• Stream & batch processing
• Read-only-data structures
• Data Lakes vs Data Warehouse
• Event-driven data architectures

Key Information

Learning Objectives

This module covers the fundamental conceptual, systemic, and practical aspects of the sustainable use of data—from its creation and publication to its long-term preservation. A structured approach to data involves the processes of archiving and reusing data. This includes the strategic planning of research projects (research data management), the administration of technical infrastructure, and the collection, organization, and linking of metadata. Participants will learn approaches to handling big data, including all aspects of heterogeneous or rapidly changing data. In addition, we will explore the concepts of (Web) APIs to enable participants to collect and combine their own datasets. The latter requires an understanding of standard processes such as Extract-Transform-Load (ETL). Data integration and the interoperability of various data sources represent the central challenge. The concepts learned will be tested and applied using advanced solutions. We will examine the current market for data management tools, data warehouse solutions, and data processing platforms. Students will develop the ability to think in a system- and process-oriented manner. They will be able to apply their acquired knowledge and problem-solving skills to new areas of work, collaborate in groups, and jointly tackle new topics.

Recommended Prerequisites

None

Exam

Exam: Written exam (120 minutes)

Exam Prerequisites:

None

Exam Requirements:

• Description of the data lifecycle
• Understanding of various approaches to data archiving
• Explanation of the structure, functionality, and use of practical systems for data management, storage, and archiving
• Understanding of ETL/ELT processes for data processing
• Description of the concepts of data warehousing and data lakes
• Description of the concepts and challenges associated with big data and large-scale data
• Understanding of the architecture of read-only data stores