Dataflow Autoscaling Pipelines

Dataflow Autoscaling Pipelines

Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number
Expertise Level

Overview/Description
Apache Beam, Cloud Dataflow, and Cloud Dataprep can be used to create data pipelines. In this course, you will learn how areas of Beam, Apache Beam SDK, Cloud Dataflow, and Cloud Dataprep assist in pipeline management.

Target Audience
Data professionals who are responsible for provisioning and optimizing big data solutions, and data enthusiasts getting started with Google Cloud Platform

Prerequisites
None

Expected Duration (hours)
0.8

Lesson Objectives

Dataflow Autoscaling Pipelines

start the course

define Apache Beam concepts and SDKs

describe the Python SDK and its connection with data pipelines

describe the Java SDK and its connection with data pipelines

initialize Cloud Dataprep

demonstrate how to ingest data into a pipeline

create recipes in a Cloud Dataprep pipeline

work with the import/export process and demonstrate how to run Dataflow jobs in Cloud Dataprep

describe MapReduce and the benefits of Cloud Dataflow over MapReduce

outline serverless architecture and some of the GCP products supporting data analytics

describe the use of Apache Beam, Cloud Dataflow, and Cloud Dataprep in GCP to create and manage pipelines

Course Number:
cl_gcde_a11_it_enus

Expertise Level
Intermediate