Structured Streaming

Structured Streaming

Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number
Expertise Level

Overview/Description
In this course, you will learn about the concepts of Structured Steaming such as Windowing, DataFrame, and SQL Operations. You will also learn about File Sinks, Deduplication, and Checkpointing.

Target Audience
Programmers and Developers wishing to perform big data development using Apache Spark 2.2

Prerequisites
None

Expected Duration (hours)
1.2

Lesson Objectives

Structured Streaming

start the course

describe Structured Streaming

read stream input using readStream

write stream data using writeStream

apply window operations on event time

describe continuous applications in terms of structured streaming

implement deduplication with and without watermarking

store stream output to a directory using a file sink

use streaming query objects

manage streaming queries

enable checkpointing in structured streaming

use structured streaming to implement a word count on a text stream

describe the basics of Spark Streaming

Course Number:
df_apsk_a03_it_enus

Expertise Level
Intermediate