Streaming Data Architectures: Processing Streaming Data
Overview/Description
Expected Duration
Lesson Objectives
Course Number
Expertise Level
Overview/Description
Process streaming data with Spark, the analytic engine built on Hadoop. In this course, you will discover how to develop applications in Spark to work with streaming data and generate output. Topics include the following: Configure a streaming data source; Use Netcat and write applications to process the data stream; Learn the effects of using the Update mode on your stream processing application's output; Write a monitoring application that listens for new files added to a directory; Compare the append output with the update mode; Develop applications to limit files processed in each trigger; Use Spark's Complete mode for output; Perform aggregation operations on streaming data with the DataFrame API; Process streaming data with Spark SQL queries.
Expected Duration (hours)
0.9
Lesson Objectives
Streaming Data Architectures: Processing Streaming Data
it_dssdardj_02_enus
Expertise Level
Intermediate