Kafka Integration with Spark

Kafka Integration with Spark

Overview/Description
Target Audience
Prerequisites
Expected Duration
Lesson Objectives
Course Number
Expertise Level

Overview/Description
Apache Kafka can easily integrate with Apache Spark to allow processing of the data entered into Kafka. In this course, you will discover how to integrate Kafka with Spark.

Target Audience
Developers, IT Operations engineers, and DevOPs engineers looking to implement and manage Apache Kafka

Prerequisites
None

Expected Duration (hours)
1.1

Lesson Objectives

Kafka Integration with Spark

start the course

install and configure the Spark Streaming package for Kafka

read data into Spark from Kafka

read data in parallel into Spark from Kafka

write data back to Kafka from Spark

write data back to Kafka from Spark in parallel

create a direct stream to access Kafka data from Spark

use LocationStrategies and ConsumerStrategies to improve performance

use an RDD in cases where batch processing would be a better solution

use offsets to handle exactly-once semantics

use Kafka and Spark to split words from sentences

Course Number:
df_apka_a03_it_enus

Expertise Level
Intermediate