Kafka Developer is a practical, hands-on training program designed to help participants develop the skills needed to build scalable and reliable event-driven applications using Apache Kafka. The course covers essential Kafka concepts, including topics, partitions, brokers, producers, consumers, message delivery, data streaming, and distributed messaging. Participants will gain practical experience developing Kafka-based solutions, processing real-time data, managing event streams, and applying Kafka in modern distributed systems, microservices, and big data environments.
Course Overview:
Kafka Developer is a comprehensive, hands-on training program designed to develop practical skills in Apache Kafka, Big Data processing, distributed messaging, and Scala programming. The course begins with the fundamentals of Big Data and Kafka, introducing participants to Kafka architecture, topics, partitions, producers, consumers, brokers, and ZooKeeper.
Participants will progress into advanced Kafka concepts, including broker configuration and tuning, replication, persistence, offsets, partition management, MirrorMaker, serializers, security, and Producer and Consumer APIs. Through practical exercises, they will learn how to build Kafka producers and consumers using Scala and implement reliable publish-subscribe messaging solutions.
The program also provides a strong foundation in Scala programming and functional programming, covering collections, functions, higher-order functions, classes, objects, traits, inheritance, pattern matching, exception handling, and other core Scala concepts required for Kafka development. Participants will gain the knowledge and hands-on experience needed to develop scalable event-streaming solutions and work effectively with Kafka in modern Big Data and distributed application environments.
Course Objectives:
- Traits
- Traits in Scala
- Mixing traits into Objects
- Layered Traits
- Traits for Rich Interfaces
- Scala class hierarchy
- Functions & Procedures
- Local functions
- First class functions
- Partially applied functions
- Currying
- Default & Named Arguments
- Varargs
- Procedures
- Closures
- Classes in Scala
- Simple Classes and Parameter less Methods
- Access modifiers and rules
- Type parameters
- Object-Private Fields
- Auxiliary Constructors
- The Primary Constructor
- Nested Classes
- Higher Order functions in Collections
- Anonymous Functions (aka Function Literals)
- Pattern Matching in Depth
- Using the match keyword to return a value
- Using case classes for pattern matching
- Adding pattern guards to match conditions
- Partially specifying matches with wildcards
- Deep matching using case constructors
- Matching against collections of items
- Using extractors instead of case classes
- Exception Handling
Pre-requisites:
- Software Development
- Unix and Hadoop
Target Audience:
- Hadoop Developers
- Data Engineers
- Integration Engineers
- Architects
- Data Steward
Course Duration:
- 35 hours – 5 days
Course Content:
Phase 1: Understanding Big Data and Kafka (Day1)
- What is Big Data & Dimensions of Big Data
- Big Data in Advertising, Banking, Telecom, eCommerce, Healthcare, Defense
- Input options of Big Data
- Associated Challenges
- Introduction to Kafka
- Why Kafka
- Kafka Use Cases
- Introduction to Zookeeper
- Zookeeper: Distributed Applications and Challenges
- Zookeeper: Handling Partial Failures
- Zookeeper: Leader Election
- Aggregating User Activity Using Kafka
- Zookeeper for Kafka
- Kafka Architecture
- What is Kafka Data Model?
- What is a topic
- What is a partition and Partition Distribution
- What is a producer
- What is a consumer
Phase 2: Kafka Deep Dive (Day 2 and 3)
- Zookeeper Architecture
- Zookeeper watcher
- Kafka broker tuning parameters
- Kafka Mirror Maker
- Kafka Brokers
- Kafka Replication
- Persistence and Offsets
- The Kafka broker property list
- Customer Key and Value Serializer
- Replication and Partition management in Kafka
- Writing Producers– The Scala producer API
- Creating a Scala producer with custom partitioning and Serializers The Kafka producer property list
- Writing Consumers Kafka consumer APIs
- Simple Scala consumers Reading messages from a topic and printing them
- Publish-Subscribe System
- Configuring Producers with Scala
- Security options in Kafka
Phase 3: Understanding Scala Programming (Day 4 and 5)
- Introduction to Functional Programming
- Introduction to Scala
- Scala Fundamentals
- Functions & Procedures
- Collections
- Sets
- Maps
- Lists
- Tuples
- Arrays
- Higher Order Functions on Collections
- Objects in Scala oSingletons
- Companion Objects
- Objects Exten ding a Class or Trait
- Application Objects
- Inheritance
- Extending a Class
- Method Overriding
- Protected Fields and Methods
- Overriding
- Anonymous Subclasses
- Abstract Classes
- Abstract Fields
- Inheritance Hierarchy
- Object Equality
- Value Classes
