Hadoop Development 0 to 100 is a comprehensive, hands-on training program designed to take participants from the fundamentals of Apache Hadoop and big data development to practical implementation of distributed data processing solutions.
Course Overview:
Hadoop Development, Administration and BI Program is a one-stop course that introduces you to the domain of Hadoop Development as well as gives you technical know-how of the same.
Course Objectives:
By the end of this course, participants will be able to:
- Understand the fundamentals of Big Data, Apache Hadoop, and distributed data processing.
- Explain how Hadoop and its ecosystem support large-scale data storage and processing.
- Work with HDFS and execute essential Hadoop file system operations.
- Develop and run MapReduce programs for processing large datasets.
- Use Hadoop ecosystem tools including Sqoop, Oozie, Hive, Pig, and Flume.
- Develop data processing applications using Apache Spark.
- Work with RDDs, transformations, actions, Pair RDDs, Spark SQL, and DataFrames.
- Build and understand basic Spark Streaming applications.
- Analyze and manage data using Hive, Impala, and Pig.
- Understand the role of NoSQL databases in big data environments.
- Gain practical exposure to HBase, Cassandra, and MongoDB.
- Understand Hadoop cluster administration and use Cloudera Manager for cluster setup and management.
- Apply Hadoop and Spark technologies to practical big data development and analytics scenarios.
Pre-requisites:
- Typically, professionals with basic knowledge of software development, programming languages, and databases will find this course helpful. Basic knowledge should be enough to succeed at this course
- Not For: Students who are absolute beginners at software development as a discipline will find it difficult to follow the course
Target Audience:
- Developers
Course Duration:
- 35 hours – 5days
Course Content:
Phase 1: Hadoop Fundamentals with multinode setup (Day 1,2)
Laying the foundation
Big Data
- What is Big Data
- Dimensions of Big Data
- Big Data in Advertising
- Big Data in Banking
- Big Data in Telecom
- Big Data in eCommerce
- Big Data in Healthcare
- Big Data in Defense
- Processing options of Big Data
- Hadoop as an option
Hadoop
- What is Hadoop
- How Hadoop Works
- HDFS
- Mapreduce Deep Dive
- How Hadoop has an edge
Hadoop Ecosystem
- Sqoop
- Oozie
- Pig
- Hive
- Flume
Hadoop Hands On
- Running HDFS commands
- Running your Mapreduce program
- Running Sqoop Import and Sqoop Export
- Creating Hive tables directly from Sqoop
- Creating Hive tables
- Querying Hive tables
- Running an Oozie workflow
Phase 2: Hadoop Development (Day 3)
Become a Pro developer
Apache Spark
- What is Spark?
- Using Spark Shell
- RDD Fundamentals
- Functional Programming
- Program
RDD in Depth
- RDDs
- Creating RDDs from files
- Creating RDDs for another RDDs
- RDD operations
- Actions
- Transformations
- Pair RDDs
- Joins using RDD
Spark platforms
- Spark local mode
- Spark standalone mode
- Spark on YARN
- Spark on Mesos
Spark Hands On
- Python Spark Shell
- Scala Spark Shell
- Basic operations on RDDs
- Pair RDD Hands On
Spark SQL & Dataframes
- Spark SQL and the SQL Context
- Creating Dataframes
- Dataframe Queries and Transformations
- Saving Dataframes
- Dataframes and RDDs
Spark Dataframes Hands On
- Dataframes on a JSON file
- Dataframes on hive tables
- Dataframes on JSON
- Querying operations dataframes
Spark SQL & Dataframes
- What is Spark Streaming
- How it works
- DStreams
- Developing Spark Streaming Applications
Phase 3: Hadoop BI (Day 4) Analyze data using Pig and Hive
Hive
- Introduction
- Basic Data Analysis with Hive
- Hive Data Management
- Text Processing with Hive
- Transformations in Hive
- Optimizing Hive
- Hive Hands On
- Extending Hive
Impala
- Introduction
- Basic Data Analysis with Impala
- Text Processing with Impala
- Optimizing Impala
- Impala Hands On
Pig
- Introduction
- How Pig works
- Pig Hands On
Phase 4: NoSQL and Cluster Walkthrough (Day 5) Cluster Administration
NoSQL Databases
- Why NoSQL
- What is NoSQL Databases
- Types of NoSQL Databases
- Introduction to Cassandra and MongoDB and Hbase
- Hbase Hands On
Cloudera Manager Setup
- Why Cluster Manager software
- Cloudera Manager
- Using Cloudera Manager to setup cluster
- Cluster walkthrough
Final Test
Course Customization Options
To request a customized training for this course, please contact us to arrange.
