Advanced Certification in Big Data Analytics

• Introduction to Python
• Python basic constructs
• OOPs in Python
• NumPy for mathematical computing
• SciPy for scientific computing
• Data manipulation
• Data visualization with Matplotlib
• Implementing statistical algorithms using Python

Tools covered

• Hadoop installation and setup
• Introduction to Big Data and Hadoop
• Understanding HDFS and MapReduce
• Deep dive into MapReduce
  • Introduction to Hive
  • Advanced Hive and Impala
  • Introduction to Pig
  • Flume and Sqoop

Tools covered

• Scala programming
• Spark framework
• RDD in Spark
• DataFrames and Spark SQL
• Machine Learning using Spark (MLlib)

Tools covered

• Introduction to PySpark
• Who uses PySpark?
• Why Python for Spark?
• Values, Types, Variables
• Operands and Expressions
• Conditional Statements
• Loops
• Numbers
• Python files I/O Functions
• Strings and associated operations
• Sets and associated operations
• Lists and associated operations
• Tuples and associated operations
• Dictionaries and associated operations
• Functions
• Lambda Functions
• Global Variables, its Scope, and Returning Values
• Standard Libraries
• Object-Oriented Concepts
• Modules Used in Python
• The Import Statements
• Module Search Path
• Package Installation Ways
• Introduction to Spark Streaming
• Features of Spark Streaming
• Spark Streaming Workflow
• StreamingContext Initializing
• Discretized Streams (DStreams)
• Input DStreams, Receivers
• Transformations on DStreams
• DStreams Output Operations
• Describe Windowed Operators and Why it is Useful
• Stateful Operators
• Vital Windowed Operators
• Twitter Sentiment Analysis
• Streaming using Netcat server
• WordCount program using Kafka-Spark Streaming

Hands-On:

• Twitter Sentiment Analysis
• Streaming using Netcat server
• WordCount program using Kafka-Spark Streaming
• Spark-flume Integration
• Demonstrating Loops and Conditional Statements
• Tuple – related operations, properties, list, etc.
• List – operations, related properties
• Set – properties, associated operations
• Dictionary – operations, related properties
• Lambda – Features, Options, Syntax, Compared with the Functions
• Functions – Syntax, Return Values, Arguments, and Keyword Arguments
• Errors and Exceptions – Issue Types, Remediation
• Packages and Modules – Import Options, Modules, sys Path

Tools covered

• Spark Components & its Architecture
• Spark Deployment Modes
• Spark Web UI
• Introduction to PySpark Shell
• Submitting PySpark Job
• Writing your first PySpark Job Using Jupyter Notebook
• What is Spark RDDs?
• Stopgaps in existing computing methodologies
• How RDD solve the problem?
• What are the ways to create RDD in PySpark?
• RDD persistence and caching
• General operations: Transformation, Actions, and Functions
• Concept of Key-Value pair in RDDs
• Other pair, two pair RDDs
• RDD Lineage
• RDD Persistence
• WordCount Program Using RDD Concepts
• RDD Partitioning & How it Helps Achieve Parallelization
• Passing Functions to Spark

Hands-On:

• Building and Running Spark Application
• Spark Application Web UI
• Loading data in RDDs
• Saving data through RDDs
• RDD Transformations
• RDD Actions and Functions
• RDD Partitions
• WordCount program using RDD’s in Python

Tools covered

• Introduction to Machine Learning- What, Why and Where?
• Use Case
• Types of Machine Learning Techniques
• Why use Machine Learning for Spark?
• Applications of Machine Learning (general)
• Applications of Machine Learning with Spark
• Introduction to MLlib
• Features of MLlib and MLlib Tools
• Various ML algorithms supported by MLlib
• Supervised Learning Algorithms
• Unsupervised Learning Algorithms
• ML workflow utilities

Hands-On:

• K- Means Clustering
• Linear Regression
• Logistic Regression
• Decision Tree
• Random Forest

Tools covered

• Apache Flume and Apache Kafka
• Spark Streaming
• Case Study: Spark vs Kafka and when to use them

Tools covered