Hadoop Testing Training

1. A clear understanding of the Hadoop and Hadoop ecosystems
2. HDFS architecture, flow of data, data replication, name node and data node
3. MapReduce concepts, Mapper and Reducer functions, concurrency, shuffle and ordering
4. Unit testing of Hadoop Mapper on a MapReduce application
5. Deploy Pig for Big Data analysis and Hive for relational data analysis and test the application
6. Deep understanding of Hadoop testing and the workflow process
7. Design, formulate and implement Hadoop test scenarios, test cases and test scripts
8. Using Big Data testing tools for detecting bugs and rectifying it
9. MRUnit framework for testing MapReduce jobs without Hadoop clusters
10. Get trained for the Cloudera Hadoop Certification

• Big Data and Hadoop Developers
• Quality Assurance, Tester, Tech Support and System Administrators

No prerequisite is required to learn Hadoop testing.

• Global Hadoop market to reach $84.6 billion in 2 years – Allied Market Research
• The number of jobs for all USA data professionals will increase to 2.7 million per year – IBM
• Hadoop Testing Professionals in the US can get a salary of $132,000 – indeed.com

Hadoop is being deployed across the board in enterprises around the world. With each passing day the scale and complexities of the task that Hadoop Big Data is expected to achieve is getting bigger. With more and more Hadoop developers and Hadoop architects deployed on Hadoop projects, there is an equal and urgent necessity of Hadoop testers. This Big Data and Hadoop testing training will ensure that you gain the right skills which will open up opportunities in the Big Data testing domain as a Hadoop Tester.

Introduction to Hadoop and its constituent ecosystem, understanding MapReduce and HDFS, Big Data, factors constituting Big Data, Hadoop and Hadoop Ecosystem, MapReduce: concepts of Map, Reduce, ordering, concurrency, shuffle and reducing, Hadoop Distributed File System (HDFS) concepts and its importance, deep dive into MapReduce, execution framework, partitioner, combiner, data types, key pairs, HDFS deep dive: architecture, data replication, name node, data node, dataflow, parallel copying with DISTCP and Hadoop archives

Hands-on Exercises:

Installing Hadoop in pseudo-distributed mode, understanding important configuration files, their properties and Demon Threads, accessing HDFS from Command Line, MapReduce: basic exercises, understanding Hadoop ecosystem, introduction to Sqoop, use cases and installation, introduction to Hive, use cases and installation, introduction to Pig, use cases and installation, introduction to Oozie, use cases and installation, introduction to Flume, use cases and installation and introduction to YarnMini Project:

Importing MySQL data using Sqoop and querying it using Hive

How to develop a MapReduce application, writing unit test, the best practices for developing and writing and debugging MapReduce applications

What is Pig, Pig’s features, Pig use cases, interacting with Pig, basic data analysis with Pig, Pig Latin Syntax, loading data, simple data types, field definitions, data output, viewing the schema, filtering and sorting data and commonly-used functions

Hands-on Exercise: Using Pig for ETL processing

What is Hive, Hive schema and data storage, comparing Hive to traditional databases, Hive vs. Pig, Hive use cases, interacting with Hive, relational data analysis with Hive, Hive databases and tables, Basic HiveQL Syntax, data types, joining data sets and common built-in functions

Hands-on Exercise: Running Hive queries on the Shell, Scripts and Hue

Why Hadoop testing is important, unit testing, integration testing, performance testing, diagnostics, nightly QA test, benchmark and end-to-end tests, functional testing, release certification testing, security testing, scalability testing, commissioning and decommissioning of data nodes testing, reliability testing and release testing

Understanding the requirement, preparation of the testing estimation, test cases, test data, test bed creation, test execution, defect reporting, defect retest, daily status report delivery, test completion, ETL testing at every stage (HDFS, Hive and HBase) while loading the input (logs, files, records, etc.) using Sqoop/Flume which includes but not limited to data verification, reconciliation, user authorization and authentication testing (groups, users, privileges, etc.), report defects to the development team or manager and driving them to closure, consolidate all the defects and create defect reports and validating new feature and issues in core Hadoop

Report defects to the development team or manager and driving them to closure, consolidate all the defects and create defect reports, validating new feature and issues in core Hadoop and responsible for creating a testing framework called MRUnit for testing of MapReduce programs

Automation testing using the Oozie and data validation using the query surge tool

Test plan for HDFS upgrade and test automation and result

How to test install and configure

Project Works

Project 1: Working with MapReduce, Hive and Sqoop

Problem Statement: It describes how to import MySQL data using Sqoop and querying it using hive and also describes how to run the word count MapReduce job.

Project 2: Testing Hadoop Using MRUnit

Industry: General

Problem Statement: How to test the Hadoop application using MRUnit testing

Topics: This project involves working with MRUnit for testing the Hadoop application without spinning a cluster. You will learn how to do the map and reduce test in an application.

Highlights:

• Hadoop testing in isolation using MRUnit
• Craft the test input and push through mapper and reducer
• Deploy MapReduce driver