Intellipaat’s Big Data and Data Science master’s course will provide you with in-depth knowledge of designing, developing, and deploying Data Science and Big Data applications in the real world, along with performance tuning of applications. This course will make you a Big Data and Data Science architect, and by the end of the course, you will have expertise on Hadoop Developer, Administration, Testing, and Analysis modules, working with real-time analytics, statistical computing, parsing machine-generated data, creating NoSQL applications, and finally in the domain of Deep Learning in Artificial Intelligence. This program is specially designed by industry experts, and you will get 20 courses with 56 industry-based projects.
Online Instructor-led Courses:
Self-paced Courses:
There are no prerequisites for taking up this training program.
This Intellipaat training program has been created keeping in mind the needs of the industry. You will gain mastery in the complete aspects of Data Science and Hadoop ecosystem to take on various roles and responsibilities in the Big Data and Data Science domains at top-notch salaries.
The classes were highly interactive and also practical oriented. The office staff was cordial and co-operative. Every teaching session was recorded each day and was put on-line by the institute which was really helpful. The trainer was very patient and able to solve or give some hints to solve all the questions posed to him.
As a Big Data Engineer, this masters course doubled my interest in various technologies specifically Hadoop, Spark, Storm Scala and others. Very beneficial for those who are passionate about Big Data and Data science and want to learn everything at one place. The courses in this Master are well constructed and each topic is well explained. Thanks Intellipaat!!!
Nothing better than a master like this! Being a Data Scientist, I could gain insights into various Big Data platforms like Hadoop, Spark and Scala, which has really enriched my skill set and gave me an edge amongst coworkers. The idea of learning the most demanding Big Data and Data Science technologies through a single course is just wonderful. The trainers are doing a great job. I just love the Integration of various technologies together.
The architecture of Hadoop cluster, what is High Availability and Federation, how to setup a production cluster, various shell commands in Hadoop, understanding configuration files in Hadoop, installing single node cluster with Cloudera Manager and understanding Spark, Scala, Sqoop, Pig and Flume
Introducing Big Data and Hadoop, what is Big Data and where does Hadoop fit in, two important Hadoop ecosystem components, namely, MapReduce and HDFS, in-depth Hadoop Distributed File System – Replications, Block Size, Secondary Name node, High Availability and in-depth YARN – resource manager and node manager
Hands-on Exercise: HDFS working mechanism, data replication process, how to determine the size of the block, understanding a data node and name node
Learning the working mechanism of MapReduce, understanding the mapping and reducing stages in MR, various terminologies in MR like Input Format, Output Format, Partitioners, Combiners, Shuffle and Sort
Hands-on Exercise: How to write a Word Count program in MapReduce, how to write a Custom Partitioner, what is a MapReduce Combiner, how to run a job in a local job runner, deploying unit test, what is a map side join and reduce side join, what is a tool runner, how to use counters, dataset joining with map side and reduce side joins
Introducing Hadoop Hive, detailed architecture of Hive, comparing Hive with Pig and RDBMS, working with Hive Query Language, creation of database, table, Group by and other clauses, various types of Hive tables, HCatalog, storing the Hive Results, Hive partitioning and Buckets
Hands-on Exercise: Database creation in Hive, dropping a database, Hive table creation, how to change the database, data loading, Hive table creation, dropping and altering table, pulling data by writing Hive queries with filter conditions, table partitioning in Hive and what is a Group by clause
Indexing in Hive, the Map Side Join in Hive, working with complex data types, the Hive User-defined Functions, Introduction to Impala, comparing Hive with Impala, the detailed architecture of Impala
Hands-on Exercise: How to work with Hive queries, the process of joining table and writing indexes, external table and sequence table deployment and data storage in a different table
Apache Pig introduction, its various features, various data types and schema in Hive, the available functions in Pig, Hive Bags, Tuples and Fields
Hands-on Exercise: Working with Pig in MapReduce and local mode, loading of data, limiting data to 4 rows, storing the data into files and working with Group By, Filter By, Distinct, Cross, Split in Hive
Apache Sqoop introduction, overview, importing and exporting data, performance improvement with Sqoop, Sqoop limitations, introduction to Flume and understanding the architecture of Flume and what is HBase and the CAP theorem
Hands-on Exercise: Working with Flume to generating of Sequence Number and consuming it, using the Flume Agent to consume the Twitter data, using AVRO to create Hive Table, AVRO with Pig, creating Table in HBase and deploying Disable, Scan and Enable Table
Using Scala for writing Apache Spark applications, detailed study of Scala, the need for Scala, the concept of object oriented programming, executing the Scala code, various classes in Scala like Getters, Setters, Constructors, Abstract, Extending Objects, Overriding Methods, the Java and Scala interoperability, the concept of functional programming and anonymous functions, Bobsrockets package and comparing the mutable and immutable collections, Scala REPL, Lazy Values, Control Structures in Scala, Directed Acyclic Graph (DAG), first Spark application using SBT/Eclipse, Spark Web UI, Spark in Hadoop ecosystem.
Hands-on Exercise: Writing Spark application using Scala, understanding the robustness of Scala for Spark real-time analytics operation
Detailed Apache Spark, its various features, comparing with Hadoop, various Spark components, combining HDFS with Spark, Scalding, introduction to Scala and importance of Scala and RDD
Hands-on Exercise: The Resilient Distributed Dataset in Spark and how it helps to speed up Big Data processing
Understanding the Spark RDD operations, comparison of Spark with MapReduce, what is a Spark transformation, loading data in Spark, types of RDD operations viz. transformation and action and what is a Key/Value pair
Hands-on Exercise: How to deploy RDD with HDFS, using the in-memory dataset, using file for RDD, how to define the base RDD from external file, deploying RDD via transformation, using the Map and Reduce functions and working on word count and count log severity
The detailed Spark SQL, the significance of SQL in Spark for working with structured data processing, Spark SQL JSON support, working with XML data and parquet files, creating Hive Context, writing Data Frame to Hive, how to read a JDBC file, significance of a Spark Data Frame, how to create a Data Frame, what is schema manual inferring, how to work with CSV files, JDBC table reading, data conversion from Data Frame to JDBC, Spark SQL user-defined functions, shared variable and accumulators, how to query and transform data in Data Frames, how Data Frame provides the benefits of both Spark RDD and Spark SQL and deploying Hive on Spark as the execution engine
Hands-on Exercise: Data querying and transformation using Data Frames and finding out the benefits of Data Frames over Spark SQL and Spark RDD
Introduction to Spark MLlib, understanding various algorithms, what is Spark iterative algorithm, Spark graph processing analysis, introducing Machine Learning, K-Means clustering, Spark variables like shared and broadcast variables and what are accumulators, various ML algorithms supported by MLlib, Linear Regression, Logistic Regression, Decision Tree, Random Forest, K-means clustering techniques, building a Recommendation Engine
Hands-on Exercise: Building a Recommendation Engine
Why Kafka, what is Kafka, Kafka architecture, Kafka workflow, configuring Kafka cluster, basic operations, Kafka monitoring tools, integrating Apache Flume and Apache Kafka
Hands-on Exercise: Configuring Single Node Single Broker Cluster, Configuring Single Node Multi Broker Cluster, Producing and consuming messages, Integrating Apache Flume and Apache Kafka.
Introduction to Spark streaming, the architecture of Spark streaming, working with the Spark streaming program, processing data using Spark streaming, requesting count and DStream, multi-batch and sliding window operations and working with advanced data sources, Introduction to Spark Streaming, features of Spark Streaming, Spark Streaming workflow, initializing StreamingContext, Discretized Streams (DStreams), Input DStreams and Receivers, transformations on DStreams, Output Operations on DStreams, Windowed Operators and why it is useful, important Windowed Operators, Stateful Operators.
Hands-on Exercise: Twitter Sentiment Analysis, streaming using netcat server, Kafka-Spark Streaming and Spark-Flume Streaming
Create a 4-node Hadoop cluster setup, running the MapReduce Jobs on the Hadoop cluster, successfully running the MapReduce code and working with the Cloudera Manager setup
Hands-on Exercise: The method to build a multi-node Hadoop cluster using an Amazon EC2 instance and working with the Cloudera Manager
The overview of Hadoop configuration, the importance of Hadoop configuration file, the various parameters and values of configuration, the HDFS parameters and MapReduce parameters, setting up the Hadoop environment, the Include and Exclude configuration files, the administration and maintenance of name node, data node directory structures and files, what is a File system image and understanding Edit log.
Hands-on Exercise: The process of performance tuning in MapReduce
Introduction to the checkpoint procedure, name node failure and how to ensure the recovery procedure, Safe Mode, Metadata and Data backup, various potential problems and solutions, what to look for and how to add and remove nodes
Hands-on Exercise: How to go about ensuring the MapReduce File System Recovery for different scenarios, JMX monitoring of the Hadoop cluster, how to use the logs and stack traces for monitoring and troubleshooting, using the Job Scheduler for scheduling jobs in the same cluster, getting the MapReduce job submission flow, FIFO schedule and getting to know the Fair Scheduler and its configuration
How ETL tools work in Big Data industry, introduction to ETL and data warehousing, working with prominent use cases of Big Data in ETL industry and end-to-end ETL PoC showing Big Data integration with ETL tool
Hands-on Exercise: Connecting to HDFS from ETL tool and moving data from Local system to HDFS, moving data from DBMS to HDFS, working with Hive with ETL Tool and creating MapReduce job in ETL tool
Working towards the solution of the Hadoop project solution, its problem statements and the possible solution outcomes, preparing for the Cloudera certifications, points to focus for scoring the highest marks and tips for cracking Hadoop interview questions
Hands-on Exercise: The project of a real-world high value Big Data Hadoop application and getting the right solution based on the criteria set by the Intellipaat team
Why testing is important, Unit testing, Integration testing, Performance testing, Diagnostics, Nightly QA test, Benchmark and end-to-end tests, Functional testing, Release certification testing, Security testing, Scalability testing, Commissioning and Decommissioning of data nodes testing, Reliability testing and Release testing
Understanding the Requirement, preparation of the Testing Estimation, Test Cases, Test Data, Test Bed Creation, Test Execution, Defect Reporting, Defect Retest, Daily Status report delivery, Test completion, ETL testing at every stage (HDFS, Hive and HBase) while loading the input (logs, files, records, etc.) using Sqoop/Flume which includes but not limited to data verification, Reconciliation, User Authorization and Authentication testing (Groups, Users, Privileges, etc.), reporting defects to the development team or manager and driving them to closure, consolidating all the defects and create defect reports, validating new feature and issues in Core Hadoop
Report defects to the development team or manager and driving them to closure, consolidate all the defects and create defect reports, responsible for creating a testing framework called MRUnit for testing of MapReduce programs
Automation testing using the OOZIE and data validation using the query surge tool
Test plan for HDFS upgrade, test automation and result
How to test, install and configure
Project 1: Working with MapReduce, Hive and Sqoop
Industry: General
Problem Statement: How to successfully import data using Sqoop into HDFS for data analysis
Topics: As part of this project, you will work on the various Hadoop components like MapReduce, Apache Hive and Apache Sqoop. You will have to work with Sqoop to import data from relational database management system like MySQL data into HDFS. You need to deploy Hive for summarizing data, querying and analysis. You have to convert SQL queries using HiveQL for deploying MapReduce on the transferred data. You will gain considerable proficiency in Hive and Sqoop after the completion of this project.
Highlights:
Project 2: Work on MovieLens data for finding the top movies
Industry: Media and Entertainment
Problem Statement: How to create the top-ten-movies list using the MovieLens data
Topics: In this project you will work exclusively on data collected through MovieLens available rating data sets. The project involves writing MapReduce program to analyze the MovieLens data and creating the list of top ten movies. You will also work with Apache Pig and Apache Hive for working with distributed datasets and analyzing it.
Highlights:
Project 3: Hadoop YARN Project; End-to-end PoC
Industry: Banking
Problem Statement: How to bring the daily data (incremental data) into the Hadoop Distributed File System
Topics: In this project, we have transaction data which is daily recorded/stored in the RDBMS. Now this data is transferred everyday into HDFS for further Big Data Analytics. You will work on live Hadoop YARN cluster. YARN is part of the Hadoop ecosystem that lets Hadoop to decouple from MapReduce and deploy more competitive processing and wider array of applications. You will work on the YARN central resource manager.
Highlights:
Project 4: Table Partitioning in Hive
Industry: Banking
Problem Statement: How to improve the query speed using Hive data partitioning
Topics: This project involves working with Hive table data partitioning. Ensuring the right partitioning helps to read the data, deploy it on the HDFS and run the MapReduce jobs at a much faster rate. Hive lets you partition data in multiple ways. This will give you hands-on experience in partitioning of Hive tables manually, deploying single SQL execution in dynamic partitioning and bucketing of data so as to break it into manageable chunks.
Highlights:
Project 5: Connecting Pentaho with Hadoop Ecosystem
Industry: Social Network
Problem Statement: How to deploy ETL for data analysis activities
Topics: This project lets you connect Pentaho with the Hadoop ecosystem. Pentaho works well with HDFS, HBase, Oozie and ZooKeeper. You will connect the Hadoop cluster with Pentaho data integration, analytics, Pentaho server and report designer. This project will give you complete working knowledge on the Pentaho ETL tool.
Highlights:
Project 6: Multi-node Cluster Setup
Industry: General
Problem Statement: How to setup a Hadoop real-time cluster on Amazon EC2
Topics: This is a project that gives you opportunity to work on real world Hadoop multi-node cluster setup in a distributed environment. You will get a complete demonstration of working with various Hadoop cluster master and slave nodes, installing Java as a prerequisite for running Hadoop, installation of Hadoop and mapping the nodes in the Hadoop cluster.
Highlights:
Project 7: Hadoop Testing Using MRUnit
Industry: General
Problem Statement: How to test MapReduce applications
Topics: In this project, you will gain proficiency in Hadoop MapReduce code testing using MRUnit. You will learn about real-world scenarios of deploying MRUnit, Mockito and PowerMock. This will give you hands-on experience in various testing tools for Hadoop MapReduce. After completion of this project you will be well-versed in test-driven development and will be able to write light-weight test units that work specifically on the Hadoop architecture.
Highlights:
Project 8: Hadoop Web Log Analytics
Industry: Internet Services
Problem Statement: How to derive insights from web log data
Topics: This project is involved with making sense of all the web log data in order to derive valuable insights from it. You will work with loading the server data onto a Hadoop cluster using various techniques. The web log data can include various URLs visited, cookie data, user demographics, location, date and time of web service access, etc. In this project, you will transport the data using Apache Flume or Kafka, workflow and data cleansing using MapReduce, Pig or Spark. The insight thus derived can be used for analyzing customer behavior and predict buying patterns.
Highlights:
Project 9: Hadoop Maintenance
Industry: General
Problem Statement: How to administer a Hadoop cluster
Topics: This project is involved with working on the Hadoop cluster for maintaining and managing it. You will work on a number of important tasks that include recovering of data, recovering from failure, adding and removing of machines from the Hadoop cluster and onboarding of users on Hadoop.
Highlights:
Project 10: Twitter Sentiment Analysis
Industry: Social Media
Problem Statement: Find out what is the reaction of the people to the demonetization move by India by analyzing their tweets
Topics: This Project involves analyzing the tweets of people by going through what they are saying about the demonetization decision taken by the Indian government. Then you look for key phrases and words and analyze them using the dictionary and the value attributed to them based on the sentiment that they are conveying.
Highlights:
Project 11: Analyzing IPL T20 Cricket
Industry: Sports and Entertainment
Problem Statement: Analyze the entire cricket match and get answers to any question regarding the details of the match
Topics: This project involves working with the IPL dataset that has information regarding batting, bowling, runs scored, wickets taken and more. This dataset is taken as input, and then it is processed so that the entire match can be analyzed based on the user queries or needs.
Highlights:
Project 1: Movie Recommendation
Industry: Entertainment
Problem Statement: How to recommend the most appropriate movie to a user based on his taste
Topics: This is a hands-on Apache Spark project deployed for the real-world application of movie recommendations. This project helps you gain essential knowledge in Spark MLlib which is a Machine Learning library; you will know how to create collaborative filtering, regression, clustering and dimensionality reduction using Spark MLlib. Upon finishing the project, you will have first-hand experience in the Apache Spark streaming data analysis, sampling, testing and statistics, among other vital skills.
Highlights:
Project 2: Twitter API Integration for Tweet Analysis
Industry: Social Media
Problem Statement: Analyzing the user sentiment based on the tweet
Topics: This is a hands-on Twitter analysis project using the Twitter API for analyzing of tweets. You will integrate the Twitter API and do programming using Python or PHP for developing the essential server-side codes. Finally, you will be able to read the results for various operations by filtering, parsing and aggregating it depending on the tweet analysis requirement.
Highlights:
Project 3: Data Exploration Using Spark SQL – Wikipedia Data Set
Industry: Internet
Problem Statement: Making sense of Wikipedia data using Spark SQL
Topics: In this project you will be using the Spark SQL tool for analyzing the Wikipedia data. You will gain hands-on experience in integrating Spark SQL for various applications like batch analysis, Machine Learning, visualizing and processing of data and ETL processes, along with real-time analysis of data.
Highlights:
Introducing Scala, deployment of Scala for Big Data applications and Apache Spark analytics, Scala REPL, Lazy Values, Control Structures in Scala, Directed Acyclic Graph (DAG), First Spark Application Using SBT/Eclipse, Spark Web UI and Spark in Hadoop Ecosystem.
The importance of Scala, the concept of REPL (Read Evaluate Print Loop), deep dive into Scala pattern matching, type interface, higher-order function, currying, traits, application space and Scala for data analysis
Learning about the Scala Interpreter, static object timer in Scala and testing string equality in Scala, implicit classes in Scala, the concept of currying in Scala and various classes in Scala
Learning about the Classes concept, understanding the constructor overloading, various abstract classes, the hierarchy types in Scala, the concept of object equality and the val and var methods in Scala
Understanding sealed traits, wild, constructor, tuple, variable pattern and constant pattern
Understanding traits in Scala, the advantages of traits, linearization of traits, the Java equivalent and avoiding of boilerplate code
Implementation of traits in Scala and Java and handling of multiple traits extending
Introduction to Scala collections, classification of collections, the difference between Iterator and Iterable in Scala and example of list sequence in Scala
The two types of collections in Scala, Mutable and Immutable collections, understanding lists and arrays in Scala, the list buffer and array buffer, queue in Scala and double-ended queue Deque, Stacks, Sets, Maps and Tuples in Scala
Introduction to Scala packages and imports, the selective imports, the Scala test classes, introduction to JUnit test class, JUnit interface via JUnit 3 suite for Scala test, packaging of Scala applications in Directory Structure and examples of Spark Split and Spark Scala
Introduction to Spark, how Spark overcomes the drawbacks of working on MapReduce, understanding in-memory MapReduce, interactive operations on MapReduce, Spark stack, fine vs. coarse-grained update, Spark stack, Spark Hadoop YARN, HDFS Revision, YARN Revision, the overview of Spark and how it is better than Hadoop, deploying Spark without Hadoop, Spark history server and Cloudera distribution
Spark installation guide, Spark configuration, memory management, executor memory vs. driver memory, working with Spark Shell, the concept of resilient distributed datasets (RDD), learning to do functional programming in Spark and the architecture of Spark
Spark RDD, creating RDDs, RDD partitioning, operations and transformation in RDD, deep dive into Spark RDDs, the RDD general operations, a read-only partitioned collection of records, using the concept of RDD for faster and efficient data processing, RDD action for collect, count, collects map, save-as-text-files and pair RDD functions
Understanding the concept of Key–Value pair in RDDs, learning how Spark makes MapReduce operations faster, various operations of RDD, MapReduce interactive operations, fine and coarse-grained update and Spark stack
Comparing the Spark applications with Spark Shell, creating a Spark application using Scala or Java, deploying a Spark application, Scala built application, creation of mutable list, set and set operations, list, tuple, concatenating list, creating application using SBT, deploying application using Maven, the web user interface of Spark application, a real-world example of Spark and configuring of Spark
Learning about Spark parallel processing, deploying on a cluster, introduction to Spark partitions, file-based partitioning of RDDs, understanding of HDFS and data locality, mastering the technique of parallel operations, comparing repartition and coalesce and RDD actions
The execution flow in Spark, understanding the RDD persistence overview, Spark execution flow and Spark terminology, distribution shared memory vs. RDD, RDD limitations, Spark shell arguments, distributed persistence, RDD lineage, Key–Value pair for sorting implicit conversions like CountByKey, ReduceByKey, SortByKey and AggregateByKey
Introduction to Machine Learning, types of Machine Learning, introduction to MLlib, various ML algorithms supported by MLlib, Linear Regression, Logistic Regression, Decision Tree, Random Forest, K-means clustering techniques and building a Recommendation Engine
Hands-on Exercise: Building a Recommendation Engine
Why Kafka, what is Kafka, Kafka architecture, Kafka workflow, configuring Kafka cluster, basic operations, Kafka monitoring tools and integrating Apache Flume and Apache Kafka
Hands-on Exercise: Configuring Single Node Single Broker Cluster, Configuring Single Node Multi Broker Cluster, Producing and consuming messages and integrating Apache Flume and Apache Kafka
Introduction to Spark Streaming, features of Spark Streaming, Spark Streaming workflow, initializing StreamingContext, Discretized Streams (DStreams), Input DStreams and Receivers, transformations on DStreams, Output Operations on DStreams, Windowed Operators and why it is useful, important Windowed Operators and Stateful Operators
Hands-on Exercise: Twitter Sentiment Analysis, streaming using netcat server, Kafka–Spark Streaming and Spark–Flume Streaming
Introduction to various variables in Spark like shared variables and broadcast variables, learning about accumulators, the common performance issues and troubleshooting the performance problems
Learning about Spark SQL, the context of SQL in Spark for providing structured data processing, JSON support in Spark SQL, working with XML data, parquet files, creating Hive context, writing Data Frame to Hive, reading JDBC files, understanding the Data Frames in Spark, creating Data Frames, manual inferring of schema, working with CSV files, reading JDBC tables, Data Frame to JDBC, user-defined functions in Spark SQL, shared variables and accumulators, learning to query and transform data in Data Frames, how Data Frame provides the benefit of both Spark RDD and Spark SQL and deploying Hive on Spark as the execution engine
Learning about the scheduling and partitioning in Spark, hash partition, range partition, scheduling within and around applications, static partitioning, dynamic sharing, fair scheduling, Map partition with index, the Zip, GroupByKey, Spark master high availability, standby masters with ZooKeeper, Single-node Recovery with Local File System and High Order Functions
Project 1: Movie Recommendation
Topics: This is a project wherein you will gain hands-on experience in deploying Apache Spark for the movie recommendation. You will be introduced to the Spark Machine Learning Library, a guide to MLlib algorithms and coding which is a Machine Learning library. You will understand how to deploy collaborative filtering, clustering, regression and dimensionality reduction in MLlib. Upon the completion of the project, you will gain experience in working with streaming data, sampling, testing and statistics.
Project 2: Twitter API Integration for Tweet Analysis
Topics: With this project, you will learn to integrate Twitter API for analyzing tweets. You will write codes on the server side using any of the scripting languages, like PHP, Ruby or Python, for requesting the Twitter API and get the results in JSON format. You will then read the results and perform various operations like aggregation, filtering and parsing as per the need to come up with tweet analysis.
Project 3: Data Exploration Using Spark SQL – Wikipedia Dataset
Topics: This project lets you work with Spark SQL. You will gain experience in working with Spark SQL for combining it with ETL applications, real-time analysis of data, performing batch analysis, deploying Machine Learning, creating visualizations and processing of graphs.
1.1 What is Data Science
1.2 Significance of Data Science in today’s digitally-driven world, applications of Data Science, lifecycle of Data Science, components of the Data Science lifecycle
1.3 Introduction to big data and Hadoop, introduction to Machine Learning and Deep Learning,
1.4 Introduction to R programming and R Studio
Hands-on Exercise
1. Installation of R Studio
2. Implementing simple mathematical operations and logic using R operators, loops, if statements and switch cases.
2.1 Introduction to data exploration
2.2 Importing and exporting data to/from external sources
2.3 What is data exploratory analysis, data importing, dataframes
2.4 working with dataframes, accessing individual elements, vectors and factors, operators, in-built functions, conditional, looping statements and user-defined functions, matrix, list and array.
Hands-on Exercise
1. Accessing individual elements of customer churn data
2. Modifying and extracting the results from the dataset using user-defined functions in R.
3.1 Need for Data Manipulation
3.2 Introduction to dplyr package
3.3 Selecting one or more columns with select() function, Filtering out records on the basis of a condition with filter() function, Adding new columns with the mutate() function, Sampling & Counting
3.4 Combining different functions with the pipe operator, Implementing sql like operations with sqldf.
Hands-on Exercise
1. Implementing dplyr
2. perform various operations for abstracting over how data is manipulated and stored.
4.1 Introduction to visualization
4.2 Different types of graphs, Introduction to grammar of graphics & ggplot2 package, Understanding categorical distribution with geom_bar() function, understanding numerical distribution with geom_hist() function, building frequency polygons with geom_freqpoly(), making a scatter-plot with geom_pont() function
4.3 Multivariate analysis with geom_boxplot
4.4 Univariate Analysis with Bar-plot, histogram and Density Plot, multivariate distribution
4.5 Bar-plots for categorical variables using geom_bar(), adding themes with the theme() layer
4.6 Visualization with plotly package & building web applications with shinyR, frequency-plots with geom_freqpoly(), multivariate distribution with scatter-plots and smooth lines, continuous vs categorical with box-plots, subgrouping the plots
4.7 Working with co-ordinates and themes to make the graphs more presentable, Intro to plotly & various plots, visualization with ggvis package
4.8 Geographic visualization with ggmap(), building web applications with shinyR.
Hands-on Exercise
1. Creating data visualization to understand the customer churn ratio using charts using ggplot2
2. Plotly for importing and analyzing data into grids
3. Visualize tenure, monthly charges, total charges and other individual columns by using the scatter plot.
5.1 Why do we need Statistics?
5.2 Categories of Statistics, Statistical Terminologies, Types of Data, Measures of Central Tendency, Measures of Spread
5.3 Correlation & Covariance,Standardization & Normalization,Probability & Types of Probability, Hypothesis Testing, Chi-Square testing, ANOVA, normal distribution, binary distribution.
Hands-on Exercise
1. Building a statistical analysis model that uses quantifications, representations, experimental data for gathering
2. Reviewing, analyzing and drawing conclusions from data.
6.1 Introduction to Machine Learning
6.2 Introduction to Linear Regression, predictive modeling with Linear Regression, simple Linear and multiple Linear Regression, concepts and formulas, assumptions and residual diagnostics in Linear Regression, building simple linear model
6.3 Predicting results and finding p-value, introduction to logistic regression
6.4 Comparing linear regression and logistics regression, bivariate & multi-variate logistic regression
6.5 Confusion matrix & accuracy of model, threshold evaluation with ROCR, Linear Regression concepts and detailed formulas, various assumptions of Linear Regression,residuals, qqnorm(), qqline(), understanding the fit of the model, building simple linear model, predicting results and finding p-value
6.6 understanding the summary results with Null Hypothesis, p-value & F-statistic,
building linear models with multiple independent variables.
Hands-on Exercise
1. Modeling the relationship within the data using linear predictor functions.
2. Implementing Linear & Logistics Regression in R by building model with ‘tenure’ as dependent variable and multiple independent variables.
7.1 Introduction to Logistic Regression
7.2 Logistic Regression Concepts, Linear vs Logistic regression, math behind Logistic Regression
7.3 Detailed formulas, logit function and odds, Bi-variate logistic Regression, Poisson Regression
7.4 Building simple “binomial” model and predicting result, confusion matrix and Accuracy, true positive rate, false positive rate, and confusion matrix for evaluating built model, threshold evaluation with ROCR
7.5 Finding the right threshold by building the ROC plot, cross validation & multivariate logistic regression, building logistic models with multiple independent variables
7.6 Real-life applications of Logistic Regression.
Hands-on Exercise
1. Implementing predictive analytics by describing the data
2. explaining the relationship between one dependent binary variable and one or more binary variables.
3. You will use glm() to build a model and use ‘Churn’ as the dependent variable.
8.1 What is classification and different classification techniques
8.2 Introduction to Decision Tree
8.3 Algorithm for decision tree induction, building a decision tree in R
8.4 Creating a perfect Decision Tree, Confusion Matrix, Regression trees vs Classification trees
8.5 Introduction to ensemble of trees and bagging
8.6 Random Forest concept, implementing Random Forest in R
8.7 what is Naive Bayes, Computing Probabilities, Impurity Function – Entropy, understand the concept of information gain for right split of node
8.8 Impurity Function – Information gain, understand the concept of Gini index for right split of node
8.9 Impurity Function – Gini index, understand the concept of Entropy for right split of node, overfitting & pruning, pre-pruning, post-pruning, cost-complexity pruning, pruning decision tree and predicting values, find the right no of trees and evaluate performance metrics.
Hands-on Exercise
1. Implementing Random Forest for both regression and classification problems.
2. You will build a tree, prune it by using ‘churn’ as the dependent variable and build a Random Forest with the right number of trees,
3. using ROCR for performance metrics.
9.1 What is Clustering & it’s Use Cases, what is K-means Clustering
9.2 What is Canopy Clustering
9.3 What is Hierarchical Clustering
9.4 Introduction to Unsupervised Learning
9.5 Feature extraction & clustering algorithms, k-means clustering algorithm
9.6 Theoretical aspects of k-means, and k-means process flow, K-means in R, implementing K-means on the data-set and finding the right no. of clusters using Scree-plot
9.7 Hierarchical clustering & Dendogram, understand Hierarchical clustering, implement it in R and have a look at Dendograms
9.8 Principal Component Analysis, explanation of Principal Component Analysis in detail, PCA in R, implementing PCA in R.
Hands-on Exercise
1. Deploying unsupervised learning with R to achieve clustering and dimensionality reduction
2. K-means clustering for visualizing and interpreting results for the customer churn data.
10.1 Introduction to association rule Mining & Market Basket Analysis
10.2 Measures of Association Rule Mining: Support, Confidence, Lift, Apriori algorithm & implementing it in R
10.3 Introduction to Recommendation Engine
10.4 User-based collaborative filtering & Item-Based Collaborative Filtering, implementing Recommendation Engine in R, user-Based and item-Based
10.5 Recommendation Use-cases
Hands-on Exercise
1. Deploying association analysis as a rule-based machine learning method,
2. Identifying strong rules discovered in databases with measures based on interesting discoveries.
11.1 Introducing Artificial Intelligence and Deep Learning
11.2 what is an Artificial Neural Network, TensorFlow – computational framework for building AI models
11.3 Fundamentals of building ANN using TensorFlow, working with TensorFlow in R.
12.1 What is Time Series
12.2 Techniques and applications, components of Time Series, moving average, smoothing techniques, exponential smoothing
12.3 Univariate time series models, multivariate time series analysis
12.4 Arima model
12.5 Time Series in R, sentiment analysis in R (Twitter sentiment analysis), text analysis.
Hands-on Exercise
1. Analyzing time series data
2. Sequence of measurements that follow a non-random order to identify the nature of phenomenon and to forecast the future values in the series.
13.1 Introduction to Support Vector Machine (SVM)
13.2 Data classification using SVM
13.3 SVM Algorithms using Separable and Inseparable cases
13.4 Linear SVM for identifying margin hyperplane.
14.1 What is Bayes theorem
14.2 What is Naïve Bayes Classifier
14.3 Classification Workflow
14.4 How Naive Bayes classifier works, Classifier building in Scikit-learn
14.5 Building a probabilistic classification model using Naïve Bayes, Zero Probability Problem.
15.1 Introduction to concepts of Text Mining
15.2 Text Mining use cases, understanding and manipulating text with ‘tm’ & ‘stringR’
15.3 Text Mining Algorithms, Quantification of Text
15.4 Term Frequency-Inverse Document Frequency (TF-IDF), After TF-IDF.
01 – The Market Basket Analysis (MBA) case study
1.1 This case study is associated with the modeling technique of Market Basket Analysis where you will learn about loading of data, various techniques for plotting the items and running the algorithms.
1.2 It includes finding out what are the items that go hand in hand and hence can be clubbed together.
1.3 This is used for various real world scenarios like a supermarket shopping cart and so on.
02 – Logistic Regression Case Study
2.1 In this case study you will get a detailed understanding of the advertisement spends of a company that will help to drive more sales
2.2 You will deploy logistic regression to forecast the future trends
2.3 Detect patterns, uncover insights and more all through the power of R programming.
2.4 Due to this the future advertisement spends can be decided and optimized for higher revenues.
03 – Multiple Regression Case Study
3.1 You will understand how to compare the miles per gallon (MPG) of a car based on the various parameters.
3.2 You will deploy multiple regression and note down the MPG for car make, model, speed, load conditions, etc.
3.3 It includes the model building, model diagnostic, checking the ROC curve, among other things.
04 – Receiver Operating Characteristic (ROC) case study
4.1 You will work with various data sets in R,
4.2 Deploy data exploration methodologies,
4.3 Build scalable models
4.4 Predict the outcome with highest precision, diagnose the model that you have created with various real world data, check the ROC curve and more.
Project 01 – Market Basket Analysis
Domain – Inventory Management
Problem Statement – As a new manager in the company, you are assigned the task of increasing cross selling
Topics – Association Rule Mining, Data Extraction, Data Manipulation
Highlights
Project 02 – Credit Card Fraud Detection
Domain – Banking
Problem Statement – Analysis of probability of being involved in a fraudulent operation
Topics – Algorithms, V17 Predictor, Data Visualization, R Language
Highlights
Project 03 – Data Cleaning using Census Dataset
Domain – Government
Problem Statement – Performing Data Cleansing operation on a raw dataset
Topics – Data Analysis, Data preprocessing, Cleaning Ops, Data Visualization, R Language
Highlights
Project 04 – Loan Approval Prediction
Domain -Banking
Problem Statement – Prediction of approval rate of a loan by using multiple labels
Topics – Data Analysis, Data preprocessing, Cleaning Ops, Data Visualization, R Language
Highlights
Project 05 – Book Recommendation System
Domain – E-Commerce
Problem Statement – Creating a model, which can recommend books, based on user interest
Topics – Data Cleaning, Data Visualization, User Based Collaborative Filtering
Highlights
Project 06 – Netflix Recommendation System
Domain – E-Commerce
Problem Statement Simulating the Netflix Recommendation System
Topics – Data Cleaning, Data Visualization, Distribution, Recommender Lab
Highlights
Project 07 – Creating a Pokemon Game using Machine Learning
Domain – Gaming
Problem Statement – Creating a game engine for Pokemon using Machine Learning
Topics – Decision Tress, Regression, Data Cleaning, Data Visualization
Highlights
Case Study 01 – Introduction to R Programming
Problem Statement – Working with various operators in R
Topics – Arithmetic Operators, Relational Operators, Logical Operators
Highlights
Case Study 02 – Solving Customer Churn using Data Exploration
Problem Statement – Understanding what to do to reduce customer churn using Data Exploration
Topics – Data Exploration
Highlights
Case Study 03 – Creating Data Structures in R
Problem Statement – Implementing various Data Structures in R for various scenarios
Topics – Vectors, list, Matrix, Array
Highlights
Case Study 04 – Implementing SVD in R
Problem Statement – Understanding the use Single Value Decomposition in R by making use of the MovieLense Dataset
Topics – 5-fold cross validation, Real Rating Matrix
Highlights
Case Study 05 – Time Series Analysis
Problem Statement – Performing TSA and understanding concepts of ARIMA for a given scenario
Topics – Time Series Analysis, R Language, Data Visualization, ARIMA model
Highlights
1.1 What is Data Science, what does a data scientist do
1.2 Various examples of Data Science in the industries
1.3 How Python is deployed for Data Science applications
1.4 Various steps in Data Science process like data wrangling, data exploration and selecting the model.
1.5 Introduction to Python programming language
1.6 Important Python features, how is Python different from other programming languages
1.7 Python installation, Anaconda Python distribution for Windows, Linux and Mac
1.8 How to run a sample Python script, Python IDE working mechanism
1.9 Running some Python basic commands
1.10 Python variables, data types and keywords.
Hands-on Exercise – Installing Python Anaconda for the Windows, Linux and Mac
2.1 Introduction to a basic construct in Python
2.2 Understanding indentation like tabs and spaces
2.3 Python built-in data types
2.4 Basic operators in Python
2.5 Loop and control statements like break, if, for, continue, else, range() and more.
Hands-on Exercise –
1.Write your first Python program
2. Write a Python function (with and without parameters)
3. Use Lambda expression
4. Write a class
5. Create a member function and a variable
6. Create an object and write a for loop to print all odd numbers
3.1 Central Tendency
3.2 Variabiltiy
3.3 Hypothesis Testing
3.4 Anova
3.5 Correlation
3.6 Regression
3.7 Probability Definitions and Notation
3.8 Joint Probabilities
3.9 The Sum Rule, Conditional Probability, and the Product Rule
3.10 Baye’s Theorem
Hands-on Exercise –
1. We will analyze both categorical data and quantitative data
2. Focusing on specific case studies to help solidify the week’s statistical concepts
4.1 Understanding the OOP paradigm like encapsulation, inheritance, polymorphism and abstraction
4.2 What are access modifiers, instances, class members
4.3 Classes and objects
4.4 Function parameter and return type functions
4.5 Lambda expressions.
Hands-on Exercise –
1. Writing a Python program and incorporating the OOP concepts
5.1 Introduction to mathematical computing in Python
5.2 What are arrays and matrices, array indexing, array math, Inspecting a numpy array, Numpy array manipulation
Hands-on Exercise –
1. How to import numpy module
2. Creating array using ND-array
3. Calculating standard deviation on array of numbers and calculating correlation between two variables.
6.1 Introduction to scipy, building on top of numpy
6.2 What are the characteristics of scipy
6.3 Various subpackages for scipy like Signal, Integrate, Fftpack, Cluster, Optimize, Stats and more, Bayes Theorem with scipy.
Hands-on Exercise:
1. Importing of scipy
2. Applying the Bayes theorem on the given dataset.
7.1 What is a data Manipulation. Using Pandas library
7.2 Numpy dependency of Pandas library
7.3 Series object in pandas
7.4 Dataframe in Pandas
7.5 Loading and handling data with Pandas
7.6 How to merge data objects
7.7 Concatenation and various types of joins on data objects, exploring dataset
Hands-on Exercise –
1. Doing data manipulation with Pandas by handling tabular datasets that includes variable types like float, integer, double and others.
2. Cleaning dataset, Manipulating dataset, Visualizing dataset
8.1 Introduction to Matplotlib
8.2 Using Matplotlib for plotting graphs and charts like Scatter, Bar, Pie, Line, Histogram and more
8.3 Matplotlib API
Hands-on Exercise –
1. Deploying Matplotlib for creating pie, scatter, line and histogram.
2. Subplots and Pandas built-in data visualization.
9.1 Revision of topics in Python (Pandas, Matplotlib, numpy, scikit-Learn)
9.2 Introduction to machine learning
9.3 Need of Machine learning
9.4 Types of machine learning and workflow of Machine Learning
9.5 Uses Cases in Machine Learning, its various arlogrithms
9.6 What is supervised learning
9.7 What is Unsupervised Learning
Hands-on Exercise –
1. Demo on ML algorithms
10.1 What is linear regression
10.2 Step by step calculation of Linear Regression
10.3 Linear regression in Python
10.4 Logistic Regression
10.5 What is classification
10.6 Decision Tree, Confusion Matrix, Random Forest, Naïve Bayes classifier (Self paced), Support Vector Machine(self paced), xgboost(self paced)
Hands-on Exercise – Using Python library Scikit-Learn for coming up with Random Forest algorithm to implement supervised learning.
11.1 Introduction to unsupervised learning
11.2 Use cases of unsupervised learning
11.3 What is clustering
11.4 Types of clustering(self-paced)-Exclusive clustering, Overlapping Clustering, Hierarchical Clustering(self-paced)
11.5 What is K-means clustering
11.6 Step by step calculation of k-means algorithm
11.7 Association Rule Mining(self-paced), Market Basket Analysis(self-paced), Measures in association rule mining(self-paced)-support, confidence, lift
11.8 Apriori Algorithm
Hands-on Exercise –
1. Setting up the Jupyter notebook environment
2. Loading of a dataset in Jupyter
3. Algorithms in Scikit-Learn package for performing Machine Learning techniques and training a model to search a grid.
4. Practice on k-means using Scikit
5. Practice on Apriori
12.1 Introduction to pyspark
12.2 Who uses pyspark, need of spark with python
12.3 Pyspark installation
12.4 Pyspark fundamentals
12.5 Advantage over mapreduce, pyspark
12.6 Use-cases pyspark and demo.
Hands-on Exercise:
1. Demonstrating Loops and Conditional Statements
2. Tuple – related operations, properties, list, etc.
3. List – operations, related properties
4. Set – properties, associated operations, dictionary – operations, related properties.
13.1 Introduction to Dimensionality
13.2 Why Dimensionality Reduction
13.3 PCA
13.4 Factor Analysis
13.5 LDA
Hands-on Exercise –
Practice Dimensionality reduction Techniques : PCA, Factor Analysis, t-SNE, Random Forest, Forward and Backward feature
14.1 White Noise
14.2 AR model
14.3 MA model
14.4 ARMA model
14.5 ARIMA model
14.6 Stationarity
14.7 ACF & PACF
Hands-on Exercise –
1. Create AR model
2. Create MA model
3. Create ARMA model
Project 01: Analysing the trends of COVID-19 with Python
Industry: Analytics
Problem Statement: Understanding, the trend of covid 19 spread and if the restrictions imposed by governments around the world has helped us curb the COVID cases and by what degree
Topics: In this project we will use Data Science and Python perfrom visualization to better understand the data we currently have on COVID 19 as well as using Time Series Analysis in order to make a perdiction about future cases if the current trend as observed thus far continues.
Highlights:
Project 02: Analyzing the naming trends using Python
Problem Statement: The dataset is in Zipped format, we have to extract the dataset in the program, visualize the number of male and female babies born in a particular year, and find out popular baby names.
Topics: Algorithms, Python programming
Highlights:
Project 03: Performing Analysis on Customer Churn Dataset
Problem Statement: Analysis of Employment reliability of employees in the telecom industry
Topics: Algorithms, Manipulation, Data Visualization, Python Language
Highlights:
Project 04: Netflix-Recommendation system
Problem Statement: Analysis of movies dataset and recommendation of movies with respect to ratings.
Topics: Algorithms, Python, Recommendation engine
Highlights:
Project 05: Python Web Scraping for Data Science
In this project you will be introduced to the process of web scraping using Python. It involves installation of Beautiful Soup, web scraping libraries, working on common data and page format on the web, learning the important kinds of objects, Navigable String, deploying the searching tree, navigation options, parser, search tree, searching by CSS class, list, function and keyword argument.
Case Study 01: OOPS in Python
Problem Statement: Create multiple methods using OOPS concept
Topics: Parameterization, OOPS, Classes
Highlights:
Case Study 02: Working with NumPy
Problem Statement: Working with NumPy library to solve various problems in Python
Topics: NumPy
Highlights:
Case Study 03: Visualizing and Analyzing the Customer Churn dataset using R.
Problem Statement: Analyzing the data by building some aesthetic graphs to make better sense of the data.
Topics: Plots, ggplot2, Python Language
Highlights:
Case Study 04: Building models with the help of Machine Learning Algorithms
Problem Statement: designing tree-based models on ‘Heart’ dataset.
Topics: ML Algorithms, Python Language
Highlights:
Case Study 05: Visualizing and Analyzing the Customer Churn dataset using R.
Problem Statement: Analyzing the data by building some aesthetic graphs to make better sense of the data.
Topics: Plots, ggplot2, Python Language
Highlights
Case Study 06: Building models with the help of Machine Learning Algorithms
Problem Statement: designing tree-based models on ‘Heart’ dataset.
Topics: ML Algorithms, Python Language
Highlights:
What is data visualization?, comparison and benefits against reading raw numbers, real use cases from various business domains, some quick and powerful examples using Tableau without going into the technical details of Tableau, installing Tableau, Tableau interface, connecting to DataSource, Tableau data types, and data preparation.
Installation of Tableau Desktop, architecture of Tableau, interface of Tableau (Layout, Toolbars, Data Pane, Analytics Pane, etc.) how to start with Tableau, and the ways to share and export the work done in Tableau.
Hands-on Exercise: Play with Tableau desktop, learn about the interface, and share and export existing works.
Connection to Excel, cubes and PDFs, management of metadata and extracts, data preparation, Joins (Left, Right, Inner, and Outer) and Union, dealing with NULL values, cross-database joining, data extraction, data blending, refresh extraction, incremental extraction, how to build extract , etc.
Hands-on Exercise: Connect to Excel sheet to import data, use metadata and extracts, manage NULL values, clean up data before using, perform the join techniques, execute data blending from multiple sources , etc.
Mark, highlight, sort, group, and use sets (creating and editing sets, IN/OUT, sets in hierarchies), constant sets, computed sets, bins, etc.
Hands-on Exercise: Use marks to create and edit sets, highlight the desired items, make groups, apply sorting on results, and make hierarchies among the created sets.
Filters (addition and removal), filtering continuous dates, dimensions, and measures, interactive filters, marks card, hierarchies, how to create folders in Tableau, sorting in Tableau, types of sorting, filtering in Tableau, types of filters, filtering the order of operations, etc.
Hands-on Exercise: Use the data set by date/dimensions/measures to add filter, use interactive filter to view the data, customize/remove filters to view the result, etc.
Using Formatting Pane to work with menu, fonts, alignments, settings, and copy-paste; formatting data using labels and tooltips, edit axes and annotations, k-means cluster analysis, trend and reference lines, visual analytics in Tableau, forecasting, confidence interval, reference lines, and bands.
Hands-on Exercise: Apply labels and tooltips to graphs, annotations, edit axes’ attributes, set the reference line, and perform k-means cluster analysis on the given dataset.
Working on coordinate points, plotting longitude and latitude, editing unrecognized locations, customizing geocoding, polygon maps, WMS: web mapping services, working on the background image, including add image, plotting points on images and generating coordinates from them; map visualization, custom territories, map box, WMS map; how to create map projects in Tableau, creating dual axes maps, and editing locations.
Hands-on Exercise: Plot longitude and latitude on a geo map, edit locations on the geo map, custom geocoding, use images of the map and plot points, find coordinates, create a polygon map, and use WMS.
Calculation syntax and functions in Tableau, various types of calculations, including Table, String, Date, Aggregate, Logic, and Number; LOD expressions, including concept and syntax; aggregation and replication with LOD expressions, nested LOD expressions; levels of details: fixed level, lower level, and higher level; quick table calculations, the creation of calculated fields, predefined calculations, and how to validate.
Creating parameters, parameters in calculations, using parameters with filters, column selection parameters, chart selection parameters, how to use parameters in the filter session, how to use parameters in calculated fields, how to use parameters in reference line, etc.
Hands-on Exercise: Creating new parameters to apply on a filter, passing parameters to filters to select columns, passing parameters to filters to select charts, etc.
Dual axes graphs, histograms: single and dual axes; box plot; charts: motion, Pareto, funnel, pie, bar, line, bubble, bullet, scatter, and waterfall charts; maps: tree and heat maps; market basket analysis (MBA), using Show me; and text table and highlighted table.
Hands-on Exercise: Plot a histogram, tree map, heat map, funnel chart, and more using the given dataset and also perform market basket analysis (MBA) on the same dataset.
Building and formatting a dashboard using size, objects, views, filters, and legends; best practices for making creative as well as interactive dashboards using the actions; creating stories, including the intro of story points; creating as well as updating the story points, adding catchy visuals in stories, adding annotations with descriptions; dashboards and stories: what is dashboard?, highlight actions, URL actions, and filter actions, selecting and clearing values, best practices to create dashboards, dashboard examples; using Tableau workspace and Tableau interface; learning about Tableau joins, types of joins; Tableau field types, saving as well as publishing data source, live vs extract connection, and various file types.
Hands-on Exercise: Create a Tableau dashboard view, include legends, objects, and filters, make the dashboard interactive, and use visual effects, annotations, and description s to create and edit a story.
Introduction to Tableau Prep, how Tableau Prep helps quickly combine join, shape, and clean data for analysis, creation of smart examples with Tableau Prep, getting deeper insights into the data with great visual experience, making data preparation simpler and accessible, integrating Tableau Prep with Tableau analytical workflow, and understanding the seamless process from data preparation to analysis with Tableau Prep.
Introduction to R language, applications and use cases of R, deploying R on the Tableau platform, learning R functions in Tableau, and the integration of Tableau with Hadoop.
Hands-on Exercise: Deploy R on Tableau, create a line graph using R interface, and also connect Tableau with Hadoop to extract data.
Project 1: Working with Tableau Interactive Dashboard
Domain: Sales
Problem Statement: How to make an interactive dashboard with Tableau?
Description:
Upon the completion of this project, you will understand how to create a single point of access for all your sales data, ways of dissecting and analyzing sales from multiple angles, coming up with a sales strategy for improved business revenues.
Project 2: Tableau for Crime Statistics Analysis
Domain: Crime Statistics (Public Domain)
Problem Statement: Showing the types of crimes and their frequency in the District of Columbia and providing the details of the crimes
Description: Police authorities are often called on to “put more feet on the street” to prevent crime and ensure order. However, due to limited resources, it is almost impossible to use them anytime, anywhere. In this project, you will work on the crime data, in the District of Columbia and analyze it using the Tableau tool. During visualization, you can use ‘crime categories’ and ‘days of the week’ as data types to see when and where crimes have occurred. In this project, you will also analyze the details of the crimes, such as, the area/location and the day of the week it has happened, etc. This project will help the local police get insightful information on where to put their crime prevention efforts on.
Highlights:
Project 3: Analyzing Economic Data
Domain: Government
Problem Statement: How is unemployment affecting global malnutrition?
Description: In this Tableau project, you will be working on vast amounts of data and analyzing it to come up with trends, insights, and correlations. Datasets include the global unemployment figures for multiple years, world population statistics across several years, and the worldwide nutritional data. By analyzing this data, you will correlate the malnutrition problem with the unemployment rates using Tableau.
Highlights:
Project 4: Analyzing Market Performance
Domain: Retail
Problem Statement: Using the Consumer Packaged Goods data to analyze which are the markets that are performing well for a particular retail enterprise using Tableau Desktop
Description: This Tableau Desktop project involves working with the complex Consumer Packaged Goods data to come up with the brand performance analysis, regions that are contributing good to the revenues, where there is a need to offer more discounts to spur sales, and making in-depth budget vs spend analysis for any particular year.
Highlights:
Introduction to Splunk and Splunk developer roles and responsibilities
Writing Splunk query for search, auto-complete to build a search, time range, refine search, working with events, identifying the contents of search and controlling a search job
Hands-on Exercise – Write a basic search query
What is a Field, how to use Fields in search, deploying Fields Sidebar and Field Extractor for REGEX field extraction and delimiting Field Extraction using FX
Hands-on Exercise – Use Fields in Search, use Fields Sidebar, use Field Extractor (FX) and delimit field Extraction using FX
Writing Splunk query for search, sharing, saving, scheduling and exporting search results
Hands-on Exercise – Schedule a search, save a search result and share and export a search result
How to create alerts, understanding alerts and viewing fired alerts.
Hands-on Exercise –Create an alert in Splunk and view the fired alerts
Describe and configure scheduled reports
Introduction to Tags in Splunk, deploying Tags for Splunk search, understanding event types and utility and generating and implementing event types in search
Hands-on Exercise – Deploy tags for Splunk search and generate and implement event types in search
What is a Macro and what are variables and arguments in Macros
Hands-on Exercise –First, you define a Macro with arguments and then use variables with in it
Creating get, post and search workflow actions
Hands-on Exercise – Create get, post and search workflow actions
Studying the search command, the general search practices, what is a search pipeline, how to specify indexes in search, highlighting the syntax and deploying the various search commands like fields, tables, sort, rename, rex and erex
Hands-on Exercise –Steps to create a search pipeline, search index specification, how to highlight syntax, using the auto complete feature and deploying the various search commands like sort, fields, tables, rename, rex and erex
Using top, rare and stats commands
Hands-on Exercise – Use top, rare and stats commands
Using following commands and their functions: addcoltotals, addtotals,top, rare and stats
Hands-on Exercise – Create reports using following commands and their functions: addcoltotals and addtotals
iplocation, geostats, geom and addtotals commands
Hands-on Exercise – Track IP using iplocation and get geo data using geostats
Explore the available visualizations, create charts and time charts, omit null values and format results
Hands-on Exercise – Create time charts, omit null values and format results
Calculating and analyzing results, value conversion, roundoff and format values, using the eval command, conditional statements and filtering calculated search results
Hands-on Exercise – Calculate and analyze results, perform conversion on a data value, roundoff numbers, use the eval command, write conditional statements and apply filters on calculated search results
How to search the transactions, creating report on transactions, grouping events using time and fields and comparing transactions with stats
Hands-on Exercise – Generate report on transactions and group events using fields and time
Learning data lookups, examples and lookup tables, defining and configuring automatic lookups and deploying lookups in reports and searches
Hands-on Exercise – Define and configure automatic lookups and deploy lookups in reports and searches
Creating search charts, reports and dashboards, editing reports and dashboards and adding reports to dashboards
Hands-on Exercise – Create search charts, reports and dashboards, edit reports and dashboards andadd reports to dashboards
Working with raw data for data extraction, transformation, parsing and preview
Hands-on Exercise – Extract useful data from raw data, perform transformation and parse different values and preview
Describe pivot, relationship between data model and pivot, select a data model object, create a pivot report, create in stant pivot from a search and add a pivot report to dashboard
Hands-on Exercise – Select a data model object, create a pivot report, create instant pivot from a search and add a pivot report to dashboard
What is a Splunk CIM and using the CIM Add-On to normalize data
Hands-on Exercise – Use the CIM Add-On to normalize data
Introduction to the architecture of Splunk, various server settings, how to set up alerts, various types of licenses, important features of Splunk tool, the requirements of hardware and conditions needed for installation of Splunk
How to install and configure Splunk, the creation of index, standalone server’s input configuration, the preferences for search, Linux environment Splunk installation and the administering and architecting of Splunk
How to install Splunk in the Linux environment, the conditions needed for Splunk and configuring Splunk in the Linux environment
Introducing Splunk distributed management console, indexing of clusters,how to deploy distributed search in Splunk environment, forwarder management, user authentication and access control
Introduction to the Splunk app, how to develop Splunk apps, Splunk app management, Splunk app add-ons, using Splunk-base for installation and deletion of apps, different app permissions and implementation and how to use the Splunk app and apps on forwarder
Details of the index time configuration file and the search time configuration file
Understanding of Index time and search time configuration filesin Splunk, forwarder installation, input and output configuration, Universal Forwarder management and Splunk Universal Forwarder highlights
Implementing the Splunk tool, deploying it on the server, Splunk environment setup and Splunk client group deployment
Understanding the Splunk Indexes, the default Splunk Indexes, segregating the Splunk Indexes, learning Splunk Buckets and Bucket Classification, estimating Index storage and creating new Index
Understanding the concept of role inheritance, Splunk authentications, native authentications and LDAP authentications
Splunk installation, configuration, data inputs, app management, Splunk important concepts, parsing machine-generated data, search indexer and forwarder
Introduction to Splunk Configuration Files, Universal Forwarder, Forwarder Management, data management, troubleshooting and monitoring
Converting machine-generated data into operational intelligence, setting up the dashboard, reports and charts and integrating Search Head Clustering and Indexer Clustering
Understanding the input methods, deploying scripted, Windows and network and agentless input types and fine-tuning them all
Splunk user authentication and job role assignment and learning to manage, monitor and optimize Splunk Indexes
Understanding parsing of machine-generated data, manipulation of raw data, previewing and parsing, data field extraction and comparing single-line and multi-line events
Distributed search concepts, improving search performance, large-scale deployment and overcoming execution hurdles and working with Splunk Distributed Management Console for monitoring the entire operation
Cluster indexing, configuring individual nodes, configuring the cluster behavior, index and search behavior, setting node type to handle different aspects of cluster like master node, peer node and search head
Project 1 : Creating an Employee Database of a Company
Industry : General
Problem Statement : How to build a Splunk dashboard where employee details are readily available
Topics : In this project, you will create a text file of employee data with details like full name, salary, designation, ID and so on. You will index the data based on various parameters, use various Splunk commands for evaluating and extracting the information. Finally, you will create a dashboard and add various reports to it.
Highlights :
Project 2 : Building an Organizational Dashboard with Splunk
Industry : E-commerce
Problem Statement : How to analyze website traffic and gather insights
Topics : In this project, you will build an analytics dashboard for a website and create alerts for various conditions. You will capture access logs of the web server andthe sample logs and then the sample are uploaded. You will analyze the top ten users, the average time spent, peak response time of the website, the top ten errors and error code description. You will also create a Splunk dashboard for reporting and analyzing.
Highlights :
Project 3 : Field Extraction in Splunk
Industry : General
Problem Statement :How to extract the fields from event data in Splunk
Topics : In this project, you will learn to extract fields from events using the Splunk field extraction technique. You will gain knowledge in the basics of field extractions, understand the use of the field extractor, the field extraction page in Splunk web and field extract configuration in files. You will learn the regular expression and delimiters method of field extraction. Upon the completion of the project, you will gain expertise in building Splunk dashboard and use the extracted fields data in it to create rich visualizations in an enterprise setup.
Highlight :
Installation and introduction to SAS, how to get started with SAS, understanding different SAS windows, how to work with data sets, various SAS windows like output, search, editor, log and explorer and understanding the SAS functions, which are various library types and programming files
How to import and export raw data files, how to read and subset the data sets, different statements like SET, MERGE and WHERE
Hands-on Exercise: How to import the Excel file in the workspace and how to read data and export the workspace to save data
Different SAS operators like logical, comparison and arithmetic, deploying different SAS functions like Character, Numeric, Is Null, Contains, Like and Input/Output, along with the conditional statements like If/Else, Do While, Do Until and so on
Hands-on Exercise: Performing operations using the SAS functions and logical and arithmetic operations
Understanding about input buffer, PDV (backend) and learning what is Missover
Defining and using KEEP and DROP statements, apply these statements and formats and labels in SAS
Hands-on Exercise: Use KEEP and DROP statements
Understanding the delimiter, dataline rules, DLM, delimiter DSD, raw data files and execution and list input for standard data
Hands-on Exercise: Use delimiter rules on raw data files
Various SAS standard procedures built-in for popular programs: PROC SORT, PROC FREQ, PROC SUMMARY, PROC RANK, PROC EXPORT, PROC DATASET, PROC TRANSPOSE, PROC CORR, etc.
Hands-on Exercise: Use SORT, FREQ, SUMMARY, EXPORT and other procedures
Reading standard and non-standard numeric inputs with formatted inputs, column pointer controls, controlling while a record loads, line pointer control/absolute line pointer control, single trailing, multiple IN and OUT statements, dataline statement and rules, list input method and comparing single trailing and double trailing
Hands-on Exercise: Read standard and non-standard numeric inputs with formatted inputs, control while a record loads, control a line pointer and write multiple IN and OUT statements
SAS Format statements: standard and user-written, associating a format with a variable, working with SAS Format, deploying it on PROC data sets and comparing ATTRIB and Format statements
Hands-on Exercise: Format a variable, deploy format rule on PROC data set and use ATTRIB statement
Understanding PROC GCHART, various graphs, bar charts: pie, bar and 3D and plotting variables with PROC GPLOT
Hands-on Exercise: Plot graphs using PROC GPLOT and display charts using PROC GCHART
SAS advanced data discovery and visualization, point-and-click analytics capabilities and powerful reporting tools
Character functions, numeric functions and converting variable type
Hands-on Exercise: Use functions in data transformation
Introduction to ODS, data optimization and how to generate files (rtf, pdf, html and doc) using SAS
Hands-on Exercise: Optimize data and generate rtf, pdf, html and doc files
Macro Syntax, macro variables, positional parameters in a macro and macro step
Hands-on Exercise: Write a macro and use positional parameters
SQL statements in SAS, SELECT, CASE, JOIN and UNION and sorting data
Hands-on Exercise: Create SQL query to select and add a condition and use a CASE in select query
Base SAS web-based interface and ready-to-use programs, advanced data manipulation, storage and retrieval and descriptive statistics
Hands-on Exercise: Use web UI to do statistical operations
Report enhancement, global statements, user-defined formats, PROC SORT, ODS destinations, ODS listing, PROC FREQ, PROC Means, PROC UNIVARIATE, PROC REPORT and PROC PRINT
Hands-on Exercise: Use PROC SORT to sort the results, list ODS, find mean using PROC Means and print using PROC PRINT
Project 1: Categorization of Patients Based on the Count of Drugs for Their Therapy
Domain: Healthcare
Objective: This project aims to find out descriptive statistics and subset for specific clinical data problems. It will give them brief insight about Base SAS procedures and data steps.
Problem Statement:
Count the number of patients,
Output should be four datasets
Project 2: Build Revenue Projections Reports
Domain: Sales
Objective: This project will give you hands-on experience in working with the SAS data analytics and business intelligence tool. You will be working on the data entered in a business enterprise setup and will aggregate, retrieve and manage that data. You will learn to create insightful reports and graphs and come up with statistical and mathematical analysis to scientifically predict the revenue projection for a particular future time frame. Upon the completion of the project, you will be well-versed in the practical aspects of data analytics, predictive modeling and data mining.
Project 3: Impact of Pre-paid Plans on the Preferences of Investors
Domain: Finance Market
Objective: The project aims to find the most impacting factors in preferences of pre-paid model; it also identifies which all are the variables highly correlated with impacting factors.
Problem Statement:
Project 4:K-Means Cluster Analysis on Iris Dataset
Domain: Analytics
Objective: K-Means cluster analysis on Iris dataset to predict about the class of a flower using its petal’s dimensions
Requirements:
1.1 Field of machine learning, its impact on the field of artificial intelligence
1.2 The benefits of machine learning w.r.t. Traditional methodologies
1.3 Deep learning introduction and how it is different from all other machine learning methods
1.4 Classification and regression in supervised learning
1.5 Clustering and association in unsupervised learning, algorithms that are used in these categories
1.6 Introduction to ai and neural networks
1.7 Machine learning concepts
1.8 Supervised learning with neural networks
1.9 Fundamentals of statistics, hypothesis testing, probability distributions, and hidden markov models.
2.1 Multi-layer network introduction, regularization, deep neural networks
2.2 Multi-layer perceptron
2.3 Overfitting and capacity
2.4 Neural network hyperparameters, logic gates
2.5 Different activation functions used in neural networks, including relu, softmax, sigmoid and hyperbolic functions
2.6 Back propagation, forward propagation, convergence, hyperparameters, and overfitting.
3.1 Various methods that are used to train artificial neural networks
3.2 Perceptron learning rule, gradient descent rule, tuning the learning rate, regularization techniques, optimization techniques
3.3 Stochastic process, vanishing gradients, transfer learning, regression techniques,
3.4 Lasso l1 and ridge l2, unsupervised pre-training, xavier initialization.
4.1 Understanding how deep learning works
4.2 Activation functions, illustrating perceptron, perceptron training
4.3 multi-layer perceptron, key parameters of perceptron;
4.4 Tensorflow introduction and its open-source software library that is used to design, create and train
4.5 Deep learning models followed by google’s tensor processing unit (tpu) programmable ai
4.6 Python libraries in tensorflow, code basics, variables, constants, placeholders
4.7 Graph visualization, use-case implementation, keras, and more.
5.1 Keras high-level neural network for working on top of tensorflow
5.2 Defining complex multi-output models
5.3 Composing models using keras
5.3 Sequential and functional composition, batch normalization
5.4 Deploying keras with tensorboard, and neural network training process customization.
6.1 Using tflearn api to implement neural networks
6.2 Defining and composing models, and deploying tensorboard
7.1 Mapping the human mind with deep neural networks (dnns)
7.2 Several building blocks of artificial neural networks (anns)
7.3 The architecture of dnn and its building blocks
7.4 Reinforcement learning in dnn concepts, various parameters, layers, and optimization algorithms in dnn, and activation functions.
8.1 What is a convolutional neural network?
8.2 Understanding the architecture and use-cases of cnn
8.3‘What is a pooling layer?’ how to visualize using cnn
8.4 How to fine-tune a convolutional neural network
8.5 What is transfer learning?
8.6 Understanding recurrent neural networks, kernel filter, feature maps, and pooling, and deploying convolutional neural networks in tensorflow.
9.1 Introduction to the rnn model
9.2 Use cases of rnn, modeling sequences
9.3 Rnns with back propagation
9.4 Long short-term memory (lstm)
9.5 Recursive neural tensor network theory, the basic rnn cell, unfolded rnn, dynamic rnn
9.6 Time-series predictions.
10.1 Gpu’s introduction, ‘how are they different from cpus?,’ the significance of gpus
10.2 Deep learning networks, forward pass and backward pass training techniques
10.3 Gpu constituent with simpler core and concurrent hardware.
11.1 Introduction rbm and autoencoders
11.2 Deploying rbm for deep neural networks, using rbm for collaborative filtering
11.3 Autoencoders features and applications of autoencoders.
12.1 Image processing
12.2 Natural language processing (nlp) – Speech recognition, and video analytics.
13.1 Automated conversation bots leveraging any of the following descriptive techniques: Ibm watson, Microsoft’s luis, Open–closed domain bots,
13.2 Generative model, and the sequence to sequence model (lstm).
Project 01: Image Recognition with TensorFlow
Industry: Internet Search
Problem Statement: Creating a Deep Learning model to identify the right object on the Internet as per the user search for the corresponding image
Description: In this project, you will learn how to build a convolutional neural network using Google TensorFlow. You will do the visualization of images using training, providing input images, losses, and distributions of activations and gradients. You will learn to break each image into manageable tiles and input them to the convolutional neural network for the desired result.
Highlights:
Project 02: Building an AI-based Chatbot using IBM watson LAB
Industry: Ecommerce
Problem Statement: Building a chatbot using Artificial Intelligence
Description: In this project, by understanding the customer needs, you will be able to offer the right services through Artificial Intelligence chatbots. You will learn how to create the right artificial neural network with the right amount of layers to ensure that the customer queries are comprehensible to the Artificial Intelligence chatbot. This will help you understand Natural Language Processing, going beyond keywords, data parsing, and providing the right solutions.
Highlights:
Project 03: Ecommerce Product Recommendation
Industry: Ecommerce
Problem Statement: Recommending the right products to customers using Artificial Intelligence with TensorFlow
Description: This project involves working with recommender systems to provide the right product recommendation to customers with TensorFlow. You will learn how to use Artificial Intelligence to check for users’ past buying habits, find out the products that go hand-in-hand, and recommend the best products that can be bought together with a particular product.
Highlights:
RDBMS, types of relational databases, challenges of RDBMS, NoSQL database, its significance, how NoSQL suits Big Data needs, introduction to MongoDB and its advantages, MongoDB installation, JSON features, data types and examples
Installing MongoDB, basic MongoDB commands and operations, MongoChef (MongoGUI) installation and MongoDB data types
Hands-on Exercise: Install MongoDB and install MongoChef (MongoGUI)
The need for NoSQL, types of NoSQL databases, OLTP, OLAP, limitations of RDBMS, ACID properties, CAP Theorem, Base property, learning about JSON/BSON, database collection and documentation, MongoDB uses, MongoDB write concern—acknowledged, replica acknowledged, unacknowledged, journaled—and Fsync
Hands-on Exercise: Write a JSON document
Understanding CRUD and its functionality, CRUD concepts, MongoDB query and syntax and read and write queries and query optimization
Hands-on Exercise:Use insert query to create a data entry, use find query to read data, use update and replace queries to update and use delete query operations on a DB file
Concepts of data modelling, difference between MongoDB and RDBMS modelling, model tree structure, operational strategies, monitoring and backup
Hands-on Exercise: Write a data model tree structure for a family hierarchy
In this module, you will learn MongoDB® Administration activities such as health check, backup, recovery, database sharding and profiling, data import/export, performance tuning, etc.
Hands-on Exercise: Use shard key and hashed shard keys, perform backup and recovery of a dummy dataset, import data from a CSV file and export data to a CSV file
Concepts of data aggregation and types and data indexing concepts, properties and variations
Hands-on Exercise: Do aggregation using pipeline, sort, skip and limit and create index on data using single key and using multi-key
Understanding database security risks, MongoDB security concept and security approach and MongoDB integration with Java and Robomongo
Hands-on Exercise: MongoDB integration with Java and Robomongo
Implementing techniques to work with variety of unstructured data like images, videos, log data and others and understanding GridFS MongoDB file system for storing data
Hands-on Exercise: Work with variety of unstructured data like images, videos, log data and others
Project: Working with the MongoDB Java Driver
Industry: General
Problem Statement: How to create table for video insertion using Java
Topics: In this project, you will work with MongoDB Java Driver and become proficient in creating a table for inserting video using Java programming. You will work with collections and documents and understand the read and write basics of MongoDB database and the Java virtual machine libraries.
Highlights:
1.1 Introduction to cloud computing
1.2 What is Microsoft Azure?
1.3 Microsoft Azure Services
1.4 Creating a Microsoft Azure Account
1.5 Azure CLI, Azure Powershell
1.6 Managing Azure Resources & Subscriptions
1.7 Azure Resource Manager
1.8 Microsoft Azure Architecture
Hands-on Exercise:
1. Move and organised resources in the resource groups
2. Create a resource Groups and deploy resources in it.
2.1 Introduction Azure Storage Account
2.2 Explain types -Azure Blob Storage, Azure Files Storage,Azure Table Storage, Azure Queue Storage, Azure Storage Explorer
2.3 Attach & Detach an External Storage Account
2.4 Azure Shared Access Sgnature (SAS).
Hands-on Exercise:
1. Create Storage Account
2. Create and manage Azure Blob
3. Create and manage Azure File Storage
4. Create and manage Azure Queue Storage
5. Create and manage Azure Table Storage
6. Download and Set Up Storage Explorer
7. Attach and detach external storage account in storage Explorer
8. Access Storage Services using Storage Explorer.
3.1 Azure Storage Replication
3.2 Why Azure Storage Replication?
3.3 Data Replication Options
3.4 Azure Import/Export Service
3.5 How Import/Export Service Works
3.6 Azure File Sync, Azure Backup Service
3.7 Azure Content Delivery Network (CDN).
Hands-on Exercise:
1. Create Archive Backup
2. Create SNapshot Backup
3. Create AZ Copy Backup
4. Create and Use CDN Endpoints.
4.1 Introduction to Azure Virtual Machines
4.2 Virtual Machine Configurations
4.3 Creating and Configuring Azure VM
4.4 Data Disks in Azure
4.5 Creating VM from an existing OS disk
4.6 Azure VMs & Interfaces
4.7 Introduction to ARM templates
4.8 Introduction to VHD templates
4.9 Creating Virtual Machines using templates
4.10 Introduction to Custom Images of Azure VM.
Hands-on Exercise:
1. Create and congfigure a Virtual Machine
2. Creating a VM from VHD Template
3. Deploying a VM from a Custom Image.
5.1 Why Use Virtual Machine Scale Sets
5.2 Virtual Machine Availability Sets
5.3 Backing Up Azure Virtual Machines
5.4 VM Back Up Policies
5.5 Restoring Azure Virtual machines
5.6 Azure Site Recovery.
Hands-on Exercise:
1. Create and configure a Virtual Machine
2. Create a Scale Set
3. Creating a VM from VHD Template
4. Deploying a VM from a Custom Image
5. Take a back up of an Azure VM
6. Restore the VM
7. Create a Scale Set.
6.1 Introduction to Azure Virtual Networks
6.2 Azure Vnet Components
6.3 IP Addresess – Public & Private IPs
6.4 Azure Vnet Subnets
6.5 Azure Network Interface Cards (NIC)
6.6 Network Security Group (NSG)
6.7 NSG Rules
6.8 Creating Route Tables, Service Tags
6.9 Introduction to Azure DNS and Private DNS.
Hands-on Exercise:
1. Assigning Static IP
2. Creating Azure Vnet from Scratch
3. Creating and Attaching NIC to VM
4. Creating and using NSG
5. Create and configure DNS.
7.1 Introduction to Azure Load Balancers
7.2 Troubleshooting Azure Load Balancers
7.3 Azure Network Watcher
7.4 Virtual Network Gateway
7.5 Vnet Peering, Vnet to Vnet Connection Gateway, Vnet Site to Site Connection
7.6 Azure Express Route, Express Route Circuits, Express Route Peering.
Hands-on Exercise:
1. Create and Configure Vnet to Vnet Peering
2. Create and Configure Vnet to Vnet Gateway Connection
3. Create Internal Load Balancer
4. Create Public Load Balancer
5. Estabalish site to site VPN gateway Connection.
8.1 What is Access Management in Azure?
8.2 Role Based Access Management (RBAC)
8.3 Built in Roles in Access Management
8.4 Structure of Role Definitions
8.5 Creating a Custom Role
8.6 Role Assignment in Azure Resources, Azure Users & Groups, RBAC Policies.
Hands-on Exercise:
1. Creating a Custom Role
2. Attach policies to the role
3. Assigning role to Azure resources.
9.1 What is Active Directory
9.2 Why Azure Active Directory?
9.3 What is Windows Active Directory?
9.4 Windows AD Vs Azure AD, Azure AD Users, Azure AD Groups,Azure AD Domains, Azure AD Tenants.
Hands-on Exercise:
1. Create an Azure Active Directory
2. Create Azure AD Users – Guests and Members
3. Assign roles to Users, Create Azure AD Groups
4. Create a Custom Domain in Azure Active Directory.
10.1 What is Azure AD Connect?
10.2 Features of Azure AD Connect
10.3 Authentication Options, AD Connect Password hash Synchronization
10.4 AD Connect Federation Integration, Azure AD Connect Health Monitoring
10.5 Self Service password Reset (SSPR)
10.6 Implementing Multi Factor Authentication (MFA).
Hands-on Exercise:
1. Configure Azure AD Connect
2. Manage Azure AD Connect Health operations
3. Enable and implement Self service password reset for a certain user
4. Enable and implement Multi factor Authentication for users.
11.1 Data Disks in Azure
11.2 Azure VMs & Interfaces
11.3 ARM templates and VHD templates
11.4 Custom Images of Azure VM
11.5 Virtual Machine Scale Sets
11.6 Virtual Machine Availability Sets
11.7 Backing Up Azure Virtual Machines
11.8 VM Back Up Policies, Restoring Azure Virtual machines
11.9 Azure Site Recovery
11.10 App Service Web App for Containers
11.11 App Service plan
11.12 Networking for an App Service
11.13 Deployment slots, Container image
11.14 Azure Kubernetes Service
11.15 Azure Container Registry
Hands-on Exercise:
1. Create Networking for an App Service
2. Create Azure Container Registry
3. Backing Up Azure Virtual Machines
4. VM Back Up Policies
5. Restoring Azure Virtual machines
Project 01: Deploying a website using Microsoft Azure
Topics: Traffic manager, Application gateway, Virtual Machines. Blob storage, Virtual Network Peering
Highlights:
1.1 Deploying the website servers in two different regions
1.2 Controlling, managing and monitoring the traffic geographically
1.3 Enabling Path based Routing for the website
1.4 Enabling communication between the servers
1.5 Storing data on the cloud
Project 02: Queues using Azure
Topics: Queues
Highlights:
2.1 Setting up an application in Azure
2.2 Using azure SDK to set up queues
2.3 Deploying queues to pass messages
Case Study 01: Introduction to Cloud computing
Problem Statement: Solving the issue of not wanting the corporation’s confidential data on the cloud while migrating to Microsoft Azure
Topics: Azure Resource manager, Azure subscriptions
Highlights:
1.1 Govern all resources separately
1.2 Tracking cost and billing of each service being used separately
1.3 Accessing and managing resource groups
Case Study 02: Microsoft Azure Storage
Problem Statement: Solving latency issues and difficulty accessing common files and tools
Topics: Azure storage account, CDN endpoint, Azure file share
Highlights:
2.1 Uploading static content to azure
2.2 Creating a CDN Endpoint and configuring it to serve the static files that have been uploaded
2.3 Creating an azure file share and uploading content in it
2.4 Connecting a Linux and Windows box to File share
Case Study 03: Azure Virtual Machine
Problem Statement: Managing scaling requirements and using custom image to create a Virtual machine
Topics: Virtual Machines, Custom images
Highlights:
3.1 Automating the scaling of Virtual Machines as required
3.2 Deploying multiple identical Virtual Machines using custom VM image
Case Study 04: Azure networking
Problem Statement: Deploy a virtual network with subnets in it and enable them to communicate privately
Topics: Virtual network, Virtual network peering
Highlights:
4.1 Creating a virtual network with subnets and deploying Virtual Machines in it
4.2 Establishing a connection between these subnets
Case Study 05: Load balancing and Network watcher
Problem Statement: Setting up a load balancer and a network watcher
Topics: Azure load balancer, Network performance monitor
Highlights:
5.1 Deploying a load balancer for the backend resources such that a single frontend IP is exposed
5.2 Setting up a Network performance manager to generate alerts
Case Study 06: Access management in Azure
Problem Statement: Providing access to some of the services managed by your organization’s active directory
Topics: Azure active directory, Azure Multi-Factor Authentication
Highlights:
6.1 Adding users in active directory and giving them access
6.2 Create users in custom active directory domain
6.3 Setting up a password authentication method
6.4 Setting up MFA with a verification option
1.1 What is Cloud Computing
1.2 Cloud Service & Deployment Models
1.3 How AWS is the leader in the cloud domain
1.4 Various cloud computing products offered by AWS
1.5 Introduction to AWS S3, EC2, VPC, EBS, ELB, AMI
1.6 AWS architecture and the AWS Management Console, virtualization in AWS (Xen hypervisor)
1.7 What is auto-scaling
1.8 AWS EC2 best practices and cost involved.
Hands-on Exercise – Setting up of AWS account, how to launch an EC2 instance, the process of hosting a website and launching a Linux Virtual Machine using an AWS EC2 instance.
2.1 Introduction to EC2
2.2 Regions & Availability Zones(AZs)
2.3 Pre-EC2, EC2 instance types
2.4 Comparing Public IP and Elastic IP
2.5 Demonstrating how to launch an AWS EC2 instance
2.6 Introduction to AMIs, Creating and Copying an AMI
2.7 Introduction to EBS
2.8 EBS volume types
2.9 EBS Snapshots
2.10 Introduction to EFS
2.11 Instance tenancy- Reserved and Spot instances
2.12 Pricing and Design Patterns.
Hands-on Exercise –
1. Launching an EC2 instance
2. Creating an AMI of the launched instance
3. Copying the AMI to another region
4. Creating an EBS volume
5. Attaching the EBS volume with an instance
6. Taking backup of an EBS volume
7. Creating an EFS volume and mounting the EFS volume to two instances.
3.1 Introduction to Elastic Load Balancer
3.2 Types of ELB – Classic, Network and Application
3.3 Load balancer architecture
3.4 Cross-zone load balancing
3.5 Introduction to Auto Scaling, vertical and horizontal scaling, the lifecycle of Auto Scaling
3.6 Components of Auto Scaling, scaling options and policy, instance termination
3.7 Using load balancer with Auto Scaling
3.8 Pre-Route 53 – how DNS works
3.9 Routing policy, Route 53 terminologies, Pricing.
Hands-on Exercise –
1. Creating a Classic ELB
2. Creating an Application ELB
3. Creating an auto-scaling group
4. Configuring an auto-scaling group
5. Integrating ELB with Auto Scaling
6. Redirect traffic from domain name to ELB using Route 53.
4.1 What is Amazon VPC,
4.2 VPC as a networking layer for EC2,
4.3 IP address and CIDR notations,
4.4 Components of VPC – network interfaces, route tables, internet gateway, NAT,
4.5 Security in VPC – security groups and NACL, types of VPC, what is a subnet, VPC peering with scenarios, VPC endpoints, VPC pricing and design patterns.
Hands-on Exercise –
1. Creating a VPC and subnets,
2. Creating a 3 Tier architecture with security groups,
3. NACL, Internet gateway and NAT gateway,
4. Creating a complete VPC architecture.
5.1 Introduction to AWS storage
5.2 Pre-S3 – online cloud storage
5.3 API, S3 consistency models
5.4 Storage hierarchy, buckets in S3
5.5 Objects in S3, metadata and storage classes, object versioning, object lifecycle management, cross-region replication, data encryption, connecting using VPC endpoint, S3 pricing.
Hands-on Exercise –
1. Creating an S3 bucket
2. Uploading objects to the S3 bucket
3. Enabling object versioning in the S3 bucket
4. Setting up lifecycle management for only a few objects
5. Setting up lifecycle management for all objects with the same tag
6. Static website hosting using S3.
6.1 What is a database, types of databases, databases on AWS
6.2 Introduction to Amazon RDS
6.3 Multi-AZ deployments, features of RDS
6.4 Read replicas in RDS, reserved DB instances
6.5 RDS pricing and design patterns
6.6 Introduction to Amazon Aurora, benefits of Aurora, Aurora pricing and design patterns
6.7 Introduction to DynamoDB, components of DynamoDB, DynamoDB pricing and design patterns
6.8 What is Amazon Redshift, advantages of Redshift
6.9 What is ElastiCache, why ElastiCache.
Hands-on Exercise –
1. Launching a MySQL RDS instance
2. Modifying an RDS instance
3. Connecting to the DB instance from your machine
4. Creating a multi-az deployment
5. Create an Aurora DB cluster
6. Creating an Aurora replica
7. Creating a DynamoDB table.
7.1 Introduction to CloudFormation
7.2 CloudFormation components
7.3 CloudFormation templates
7.4 The concept of Infrastructure-as-a-code
7.5 Functions and pseudo parameters
7.6 Introduction to Simple Notification Service, how does SNS work
7.7 Introduction to Simple Email Service, how does SES work
7.8 Introduction to Simple Queue Service, how does SQS work.
Hands-on Exercise –
1. Creating a CloudFormation stack
2. Launching a t2.micro
3. EC2 instance using CloudFormation
4. Using CloudFormation to automate an architectural deployment
5. Creating an SNS topic, creating a subscription within the topic
6. Setting up SES and sending a mail
7. Creating an SQS queue and sending a sample message.
8.1 Pre-IAM, why access management
8.2 Amazon Resource Name (ARN), IAM features
8.3 Multi-factor authentication (MFA) in IAM, JSON
8.4 IAM policies, IAM permissions, IAM roles, identity federation, pricing
8.5 Introduction to CloudWatch, metrics and namespaces, CloudWatch architecture, dashboards in CW, CloudWatch alarms, CloudWatch logs, pricing and design patterns
8.6 Introduction to CloudTrail, tracking API usage.
Hands-on Exercise –
1. Creating IAM users and a group
2. creating an IAM policy and attach it to the group
3. creating an IAM role
4. Setup MFA for a user
5. Creating a CloudWatch dashboard and add metrics
6. Create a CloudWatch alarm which triggers according to CPU Utilization of an EC2 instance
7. Creating a billing alarm
8. Creating a log group
9. Creating a trail.
9.1 What is AWS Lambda
9.2 How Lambda is different from EC2
9.3 Benefits and limitations of Lambda
9.4 How does Lambda work
9.5 Use cases of Lambda, Lambda concepts
9.6 Integration S3 with Lambda
9.7 What is Elastic Beanstalk, how does Beanstalk work, Beanstalk concepts, Beanstalk pricing
9.8 What is configuration management
9.9 What is AWS OpsWorks, AWS OpsWorks benefits
9.10 CloudFormation vs OpsWorks, services in OpsWorks, AWS OpsWorks Stacks, OpsWorks pricing.
Hands-on Exercise –
1. Creating a Lambda function
2. Setting up Lambda triggers and destinations
3. Creating an Elastic Beanstalk application
4. Uploading a new version of the application to Beanstalk
5. Creating a stack in OpsWorks
6. Launching the instance using OpsWorks and automatically installing the application.
10.1 What is FSx
10.2 Types of FSx,FSx for Windows server
10.3 How does FSx for Windows File Server work, FSx for Lustre
10.4 Use cases of FSx
10.5 Automatic failover process
10.6 Supported clients and access methods
10.7 What is a Global Accelerator, How Global Accelerator works, Listeners and Endpoints
10.8 What are AWS Organizations, Features of AWS Organizations, Managing multiple accounts
10.9 What are ENIs, ENAs and EFAs, Working with network interfaces
10.10 Enhanced Networking with ENA, EFA with MPI, Monitoring an EFA
Hands-on Exercise:
1. Creating a shared FSx file system between two windows instances
2. Accessing one instance with multiple Elastic IPS using ENI
3. Using Global Accelerator to map instances from 2 regions into one domain name
4. Enabling Enhanced Networking on an Ubuntu instance
11.1 Important guidelines for creating a well-architected AWS framework that is resilient and performant
11.2 Designing of fault-tolerant and high-availability architecture
11.3 Resilient storage
11.4 Decoupling mechanism
11.5 Multi-tier architecture solution
11.6 Disaster recovery solution
11.7 Scalable and elastic solutions.
12.1 What is DevOps,
12.2 Introduction to AWS DevOps,
12.3 AWS Developer tools – CodeCommit, CodeBuild, CodeDeploy and CodePipeline, integrating GitHub with CodePipeline,
12.4 Creating a DevOps lifecycle using AWS DevOps tools.
13.1 What is Cloud migration
13.2 Why migration is important
13.3 Migration process in AWS, the 6 R’s migration strategy
13.4 Virtual machine migration, migrating a local vm onto the AWS cloud
13.5 Migrating databases using Database Migration Service (DMS)
13.6 Migrating a local database to RDS
13.7 Migrating an on-premises database server to RDS using DMS, other migration services.
14.1 Guidance for clearing the exam, most probable interview questions and other helpful tips for clearing the exam and interview.
Project 01 – Deploy a multi-tier website on AWS
Problem Statement: Deploying a Custom PHP Website to AWS with functionalities for SQL, NoSQL and file storage
Topics – RDS, SNS, DynamoDB, S3, VPC, EC2, NAT Gateways, Load Balancer and Auto Scaling
Highlights:
1.1 Configuring AWS to send emails for every operation using the website
1.2 Deploying the web application in private subnet with no internet access
1.3 Using Load Balancer to expose the application in the private subnet
1.4 Using NoSQL database for metadata storage
1.5 Using Auto Scaling for varying traffic workloads
Project 02 – Deploying a website for High Availability and High Resilience
Problem Statement: Design an architecture which can automatically scale up and down based on traffic and is de coupled for components like Database, webapp etc.
Topics: Auto Scaling, Target Groups, Load Balancing, RDS, PaaS, Elastic Beanstalk
Highlights:
2.1 Designing Architectures which are self-healing and auto scale
2.2 Load Balancing across multiple instances using Load Balancer
2.3 Deploying AWS RDS for connecting to Elastic Beanstalk
Project 03 – Sending Notifications to patients using push notifications
Problem Statement – Design an architecture on AWS, which can send notifications to patients based on Doctor Feedback
Topics: AWS SNS, VPC, EC2
Highlights:
3.1 Sending messages through SNS for resiliency
3.2 Integrating EC2 and SNS for message storing
3.3 Securing the instances using Private and Public subnets in VPC.
Case Study 01 – Using different operations on EC2 and EBS
Problem Statement: Replicate EC2 instances to other regions for High Availability, extending the size of EBS volumes without losing data
Topics – EC2, EBS, AMIs
Highlights:
1.1 Copying EC2 instance to another region
1.2 Scaling EBS Volumes
1.3 Mounting the same EBS Volumes to different EC2 instances one at a time
Case Study 02 – Autoscaling compute capacity in AWS
Problem Statement: Autoscaling and Load Balancing EC2 instances in AWS based on different metrics. Routing custom domain to AWS resources
Topics – Autoscaling, Load Balancing, Route 53
Highlights:
2.1 Load Balancing among multiple EC2 instances
2.2 Scaling up and scaling down instances automatically using Autoscaling
2.3 Defining metrics for Autoscaling instances
2.4 Routing custom domain to AWS resources
Case Study 03 – Creating custom VPCs in AWS
Problem Statement: Create a custom VPC in AWS, with multiple subnets having private and public access
Topics: VPC, Subnets, Internet Gateway, NAT Gateway, Route Tables
Highlights:
3.1 Creating a custom VPC with various subnets
3.2 Configuring routes to the subnets
3.3 Using and configuring NAT Gateways
Case Study 04 – Using AWS S3 for Lifecycle Access Management
Problem Statement: Moving artifacts from on-premise to S3 in the most cost efficient manner
Topics: Lifecycle Management, Glacier, Access Tiers, Event in S3
Highlights:
4.1 Creating Lifecycle rules for S3 objects
4.2 Hosting Static Website
4.3 Using Route 53
Case Study 05 – Highly available relational database in AWS
Problem Statement: Create a highly available and scalable AWS Database service in AWS using RDS
Topics – Multi Region Deployment, High Availability, RDS, Backups
Highlights:
5.1 Build a highly scalable database
5.2 Creating Database Architecture
5.3 Collect data for real time analysis
5.4 Resolving Latency Issues
Case Study 06 – CloudFormation for Infrastructure as a Code
Problem Statement: Provisioning AWS Resources using AWS CloudFormation
Topics: CloudFormation, VPC, Route 53, RDS
Highlights:
6.1 Using CloudFormation to deploy resources
6.2 Defining rules for deletion using IaC
6.3 Minimize deployment time
Case Study 07 – Administering user access using AWS IAM
Problem Statement: Creating users for defining granular access to different users
Topics – AWS IAM Users, IAM Groups, Policies
Highlights:
7.1 Creating users in IAM
7.2 Defining custom policies
7.3 Adding users to groups
Case Study 08 – Appplication Services in AWS and Configuration Management
Problem Statement: Using application services in AWS to deploy code, configuration management using OpsWorks
Topics: Elastic Beanstalk, AWS Lambda, SNS
Highlights:
8.1 Deploying code to AWS Lambda
8.2 Deploying webapp to Elastic Beanstalk
8.3 Configuration Management using OpsWorks
Getting started with HBase, core concepts of HBase and understanding HBase with an example
Why HBase, where to use HBase and what is NoSQL
HDFS vs. HBase, HBase Use Cases and Data Modeling HBase
HBase Architecture and main components of HBase cluster
HBase Shell, HBase API, primary operations and advanced operations
Create a table and insert data into it and integration of Hive with HBase and Load Utility
Putting Folder to VM and file loading with both load utilities
Project: Integrate Hive and Java with HBase
Industry: General
Problem Statement: How to create HBase table with Hive
Topics: This is a project that gives you hands-on experience to connect Hive and Java with HBase. Hive is used for querying data using HiveQL. In this project, you will do HBase installation, create Hive tables, import the data onto Hive from HBase and use HiveQL for Hive table data querying, analyzing and managing the HBase table. You will also learn to write Java code for writing HBase queries.
Highlight:
Introduction to Cassandra, its strengths and deployment areas
Significance of NoSQL, RDBMS Replication, Key Challenges, types of NoSQL, benefits and drawbacks, salient features of NoSQL database. CAP Theorem, Consistency.
Installation, introduction to Cassandra, key concepts and deployment of non relational database, column-oriented database, Data Model – column, column family,
Token calculation, Configuration overview, Node tool, Validators, Comparators, Expiring column, QA
How Cassandra modelling varies from Relational database modelling, Cassandra modelling steps, introduction to Time Series modelling, comparing Column family Vs. Super Column family, Counter column family, Partitioners, Partitioners strategies, Replication, Gossip protocols, Read operation, Consistency, Comparison
Creation of multi node cluster, node settings, Key and Row cache, System Key space, understanding of Read Operation, Cassandra Commands overview, VNodes, Column family
JSON, Hector client, AVRO, Thrift, JAVA code writing method, Hector tag
Cassandra management, commands of node tool, MapReduce and Cassandra, Secondary index, Datastax Installation
Rules of Cassandra data modelling, increasing data writes, duplication, and reducing data reads, modelling data around queries, creating table for data queries
Understanding the Java application creation methodology, learning key drivers, deploying the IDE for Cassandra applications,cluster connection and data query implementation
Learning about Node Tool Utility, cluster management using Command Line Interface, Cassandra management and monitoring via DataStax Ops Center.
Cassandra client connectivity, connection pool internals, API, important features and concepts of Hector client, Thrift, JAVA code, Summarization.
Type : Deploying the IDE for Cassandra applications
Topics : This project gives you a hands-on experience in installing and working with Apache Cassandra which is a high performance and extremely scalable database for distributed data with no single point of failure. You will deploy the Java Integrated Development Environment for running Cassandra, learn about the key drivers, work with Cassandra applications in a cluster setup and implement data querying techniques.
The architecture of Couchbase, understanding Couchbase distributed NoSQL database engine, vBuckets for information distribution on Couchbase cluster, user and system requirements and Couchbase downloading and installation
Couchbase single-node deployment for development purpose
Managing the Couchbase environment with the Web Console tool, configuring the Couchbase server and management and working with Couchbase data buckets, default bucket sizing and administration
Methods for deploying Couchbase in multi-node cluster: first, all Couchbase Servers on one machine and, second, with each Couchbase Server on own machine
The Couchbase Command-line Interface tools for managing and monitoring single-node and multi-node clusters, Severs and vBuckets and developing reports for log data collection
Topics: This project involves working with the Couchbase command-line interface tools that are used for managing of clusters in a multi-node or single-node setup, working with vBuckets in Couchbase Server, deploying reports for log data collection. You will gain hands-on experience in deploying commands like start, stop and report status for log collection. It also includes working with Couchbase-cli, cbcollect_info tool and so on. Upon the completion of the project, you will be proficient in using Couchbase CLI for managing and monitoring clusters and data replication using XDCR.
1.1 Need of Machine Learning
1.2 Introduction to Machine Learning
1.3 Types of Machine Learning, such as supervised, unsupervised and reinforcement learning, why Machine Learning with Python and applications of Machine Learning.
2.1 Introduction to supervised learning, types of supervised learning, such as regression and classification
2.2 Introduction to regression,
2.3 simple linear regression,
2.4 Multiple linear regression, assumptions in linear regression
2.5 Math behind linear regression.
Hands-on Exercise
1. Implementing linear regression from scratch with Python
2. Using Python library Scikit-learn to perform simple linear regression and multiple linear regression
3. Implementing train–test split and predicting the values on the test set.
3.1 Introduction to classification
3.2 Linear regression vs Logistic regression
3.3 Math behind logistic regression, detailed formulas, log it function and odds, confusion matrix and accuracy, true positive rate, false positive rate, and threshold evaluation with ROCR.
Hands-on Exercise
1. Implementing logistic regression from scratch with Python
2. Using Python library Scikit-learn to perform simple logistic regression and multiple logistic regression
3. Building a confusion matrix to find out accuracy, true positive rate, and false positive rate.
4.1 Introduction to tree-based classification
4.2 Understanding a decision tree, impurity function, entropy, to understand the concept of information gain for the right split of node, impurity function, information gain
4.3 Understand the concept of information gain for the right split of node, impurity function, Gini index, to understand the concept of Gini index for the right split of node, overfitting, pruning, pre-pruning, post-pruning, cost-complexity pruning
4.4 Introduction to ensemble techniques, understanding bagging, introduction to random forests, and finding the right number of trees in a random forest.
Hands-on Exercise
1. Implementing a decision tree from scratch in Python
2. Using Python library Scikit-learn to build a decision tree and a random forest.
3. Visualizing the tree and changing the hyper parameters in the random forest.
5.1 Introduction to probabilistic classifiers,
5.2 Understanding Naïve Bayes, math behind the Bayes theorem
5.3 Understanding a support vector machine (SVM)
5.4 Kernel functions in SVM, and math behind SVM.
Hands-on Exercise
1. Using Python library Scikit-learn to build a Naïve Bayes classifier and a support vector classifier.
6.1 Types of unsupervised learning, such as clustering and dimensionality reduction, types of clustering
6.2 Introduction to k-means clustering
6.3 Math behind k-means
6.4 Dimensionality reduction with PCA.
Hands-on Exercise
1. Using Python library Scikit-learn to implement K-means clustering
2. Implementing PCA (principal component analysis) on top of a dataset.
7.1 Introduction to Natural Language Processing (NLP)
7.2 Introduction to text mining
7.3 Importance and applications of text mining
7.4 How NPL works with text mining
7.5 Writing and reading to word files
7.6 OS modules, Natural Language Toolkit (NLTK) environment,
7.7 Text mining: its cleaning and pre-processing and text classification.
Hands-on Exercise
1. Learning Natural Language Toolkit and NLTK Corpora
2. Reading and writing .txt files from/to a local drive
3. Reading and writing .docx files from/to a local drive.
8.1 Introduction to Deep Learning with neural networks
8.2 Biological neural network vs artificial neural network
8.3 Understanding perception learning algorithm, introduction to Deep Learning frameworks, and Tensor Flow constants, variables and place-holders.
9.1 What is time series?, its techniques and applications
9.2 Time series components
9.3 Moving average, smoothing techniques, exponential smoothing
9.4 Univariate time series models
9.5 Multivariate time series analysis
9.6 ARIMA model, time series in Python
9.7 Sentiment analysis in Python (Twitter sentiment analysis), and text analysis.
Hands-on Exercise
1. Analyzing time series data
2. The sequence of measurements that follow a non-random order to recognize the nature of phenomenon
3. Forecasting the future values in the series.
Project 01: Analysing the trends of COVID-19 with Python
Industry: Analytics
Problem Statement: Understanding, the trend of covid 19 spread and if the restrictions imposed by governments around the world has helped us curb the COVID cases and by what degree
Topics: In this project we will use Data Science and Python perfrom visualization to better understand the data we currently have on COVID 19 as well as using Time Series Analysis in order to make a perdiction about future cases if the current trend as observed thus far continues.
Highlights:
Project 02 – Customer Churn Classification
Topics – This is a real-world project that gives you hands-on experience in working with most of the Machine Learning algorithms.
The main components of the project include the following:
Project 03 – Recommendation for Movie, Summary
Topics – This is a real-world project that gives you hands-on experience in working with a movie recommender system. Depending on what movies are liked by a particular user, you will be in a position to provide data-driven recommendations. This project requires you to deeply understand information filtering, recommender systems, user ‘preference’, and more. You will exclusively work on data related to user details, movie details, and others.
The main components of the project include the following:
Case Study 01 – Decision Tree
Topics – To understand the structure of a dataset (PIMA Indians Diabetes database) and create a decision tree model based on it by using Scikit-learn
Case Study 02 – Insurance Cost Prediction (Linear Regression)
Topics – To understand the structure of a medical insurance dataset, implement both simple and multiple linear regression, and predict values
Case Study 03 – Diabetes Classification (Logistic Regression)
Topics – To understand the structure of a dataset (PIMA Indians Diabetes dataset), and implement multiple logistic regression and classify[I3] . Fit your model on the test and train data for prediction and evaluate your model using confusion matrix and then visualize it
Case Study 04 – Random Forest
Topics – To create a model that would help in classifying whether a patient is ‘Normal’, ‘Suspected to have disease,’ or in actuality ‘Has the disease’ on the ‘Cardiotocography’ dataset
Case Study 05 – Principal Component Analysis (PCA)
Topics – Read the sample iris dataset given to you, use PCA to figure out the number of most important principal features, and then reduce the number of features using PCA. Train and test the Random Forest Classifier algorithm to check if reducing the number of dimensions is causing the model to perform poorly. Figure out the most optimal number that produces good quality results and predicts accuracy
Case Study 06 – K-means Clustering
Topics
Introduction to the search engine, the Apache Lucene, understanding the inverted index, documents and fields & documents.
Introduction to the various query types available in Lucene and clear understanding of these.
Understanding the prerequisites for using Apache Lucene, learning about the querying process, analyzers, scoring boosting, faceting, grouping, highlighting, the various types of geographical and spatial searches, introduction to Apache Tika.
Demonstration of the Apache Lucene workings.
Understanding the Analyzer, Query Parser in Apache Lucene, Query Object, Stopword.
Understanding the various aspects of Apache Lucene like Scoring, Boosting, Highlighting, Faceting and Grouping
Introduction to Apache Solr, the advantages of Apache Solr over Apache Lucene, the basic system requirements for using Apache Solr, introduction to Cores in Apache Solr.
Introduction to the Apache Solr indexing, index using built-in data import handler and post tool, understanding the Solrj Client and configuration of Solrj Client.
Demonstrating the Book Store use cases with Solr Indexing with practical examples, learning to build Schema, the field, field types, CopyField and Dynamic Field, understanding how to add, explore, update, and delete using Solrj.
The various aspects of Apache Solr search like sorting, pagination, an overview of the request parameters, faceting and highlighting.
Understanding the Request Handlers, defining and mapping to search components, highlighting and faceting, updating managed schemas, request parameters hardwiring, adding fields to default search, the various types of Analyzers, Parsers, Tokenizers.
Grouping of results in Apache Solr, Parse queries functions, fuzzy query in Apache Solr.
The extended features in Apache Solr, learning about Pseudo-fields, Pseudo-Joins, Spell Check, suggestions, Geospatial Search, multi-language search, stop words and synonyms.
Understanding the concept of Multicore in Solr, the creation of Multicore in Solr, the need of Multicore, Joining of data, Replication and Ping Handler.
Understanding the SolrCloud, the concept of Sharding, indexing, and replication in Apache SolrCloud, the working of Apache SolrCloud, distributed requests, reading and writng slide fault tolerance, cluster coordination using Apache ZooKeeper.
Project – Designing and developing simple ecommerce search engine database
Topics : This project involves deploying Apache Solr for search engine database design. You will be exposed to real world scenarios in the eCommerce domains like searching customer history, purchase order, personalization, targeted marketing opportunities and more using Solr. Get hands-on experience in Solr Multi Core, Creation Core, Solrj, Stop words, Auto complete, Auto Suggest, Synonyms, and Managed schema. Upon completion of project you will understand about distributed indexing, and other salient features of Solr like resilience, scalability, replication, reliability, centralized configuration and recovery.
Introduction to Linux, Basics of Shell, Basics of Kernel, CentOS 8 installation and VBox additions, Basic Linux Commands, ECHO and EXPR command, Set and unset a variable, Header of a shell script (#!).
Hands-on Exercise – Executing basic Linux commands, Installing CentOS 8 on VirtualBox and adding guest additions to the installed OS.
Text editors and file creation; Users, Groups and Processes; Root and Linux file hierarchy, Understanding file hierarchy, Understanding file permissions, chmod and chown commands, the LS command, Metacharacters, Editing a file using VIM, Displaying contents of a file, Copy, Move and Remove files.
Hands-on Exercise – Using VIM, Creating users and groups, Creating files and directories, Assigning file permissions and ownership using chmod and chown, Editing files in VIM.
Everything is a file in UNIX/Linux (files, directories, executables, processes), Process control commands (ps and kill), other process control tools (top, nice, renice).
Hands-on Exercise – Executing ps and kill commands on running services, Monitoring the OS using top.
What is shell scripting, Types of shell, Creating and writing a shell script, Changing the permission of the shell script, Executing the script, Environment variables, Defining a local and a global variable, User input in a shell script.
Hands-on Exercise – Creating a shell script, Writing and executing the shell script, creating a local and a global variable, taking input from the user in a shell script.
What are Conditional statements, Using IF, IF-ELSE, Nested IF statements, What are Looping statements, Using WHILE, UNTIL and FOR statements, Using the case…esac statement, What is a Function, Creating a function in Linux, Calling functions.
Hands-on Exercise – Executing IF, IF-ELSE, Nested IF statements, Executing WHILE, UNTIL and FOR statements, Executing the case…..esac statement, creating a function in multiple ways, calling a function in a file, calling a function from another file.
Using GREP command, Using SED command, Using AWK command, Mounting a file to the virtual box, Creating a shared folder (mounting a folder), Using SORT command and Using pipes to combine multiple Commands.
Hands-on Exercise – Executing commands using GREP, Executing commands using SED, Executing commands using AWK, Mounting a folder in the Windows OS to the Linux OS, Installing VirtualBox guest additions on CentOS 8, Extracting zipped files.
What are Daemons, Introduction to Task scheduling in Linux, Scheduling a job in Linux, What is Cron and Crontab, How to use cron, Using the AT command.
Hands-on Exercise – Starting, Stopping and Restarting Daemon processes, Scheduling jobs using cron and crontab, Scheduling a one time task using AT, Managing scheduled tasks using ATQ and ATRM.
Why monitoring, Introduction to process monitoring, Top vs HTop, What does PGREP do, Introduction to file and folder monitoring, Monitoring tool inotifywait, inotifywait options for folder monitoring, Events of a folder in inotify, the FREE command.
Hands-on Exercise – Using Top to moniter the OS, Installing Htop, Using Htop to monitor the OS, Filtering and sorting using Htop, Installing inotify tools, monitoring a folder using inotifywait, monitoring a folder only for certaing events, using the FREE command.
Installing and configuring MySQL, Securing MySQL, Running Queries from terminal, Running Queries from a shell script.
Hands-on Exercise – Downloading and installing MySQL, Connecting to MySQL from terminal, Querying directly from the terminal, Pushing the query result inside a file, CRUD operations from a shell script.
What is networking in Linux, Why do we need networking, Using networking commands – IFCONFIG, PING, Wget and cURL, SSH, SCP and FTP, Learning Firewall tools – iptables and firewalld, DNS and Resolving IP address, /etc/hosts and /etc/hostname, nslookup and dig.
Hands-on Exercise – Executing all the networking commands, Using iptables and firewalld, Adding and removing ports, Resolving IP address in /etc/hosts, looking into a websites IP and nameservers using nslookup and dig.
Project: Installing WordPress on Centos7
Industry: Internet related
Problem Statement: How to install LAMP stack on Centos7 and creating a database for WordPress
Topics: In this project you will be working on creating your account on WordPress (with Database), then flush it using Flush Privileges and Install a PHP Module. We can get that package directly from CentOS’s default repositories using yumand also we will install and update the WordPress for the latest Template & Formats.
Highlight
Introduction to Java Programming, Defining Java, Need for Java, Platform Independent in Java, Define JRE,JVM, JDK, Important Features and Evolution of Java
Overview of Coding basics, Setting up the required environment, Knowing the available IDEs, Writing a Basic-level Java Program, Define Package, What are Java Comments?, Understanding the concept of Reserved Words, Introduction to Java Statements, What are Blocks in Java, Explain a Class, Different Methods
Overview of the Language, Defining Identifiers, What are Constraints and Variables, What is an Encoding Set?, Concept of Separators, Define Primitives, How to make Primitive Conversions?, Various Operators in Java
Module Overview, Learn how to write If Statement, Understanding While Statement, Working with Do-while Statement, How to use For Statement?, Using Break Statement, What is Continue Statement, Working of Switch Statement
General Review of the Module, Defining Object and Classes in Java, What are Encapsulation, Static Members and Access Control?, Use and importance of ‘this’ Keyword, Deining Method Overloading with an example, ‘By Value’ vs. ‘By Reference’, Loading, Defining Initialization and Linking, How to Compare Objects in Java?, What is Garbage Collector?
General Review, Concept of Object in Java, Define Core Class, What is System?, Explain String Classes, How do Arrays work?, Concept of Boxing & Unboxing, Use of ‘varargs’, ‘format’ and ‘printf’ Methods
Introduction, Define Inheritance with an example, Accessibility concept, Method Overriding, Learning how to call a Superclass’ Constructor, What is Type Casting?, Familiarity with ’instanceof’ Keyword
Getting started with exception Handling, Defining an Exception, How to use Constructs to deal with exceptions?, Classification of exceptions, Throw Exceptions, How to create an exception class?, stack Trace analysis
General Review, Defining Interface, Use and Create and Interface, Concept of Extending interfaces, How to implement multiple interfaces?, What are abstract classes?, How to create and use abstract classes?, Comparison between interface and abstract classes, Concept of Nested Classes, What are Nested Classes?, Nested Classes Types, Working of an Inner Class, What is a Local Inner Class?, Anonymous Classes in java, What is a Static Nested Class
What are Nested Classes?, Types of Nested Classes, What is an Inner Class?, Understanding local inner class, Anonymous Inner Class, Nested Class – Static
What is a Thread?, How to create and start a Thread?, States of a Thread, Blocking the Execution of a Thread, Concept of Sleep Thread, Understanding the priorities in a thread, Synchronisation in Java Threads, Interaction between threads
Introduction to Collection Framework, Preeminent Interfaces, What are Comparable and Comparator?, Working with Lists, Working with Maps, Working with Sets, Working with Queues
Define JDBC, Different types of Drivers, How to access the drivers?, What is Connection in Java?, What is a Statement?, Explaining CRUD Operations with examples, Prepared Statement and Callable Statement
Overview of important topics included, Important and Frequently-Used Features, Defining Generic List, What is Generic Map in Java?, Java Generic Classes & Methods, For Loop Generic, What is Generic Wild Card?
Brief Introduction, Learning about Input and output streams in java, Concept of byte Oriented Streams, Defining Character Oriented Streams?, Explain Object Serialisation, Input and Output Based on Channel
Introduction and Definition of Annotations, How they are useful for Java programmers?, Placements in Annotations, What are Built-in Java Annotations, Defining Custom Annotations
Getting started, Define Java Reflection?, What is a Class Object?, Concept of Constructors, Using Fields, Applying Methods, Implementing Annotations in Your Java Program
Project – Library Management System
Problem Statement – It creates library management system project which includes following functionalities:
Add book, Add Member, Issue Book, Return Book, Available Book etc.
Understanding what is Apache Kafka, the various components and use cases of Kafka, implementing Kafka on a single node.
Learning about the Kafka terminology, deploying single node Kafka with independent Zookeeper, adding replication in Kafka, working with Partitioning and Brokers, understanding Kafka consumers, the Kafka Writes terminology, various failure handling scenarios in Kafka.
Introduction to multi node cluster setup in Kafka, the various administration commands, leadership balancing and partition rebalancing, graceful shutdown of kafka Brokers and tasks, working with the Partition Reassignment Tool, cluster expending, assigning Custom Partition, removing of a Broker and improving Replication Factor of Partitions.
Understanding the need for Kafka Integration, successfully integrating it with Apache Flume, steps in integration of Flume with Kafka as a Source.
Detailed understanding of the Kafka and Flume Integration, deploying Kafka as a Sink and as a Channel, introduction to PyKafka API and setting up the PyKafka Environment.
Connecting Kafka using PyKafka, writing your own Kafka Producers and Consumers, writing a random JSON Producer, writing a Consumer to read the messages from a topic, writing and working with a File Reader Producer, writing a Consumer to store topics data into a file.
Type : Multi Broker Kafka Implementation
Topics : In this project you will learn about the Apache Kakfa which is a platform for handling real-time data feeds. You will exclusively work with Kafka brokers, understand partitioning, Kafka consumers, the terminology used for Kafka writes and failure handling in Kafka, understand how to deploy a single node Kafka with independent Zookeeper. Upon completion of the project you will gain considerable experience in working in a real world scenario for processing streaming data within an enterprise infrastructure.
Various types of databases, introduction to Structured Query Language, distinction between client server and file server databases, understanding SQL Server Management Studio, SQL Table basics, data types and functions, Transaction-SQL, authentication for Windows, data control language, and the identification of the keywords in T-SQL, such as Drop Table.
Data Anomalies, Update Anomalies, Insertion Anomalies, Deletion Anomalies, Types of Dependencies, Functional Dependency, Fully functional dependency, Partial functional dependency, Transitive functional dependency, Multi-valued functional dependency, Decomposition of tables, Lossy decomposition, Lossless decomposition, What is Normalization?, First Normal Form, Second Normal Form, Third Normal Form, Boyce-Codd Normal Form(BCNF), Fourth Normal Form, Entity-Relationship Model, Entity and Entity Set, Attributes and types of Attributes, Entity Sets, Relationship Sets, Degree of Relationship, Mapping Cardinalities, One-to-One, One-to-Many, Many-to-one, Many-to-many, Symbols used in E-R Notation.
Introduction to relational databases, fundamental concepts of relational rows, tables, and columns; several operators (such as logical and relational), constraints, domains, indexes, stored procedures, primary and foreign keys, understanding group functions, the unique key, etc.
Advanced concepts of SQL tables, SQL functions, operators & queries, table creation, data retrieval from tables, combining rows from tables using inner, outer, cross, and self joins, deploying operators such as ‘intersect,’ ‘except,’ ‘union,’ temporary table creation, set operator rules, table variables, etc.
Understanding SQL functions – what do they do?, scalar functions, aggregate functions, functions that can be used on different datasets, such as numbers, characters, strings, and dates, inline SQL functions, general functions, and duplicate functions.
Understanding SQL subqueries, their rules; statements and operators with which subqueries can be used, using the set clause to modify subqueries, understanding different types of subqueries, such as where, select, insert, update, delete, etc., and methods to create and view subqueries.
Learning SQL views, methods of creating, using, altering, renaming, dropping, and modifying views; understanding stored procedures and their key benefits, working with stored procedures, studying user-defined functions, and error handling.
User-defined functions; types of UDFs, such as scalar, inline table value, multi-statement table, stored procedures and when to deploy them, what is rank function?, triggers, and when to execute triggers?
SQL Server Management Studio, using pivot in MS Excel and MS SQL Server, differentiating between Char, Varchar, and NVarchar, XL path, indexes and their creation, records grouping, advantages, searching, sorting, modifying data; clustered indexes creation, use of indexes to cover queries, common table expressions, and index guidelines.
Creating Transact-SQL queries, querying multiple tables using joins, implementing functions and aggregating data, modifying data, determining the results of DDL statements on supplied tables and data, and constructing DML statements using the output statement.
Querying data using subqueries and APPLY, querying data using table expressions, grouping and pivoting data using queries, querying temporal data and non-relational data, constructing recursive table expressions to meet business requirements, and using windowing functions to group and rank the results of a query.
Creating database programmability objects by using T-SQL, implementing error handling and transactions, implementing transaction control in conjunction with error handling in stored procedures, and implementing data types and NULL.
Designing and implementing relational database schema; designing and implementing indexes, learning to compare between indexed and included columns, implementing clustered index, and designing and deploying views and column store views.
Explaining foreign key constraints, using T-SQL statements, usage of Data Manipulation Language (DML), designing the components of stored procedures, implementing input and output parameters, applying error handling, executing control logic in stored procedures, and designing trigger logic, DDL triggers, etc.
Applying transactions, using the transaction behavior to identify DML statements, learning about implicit and explicit transactions, isolation levels management, understanding concurrency and locking behavior, and using memory-optimized tables.
Accuracy of statistics, formulating statistics maintenance tasks, dynamic management objects management, identifying missing indexes, examining and troubleshooting query plans, consolidating the overlapping indexes, the performance management of database instances, and SQL server performance monitoring.
Corelated Subquery, Grouping Sets, Rollup, Cube
Hands-on Exercise
Implementing Corelated Subqueries, Using EXISTS with a Correlated subquery, Using Union Query, Using Grouping Set Query, Using Rollup, Using CUBE to generate four grouping sets, Perform a partial CUBE.
Project 1: Writing Complex Subqueries
Industry: General
Problem Statement: How to create subqueries using SQL?
Topics: This project will give you hands-on experience in working with SQL subqueries and utilizing them in various scenarios. Some of the subqueries that you will be working with and gaining hands-on experience in are: IN or NOT IN, ANY or ALL, EXISTS or NOT EXISTS, and other major queries.
Highlights:
Project 2: Querying a Large Relational Database
Industry: General
Problem Statement: How to get details about customers by querying the database?
Topics: In this project, you will work on downloading a database and restoring it on the server. You will then query the database to get customer details like name, phone number, email ID, sales made in a particular month, increase in month-on-month sales, and even the total sales made to a particular customer.
Highlights:
Project 3: Relational Database Design
Industry: General
Problem Statement: How to convert a relational design into a table in SQL Server?
Topics: In this project, you will work on converting a relational design that has enlisted within it various users, user roles, user accounts, and their statuses. You will create a table in SQL Server and insert data into it. With at least two rows in each of the tables, you will ensure that you have created respective foreign keys.
Highlights:
This is a comprehensive course that is designed to clear multiple certifications, namely:
The entire training course content is in line with respective certification exams. At the end of this training course, there will be quizzes that perfectly reflect the type of questions asked in the respective certification exams and help you score better marks. Intellipaat Course Completion certificate will be awarded upon the completion of the project work (after expert review) and upon scoring at least 60% marks in the quiz. Intellipaat certification is well recognized in top 80+ MNCs like Ericsson, Cisco, Cognizant, Sony, Mu Sigma, Saint-Gobain, Standard Chartered, TCS, Genpact, Hexaware, etc.
Our Alumni works at top 3000+ companies
A Senior Software Architect at NextGen Healthcare who has previously worked with IBM Corporation, Suresh Paritala has worked on Big Data, Data Science, Advanced Analytics, Internet of Things and Azure, along with AI domains like Machine Learning and Deep Learning. He has successfully implemented high-impact.
An experienced Blockchain Professional who has been bringing integrated Blockchain, particularly Hyperledger and Ethereum, and Big Data solutions to the cloud, David Callaghan has previously worked on Hadoop, AWS Cloud, Big Data and Pentaho projects that have had major impact on revenues of marquee brands around the world.
A renowned Data Scientist who has worked with Google and is currently working at ASCAP, Samanth Reddy has a proven ability to develop Data Science strategies that have a high impact on the revenues of various organizations. He comes with strong Data Science expertise and has created decisive Data Science strategies for Fortune 500 corporations.
Intellipaat’s master’s course is a structured learning path especially designed by industry experts and ensures that you transform into a Big Data and Data Science expert. Individual courses at Intellipaat focus on one or two specializations. However, if you have to master Big Data and Data Science, then this program is for you.
