Module-1 (Overview of Course)

A discussion on course information and protocols. Evolution of technology and data generation sources. Structured, semi-structured and un-structured data. What is Data Science? Factors making data science ubiquitous. Applications of data science in the domains of social media, banking, e-commerce, web-based search engines, travelling, health care, automation, credit and insurance.

How to do data science? Who is a data scientist? Skill sets of a data scientist. Programming languages, tools and techniques involved in doing data science. 3 Vs of Big Data and tools used to handle Volume, Variety, and Velocity. Python libraries for data science tasks. Why data science is so complicated? Six phases of data science life cycle: Business Problem, Data Acquisition, Data Processing, EDA and Visualization, ML Model Creation-Training-Evaluation, Deployment and Monitoring. Industry job roles and salary trends in data science domain.

Version Control Systems:
Overview and types of versioning control systems: Local, Centralized and Distributed Data Models. Downloading, installing, and configuring git on local machine. Overview of git three tree architecture and its basic work flow. Branching, merging, and handling merge conflicts. Web Portals and Cloud hosting services for git (GitHub, Bitbucket, GitLab). Creating remote repository on GitHub. Cloning a remote repository on local machine. Pushing a local repository on remote repository. Fetch vs Pull vs Fork

Module-2 (Basics of Python Programming)

Python Environment:
History of Python, its key features, important libraries and frameworks. Applications of Python. World top organizations using Python. Downloading and installing latest version of Python. Code Editors (Notepad , Atom, Sublime), Desktop IDEs (Spyder, Pycharm), Browser based IDEs (Jupyter Notebook/Lab), and Cloud based IDEs (Google Colab, Deepnote, CoCalc). Introduction to IPython, Installing Anaconda navigator and Jupyter Notebook, Jupyter Notebook architecture, converting notebook formats and sharing notebooks. A hello world of Python Programming in Jupyter Notebook. Handling Numeric data types in Python. Arithmetic, assignment, comparison, logical, bitwise, identity and membership operators

STRING:
Defining strings in Python, Accessing characters of a string in Python, Strings are immutable, Slicing strings, String concatenation, Creating large strings, String Methods: lower(), upper(), strip(), startswith(), split(), join(), find(), replace(), format(), String Membership test

LIST:
What are Python Lists and how to create them? Different ways to access list elements, Slicing Lists, List concatenation and repetition, Modifying/Adding elements to a List using append(), extend(), and insert() methods, Removing elements from a list using pop() and remove() methods, Converting string object to list and vice-versa, The sort() and reverse() methods, Use of in Operator on Lists, Comparing objects and values, Simple Assignment (aliasing) vs Shallow Copy vs Deep Copy, Pickling vs Un-pickling in Python

TUPLE:
What are tuples and how to create them? Accessing elements of tuples? Tuple concatenation and repetition, Slicing a tuple, Converting string object to tuple and vice-versa, Misc tuple methods and Concepts

SET:
What are sets and how to create them? Accessing elements of sets? Adding elements to a sets using add, and update methods, Removing elements from a set using pop, remove and discard methods. Converting string object to set and vice-versa, Misc set methods. Union, Intersection, Difference, Symmetric Difference of sets, Subsets, Supersets and Disjoint sets

DICTIONARY:
What are dictionaries and how to create them?, Proof of concepts, Accessing elements of a dictionary, Adding/Modifying elements in a dictionary, Removing elements from a dictionary, Dictionary, tuple and list conversions, Misc Dictionary methods

SELECTION STRUCTURE:
Python Indentation, Branching with if, else, and elif, Ternary operator, Nested if-else statement, if-elif ladder, Statements and Expressions, The pass statement, Python3 input() function

REPETITION STRUCTURE:
While loop (Indefinite Iteration), Break and continue statements, While loop with else statement, For loop, Iterables and Iterators, Range and enumerate() functions, Nested for loop

FUNCTIONS:
What are functions in Python, User Defined Functions in Python, Docstring inside a function, Passing arguments and Returning value from a function, Pass by value vs Pass by reference, Arguments of a Python function, Positional/Required arguments, Default arguments, Keyword arguments, Variable length arguments, Arbitrary keyword arguments, Nested functions, Understanding Scope in Python, Local scope, Enclosing scope, Global scope, Built-in Scope, Anonymous Functions or Lambda Functions, Basic Examples of lambda functions, Using lambda function as argument to other functions, Using Lambda Function with built-in sorted(), map(), filter(), and reduce() functions

EXCEPTIONS:
What are Exceptions? Raising an Exception, Handling exceptions using try-except block, Types of exceptions in Python, Multiple except blocks, Try-except with else clause, Finally keyword in Python, User defined exceptions

BUILT IN MODULES AND PACKAGES:
What are Python packages, modules, and libraries. Python Math, Random, Time, DateTime, Calendar, OS, Threading and URLLIB Modules

CREATING MODULES AND PACKAGES:
How to create a Python Module? How to use a Module in Python? How Python locates a Module? Reloading a Python Module, What are Python Packages? How to create a Python Package? How to import modules from a Package?

FILE HANDLING:
What are Files, and Why we need them? Opening a file, Reading contents of a file using read(), readline() and readlines(). Iterating a file object, Changing file offset using seek(), Writing and appending data in a file, Close a file, Delete a file, Rename a file, Get file attributes using stat(), Checking file types, Directory handling, Create a directory, Delete a directory, Rename a directory, Changing a process cwd, Handling Image Files in Python, Opening an image file using open() and read(), Reading and displaying image files using library functions, Copying image file, Cropping image file, Transposing image file, Saving image file

REGULAR EXPRESSION:

Python re module, match() and search() function, matching vs searching, Regular expression modifiers, patterns, and special sequences. The match(), findall(), search(), split() and sub() functions

Module 3 (Python for Data Scientist)

NUMPY:
Introduction to Numpy Library. Array Creation using np.array() constructor. Miscellaneous Array Creation Functions: np.zeros(), np.ones(), np.empty(), np.full(), np.eye(), np.fromstring(), np.arange(), np.linspace(), np.random.rand(), np.random.randint(), np.zeros_like()
A Comparison between Python Lists, Python Arrays and NumPy Arrays. Memory Consumption of Python List and Numpy Array. Operation cost on Python List and Numpy Array

NUMPY:
NumPy universal functions (unary and binary), Scalar Math, Arithmetic Operations, Aggregate Functions, Comparison Operations. Searching NumPy Arrays.
Slicing NumPy Arrays. Boolean and Fancy Array Indexing.
Broadcasting NumPy Arrays. Reshaping NumPy Arrays using the shape attribute, np.reshape(), np.resize(), ndarray.transpose(), np.swapaxes(), np.flatten() methods. Sorting Arrays using np.sort(). Iterating NumPy Arrays

NUMPY:
Updating existing values of NumPy array elements. Append new elements to a NumPy array using np.append(). Insert new elements in a NumPy array using np.insert(). Delete elements of a NumPy array using np.delete()
Alias vs Shallow Copy vs Deep Copy
Array concatenation, stacking and splitting.
Reading and Writing Numeric data from text/csv Files to NumPy Arrays

PANDAS:
Overview of Pandas Library and its Data Structures. Install Pandas Library. Read Datasets into Pandas Dataframe. Python Dictionaries vs Pandas Dataframes. Anatomy of a Dataframe
Creating a Series
– From Python List
– From NumPy Arrays
– From Python Dictionary
– From a scalar value
Attributes of a Pandas Series. Understanding Index in a Series and its usage: (Identification, Selection, Subsetting, Alignment)

PANDAS:
Anatomy of a Dataframe, Creating Dataframe from an empty dataframe, from Two-Dimensional NumPy Array, from Dictionary of Python Lists, from Dictionary of Panda Series. Attributes of a Dataframe
Reading data from a CSV/TSV File Reading a CSV file from a Remote System. Writing Contents of Dataframe to a CSV File. Reading data from an EXCEL File. Writing Contents of Dataframe to an EXCEL File. Reading data from a JSON File. Writing Contents of Dataframe to a JSON File

Understanding Indices of a Dataframe, Selecting Row(s) and Column(s) of a Dataframe using `df[]`, Selecting Rows and Columns using `iloc` Method, Selecting Rows and Columns using `loc` Method, Conditional Selection, Selecting columns of a specific data type

Modifying Column labels of Dataframe. Modifying Row indices of Dataframe. Modifying Row(s) Data (Records) of a Dataframe. Modifying multiple Rows using `map()`, `df.remove()`, `df.apply()`, `df.applymap()` methods

PANDAS:
Reading a Sample Dataframe, Adding a New Column or Deleting an Existing Column from a Dataframe, Adding a New Row or Deleting an Existing Row(s) from a Dataframe, Adding a New Column with Conditional Values, Deleting Row(s) Based on Specific Condition, Delete a Column Based on Specific Condition, Change Datatype of a Pandas Series, Sorting dataframes using `df.sort_values()`, `df.sort_index()` methods.

Have an insight about the Dataset, Identify Column(s) containing Null/Missing values using df.isna() method, Handle/Impute the Null/Missing Values under the Column using `df.loc`, Handle Missing values under a Numeric/Categorical Column using fillna() method, Handle Repeating Values (for same information), Create a new Column by Modifying an Existing Column, Delete Rows containing NaN values using df.dropna() method, Convert Categorical Variables into Numerical.

Overview of Aggregation Functions and the `agg()` method, Computing the Minimum Temperature of different Cities using hard way, groupby methods. Practice GroupBy on Stack Overflow Survey Dataset.

PANDAS:
Merging DataFrames using pd.merge() method, Perform Inner Join (which is default), Perform Outer/Full Outer Join, Perform Left Outer Join, Perform Right Outer Join, Additional Parameters to pd.merge() Method, Concatenation using pd.concat() method, Appending Dataframes using df.append() method.

Reshape Data Using `pivot()` , `pivot_table()` , `melt()` and `crosstab()` methods.

Recap of Python’s Built-in Time and Datetime Modules, Overview of Pandas Time Series Data Structures, Converting Strings to Pandas DateTime64 type, Convert a Scalar String to DateTime, Convert Pandas Series to DateTime, Handling Issues of DateTime Formats, Convert a Single Integer to Pandas DateTime, Practicing with a Simple Dataset, UFO Dataset, and Crypto-Currency Dataset, Concept of Up Sampling and Down Sampling, Bonus Task.

EDA AND VISUALIZATION:
Overview of Data Visualization, Choosing an appropriate Chart for the problem at hand, Anatomy of a Figure, Download and Install Matplotlib, Programming with Matplotlib, How to draw a Line Chart, Enhance the Graph Step by Step and Saving it on Disk. Understanding Sub-Plots.

Understanding Bar-plot, Visualizing Bar-Plot for Stack Overflow Dataset, Understanding Scatter-plot, Visualizing Scatter Plot for Houses Dataset.

Understanding Pie Chart, Visualizing Pie Chart for Stack Overflow Survey Dataset, Understanding Histogram, Visualizing Histogram for Stack Overflow Survey Dataset, Visualizing Histogram of an Image.

EDA AND VISUALIZATION:
Recap of previous session, Understanding/Visualizing Box Plot, Violin Plot, Heatmap
Creating Multiple Sub-Plots within a Figure Object.

Overview of Seaborn Library, Downloading and Importing Seaborn Library, Built-in Datasets of Seaborn Library, Car_Crashes, Flights, Tips, Iris, Titanic Datasets. Three Figure Level Methods of Seaborn. Using sns.relplot() method to draw line and scatter plots. Using sns.catplot() method to draw bar, count, box, violin, strip, and swarm plots. Using sns.displot() method to draw hist, kde and ecdf plots.

Module-4 (Mathematics for Data Science)

Descriptive Statistics:
Overview of Overview of Section 4, Data and its Types, Population vs Sample.
Overview of Statistics and its Types Descriptive Statistics vs Inferential Statistics.
Measure of Center (Mean, Median and Mode). Measure of Spread (Range, IQR, Variance, and Standard Deviation). Measure of Asymmetry (Skewness and Kurtosis).
Analysis Strategy, Variance vs Covariance vs Correlation, Covariance and Covariance Matrix, Correlation and Correlation Matrix, Python Implementation of Covariance and Correlation.
Standardizing/Normalizing a Dataset, Overview of Titanic, Iris, Tips Datasets. Practicing Bivariate Data Analysis.

Inferential Statistics:
Overview of Probability, Joint Probability, Combinatorics, Practical Implementation in Python.
The Law of Large Numbers, Conditional Probability, Bayes’ Theorem, Applications of Bayes’ Theorem, Naive Bayes’ Classifier, How Probability Relates with Statistics.
Probability Distributions for Discrete Random Variables, Probability Distributions for Continuous Random Variables, Normal Distribution, Standard Normal or Z-Distribution, Practical Implementation of Distributions in Python.
Z-Score vs P-Value, Central Limit Theorem.

How to formulate a Hypothesis? Types of Hypothesis Tests. Related Terminologies (Rejection region, significance level, test scores, p-value, and error types).
Student’s Single Sample T-Test Student’s Two Independent Samples T-Test, Student’s Two Paired Samples T-Test.

Variance and Standard Deviation, Covariance and Covariance Matrix, Correlation and Correlation Matrix, Regression Analysis, Linear Regression, Fitting a Line (Gradient Descent), Fitting a Line (Linear Least Squares).

LINEAR ALGEBRA I:
Overview of Vectors, Mathematical and Graphical Representation of Vectors in R2, Mathematical and Graphical Representation of Vectors in R3, Hands on Implementation in Python, Magnitude of a Vector (Vector Norms), Direction of a Vector, Components of a Vector in R2, Components of a Vector in R3, Vector Addition, Multiplying a Vector with Scalar value, Basis and Unit Vectors, Linear combination and Span of Vectors, Vector Dot Product, Vector Cross Product, Orthogonal and Orthonormal Vectors.

Row Vector vs Column Vector, Zero Matrix, Ones Matrix, Random Integer Matrix, Square Matrix, Symmetric Matrix, Triangular Matrix, Diagonal Matrix, Identity Matrix, Scalar Matrix, Orthogonal Matrix, Matrix Addition, Matrix-Scalar Multiplication, Matrix Multiplication (Hadamard Product), Matrix Multiplication (Dot Product), Matrix-Vector Multiplication, Frobenius Norms, Transpose of a Matrix, Determinant of a Matrix, Inverse of a Matrix, Trace of a Matrix, Rank of a Matrix.

An overview of Linear Equations, What is a system of Linear Equations? How to solve a system of Linear Equations? Substitution, Elimination and Graphing strategy, Consistent vs Inconsistent System of Linear Equations, Plotting a Linear Equation with Three Variables, Solving set of Three Linear Equations with Three variables, Solving System of Linear Equations using Matrix Algebra, Writing a system of Linear Equations in Matrix form, Solving system of Linear Equations using Gaussian Elimination Method, Solving system of Linear Equations using Gauss Jordan Method, Solving system of Linear Equations using Cramer’s Rule, Solving system of Linear Equations using Matrix Inverse Method, Limitations of Matrix Inversion Method, Categories of System of Linear Equations, Overdetermined systems, Underdetermined systems, Solving Inconsistent Overdetermined System of Linear Equations using Least Squares Method, Modeling Linear Equations in Machine Learning with `2` variables, Modeling Linear Equations in Machine Learning with `m` variables, Simple Linear Regression using Least Squares Method, Multiple Linear Regression using Ordinary Least Squares (OLS) Method.

LINEAR ALGEBRA II:
What do you mean by applying a Matrix to a Vector? Algebra Behind Linear Transformations in 2-D Space, Applying an Identity Matrix, Applying a Scaling Matrix, Applying a Reflection Matrix, Applying a Shear Matrix, Applying a Rotation Matrix, Applying a Translation Matrix, Apply Multiple Transformations Simultaneously. Scaling, Translation and Rotation in Three Dimensional Space.

Eigenvalues and Eigenvectors (An abstract view), Calculating Eigenvalues and Eigenvectors of a Matrix using Paper-Pencil, Calculating Eigenvalues and Eigenvectors of a Matrix in Python, Relationship between Determinant and Eigenvalues of a Matrix, Relationship between Trace and Eigenvalues of a Matrix, Not all Transformation Matrices have Real Eigenvalues, Eigenvectors and Eigenvalues of a Diagonal Transformation Matrix, Eigendecomposition, Eigendecomposition for Symmetric Matrices, What is Singular Value Decomposition? Calculating SVD using NumPy, Calculating SVD using SciPy, Calculating SVD using Scikit-learn.

Image Compression, What is Principal Component Analysis?
Example 1: Dimensionality Reduction of Housing Dataset using NumPy,
Example 2: Dimensionality Reduction of Digits Dataset using Scikit-learn,
Example 3: Dimensionality Reduction of Iris Dataset using Scikit-learn (Data Visualization), The Moore-Penrose Pseudoinverse.

CALCULUS FOR MACHINE LEARNING

Module-5 (Web Scrapping)

Data Acquisition:
Overview of Web Scraping Using APIs to Acquire Data from Websites, What is Web Scraping? How Web Scraping Works? BeautifulSoup vs Selenium vs Scrapy, Is Web Scraping Legal? A bit of Motivation, Overview of a Web Application Architecture, The Anatomy of a Web Page, The Document Object Model (DOM), Google Chrome Developer Tools, The Python urllib Package, The requests Library.

Overview of BeautifulSoup and Requests Library, Download and Install BeautifulSoup, Playing with BeautifulSoup and Requests, Reviewing the Books Scraping Website, Fetching HTML contents using Requests Library, Creating Soup object and playing with it using the soup.find() and soup.find_all() methods. Example 1: Scraping data of a single page, Saving Data into a CSV file, Scraping Data from Multiple web pages. Example 2: Scraping data from a website that employs pagination, Limitations of Requests and BeautifulSoup.

Data Acquisition:
Overview of Web Scraping using Selenium, Selenium (Why, What and How), Architecture of Selenium, Download and Install Selenium and its WebDriver Component, A step-by-step Hello World with Selenium, Create an Instance of a Web Driver, load a web page, play and quit.
Example 1: Scraping JS Driven Website.
Example 2: Scraping Website behind a login. Selenium find_element() and find_elements() methods. Understanding Locators in Selenium. Consolidated Script for Example 2.
Example 3: Scraping Infinite Scrolling Website.
Example 4: Scraping Pagination Website.
Example 5: Scraping Website containing pop-ups. Bonus: Sending email with attachment from Python.

Best Practices for Web Scraping.
Example 1: Searching and Downloading Images
Example 2: Scrape YouTube video comments
Example 3: Scraping jobs from a Job Website
Example 4: Scraping Tweets of a celebrity
Example 5: Scraping News Paper articles.

Module-6 (Overview of Machine Learning)

Under Process

Under Process

Under Process

Under Process

Module-7 (Overview of NLP):

Overview of NLP, Challenges of NLP (Language Ambiguities), Real World Applications of NLP, Approaches used to solve NLP use cases, Heuristic Method Approach, Machine Learning Approach, Deep Learning Based Approach, NLP Pipeline.

PRE-PROCESSING:
Removing digits and words containing digits, Removing newline characters and extra spaces, Removing HTML tags, Removing URLs, Removing punctuations.

Case folding, Expand contractions, Chat word treatment, Handle emojis, Spelling correction, What is Tokenization? Tokenization using string.split() Method, Tokenization using re.split() Method, Tokenization using NLTK library, Tokenization using spaCy library, Creating N-grams, What are Stopwords? Removing Stopwords using NLTK library, Removing Stopwords using spaCy library, Text Preprocessing on IMDB Dataset.

Stemming using NLTK’s PorterStemmer, Lemmatization using NLTK’s WordNetLemmatizer, Parts of Speech Tagging using spaCy library, Named Entity Recognition using spaCy library, Coreference Resolution.

FEATURE ENGINEERING:
Overview of Feature Engineering, Label Encoding, One-Hot encoding, Bag of words, Bag of n-grams, TF-IDF, Overview of Word Embeddings, Word2Vec using spaCy Large English Model, Word2Vec using Gensim and Google News Model.

SPAM DETECTION:
Overview of Text Classification, From Bayes’ Theorem to Naïve Bayes’ Classifier, Marginal Probability, Independent Events and Joint Probability, Dependent Events and Conditional Probability, From Conditional Probability to Bayes’ Theorem, Naïve Bayes’ Classifier.
Example 1: Naïve Bayes’ Classifier (Single Input Feature)
Example 2: Naïve Bayes’ Classifier (Multiple Input Features)
Example 3: Naïve Bayes’ Classifier (Text Data)
Data Acquisition for Email Spam Classification, Adding new features in the dataset, Exploratory Data Analysis, Text Pre-Processing, Text Vectorization using Bag of Words (BoW), Model Building (Feed Training Data to Machine Learning Model), Evaluation of Machine Learning Model, Confusion Matrix, Accuracy, Accuracy Paradox, Precision, Recall, Limitations of Precision and Recall, F-1 Score, Model Deployment, Creating a PipeLine Object for Predicting New Incoming Messages.