Whatssssss up!

Welcome to guide book of Deep Learning.

Deep learning is deeply and easily summerized in this book. All the content is from 3rd parties since this book isn’t the documentation of any invention, rather it’s a documented form of already invented AI concepts.

Enjoy! :)

– Pahari

Bias and Variance

Bias and Variance two words are basic concept of ML. If you don’t know about them then malai baal.

Model Types

1. Overfit Model

When your model (say machine learning method/ algorithm or say curve) perform very well (imagine curve covering all the training points) in training dataset then there is high chance of model overfiting.

Note

Predicted value: Points of green curve
Actual value: Data points in red color

Let’s understand above figure.

There, (High) Variance is introduced. Variance can simply be understood as difference in fit of the model (how close the predicted value is from expected value between fits in 2 different test dataset) in different test dataset (or in outside ML context it is how much the predicted value is scattered).

If you have an model ( ML model/equation/ML algorithm … all similar), and you try to fit it in 2 different dataset. If there is high difference in error between there test error then it is called high variance. Here, test error varies greatly based on the selection of the training dataset.

2. Underfit Model

When your model (say machine learning method/ algorithm or say curve) perform little well only (imagine curve not covering must training points) then there is chance of model underfiting.

Let’s understand above figure.

There, (High) Bias is introduced. Bias can simply understood as how far the predicted value (curve) is from expected value(red dots).

If you have an model ( ML model/equation/ML algorithm … all similar), and you try to fit it in 2 different dataset. If there is high error in train error then it is called high bias. Here, train error is still high though different train dataset is taken.

3. Balanced Model

When your model (say machine learning method/ algorithm or say curve) perform well (imagine curve covering good number training points with less distance gap between curve and points) then there is chance of model being balanced. When it happens then in new test dataset there is possibility of less error (good prediction).

Low variance and low bias can result in creating a good model.

Remembering Tips 💡

a. Bias

How much the model fails to capture a true pattern in a training dataset. Resulting an underfit model (consistently wrong prediction in new dataset).

b. Variance

It is the amount by which the prediction would change if we fit the model to a different training data set (Bad prediction in new dataset).

Overfitting is shown. Variance is sensitivity to training data.

Note

Blue Dots: Training points
Green Dots: Testing points

References

1. Codebasics. Machine Learning Tutorial Python – 20: Bias vs Variance In Machine Learning. YouTube. Watch here

2. Josh Starmer (StatQuest). Machine Learning Fundamentals: Bias and Variance. YouTube. Watch here

Regression & Curve Fitting

Curve fitting refers to drawing a curve within a given dataset to capture a true pattern of the dataset. It is done using regression.

Types of Regression

It is about learning a general pattern and predict unseen data.

Note

Interpolation is not same as Regression. Interpolation is about drawing a curve passing exactly through all data points whereas Regression is about finding a best-fit trend in a given dataset.

1. Linear Regression Aka Least Squares

It is based on the line forming method. For a set of data in a graph, a line is drawn there and distance from that line (estimation line for output prediction of an unknown input) are sqaured and summed which is called as least squares.

And minimum least square is what we want to achive, i.e. Best Fit Model.

The final line (best fit model) minimizes the sums of squares (also known as least squares) between it and the real data.

In this situation, we optimize the value of slope(weight) and intercept(bias) for best fit.

2.

Chain Rule and Gradient Descent

Chain Rule
It is about “If something depends on something else, we calculate change step-by-step.”

Gradient Descent
It is about taking small steps downhill to reach the lowest error

Go to next for deeper understanding of both the concepts. ->

Chain Rule

Chain rule can be simply compared with the chain where say small objects are chained together.

Mathematically, one property relating to a secondary property and that another seconday property also relating with another third property implies new relation between first and the third one.

The below images can clear the concept of chain rule easily:

Things to know for the concpet of the chain rule in maths are:
a. Derivative
b. Slope

Finding best fit model

Step 1:
For each residual (x-axis is weight and y-axis is height) for a particular model(linear regression in this case), plot residuals and intercepts.

Step 2:
Plot residuals and squares of residual

Step 3:
Find the relation. Relation we got is
Weight -> Height -> Intercept (y-axis intercept/Height itself) -> Residuals -> Residual squares

Step 4:
Derivative of residual square with respect to intercept(Height) being zero means minimizing the squared residual (meaning less error)

References

1. https://www.youtube.com/watch?v=wl1myxrtQHQ&list=PLblh5JKOoLUICTaGLRoHQDuF_7q2GfuJF&index=55 | The Chain Rule, Clearly Explained!!! | StatQuest with Josh Starmer

Gradient Descent

Tip

Get familiar with the concept of Regression & Curve Fitting

It is about optimizing the model eg: Linear Regression (optimize the Intercept and Slope), Logistic Regression (optimize a squiggle), t-SNE (optimize the clusters) etc

Gradient means derivatives of the loss function with respect to the parameters whereas descent refers to descending the derivatives(slope) to near zero.

Loss Functions

Loss function are those type of functions that are used to measure how wrong the model’s prediction is (measure the difference between the actual value and the predicted value of a model)

Some type of loss functions are: Sum of Squared Residuals, Mean Absolute Error, Huber Loss, Mean Squared Log Error etc.

Working of Gradient Descent

Gradient Descent finds the optimal value (e.g. of value can be derivative of sum of square residuals curve plot with intercept on x-axis to get near zero) by taking big steps when it is far way and taking baby steps when near the optimal value.

Note

Using Least Sqaure method, we just find the slope of the curve (Sum of squared residuals vs Intercept) where it is zero. In contrast, Gradient Descent finds the minimum value by taking steps from a initial guess unit it reaches the best values. Gradient Descent is very useful when it is not possible to solve for where the derivative=0

Learning Rate

The is the rate that determines the step size of the parameter (which influence the finding of the optimal solution) like intercept.

Gradient Descent stops when the step size is very close to 0. And it happens when the slope is near to 0.

Formula:

Step Size = Slope * Learning Rate

Estimating Intercept and Slope using Gradient Descent

First, derivating the loss function (SSR) with respect to intercept and slope. Then, using learning rate increasing both two parameters Intercept and Slope. Once the step size starts getting near to 0 then it is suppose to getting near the optimal solution of both parameters.

Bird’s Eye View

Regularization

It is the technique in ML, where penalty is added to keep the model simple and avoid overfitting.

Types of Regularization

Ridge (L2) Regression

The main idea behind the ridge regression is to find a new line that doesn’t fit perfectly to the training data (done using bias).

In return, the variance is reduced and also avoid overfitting.

It basically not only minimize the sum of the squared residuals (like Least Sqaure) but also minimize (Lambda * Slope^2)

When sample size (training size) are relatively small, Ridge Regression (L2 regularization) can improve predictions made form new data (i.e. reduce variance) by making predicitons les sensitive to the training data by introducing penalty.

This is done by adding the ridge regression penalty((Lambda * Slope^2)) to the thing that must be minimized. i.e. th e sum of squared residuals (Least Sqaure) + (Lambda * Slope^2)

Lambda is determined using cross validation method. (test with the test data). Greater lambda results asymtotically zero.

Main effect is that it makes the prediction size less sensitive to the tiny training dataset.

Lasso (L1) Regression

The main idea behind the lasso regression similar to the ridge regression but here there is absolute value of slope in the penalty not square.

Similar to the Ridge regression, In return, the variance is reduced and also avoid overfitting. And main effect is that it makes the prediction size less sensitive to the tiny training dataset.

Lasso Regression can exclude useless varibles from equation due to the absolute value, it is little better than ridge regression at reducing teh variance in models that contains a lot of useless variables.

This is done by adding the lasso regression penalty((Lambda * |Slope|)) to the thing that must be minimized. i.e. th e sum of squared residuals (Least Sqaure) + (Lambda * |Slope|)

Word Embedding & Word2Vec

Long Short-Term Memory (LSTM)

Tip

Before diving into Long Short-Term Memory (LSTM), get familar with the concept of Recurrent Neural Network (RNN)

SequenceTransductionModels

Basics of Neural Network

Why is it called Neural Network? Because two fundamental components in the neural network nodes and connection are like brain neurons and synapses respectively.

Components of Neural Network

Fundamental components of NN are:

Nodes

It can be input node, output node and hidden nodes.

Hidden nodes have activation function. They are the curve from which y-axis value of the calculated x-axis from the layers (including bias and weight) is picked and plug into the dataset graph to fit the dataset.

Layers

Layers are like spider web, i.e. connections between nodes. It consist of bias and weights. 

Bias is addition (+). In ML context, bias is how much the model fails to capture a true pattern in a training dataset. Resulting an underfit model (consistently wrong prediction in new dataset).

For deeper understanding, check the blog mentioned just below.

Weight is multiplication part. It adds the importance for a particular factor.

Bias is addition part. It is like a threshold adjuster. Or think of this as shifter/base score/adjustment.

Weights and Biases helps for fitting (capture the pattern of the dataset) a model (curve) in the dataset to predict for a new case result.

Tip

Get familar with the concept of Bias and Variance

Math Behind Neural Network

Here, y-axis is for how effective the Dosage is. X-axis is the level of dosage (low,medium,high).

Stepwise Maths

It’s a septwise mathematics behind predicting if 0.5 dosage (medium) is effective or not.**

Step 1

By putting input value 0.5 (Dosage) and doing all the calculation (weights is *-34.4 and bias is +2.14), the result is corresponded to x-axis coordinate of activation funciton.

Step 2

For x-axis value (-15.06)of activation function got from step 1, y-axis value of the activation function is used to plot points in the actual data set to form a curve.

Y-axis value of activation function is calculated using equation of activation function. Used activation function is softmax [ f(x)=log(1+e^-15.06) ].

Step 3

Doing same for yellow layer till that hidden node

Step 4

Now the y-axis value from two hidden nodes (blue and orange), they both are summed and then again that value is summed with some value [ (some small number * -1.30) + (0.71 * 2.28) + (-0.58)].

This results 1.03 which is close to 1. So, which means 0.5 dosage is effective.

Note

Bias & Activation function are inside the neuron (nodes) and Weights are on connection (lines)
Still the connection is the part of the neuron

Tip

Now, get familiar with the concept of Regression & Curve Fitting

Single Perceptron

Single perceptron can’t solve non linear problem. Meaning single line can’t able to seperate classes (imagine impossible to draw a line to seperate multiple classes). e.g. XOR truth table output can’t be sepearable from a single line (meaning need multiple percentrons(and, !and, or)) to solve for the output.

** Flow of the FeedForward Network**

Input -> Weight _ Input + Bias -> Activation Function -> H (output of hidden layer) -> Weight _ H + Bias -> Activation Function -> Final Output

Training is done to adjust the weight and without training that network is nothing.

Hand Written Notes

Pahari’s Notes :)

References:

1. The Essential Main Ideas of Neural Networks | By StatQuest with Josh Starmer | https://www.youtube.com/watch?v=CqOfi41LfDw&list=PLblh5JKOoLUICTaGLRoHQDuF_7q2GfuJF&index=74
2. https://www.youtube.com/watch?v=i1G7PXZMnSc | The Perceptron Explained | Alice Heiman
3. The Main Ideas of Fitting a Line to Data (The Main Ideas of Least Squares and Linear Regression.) | https://www.youtube.com/watch?v=PaFPbb66DxQ&list=PLblh5JKOoLUICTaGLRoHQDuF_7q2GfuJF&index=9 | By StatQuest with Josh Starmer

Backpropagation in Neural Network

Tip

Before diving into Backpropagation, get familar with the concept of Chain rule and Gradient Descent

The core concepts behind the backpropagation is using gradient descent to the find the optimal value of the bias(b3) using learning rate, chain rule, derivate and loss function.

You will understand it better if you have gone through Chain Rule & Gradient Descent properly.

The screenshot below can illustrate the basic concept behind backpropagtion.

Basic understanding of Backpropagation

Assume optimal value of b3 in that figure is unknown. What we do next is, we assume 0 as initial value of b3 and find the SSR(Sum of Squared Residual) for the curve obtain from that particular bias value.

Then plot that SSR(y-axis) and Bias(x-axis) in a graph. Then take derivative of SSR with respect to the bias to find optimal value of bias.

Using Gradient Descent, we calcaulte optimal value for bias, which is obtain when step size (here it is for calculaitng the new bias value) is near 0.

Recurrent Neural Network (RNN)

Tip

Before diving into Recurrent Neural Network (RNN), get familar with the concept of Neural Network Basics

RNN is the DL architecture which is used for time series dataset where previous h (hidden state) is used in current hidden state calculation.

Convolution Neural Network

CNN is the basis of computer vision and image processing.

There is cross-correlation operation between matrices but not convolution operation. For, convolution operation, the kernal should be 180 degree rotated.

The simple architecture division looks like:

Input Image -> Convolution Layer 1 (Cross-Correlation, Pooling, ReLU) -> Activation Map -> Flattened the Activation Map -> Dense Layer -> Output

Layers:

Convolution Layers

It can thought as “eyes” of a CNN. Input image (244 x 224 x 3), get computed using filters of same channel size (i.e. 3).

Once cross-correlation opetation is done to get single output (one pixel output), then it is send thorugh an activation funciton, and that output from the activation function is save in the activation map (224 x 224 x 1), since single filter was used so only one map.

Note that since 3 channel was there, so for each channel each filter (3 channel filter with different value than each other) does the operation and summed to get a single matrix, and of elements of those get pass to the activation function to get an activation map.

Some Terminalolgy:

1. Padding:

It is inclusion of outer pixels or not.

2. Stride:

It is the step the kernal takes while doing operation.

Convolution Output Spatial Size Calcualtion

Sobel Kernel is one of the kernel example that is used to detect vertical edges.

[1,0,-1 2,0,-2 1,0,-1]

At the last stage of the convolution layer, all the activation maps (channel say) are flattened and then connected to the dense layer.

3. Max Pooling Layers

Extract the max value from the kernal size matrix from the activation maps. Pooling can be max(), avg() etc.

Fully Connected Classifier Layer

Here, dense layer is present.

Unit 1: Human Vision and Color Spaces

• 21 Jan: Introduction and Overview
• 22 Apr: Computer Vision, Ethics, and Society

This unit covers the biology of vision and the ways we encode images as numbers.

What can be done using Computer Vision?

• 3D reconstruction; eg: using 3DGS - Gausian Splatting
• 3D scenes dynamic reconstruction - capturing moving object in 3D; object moves in 3D
• Visualizing from photos - similar to 3DGS

Human Vision

This section explains how cameras and the human eye capture light.

The Eye2

The eye focuses light on the retina. The retina holds rod and cone cells.

• Rods: work in dim light, give no color information.
• Cones: work in bright light, give color information.

From Light to an Image

A digital sensor measures light at many points. Each point gives a pixel value.

• Exposure time and aperture control image brightness.
• Sensor noise and focus reduce image quality.

Limitations of the Eye

The brain completes missing information. This causes illusions.

Color Spaces and Transformations

This section covers how we represent color in numbers.

RGB

RGB stores three values for red, green, and blue light.

Each value is usually an 8-bit number from 0 to 255.

A single RGB image is a 3D array with shape H x W x 3.

HSV and HSL

HSV uses hue, saturation, and value. HSL uses hue, saturation, and lightness.

These spaces separate color from brightness. They make color thresholds easy to set.

Color Transformations

Common operations convert an image between color spaces.

• Channel swap or channel extraction.
• Histogram equalization on the value channel.
• Grayscale conversion with the luminance formula.

gray = 0.299 * R + 0.587 * G + 0.114 * B

Color Spaces in Practice

Deep learning models usually read RGB images directly. Classical methods use HSV and Lab more often.

Summary

This unit gives the base ideas of vision and color. Unit 2 uses them for image processing.

Unit 2: Image Processing

Duration: 6 hours

Lecture schedule:

• 23 Jan: Image filtering
• 28 Jan: Image filtering and edge detection
• 30 Jan: Image resampling
• 13 Feb: Image Transformations
• 20 Feb: Image Alignment
• 27 Feb: Cameras

This unit covers the core operations that change image pixels.

Image Coordinates and Resizing

This section explains how a computer stores and moves pixels.

Image Coordinates

An image is a matrix of values. The origin sits at the top left corner.

• The x-axis points right and the y-axis points down.
• A pixel has an integer position (x, y) and a value.

Interpolation

Interpolation estimates a new value from known neighbor values.

Common methods are nearest neighbor, bilinear, and bicubic.

Resizing

Resizing changes the image size. It needs an interpolation method.

• Downscaling often uses area averaging. It removes aliasing.
• Upscaling needs interpolation because the new pixels have no data.
• Resizing always changes the image aspect ratio or crop.

Filters and Convolutions

This section covers operations that compute a new pixel from a neighborhood.

Point Operators

A point operator changes each pixel on its own. It uses a function.

g(x, y) = T(f(x, y))

Examples are identity, inversion, gamma correction, and histogram equalization.

Linear Filtering

A linear filter multiplies neighbors with weights and sums the result.

g(x, y) = sum(w(i, j) * f(x + i, y + j))

Convolution flips the kernel before it slides. Cross-correlation does not flip it.

• Kernel size controls the neighborhood size.
• Padding keeps the output size equal to the input size.
• Stride controls the step size of the kernel.
• Blur, sharpen, and edge kernels are all linear filters.
• Separable kernels cut the compute cost for large kernels.

Non-Linear Filtering

A non-linear filter uses a function that is not a plain sum.

• Median filter removes salt and pepper noise.
• Bilateral filter smooths edges and keeps them sharp.
• Bilateral filter uses a spatial weight and a range weight.

Fourier Transforms

The Fourier transform changes an image from space to frequency.

• Low frequencies hold brightness and slow changes.
• High frequencies hold edges and fine noise.

Convolution in space equals multiplication in frequency.

A low pass filter blurs. A high pass filter finds edges.

Pyramids and Wavelets

A pyramid stores an image at many scales.

• Gaussian pyramid smooths and downsamples.
• Laplacian pyramid stores the detail between two levels.

A wavelet gives a local frequency representation. It gives a better time-frequency trade-off than Fourier.

Geometric Transformations

Geometric transformations change the pixel positions.

• Scaling, rotation, translation, and shear are affine transforms.
• A homography also changes perspective.
• A homography uses a 3x3 matrix and needs four point pairs.

Interpolation quality decides the output smoothness.

Summary

This unit gives the pixel operations. Unit 3 uses them for interpolation and optimization.

Unit 3: Interpolation and Optimization

Duration: 5 hours

Lecture schedule:

• No direct lecture match. This unit supports mathematical foundations for Units 2 and 6.

This unit covers how we estimate unknown values from known data.

Scattered Data Interpolation

This section covers interpolation when the data points have no grid.

The Problem

Scattered data gives values at unordered points (x, y, z). We must estimate the surface between them.

Methods

• Nearest neighbour: use the value of the closest point.
• Inverse distance weighting (IDW): weight each point by the inverse of its distance.
• Radial basis function (RBF): use a function of distance from a center point.
• Shepard interpolation: a smooth form of IDW.

Common radial basis functions are linear, cubic, thin plate spline, and Gaussian.

RBF Interpolation Formula

f(x) = sum(w(i) * phi(||x - c(i)||))

The weights come from a linear system. The system uses the values at the data points.

Variational Methods and Regularization

This section covers problems that need an optimization step to solve.

Least Squares Problems

A least squares problem finds the parameters that reduce the sum of squared errors.

E = sum((f(x, i) - y(i))^2)

The normal equations solve this problem for linear models.

Regularization

Regularization stops the model from fitting noise.

E = sum((f(x, i) - y(i))^2) + lambda * R(f)

• L1 regularization (Lasso): gives sparse solutions.
• L2 regularization (Ridge): shrinks coefficients smoothly.
• Tikhonov regularization: a general form of the same idea.

Total Variation

Total variation (TV) keeps an image sharp while it removes noise.

TV penalizes large changes between neighbor pixels.

Calculus of Variations

This field finds a whole function that reduces a cost.

• The Euler-Lagrange equation gives the first order condition.
• Gradient descent solves the problem with steps.

Markov Random Fields

This section covers how we model relations between neighbor pixels.

Markov Property

A pixel depends only on its neighbor pixels. It does not depend on far pixels.

A Markov random field (MRF) defines this rule on an image grid.

Energy Formulation

An MRF becomes a cost function over the whole image.

E = sum(data cost) + sum(smoothness cost)

The smoothness cost is low when neighbor labels are the same.

Inference

Inference finds the labelling with the lowest cost.

• Iterated Conditional Modes (ICM) changes labels one pixel at a time.
• Belief propagation and graph cuts work on larger regions.

Summary

This unit gives estimation and energy models. Unit 4 uses them to learn models from data.

Unit 4: Learning Paradigm for Computer Vision

Duration: 8 hours

Lecture schedule:

• 18 Mar: Neural Networks
• 10 Apr: Convolutional Neural Networks

This unit covers how models learn features from data.

Machine Learning

This section covers the classical supervised and unsupervised methods.

Supervised Learning

Supervised learning uses data with known labels.

• Regression: predicts a continuous value.
• Classification: predicts a category.

Unsupervised Learning

Unsupervised learning uses data without labels.

• Clustering: groups similar samples, such as k-means.
• Dimensionality reduction: makes data easier to handle, such as PCA.

Overfitting and Generalisation

A model overfits when it memorises the training data.

Use a validation set, cross validation, and regularisation to control it.

Evaluation Metrics

• Classification accuracy, precision, recall, and the F1 score.
• Mean average precision for detection tasks.
• Intersection over union (IoU) for segmentation tasks.

Neural Networks

This section covers multi-layer models of connected units.

The Perceptron

A perceptron computes a weighted sum and applies a step function.

Activation Functions

An activation function adds non-linearity.

• Sigmoid: outputs a value from 0 to 1.
• ReLU: outputs max(0, x). It is the most common function.
• Tanh: outputs a value from -1 to 1.
• Softmax: gives a probability for each class.

Forward and Backward Pass

The forward pass computes the output. The backward pass computes the gradients.

Gradient descent then updates the weights.

Optimization

• Gradient descent uses the full gradient.
• Stochastic gradient descent (SGD) uses a minibatch.
• Momentum smooths the update direction.
• Adam adapts the step size per parameter.

Overfitting Control

Dropout, weight decay, and data augmentation reduce overfitting.

Convolutional Neural Networks and Other Network Architectures

This section covers networks that use spatial structure.

Why Convolutions

Convolutions use shared weights. This cuts the parameter count.

CNN Layers

• Convolution layer: computes feature maps.
• Pooling layer: cuts the spatial size.
• Fully connected layer: gives class scores.

Famous Architectures

• LeNet-5: an early CNN for digits.
• AlexNet: won ImageNet in 2012.
• VGG-16: uses small 3x3 kernels.
• GoogLeNet: uses inception modules.
• ResNet: uses skip connections. It solves the vanishing gradient problem.

Skip Connections

A skip connection adds the input of a layer to its output.

Residual learning makes very deep networks easy to train.

Transfer Learning

This section covers reusing trained weights on a new task.

Why Transfer Learning

Large models need much data. Small datasets cannot train them from zero.

Feature Extraction

Freeze the early layers and train only the last layers.

Use the model as a fixed feature extractor on new data.

Fine-Tuning

Unfreeze some layers and train them with a low learning rate.

Fine-tune more layers when the new data is large and similar.

Autoencoder

This section covers networks that learn to compress and rebuild data.

Architecture

An encoder compresses the input into a low dimension code.

A decoder rebuilds the input from that code.

x -> Encoder -> code -> Decoder -> rebuilt x

Training

The autoencoder learns by making the rebuilt input close to the original input.

The loss is often the mean squared error or binary cross entropy.

Variants

• Denoising autoencoder: learns from noisy input.
• Variational autoencoder (VAE): makes the code a smooth distribution.
• Sparse autoencoder: keeps only few active code units.

Summary

This unit gives the learning methods. Unit 5 applies them to detection and segmentation.

Unit 5: Object Detection and Semantic Segmentation

Duration: 5 hours

Lecture schedule:

• 8 Apr: Introduction to Recognition

This unit covers tasks that find and label the objects in an image.

Object Detection

Object detection finds the objects and gives a box for each object.

Output Format

A detector returns a list of boxes, a class label, and a score for each box.

Bounding Box Encoding

A box is usually stored by its corner points or by its centre, width, and height.

• IoU measures how much two boxes overlap.
• Anchor boxes are preset box sizes and ratios.
• IoU thresholds set the match rule for ground truth.

Two-Stage Detectors

A two-stage detector first finds regions and then classifies each region.

• R-CNN uses selective search for regions.
• Fast R-CNN shares features across all regions.
• Faster R-CNN learns the region proposal network (RPN).
• Mask R-CNN adds a mask branch for segmentation.

One-Stage Detectors

A one-stage detector predicts all boxes in one step.

• YOLO divides the image into a grid.
• SSD uses multiscale feature maps.

Non-Maximum Suppression

Non-maximum suppression (NMS) removes boxes that overlap too much.

Keep the box with the highest score and drop the others.

Semantic Segmentation

Semantic segmentation gives a class label to every pixel.

It does not separate objects of the same class.

Encoder-Decoder Architectures

• Fully convolutional network (FCN): uses 1x1 convolutions for dense prediction.
• SegNet: uses an encoder-decoder with pooling and unpooling.
• U-Net: uses skip connections between encoder and decoder.

Skip connections keep sharp edges because they pass high resolution features.

Loss Functions

• Cross entropy loss: compares each pixel class.
• Dice loss: measures overlap. It helps with class imbalance.

Deep Learning Frameworks for Object Detection and Segmentation

This section covers the common tools and design choices.

Frameworks

• TensorFlow, Keras, and TensorFlow Detection API
• PyTorch, Torchvision, and Detectron2
• ONNX Runtime: runs a trained model in other languages.
• MMDetection and OpenMMLab: many detection and segmentation models.

Transfer Learning and Backbones

Pretrained backbones give strong features for small datasets.

Common backbones are ResNet, EfficientNet, and Vision Transformer.

Data Augmentation

Use flips, crops, colour changes, and scaling.

Augmentation raises the model tolerance to unseen images.

Practical Notes

• Check the data format and the annotation format first.
• Use the same preprocessing at training and at inference time.

Panoptic Segmentation and Video

This section covers methods that join detection and segmentation over time.

Panoptic Segmentation

Panoptic segmentation gives every pixel a label and an instance id.

• Thing classes count objects, such as a car or a person.
• Stuff classes count background areas, such as sky or grass.

This combines semantic segmentation and instance segmentation in one output.

Video Segmentation

Video adds a time axis to segmentation.

• Track the same object across frames.
• Use optical flow to move masks between frames.
• Smooth masks over time to avoid flicker.

Common Architectures

Panoptic FPN and Mask R-CNN are common panoptic models.

Video mask propagation and memory networks handle the time axis.

Summary

This unit gives dense prediction tasks. Unit 6 covers local features such as edges and keypoints.

Unit 6: Edges, Features, and Processing

Duration: 6 hours

Lecture schedule:

• 4 Feb: Feature Detection
• 6 Feb: Feature Invariance
• 11 Feb: Feature Descriptors and Feature Matching
• 25 Feb: RANSAC

This unit covers local image features that describe image structure.

Edges and Its Features

This section covers where object boundaries appear in an image.

Why Edges Matter

An edge marks a strong change of intensity.

Edges are stable under noise, scale, and small illumination change.

Gradient

The gradient gives the direction and size of the change.

• Use a Sobel, Scharr, or Prewitt kernel to compute the gradient.
• The gradient magnitude gives edge strength.
• The gradient angle gives the edge direction.

Edge Detection Operators

• Sobel operator: smooths noise and finds edges in one step.
• Canny detector: a complete multi-stage edge detector.
• Laplacian of Gaussian: finds edges with a second order operator.

Canny Detector Stages

1. Smooth the image with a Gaussian filter.
2. Compute the gradient magnitude and direction.
3. Apply non-maximum suppression to thin the edges.
4. Use two thresholds with hysteresis to keep strong edges.

Edge Matching and RANSAC Phenomenon

This section covers how to fit models to noisy feature sets.

The RANSAC Algorithm

Random sample consensus (RANSAC) fits a model to data with outliers.

1. Select a small random sample from the data.
2. Fit the model to that sample.
3. Count how many points fit the model. This is the inlier count.
4. Keep the model with the largest inlier count.
5. Refit the model on all inliers.

Why RANSAC Works

Only the inliers define the true model.

Random sampling increases the chance of picking an all-inlier sample.

Edge Matching

Edge matching joins edge segments between two images.

• Use a cost that measures shape and orientation similarity.
• Solve the assignment with the Hungarian algorithm.

Applications

RANSAC gives camera pose, fundamental matrix, and homography estimation.

It removes the effect of moving objects and mismatched points.

Edge Processing Algorithms

This section covers methods that describe local shape.

Histogram of Oriented Gradients (HoG)

HoG describes a shape by the direction of its gradients.

1. Compute gradients on the image.
2. Divide the image into small cells.
3. Build a histogram of gradient directions in each cell.
4. Normalise each block of cells.
5. Concatenate the blocks into one feature vector.

HoG works well for human detection.

Scale Invariant Feature Transform (SIFT)

SIFT finds keypoints that stay the same under scale and rotation change.

1. Build a scale space with a Gaussian pyramid.
2. Find keypoints as scale space local maxima of a difference of Gaussians.
3. Assign each keypoint an orientation from the gradient histogram.
4. Describe each keypoint with a 128-value gradient histogram.
5. Match keypoints with the nearest neighbour ratio test.

SIFT gives a local descriptor that is stable across viewpoints.

Optical Flow

Optical flow measures the motion of pixels between two frames.

• Brightness constancy: a pixel keeps its value after it moves.
• Small motion: the movement between nearby frames is small.
• Spatial coherence: neighbour pixels move in a similar way.

Common methods are the Lucas-Kanade method and the Horn-Schunck method.

Optical flow serves optical flow estimation, video compression, and motion segmentation.

Summary

This unit gives local features. Unit 7 uses them to find depth and build 3D scenes.

Unit 7: 3-D Vision

Duration: 5 hours

Lecture schedule:

• 11 Mar: Stereo
• 13 Mar: Multiview Stereo
• 13 Mar: Light and Perception
• 27 Mar: Light & Photometric Stereo

This unit covers how a machine understands the depth of a scene.

Introduction

This section covers the difference between 2D images and 3D scenes.

The Problem

A camera gives a 2D image. The world has 3D shape and depth.

Depth estimation recovers the missing dimension.

Sources of Depth Cues

• Occlusion: a near object hides a far object.
• Relative size: a known object looks smaller when it is far.
• Perspective: parallel lines meet at a vanishing point.
• Shading: light and shadow give the surface direction.
• Focus and motion blur: change with distance.
• Stereopsis: each eye sees a slightly different view.

Coordinate Systems

• Camera coordinate system: the origin sits at the camera centre.
• World coordinate system: a fixed frame for the whole scene.
• A transformation matrix moves points between these frames.

Depth Perception

This section covers how depth changes with viewing angle and distance.

Pinhole Camera Model

A pinhole camera maps 3D points to 2D pixels through projection.

u = f * X / Z
v = f * Y / Z

Here f is the focal length, and Z is the depth.

This model loses the depth value. Many 3D points map to one pixel.

Stereo Disparity

A stereo pair has two cameras. They have a small horizontal shift.

A point near the camera shifts more between the two images.

The shift is the disparity.

disparity = f * B / Z

Here B is the distance between the two cameras.

Depth falls when disparity grows.

Metric Depth from Disparity

The formula gives depth only for points that match well.

Occlusion and textureless areas make matching fail.

Monocular Depth Cues

One camera has no direct depth measure.

Learning based methods guess depth from cues such as size and shading.

Stereo Vision

This section covers how two images give a 3D reconstruction.

Correspondence Search

Stereo vision must match the same point in both images.

Search each pixel along an epipolar line. This cuts the search to one dimension.

Epipolar Geometry

• Epipolar line: the possible match positions in the second image.
• Fundamental matrix (F): defines the epipolar geometry of two views.
• Essential matrix (E): defines the same geometry for calibrated cameras.
• Epipole: the projection of one camera centre into the other image.

Stereo Matching Algorithms

• Block matching: compares small image patches.
• SAD and SSD: sum of absolute or squared differences of patches.
• Census transform: compares local bit patterns. It is robust to light change.
• Semi-global matching (SGM): adds a smoothness cost along paths.

Rectification

Rectification makes the epipolar lines horizontal.

Rectification makes matching faster and simpler.

Cost Volume and Deep Stereo

Learning based methods build a cost volume.

A 3D CNN then turns the cost volume into a disparity map.

Deep methods reach higher accuracy than hand made methods.

Summary

This unit gives depth from cameras. Unit 8 covers advanced topics such as SLAM and 3D models.

Unit 8: Advanced Topics in Computer Vision

Duration: 6 hours

Lecture schedule:

• 4 Mar: Panoramas
• 6 Mar: Single-view Modeling
• 20 Mar: Inverse Graphics and Neural Radiance Fields
• 25 Mar: Structure from Motion
• 15 Apr: Image Generation
• 17 Apr: Image Diffusion
• 24 Apr: Transformers
• 29 Apr: View Synthesis

This unit covers current research directions in computer vision.

Simultaneous Localization and Mapping (SLAM / SfM)

This section covers how a system maps a place and finds its own position.

Structure from Motion (SfM)

Structure from motion estimates camera poses and 3D points from many images.

1. Detect and match features across images.
2. Remove mismatches with RANSAC and geometric checks.
3. Recover camera poses with bundle adjustment.
4. Triangulate 3D points from known poses.

Bundle adjustment refines poses and points together. It reduces the reprojection error.

SLAM

Simultaneous localization and mapping (SLAM) works in real time.

The system builds a map and tracks its own position in that map.

SLAM pipelines:

• Front end: reads sensor data and extracts features.
• Back end: optimises the map with bundle adjustment.
• Loop closure: detects a visited place and corrects drift.

Loop Closure and Place Recognition

Loop closure reduces long term drift.

• BoW (Bag of Words): uses a histogram of local features.
• DBNet and learned descriptors: give better place features.

Modern Methods

Visual SLAM uses cameras only. LiDAR SLAM uses laser range data.

Learning based methods learn features and depth from data.

3D Shape and Appearance Models

This section covers methods that store and match 3D objects.

Point Clouds and Meshes

• Point cloud: an unordered set of 3D points.
• Mesh: vertices and faces that form a surface.

Shape Descriptors

• Signed distance function (SDF): the distance to the nearest surface.
• Occupancy grid: a voxel grid that stores empty and full cells.
• DeepSDF: a network that learns a continuous SDF.
• PointNet and PointNet++: networks that read point clouds directly.

Surface and Texture Mapping

Texture mapping puts a 2D image onto a 3D surface.

Unwrap the surface into a 2D plane, then sample the texture image.

Appearance Models

An appearance model stores the look of an object under many conditions.

Render many views and store them. Then match a new view to the set.

Deep learning methods use latent codes. One code gives many views of one subject.

Computational Photography

This section covers the mathematics behind camera image formation.

The Camera Model

A camera image depends on the scene, the lens, the sensor, and the exposure.

A blur kernel models the lens aperture.

Image Formation Pipeline

Scene radiance -> lens blur -> sensor noise -> quantisation -> display response.

• Demosaicing: one sensor gets only one colour per pixel.
• Colour correction: white balance and a camera matrix.
• Tone mapping: high dynamic range to displayable range.

HDR Imaging

Take several pictures at different exposures. Then combine them into one image.

This keeps detail in bright and dark areas of the scene.

Rolling Shutter and Super Resolution

• Rolling shutter gives each row a different capture time.
• Super resolution makes a higher resolution image from a low resolution one.
• Burst photography uses many photos of the same static scene.

Generative Adversarial Networks (GANs)

This section covers networks that learn to create new data.

Architecture

A GAN has two networks that compete.

• Generator (G): turns random noise into a fake sample.
• Discriminator (D): tells real samples from fake samples.

Training is a minimax game.

min_G max_D  E[log D(real)] + E[log(1 - D(G(noise)))]

The two networks improve each other during training.

Training Problems

• Mode collapse: the generator makes only a few samples.
• Unstable training: the loss value does not show real quality.
• Vanishing gradients: the gradient signal is too weak.

Improved GANs

• DCGAN: uses deep convolutional layers.
• WGAN: uses the Wasserstein loss. It gives a stable gradient signal.
• StyleGAN: gives control over the style of each generated image.

Applications

Image generation, image to image translation, super resolution, and inpainting.

Vision Transformers

This section covers transformer models for images.

Why Transformers

Transformers model relations between tokens. CNNs model local neighbours.

Transformers can use a global self-attention layer from the start.

Vision Transformer (ViT)

1. Cut the image into fixed size patches.
2. Flatten each patch into a vector.
3. Add a position embedding.
4. Add a class token for the whole image.
5. Pass the sequence through transformer encoder blocks.

A final layer head gives the class label.

Self-Attention

Each token builds a query, a key, and a value.

Attention weights show which tokens each token must read.

Attention(Q, K, V) = softmax(Q K^T / sqrt(d)) V

Improvements and Limits

• Swin Transformer: uses shifted window attention. It lowers the compute cost.
• Hybrid models: combine CNN and transformer blocks.

ViTs need large datasets or strong pretraining. CNNs often win with less data.

Summary

This unit gives advanced topics. Unit 8 closes the course.

Basics of Quantum Computing

In Quantum Computing, some of the basic terms that needs to be understood are:

• Qbits
• Superposition
• Entanglement
• Decoherence
• Interference
• Measurement
• Gates

Qbits

They are similar to the bits in classical computing, but is in quantum computing. It is the superposition of 1 and 0 but not literally 1 and 0 both (also represented as linear combination of 1 and 0 possibility).

It has amplitude and phase. But when measured, it’s value is colapsed at 1 or 0.

Qbits have possibility of having 1 or 0 based on the probabilty compution using ∣α∣^2 and ∣β∣^2

Superpostion

It is linear combination of basis state(1 and 0),output being 1 and 0 after measuring a qbit.

Here, the RHS is called as ket psi which tells the state of the Qbit. This is a wave-like state, not a classical value. And LHS is the linear combination of amplitudes corresponding to basis states to become the 0 or 1 bit after measurement.

Qbit can be in 1,0 or superpostion of 1 and 0. If there are 4 Qbits then 2^4 basic state which is 16 in this case, 0000,0001,0010,…,1111.

Before measurement, there above equation of state of Qbit is what represent the state of the Qbit. But, after measurement you get only one result (either 0000,0001,…., 1111). The state collapse to classical bits. It doesn;t mean giving all 16 outputs at the same time.

Measurement is like forcing quantum system to produce a classical outcome rather seeing the outcome as the wave.

Superpostion can’t be observed directly since any measurement destroys it which called decoherence.

Amplitude is α and β but not 1 and 0. They are complex numbers which represents wave strength and phase but not probabilities.

Note

amplitude (α and β)-> “how much wave exists” , They are also called probability weight probability (|α|^2 and |β|^2)-> “chance of outcome”

There above in note, we see absolute sign with square. That absolute is taking about the magnitude of that complex number (α and β) and squaring to get real value as probability.

Phase of state of Qbit is there in before measurement but after measurement, phase information is lost, rather it only reveals one classical outcome.

Manipulating amplitude means changing the value of α and β which is done using “quantum Gates” like hadamard, CNOT. Either increase some amplitudes, decrease others or change phase.

The idea of storing classical values is, that eg. 4 bits quantum system don’t store 16 classical values, but it stores 16 amplitudes.

Interference

Think it as waves, that can be added.

++ = biggerwave -> Constructive interference ++(-) = 0 (cancel the waves) -> Destructive interference

Wrong answers are cancel out while correct answers gets amplified.

If so than how to identify the wrong answer and write answer?

It is mentioned that algorithms are there to do so. Also, phase are not assigned rather they are controlled by quantum gate.
e.g.

• Hadamard gate → creates + and − superpositions
• Phase gate → adds phase shift e^iθ
• Oracle (in Grover’s algorithm) → flips phase of “marked” state

Example idea (Grover-style intuition)

Suppose correct answer = 1010

Oracle does:
∣1010⟩→−∣1010⟩
flips sign (phase = π)

In quantum computing, correct/wrong answers are not labeled. Instead, algorithm manupulate phases so that interfernce patterns amplify desired states (correct answer) and surppress other (wrong answer).

We only have a function which is reflected by the correct state. And this helps in amplifying the correct state.

Simple analogy

Imagine:

16 people in a dark room (states) and you cannot see them directly, but you have a rule:

“only correct person responds to a signal”

And you sen wave resulting correct person reflects differently (phase flip) and waves interfere which makes correct one becomes strongest echo

Entanglement

This is the ability of qubits to correlate their state with other Qbits. When an entangled Qbit is measured, its state become strongly correlated to other entangled Qbits which advantages on determining information immediately.

Decoherence

It is the process in which the quantum state collapse into a non-quantum state. This happens when quantum system is measured or makes a system behave as if measured (loses quantum behaviour(superposition + phase relations) when intereact with the environment). It then behaves like a classical object.

Decoherence affects quantum behavior, not the rule of measurement itself.

Quantam measurement problem

We can’t measure the full quantum state. The fundamental rule of nature is quantum state evolves smoothtly (wave-like) and measurement force a single outcome

Think of a spinning coin:

quantum state = coin spinning in air (mixture of heads & tails behavior)
measurement = catching it → only heads OR tails

Related Concepts on Quantum Mechanics & Computing

Heisenberg’s Uncertainity

It is represented by this formula:

Things to note that momentum is the inertia of motion. Simply think of it as p = m * v

Example to understand it’s affect are

• Bullet with less mass but larger velocity giving high momentum (damage to second object in contact)
• Larger ship with small velocity also giving high momentum (damage to second object in contact)

Interesting to know that, electron mass is extremely small so it can’t damage the second object in contact.

And also electron mass is constant, which mean only velocity is the variable that can be manipulated. Energey is the thing that cause the damage. Energy jumps from eV → MeV during the acceleration of electron (in nuclear plant). That’s a million times increase in energy.

Photon has speed of light then why it doesn’t damage.

It is because you are dump. See the defination of the both photon and electron.

electron: mass + velocity → kinetic energy (E = 1/2 (m* v^2)) photon: no mass → energy from frequency (E = hf)

Here, comes to a concept of quantum state. It can be imagined as a wave in water, where position is sharp bump and momentum is hidden mixture of wave components.

Fuccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccckkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkkk

Need to understand the Infamous Double Slit Experiment. Where tiny electrons acts like a wave. Crazy part is that a single atom acts like a wave, and get an interference pattern (many bright and dark stripes).

Theory of Relativity

Speed of light is constant which was predicted by the James Clerk Maxwell, whereas Isaac Newton gave a classical mechanics named Galilean relativity (theory of motion) while tells if you throw a ball at 10m/s inside a train moving at 20m/s then ground observer sees 30m/s.

If Newtonian physics predicts light speed should not be constant ( if light moves at speed c and u mode toward it then you should measure something like c+v) but that is not what happens.

Michelson-Morley experiment proved that no change in the speed of light, no matter Earth’s motion.

Einstein later made two bold statement; laws of physics are the same in all inertial frames and Speed of light is constant for all observers.

To keep light speed costant, nature adjusts time these; Time slows down (time dilation), Length contracts and simultaneity becomes relative

Core idea of Newton and Einstein are:

1. Newton: Time is absolute and speed changes (even for the light). e.g. We see light beam moving slower if we chase it.
2. Einstein: Speed of light is absolute and Time changes. e.g. We still see light at speed of c though we chase a light beam at speed of light.

Time Dilation

It is change in time when an object approach near speed of light (for that object the time of the outside world becomes very fast meaning if can travel 1 year further from only 10 sec of travel in speed of light).

Photon distance travel delay is what we see, but this is not time dilation (time dillation is what actuallly happens physically). Even if we could magically observe instantly(no delay), time dilation world still be there.

So, it is not illusion rahter it is real difference in how time flows.

Eg. Atomic Clock bouncing Top and Bottom mirror inside a train. The person inside the train see the moving of the atom in vertical distance, time is normal for him/her. But for the outside observer the atom is taking longer atom path (downward inverted (^^^^^^^^^^^^.(start))). So, for outside observe he/she conclude that time takes longer to tick.

Same analogy from the observer inside the train since he/she the outside guy moving backward and the atom in that outsider’s clock take longer distance too (downward inverted (.(start)^^^^^^^^^^^^))

Simultaneity

Interpolation Techniques

Interpolation is an estimation. It’s just about estimating the unknown value that comes either inside or outside the interval.

1. Linear Interpolation

If the point are estimated such that between two end point there form a line then we called it Linear Interpoation

Remember there is another word extrapolation which means the estimation of the values beyond the range of the data.

2. Cubic Spline interpolation. 

In short points, things to note in order to understand Cubic Spline Interpolation :

• Each interval have cubic polynomial function mapping.
• Cubic polynomial : ax³+bx²+cx+d
• For two sequential interval functions: Funcitons are equal for a interior points, it’s first derivative is also equal and it’s second derivative is also equal. [ Means Continuity]
• Extreme first and last points have second derivative zero.

Benefits of Cubic Spline and Bird Eye view

Derivation of Cubic Interpolation

Things to note from the above screenshot:

• 4 unknown coefficient need to be solved
• x0,……xn, for each point x-axis values
• n+1 points, then n polynomial function eg: above 3 point and 2 polynomial function for intervals So, 4 unknown coefficeint to be solved, we need n*4 equation ~ 4n
• Interior points: (n+1)-2 (need to exclude extreme points)= n-1 Each interior points have 2 polynomial equation and there are n polynomial from point 3 – So, 2*n equations from known points

To solve we use cubic polynomial in this format:

Si​(x)=ai​+bi​(x−xi​)+ci​(x−xi​)²+di​(x−xi​)³

Above points in below 2 screenshot which include same as above 4 points

Constraints:

Total equation that we can form are :

2n+(n-1)+(n-1)+2
= 4n-2+2
= 4n

Hence we can now plug all calculated known data 4n equations to get 4n equation for 4 unknown variables. And we solve it.

Application - Dengue Outbreak Interploation

Given:

Nepal’s District 2023 year dengue outbreak data for each month 

i.e  Jan - 1234 case Feb - 453 case ……………… Dec - 213 case

I want to interpolate in such way that I need to find out weekly interpolated data (i.e weekly estimation case from month data)

Solution:

Using Cubic Spline, 

Here, 

week_positions – created evenly spaced 52 points somthing look like [1.00, 1.21, 1.42, 1.63, …, 11.58, 11.79, 12.00]
bc_type = “natural” - second derivative of jan and dec is 0
cs – is the function for polynonimal (to estimate the case for given point which is x) 

cs behaves like a mathematical function: cs(x)

Benefits of using cubic spline in this application: (LLM)

a. Smooth and Natural Curve
Produces a smooth trend without sharp corners between data points.

b. Realistic Disease Pattern Modeling
Captures gradual rise and fall of outbreaks better than straight-line interpolation.

c. Continuity of Growth Rate
Maintains continuous first and second derivatives (smooth change in infection rate).

d. Accurate Within Data Range
Provides reliable estimates inside the observed time period (Jan–Dec).

e. Converts Monthly to Weekly Easily
Transforms discrete monthly data into a continuous function for weekly estimation.

f. Avoids High-Degree Polynomial Oscillation
Prevents extreme fluctuations seen in single high-order polynomial interpolation.

Rough Notes: 

Chebyshev Interpolation

Basic idea of Chebyshev Interpolation:

• Uses nodes with unequal distance to each other known to be chebyshev nodes.
• Node is decided from the unit circle with equally angled division to the semi circle 
• Nodes are taken from cosine formula
• Chebyshev polynomial is used to form a approx function

• Approx function is linear combination of C.polynomial

• C.polynomial is bounded on [-1,1]

Take a look into the consise note below for basics.

How to interpolate using Chebyshev Polynomial?

Let’s do it step by step:

Given -> f(x) function where x on bound [-1,1]
To find -> An approximate polynomial function to approx original function f(x)

Step 1:

Finding chebyshev nodes using chebyshev polynomial

i.e
from Tn+1​(x)=cos((n+1)arccosx) with roots when Tn+1(x)=0

Solving we get 

Chebyshev nodes are calculated using the formula :

Step 2:

Find all yk value for all C.nodes (ie x value got from step 1)

Step 3:

(Method 1): Using Lagrange interpolation to guarantee that all the found nodes passes through the approx polynomial function Pn(xk)

(Method 2): Using Chebyshev series representation

Step 4:

Coefficients are computed using following formulas and condition:

( I have not understood the math so understand yourself, I will do later)

Step 5:

Interpolation polynomial ( approx function) will be

(I have not understood the math so understand yourself, I will do later)

But need to understand that after getting general form of the polynomial in step 2; it is futher calculated to get evaluating formula that is in below llm reference

Step 6 (optional):

Calculating error using following formula

In short:

• Choose degree n -> n means n+1 coefficients & n+1 nodes
• Compute Chebyshev nodes i.e xk
• find yk
• Find chebyshev coefficients cj and construct polynomial ie Pn(x)
• Approx function: f(x) ~ Pn(x) using explicit chebyshev series form formula which is used to calculate yk for any xk

In code form

Both in Recursive way. Take reference from the hand written note

a. Polynomial Form

b. Consine Form

Application of Chebyshev Interpolation in the Dengue Case Interpolation

Given:

Nepal’s District 2023 year dengue outbreak data for each month 

i.e  Jan - 1234 case Feb - 453 case ……………… Dec - 213 case

And things to note:

F**k this can’t be applicable dude. I myself is getting confuse.

And reason is below

Note

Chebyshev interpolation = method to construct a polynomial that approximates the original function well, especially at edges, without wild oscillations.  The original function graph is “approximated” by this polynomial, not exactly reconstructed.

Not applicable for our senario dude.

Hold on guys, though Chebyshev node can’t be generated in our senario, using Chebyshev polynomials as a basis in our senerio can reduces round-off errors and instability in computation only which means using existing 12 months case data and using chebyshev polynomial instead of powers of x

What the F**k again. Dude my LLM is hallucinating yaar and forcing me too to hallucinate.

Ok, Lets do one thing, plot graph for normal polynomial built(power of x), linear interpolation, chebyshev interpolation and cublic spline too at once. 

Before that basic things need we, actually I am confused.

a. Linear Interpolation

As discussed above, straigth line joining two points is linear interpolation.

b. Polynomial Interpolation

A single polynomial function that passes a set of points (to each points). Lagrange Polynomial is one of the example of it. It introduce Runge oscillation

So, in our case with fixed monthy cases means equally spaced nodes, so no chebyshev nodes can be created so not applicable, even if we find chebyshev polynomial and plot, it is more like polynomial interpolation. That’s why there plot are overlapping with clear runge osscilation at both edge.

3. Rational Spline

It is foundational concept in the computer graphics. The standard parametric splines use standard polynomial to define their shape and doesn’t prefectly represent the conic sections like circle, ellipses, parabolas and hyperbola. 

Simply, rational function is the ratio of two polynomial.

Before diving into the rational spline, let’s have a look into the Bezier curves and B-spline.

Bezier Curves

 It is used in the computer graphics. The pen tool in the figma uses it.

For n+1 control points, Bezier curve is defined as 

This curve depends upon the control points. One point change can change whole structure of the curve.

In points:

• Global control with single control point change 
• Curve passes through first and last points
• Entire curve lies within convex hull of control points
• degree is n, and control points are n+1

B-spline

B-spline is also called basis spline which is combination of the piecewise polynomial (all with same degree )segments joined smoothly. It is controlled by control points and a knot vector. 

Attention Is All You Need

“One of the most popular and revolutionary research papers out there in the market. Today’s AI is almost 70% because of this paper,”

I assume that, because of what I have hearded about it.

If that is so, then let’s crack this paper and go deeper to understand the core concept that this paper has introduced to the world.  

Warning

This paper is absolute rabbit hole for beginners.

And yes, I am going deep into the hole.

Cracking “Abstract”

Since an abstract tells the concise summary of what the paper is trying to achieve, or say, jist of the paper, there is nothing but to understand some concepts that are introduced in this section first.

And they are:

I. Sequence Transduction Models

Sequence transduction means converting one sequence into another sequence. For example, the seq2seq model is a type of sequence transduction model.

Tip

Before diving deep into seq2seq, get familar with the concept of Long Short-Term Memory and Word Embedding & Word2Vec

seq2seq model consists of two components:

a. Encoder: 

b. Decoder:

Go Deeper into Sequence Transduction Models

II. Recurrent Neural Network

III. Convolution Neural Network

IV. Attention Mechanism

V. Transformer

What this paper achieved?

Getting Start

Below are the course contents that will be noted in this particular parent section “Deep Learning Course Content”

Course Content

Unit 1: Foundations & Applied Math (8 Hours)

• Introduction and History: Motivation for Deep Learning; Historical trends; Success stories.
• Linear Algebra & Probability: Tensors, Eigendecomposition, Information Theory, and Numerical Optimization.
• Bayesian Decision Theory: Making optimal decisions under uncertainty, inference vs. decision, and loss functions for classification/regression.
• Machine Learning Basics: Capacity, Overfitting/Underfitting, Hyperparameters, and the Bias-Variance tradeoff.

Unit 2: Deep Networks & Training Optimization (12 Hours)

• Deep Feedforward Networks: Multilayer Perceptrons (MLP); Gradient-Based Learning; Backpropagation and the Chain Rule.
• Modern Regularization: L1/L2 penalties, Dropout, Early Stopping, and Dataset Augmentation.
• Optimization & Normalization: SGD, Momentum, Adam Optimizer; Batch Normalization and Layer Normalization.

Unit 3: Convolutional Networks & Computer Vision (10 Hours)

• The Convolution Operation: Motivation, Pooling, and the Neuroscientific basis for CNNs.
• Modern Vision Architectures: Residual Networks (ResNets), Inception, and Deep CNN variants.
• Advanced Vision Tasks: Object Detection (YOLO/SSD), Semantic Segmentation, and the U-Net architecture.

Unit 4: Sequence Modeling & The Attention Revolution (10 Hours)

• Recurrent Neural Networks: RNNs, the Vanishing Gradient problem, and Gated Units (LSTM and GRU).
• The Attention Mechanism: Self-Attention, Multi-head Attention, and the “Attention is All You Need” paradigm.
• The Transformer Blueprint: Encoder-Decoder architecture, Positional Encoding, and scaling to Large Language Models (LLMs).

Unit 5: Frontiers: Generative & Graph Models (8 Hours)

• Autoencoders & Latent Spaces: Undercomplete autoencoders and Representation Learning.
• Generative AI: Variational Autoencoders (VAEs) and Diffusion Models.
• Graph Neural Networks: Message Passing, Node Embeddings, and Graph Convolutional Networks (GCNs).

Notation

Support Me — eSewa

I am raising funds to become independent. Your support helps me keep learning, sharing knowledge with others, and building innovative solutions. Scan the QR code below with your eSewa app — thank you! 🙏