Outliers

Most real-world data sets have outliers that have unusually small or big values then the typical values of the data set. Outliers can have a negative impact on data analysis if not detected correctly and also outliers can provide significant information at the same time. There are several methods to detect an outlier in the …

Logistic Regression Story

Logistic regression is one of the most commonly used algorithms for machine learning and is a building block for neural networks. It is really important to understand the concepts and the derivations of logistic regression. In this post we will explore the fundamentals of logistic regression and also concepts like maximum likelihood function, cross-entropy is …

Simple Linear Regression: An Introduction to Regression from scratch

Regression is a very fundamental concept in statistics, machine learning and in Neural Network. Imagine plotting the correlation between rainfall frequency and agriculture production in high school. Increase in rainfall generally increases agriculture production. Fitting a line to those points enables us to predict the production rate under different rain conditions. It was actually a …

Regularization in Neural Network

In the earlier era of machine learning, there used to be a lot of discussion on the bias-variance trade-off and the reason for that was we could increase bias and reduce variance, or reduce bias and increase variance. But back in the pre-deep learning era, we didn’t have many tools that just reduce bias or just reduce variance without hurting the other …

Differential Equations: Classification

Differential Equations An equation involving derivatives of one or more dependent variables with respect to one or more independent variables is called a differential equation. For example: \begin{equation} \frac{d^{2}y}{dx^2} + xy \left( \frac{dy}{dx}\right)^2 = 0 \end{equation} \begin{equation} \frac{d^{4}x}{dt^4} + 5 \frac{d^{2}x}{dt^2} + 3x = \sin t \end{equation} \begin{equation}\frac{\partial v}{\partial s} + \frac{\partial v}{\partial t} = …