IRIS Data Classification Using Neural Net

The Iris flowers dataset is one of the best-known datasets found in the classification literature. The goal is to classify Iris flowers among three species (Setosa, Versicolor or Virginica) from measurements of length and width of sepals and petals. 
The dataset is also known as Fisher’s Iris Data contains a set of 150 records under five attributes – petal length, petal width, sepal length, sepal width, and species.

Data set

First, we have to prepare the data set, which provides necessary information in a machine-readable way. Let’s First import and examine the data set.

import pandas as pd
import numpy as np
# Loading the dataset
dataset = pd.read_csv('Iris_Dataset.csv')
# Print top Five rows
dataset.head()

Here we are seeing Sepal Length, width, Petal Length and width for each flower.

Preprocessing

Now, we have to convert the flowers into numbers so that we can pass it into our neural network. The technique we are gonna using is called one-hot coding. Which is simply a multidimensional array. Where each row is representing the example and each column is representing the flowers. When the example is Setosa then only Setosa column will be 1 and the rest of the columns will be zero.

# One Hot Encoding
dataset = pd.get_dummies(dataset, columns=['Species']) 
values = list(dataset.columns.values)

Now, we have to define input and output data for the Neural network and shuffle them so that there’s no bias.

# output data
y = dataset[values[-3:]]
y = np.array(y, dtype='float32')

# input data
X = dataset[values[1:-3]]
X = np.array(X, dtype='float32')

# Shuffle Data
indices = np.random.choice(len(X), len(X), replace=False)
X_values = X[indices]
y_values = y[indices]

Neural Network Architecture

Here we will use the simplest neural network architecture with only one hidden layer which consists of 8 hidden units .

Define model Parameters

We are now ready for splitting data in training and test set with test set size 10. Will use 500 epochs for training.

# Creating a Train and a Test Dataset
test_size = 10
X_test = X_values[-test_size:]
X_train = X_values[:-test_size]
y_test = y_values[-test_size:]
y_train = y_values[:-test_size]

# Session
sess = tf.Session()

# Interval / Epochs
interval = 50
epoch = 500
hidden_layer_nodes = 8

We have to create variables for the parameters in the neural network. We must be careful about the shape of the parameters. We also have to create placeholders which will be fed eventually.

# Create variables for Neural Network layers
w1 = tf.Variable(tf.random_normal(shape=[4,hidden_layer_nodes])) # Inputs -> Hidden Layer
b1 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes]))   # First Bias
w2 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes,3])) # Hidden layer -> Outputs
b2 = tf.Variable(tf.random_normal(shape=[3]))   # Second Bias

# Initialize placeholders
X_data = tf.placeholder(shape=[None, 4], dtype=tf.float32)
y_target = tf.placeholder(shape=[None, 3], dtype=tf.float32)

We now have to define the operations for the hidden layer output and final output. Finally, we have to define our cost function and optimizer for backpropagation.

# Operations
hidden_output = tf.nn.relu(tf.add(tf.matmul(X_data, w1), b1))
final_output = tf.nn.softmax(tf.add(tf.matmul(hidden_output, w2), b2))

# Cost Function
loss = tf.reduce_mean(-tf.reduce_sum(y_target * tf.log(final_output), axis=0))

# Optimizer
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)

Training

To train the model we just have to run the session for the number of epochs in a loop.

# Training
print('Training the model...')
for i in range(1, (epoch + 1)):
    sess.run(optimizer, feed_dict={X_data: X_train, y_target: y_train})
    if i % interval == 0:
        print('Epoch', i, '|', 'Loss:', sess.run(loss, feed_dict={X_data: X_train, y_target: y_train}))

import matplotlib.pyplot as plt
iterations = list(ls.keys())
costs = list(ls.values())
plt.plot(iterations,costs)  
plt.ylabel('cost')
plt.xlabel('iterations (per tens)')
plt.title("Learning rate = " + str(.001))
plt.show() 

After training the model if we want to predict we just have use the test set.

# Prediction
for i in range(len(X_test)):
    print('Actual:', y_test[i], 'Predicted:', np.rint(sess.run(final_output, feed_dict={X_data: [X_test[i]]})))

Complete code will be in here.