The MNIST database (Modified National Institute of Standard Technology database) is an extensive database of handwritten digits, which is used for training various image processing systems. It was created by “reintegrating” samples from the original dataset of the MNIST.
If we are familiar with the building blocks of Connects, we are ready to build one with TensorFlow. We use the MNIST dataset for image classification.
Preparing the data is the same as in the previous tutorial. We can run code and jump directly into the architecture of CNN.
Here, we are executing our code in Google Colab (an online editor of machine learning).
We can go to TensorFlow editor through the below link: https://colab.research.google.com
These are the steps used to training the CNN (Convolutional Neural Network).
Steps:
Step 1: Upload Dataset
Step 2: The Input layer
Step 3: Convolutional layer
Step 4: Pooling layer
Step 5: Convolutional layer and Pooling Layer
Step 6: Dense layer
Step 7: Logit Layer

Step 1: Upload Dataset

The MNIST dataset is available with scikit for learning in this URL (Unified Resource Locator). We can download it and store it in our downloads. We can upload it with fetch_mldata ('MNIST Original').
Create a test/train set
We need to split the dataset into train_test_split.
Scale the features
Finally, we scale the function with the help of MinMax Scaler.
1. import numpy as np
2. import tensorflow as tf
3.
4. from sklearn.datasets import fetch_mldata
5. #Change USERNAME by the username of the machine
6. ##Windows USER
7. mnist = fetch_mldata('C:\\Users\\USERNAME\\Downloads\\MNIST original')
8. ## Mac User
9. mnist = fetch_mldata('/Users/USERNAME/Downloads/MNIST original')
10. print(mnist.data.shape)
11. print(mnist.target.shape)
12. from sklearn.model_selection import train_test_split
13. A_train, A_test, B_train, B_test = train_test_split(mnist.data,mnist.target, test_size=0.2, random_state=45)
14. B_train = B_train.astype(int)
15. B_test = B_test.astype(int)
16. batch_size =len(X_train)
17. print(A_train.shape, B_train.shape,B_test.shape )
18. ## rescale
19. from sklearn.preprocessing import MinMaxScaler
20. scaler = MinMaxScaler()
21. # Train the Dataset
22. X_train_scaled = scaler.fit_transform(A_train.astype(np.float65))

1. #test the dataset
2. X_test_scaled = scaler.fit_transform(A_test.astype(np.float65))
3. feature_columns = [tf.feature_column.numeric_column('x',shape=A_train_scaled.shape[1:])]
4. X_train_scaled.shape[1:]

Defining the CNN (Convolutional Neural Network)

CNN uses filters on the pixels of any image to learn detailed patterns compared to global patterns with a traditional neural network. To create CNN, we have to define:
1. A convolutional Layer: Apply the number of filters to the feature map. After convolution, we need to use a relay activation function to add non-linearity to the network.
2. Pooling Layer: The next step after the Convention is to downsampling the maximum facility. The objective is to reduce the mobility of the feature map to prevent overfitting and improve the computation speed. Max pooling is a traditional technique, which splits feature maps into subfields and only holds maximum values.
3. Fully connected Layers: All neurons from the past layers are associated with the other next layers. The CNN has classified the label according to the features from convolutional layers and reduced with any pooling layer.

CNN Architecture

• Convolutional Layer: It applies 14 5×5 filters (extracting 5×5-pixel sub-regions),
• Pooling Layer: This will perform max pooling with a 2×2 filter and stride of 2 (which specifies that pooled regions do not overlap).
• Convolutional Layer: It applies 36 5×5 filters, with ReLU activation function
• Pooling Layer: Again, performs max Pooling with a 2×2 filter and stride of 2.
• 1,764 neurons, with the dropout regularization rate of 0.4 (where the probability of 0.4 that any given element will be dropped in training)
• Dense Layer (Logits Layer): There are ten neurons, one for each digit target class (0-9).
Important modules to use in creating a CNN:
1. Conv2d (). Construct a two-dimensional convolutional layer with the number of filters, filter kernel size, padding, and activation function like arguments.
2. max_pooling2d (). Construct a two-dimensional pooling layer using the max-pooling algorithm.
3. Dense (). Construct a dense layer with the hidden layers and units
We can define a function to build CNN.
Let’s see in detail how to construct every building block before wrapping everything in the function.

Step 2: Input layer

1. #Input layer
2. def cnn_model_fn(mode, features, labels):
3. input_layer = tf.reshape(tensor= features["x"],shape=[-1, 26, 26, 1])
We need to define a tensor with the shape of the data. For that, we can use the module tf.reshape. In this module, we need to declare the tensor to reshape and to shape the tensor. The first argument is the feature of the data, that is defined in the argument of a function.
A picture has a width, a height, and a channel. The MNIST dataset is a monochromic picture with the 28x28 size. We set the batch size into -1 in the shape argument so that it takes the shape of the features ["x"]. The advantage is to tune the batch size to hyperparameters. If the batch size is 7, the tensor feeds 5,488 values (28 * 28 * 7).

Step 3: Convolutional Layer

1. # first CNN Layer
2. conv1 = tf.layers.conv2d(
3. inputs= input_layer,
4. filters= 18,
5. kernel_size= [7, 7],
6. padding="same",
7. activation=tf.nn.relu)
The first convolutional layer has 18 filters with the kernel size of 7x7 with equal padding. The same padding has both the output tensor and input tensor have the same width and height. TensorFlow will add zeros in the rows and columns to ensure the same size.
We use the Relu activation function. The output size will be [28, 28, and 14].

Step 4: Pooling layer

The next step after the convolutional is pooling computation. The pooling computation will reduce the extension of the data. We can use the module max_pooling2d with a size of 3x3 and stride of 2. We use the previous layer as input. The output size can be [batch_size, 14, 14, and 15].
1. ##first Pooling Layer
2. pool1 = tf.layers.max_pooling2d (inputs=conv1,
3. pool_size=[3, 3], strides=2)

Step 5: Pooling Layer and Second Convolutional Layer

The second CNN has exactly 32 filters, with the output size of [batch_size, 14, 14, 32]. The size of the pooling layer has the same as ahead, and output shape is [batch_size, 14, 14, and18].
1. conv2 = tf.layers.conv2d(
2. inputs=pool1,
3. filters=36,
4. kernel_size=[5, 5],
5. padding="same",
6. activation=tf.nn.relu)
7. pool2 = tf.layers.max_pooling2d (inputs=conv2, pool_size=[2, 2],strides=2).

Step6: Fully connected (Dense) Layer

We have to define the fully-connected layer. The feature map has to be compressed before to be combined with the dense layer. We can use the module reshape with a size of 7*7*36.
The dense layer will connect 1764 neurons. We add a Relu activation function and can add a Relu activation function. We add a dropout regularization term with a rate of 0.3, meaning 30 percent of the weights will be 0. The dropout takes place only along the training phase. The cnn_model_fn() has an argument mode to declare if the model needs to trained or to be evaluate.
1. pool2_flat = tf.reshape(pool2, [-1, 7 * 7 * 36])
2. dense = tf.layers.dense(inputs=pool2_flat, units=7 * 7 * 36, activation=tf.nn.relu)
3. dropout = tf.layers.dropout(
4. inputs=dense, rate=0.3, training=mode == tf.estimator.ModeKeys.TRAIN)

Step 7: Logits Layer

Finally, we define the last layer with the prediction of model. The output shape is equal to the batch size 12, equal to the total number of images in the layer.
1. #Logit Layer
2. logits = tf.layers.dense(inputs=dropout, units=12)
We can create a dictionary that contains classes and the possibility of each class. The module returns the highest value with tf.argmax () if the logit layers. The softmax function returns the probability of every class.
1. predictions= {
2. # Generate predictions
3. "classes":tf.argmax(input=logits, axis=1),
4. "probabilities":tf.nn.softmax (logits, name="softmax_tensor")}
We only want to return the dictionary prediction when the mode is set to prediction. We add these codes to display the predictions.
1. If mode== tf.estimator.ModeKeys.PREDICT:
2. return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)
The next step consists of computing the loss of the model. The loss is easily calculated with the following code:
1. # Calculate Loss (for both EVAL and TRAIN modes)
2. loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
The final step is to optimizing the model, which is to find the best values of weight. For that, we use a gradient descent optimizer with a learning rate of 0.001. The objective is to reduce losses.
1. optimizer= tf.train.GradientDescentOptimizer(learning_rate=0.0001)
2. train_op= optimizer.minimize(
3. loss=loss,
4. global_step=tf.train.get_global_step())
We are done with the CNN. However, we want to display the performance metrics during the evaluation mode. The performance metrics for the multiclass model is the accuracy metrics. TensorFlow is equipped with an accuracy model with two arguments, labels, and predicted value.
1. eval_metric_ops = {
2. "accuracy": tf.metrics.accuracy(labels=labels, predictions=predictions["classes"])}
3. return tf.estimator.EstimatorSpec(mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)
We can create our first CNN and we are ready to wrap everything in one function to use it and to train and evaluate the model.
1. def cnn_model_fn(features, labels, mode):
2. ""Model function for CNN.""
3. # Input Layer
4. input_layer = tf.reshape(features["x"], [-1, 28, 28, 1])
5.
6. # Convolutional Layer
7. conv1 = tf.layers.conv2d(
8. inputs=input_layer,
9. filters=32,
10. kernel_size=[5, 5],
11. padding="same",
12. activation=tf.nn.relu)
13.
14. # Pooling Layer
15. pool1 = tf.layers.max_pooling2d(inputs=conv1, pool_size=[2, 2], strides=2)
16.
17. # Convolutional Layer #2 and Pooling Layer
18. conv2 = tf.layers.conv2d(
19. inputs=pool1,
20. filters=36,
21. kernel_size=[5, 5],
22. padding="same",
23. activation=tf.nn.relu)
24. pool2 = tf.layers.max_pooling2d(inputs=conv2, pool_size=[2, 2], strides=2)
25.
26. # Dense Layer
27. pool2_flat = tf.reshape(pool2, [-1, 7 * 7 * 36])
28. dense = tf.layers.dense(inputs=pool2_flat, units=7 * 7 * 36, activation=tf.nn.relu)
29. dropout = tf.layers.dropout(
30. inputs=dense, rate=0.4, training=mode == tf.estimator.ModeKeys.TRAIN)
31.
32. # Logits Layer
33. logits = tf.layers.dense(inputs=dropout, units=10)
34.
35. predictions = {
36. # Generate predictions (for PREDICT and EVAL mode)
37. "classes": tf.argmax(input=logits, axis=1),
38. "probabilities": tf.nn.softmax(logits, name="softmax_tensor")
39. }
40.
41. if mode == tf.estimator.ModeKeys.PREDICT:
42. return tf.estimator.EstimatorSpec(mode=mode, predictions=predictions)
43.
44. # Calculate Loss
45. loss = tf.losses.sparse_softmax_cross_entropy(labels=labels, logits=logits)
46.
47. # Configure the Training Op (for TRAIN mode)
48. if mode == tf.estimator.ModeKeys.TRAIN:
49. optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)
50. train_op = optimizer.minimize(
51. loss=loss,
52. global_step=tf.train.get_global_step())
53. return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)
54. # Add evaluation metrics Evaluation mode
55. eval_metric_ops = {
56. "accuracy": tf.metrics.accuracy(
57. labels=labels, predictions=predictions["classes"])}
58. return tf.estimator.EstimatorSpec(
59. mode=mode, loss=loss, eval_metric_ops=eval_metric_ops)
A CNN takes many times to training, therefore, we create a logging hook to store the values of the software layers in every 50 iterations.
1. # Set up logging for predictions
2. tensors_to_log = {"probabilities": "softmax_tensor"}
3. logging_hook =tf.train.LoggingTensorHook(tensors=tensors_to_log, every_n_iter=50)
We are ready to estimator the model. We have a batch size of 100 and shuffle the data into many parts. Note that, we set training steps of 18000, it can take lots of time to train.
1. #Train the model
2. train_input_fn = tf.estimator.inputs.numpy_input_fn(
3. x={"x": X_train_scaled},
4. y=y_train,
5. batch_size=100,
6. num_epochs=None,
7. shuffle=True)
8. mnist_classifier.train(
9. input_fn=train_input_fn,
10. steps=18000,
11. hooks=[logging_hook])
Now, the model is trained, we can evaluate it and print the results easily.
1. # Evaluate the model and print the results
2. eval_input_fn = tf.estimator.inputs.numpy_input_fn(
3. x= {"x": X_test_scaled},
4. y=y_test,
5. num_epochs=1,
6. shuffle=False)
7. eval_results = mnist_classifier.evaluate(input_fn=eval_input_fn)
8. print(eval_results)

1. INFO:tensorflow:Calling model_fn
2. INFO:tensorflow:Done calling model_fn
3. INFO:tensorflow:Starting evaluation at 2019-08-10-12:53:40
4. INFO:tensorflow:Graph is finalized.
5. INFO:tensorflow:Restoring parameters from train/mnist_convnet_model/model.ckpt-15652
6. INFO:tensorflow: Running local_init_op
7. INFO:tensorflow: Running local_init_op
8. INFO:tensorflow:Finished evaluation at 2019-07-05-12:52:56
9. INFO:tensorflow: Saving dict for global step 15652: accuracy = 0.9589586, global_step = 15852, loss = 0.13894269{'accuracy': 0.9689286, 'loss': 0.13894269, 'global_step': 15652}
With the help of architecture, we get an accuracy of 97%. We can change the architecture, batch size, and number of iterations to improve accuracy. Architecture, batch size and number of iterations to improve accuracy.
CNN neural networks have performed far better than ANN or logistic regression. In the tutorial on artificial neural networks, we had an accuracy of 96%, which is low CNN. CNN's performances are impressive with an extensive image set, both in terms of speed calculation and accuracy.
To build CNN, we need to follow these six steps:
1) Input layer:
This step resets the data. Size is equal to the square root of the number of pixels. For example, if a picture has 156 pixels, the figure is 26x26. We need to specify whether the image contains color or not. If so, we had a size 3 to 3 for RGB-, otherwise 1.
1. Input_layer= tf.reshape(tensor= features["x"], shape= [-1,30,30,1])
2) Convolutional layer
We need to create consistent layers. We apply various filters to learn important features of the network. We define the size of the kernel and volume of the filter.
1. conv1= tf.layers.conv2d(
2. inputs=input_layer,
3. filters=14,
4. kernel_size=[6, 6],
5. padding="same",
6. activation= tf.nn.relu)
3) Pooling Layer
In the third step, we add a pooling layer. This layer reduces the size of the input. It does by taking the maximum value of the sub-matrix.
1. pool1 = tf.layers.max_pooling2d(inputs=conv1, strides=2, pool_size=[2, 2])
4) Add Convolutional Layer and Pooling Layer
In this step, we can add as many pooling layers as we want. It uses Google architecture with more than 20 hard layers.
5) Dense Layer
Step 5 flattens the previous to form fully joined layers. In this step, we can use a different activation function and add the dropout effect.
1. pool2_flat = tf.reshape(pool2, [-1, 8 * 8 * 36])
2. dense = tf.layers.dense(inputs=pool3_flat, units=8 * 8 * 36, activation=tf.nn.relu)
3. dropout = tf.layers.dropout(
4. Inputs=dense, rate=0.3, trainingmode == tf.estimator.ModeKeys.TRAIN)
6) Logit Layer
The final step is the prediction.
1. logits = tf.layers.dense(inputs=dropout, units=12)

So, this brings us to the end of blog. This Tecklearn ‘Training of CNN in Tensor Flow’ blog helps you with commonly asked questions if you are looking out for a job in Artificial Intelligence. If you wish to learn Artificial Intelligence and build a career in AI or Machine Learning domain, then check out our interactive, Artificial Intelligence and Deep Learning with TensorFlow Training, that comes with 24*7 support to guide you throughout your learning period. Please find the link for course details:

https://www.tecklearn.com/course/artificial-intelligence-and-deep-learning-with-tensorflow/

Artificial Intelligence and Deep Learning with TensorFlow Training

About the Course

Tecklearn’s Artificial Intelligence and Deep Learning with Tensor Flow course is curated by industry professionals as per the industry requirements & demands and aligned with the latest best practices. You’ll master convolutional neural networks (CNN), TensorFlow, TensorFlow code, transfer learning, graph visualization, recurrent neural networks (RNN), Deep Learning libraries, GPU in Deep Learning, Keras and TFLearn APIs, backpropagation, and hyperparameters via hands-on projects. The trainee will learn AI by mastering natural language processing, deep neural networks, predictive analytics, reinforcement learning, and more programming languages needed to shine in this field.

Why Should you take Artificial Intelligence and Deep Learning with Tensor Flow Training?

• According to Paysa.com, an Artificial Intelligence Engineer earns an average of $171,715, ranging from $124,542 at the 25th percentile to $201,853 at the 75th percentile, with top earners earning more than $257,530.
• Worldwide Spending on Artificial Intelligence Systems Will Be Nearly $98 Billion in 2023, According to New IDC Spending Guide at a GACR of 28.5%.
• IBM, Amazon, Apple, Google, Facebook, Microsoft, Oracle and almost all the leading companies are working on Artificial Intelligence to innovate future technologies.

What you will Learn in this Course?

Introduction to Deep Learning and AI
• What is Deep Learning?
• Advantage of Deep Learning over Machine learning
• Real-Life use cases of Deep Learning
• Review of Machine Learning: Regression, Classification, Clustering, Reinforcement Learning, Underfitting and Overfitting, Optimization
• Pre-requisites for AI & DL
• Python Programming Language
• Installation & IDE
Environment Set Up and Essentials
• Installation
• Python – NumPy
• Python for Data Science and AI
• Python Language Essentials
• Python Libraries – Numpy and Pandas
• Numpy for Mathematical Computing
More Prerequisites for Deep Learning and AI
• Pandas for Data Analysis
• Machine Learning Basic Concepts
• Normalization
• Data Set
• Machine Learning Concepts
• Regression
• Logistic Regression
• SVM – Support Vector Machines
• Decision Trees
• Python Libraries for Data Science and AI
Introduction to Neural Networks
• Creating Module
• Neural Network Equation
• Sigmoid Function
• Multi-layered perception
• Weights, Biases
• Activation Functions
• Gradient Decent or Error function
• Epoch, Forward & backword propagation
• What is TensorFlow?
• TensorFlow code-basics
• Graph Visualization
• Constants, Placeholders, Variables
Multi-layered Neural Networks
• Error Back propagation issues
• Drop outs
Regularization techniques in Deep Learning
Deep Learning Libraries
• Tensorflow
• Keras
• OpenCV
• SkImage
• PIL
Building of Simple Neural Network from Scratch from Simple Equation
• Training the model
Dual Equation Neural Network
• TensorFlow
• Predicting Algorithm
Introduction to Keras API
• Define Keras
• How to compose Models in Keras
• Sequential Composition
• Functional Composition
• Predefined Neural Network Layers
• What is Batch Normalization
• Saving and Loading a model with Keras
• Customizing the Training Process
• Using TensorBoard with Keras
• Use-Case Implementation with Keras
GPU in Deep Learning
• Introduction to GPUs and how they differ from CPUs
• Importance of GPUs in training Deep Learning Networks
• The GPU constituent with simpler core and concurrent hardware
• Keras Model Saving and Reusing
• Deploying Keras with TensorBoard
Keras Cat Vs Dog Modelling
• Activation Functions in Neural Network
Optimization Techniques
• Some Examples for Neural Network
Convolutional Neural Networks (CNN)
• Introduction to CNNs
• CNNs Application
• Architecture of a CNN
• Convolution and Pooling layers in a CNN
• Understanding and Visualizing a CNN
RNN: Recurrent Neural Networks
• Introduction to RNN Model
• Application use cases of RNN
• Modelling sequences
• Training RNNs with Backpropagation
• Long Short-Term memory (LSTM)
• Recursive Neural Tensor Network Theory
• Recurrent Neural Network Model
Application of Deep Learning in image recognition, NLP and more
Real world projects in recommender systems and others
Got a question for us? Please mention it in the comments section and we will get back to you.

623