[TensorFlow] Visualize learning by TensorBoard

 TensorFlow     TensorBoard    Python

▌Introduction

---

We will use TensorBoard to visualize the neural network in order to be easier to understand, debug, and optimize TensorFlow programs.

▋Related articles

1. [TensorFlow] Install on Windows

2. [TensorFlow] Linear Regression sample

3. [TensorFlow] Visualize learning by TensorBoard

4. [TensorFlow] Activation functions

5. [TensorFlow] Batch Normalization

6. [TensorFlow] Save and Restore model

7. [TensorFlow] Thread and Queue

▌Environment

---

▋Python 3.6.2

▋TensorFlow 1.5.0

▋matplotlib  2.1.2

▌Implement

---

We will use the Linear Regression sample in [TensorFlow] Linear Regression sample.

▋Add name_scope

First we use name_scope to pushes a name scope for a Python op into the graph.

And use property: name, to name the Variable or placeholder.

For example,

with tf.name_scope('Weights'):

    W = tf.Variable(tf.random_uniform([1], -1.0, 1.0), name='Weight')

with tf.name_scope('Biases'):   

    b = tf.Variable(tf.zeros([1]), name='Bias')

▋Output the event file

The FileWriter class can create an event file in a given directory and add summaries and events to it.

with tf.Session() as sess:

    writer = tf.summary.FileWriter("log/LinearRegression/", graph = sess.graph)

    #....

Which will generate the following file.

▋Histograms and Scalars

▋Histograms

TensorBoard Histogram Dashboard

with tf.name_scope('Weights'):

    W = tf.Variable(tf.random_uniform([1], -1.0, 1.0), name='Weight')

    tf.summary.histogram(name = 'Weights', values = W)

with tf.name_scope('Biases'):   

    b = tf.Variable(tf.zeros([1]), name='Bias')

    tf.summary.histogram(name = 'Biases', values = b)

▋Scalars

# Minimize the mean squared errors.

with tf.name_scope('Loss'):

    loss = tf.reduce_sum(tf.pow(y-train_Y, 2))/train_X.shape[0]

    tf.summary.scalar('Loss', loss)

▋Add summary FileWriter

Since we defined the Histograms or Scalars, we need to collect them and write them to event file.

with tf.Session() as sess:

    # Output graph

    merged = tf.summary.merge_all()

    writer = tf.summary.FileWriter("log/LinearRegression/", graph = sess.graph)

   

    # Fit all training data

    for step in range(training_epochs):

        sess.run(train)

        if step % display_step == 0:

            sess.run(loss, feed_dict={X: train_X, Y:train_Y})

            summary = sess.run(merged, feed_dict={X: train_X, Y:train_Y})

           writer.add_summary(summary, step)

            

You can find the complete source code here.

▋Start TensorBoard

Go to event files’ root directory, thaz  $/Samples/ in this example.

And execute the following command,

$ tensorboard --logdir='Log/LinearRegression'

to start TensorBoard.

Learning graph

Scalars

Histograms

▌Github

---

Source code

▌Reference

---

▋TensorBoard: Visualizing Learning