When building deep learning models, it is often required to check the model for consistency and proper parameters definition. In ANNdotNET, ml network models are designed using Visual Network Designer (VND), so it is easy to see the network configuration. Beside VND, in ANNdotNET there are several visualization features on different level: network preparation, model training phase, post training evaluation, performance analysis, and export results. In this blog post we will learn how to use those features when working with deep learning models.
Visualization during network preparation and model training
When preparing network and training parameters, we need information about data sets, input format and output type. This information is relevant for selecting what type of network model to configure, what types of layers we will use, and what learner to select. For example the flowing image shows network configuration containing of 2 embedding layers, 3 dense layers and 2 dropout layers. This network configuration is used to train CNTK model for mushroom data set. As can be seen network layers are arranged as listbox items, and the user has possibility to see, on the highest level, how neural networks looks like, which layers are included in the network, and how many dimensions each layer is defined. This is very helpful, since it provides the way of building network very quickly and accurately, and it requires much less times in comparisons to use traditional way of coding the network in python, or other programming language.
ANNdotNET Network Settings page provides pretty much information about the network, input and output layers, what data set are defined, as well as whole network configuration arranged in layers. Beside network related information, the Network Settings tab page also provides the learning parameters for the network training. More about Visual Network Designer the ready can find on one of the previous blog post.
Since ANNdotNET implements MLEngine which is based on CNTK, so all CNTK related visualization features could be used. The CNTK library provides rich set of visualizations. For example you can use Tensorboard in CNTK for visualization not just computational graph, but also training history, model evaluation etc. Beside Tensorboard, CNTK provides logger module which uses Graphviz tool for visualizing network graph.The bad news of this is that all above features cannot be run on C#, since those implementation are available only in python.
This is one of the main reason why ANNdotNET provides rich set of visualizations for .NET platform. This includes: training history, model evaluation for training and validation data set, as well as model performance analysis. The following image show some of the visualization features: the training history (loss and evaluation) of minibatches during training of mushroom model:
Moreover, the following image shows evaluation of training and validation set for each iteration during training:
The graphs are generated during training phase, so the user can see what is happening with the model. This is of tremendous help, when deciding when to stop the training process, or are training parameters produce good model at all, or this can be helpful in case when can stop and change parameters values. In case we need to stop the training process immediately, ANNdotNET provides Stop command which stops training process at any time.
Model performance visualization
Once the model is trained, ANNdotNET provides performance analysis tool for all three types of ML problems: regression, binary and multi class classification.
Since the mushrooms project is binary ML problem the following image shows the performance of the trained model:
Using Graphviz to visualize CNTK network graph in C#
We have seen that ANNdotNET provides all types of visualizations CNTK models, and those features are provided by mouse click through the GUI interfaces. One more feature are coming to ANNdotNET v1.1 which uses Grpahviz to visualize CNTK network graph. The feature is implemented based on original CNTK python implementation with some modification and style.
In order to use Graphviz to visualize network computation graph the following requirements must be met:
- Install Graphviz on you machine.
- Register Graphviz path as system variable. (See image below)
Now that you have install Graphviz tool, you can generate nice image of your network model directly in ANNdotNET just by click on Graph button above the Visual Network Designer (see image 1).
Here is some of nice graphs which can be generate from ANNdotNET preclaculated models.
Graphviz generated graph of mushrooms model implemented in ANNdotNET
In case you like this nice visualization features go to http://github.com/bhrnjica/anndotnet, download the latest version from release section or just download the source code and try it with Visual Studio, but don't forget to give a star.
In the next blog post I will show you how visualization of CNTK
computational graph is implemented, so you will be able to use it in
your custom solutions.