SAS^® supports the creation of deep neural network models. Examples of these models include convolutional neural networks, recurrent neural networks, feedforward neural networks and autoencoder neural networks. Let’s examine in more detail how SAS creates deep learning models using SAS^® Visual Data Mining and Machine Learning.

Deep learning models with SAS Cloud Analytic Services

SAS Visual Mining and Machine Learning takes advantage of SAS Cloud Analytic Services (CAS) to perform what are referred to as CAS actions. You use CAS actions to load data, transform data, compute statistics, perform analytics and create output. Each action is configured by specifying a set of input parameters. Running a CAS action processes the action’s parameters and data, which creates an action result. CAS actions are grouped into CAS action sets.

Deep neural net models are trained and scored using the actions in the deepLearn CAS action set. This action set consists of several actions that support the end-to-end preprocessing, developing and deploying of deep neural network models. This action set provides users with the flexibility to describe their own model directed acyclic graph (DAG) to define the initial deep net structure. There are also actions that support adding and removing of layers from the network structure.

Appropriate model descriptions and parameters are needed to build deep learning models. We first need to define the network topology as a DAG and use this model description to train the parameters of the deep net models.

A deeper dive into the deepLearn SAS CAS Action Set

An overview of the steps involved in training deep neural network models, using the deepLearn CAS action set, is as follows:

1. Create an empty deep learning model.

• • The BuildModel() CAS action in the deepLearn action set creates an empty deep learning model in the form of a CASTable object.
  • Users can choose from DNN, RNN or CNN network types to build the respective initial network.

2. Add layers to the model.

• • This can be implemented using the addLayer() CAS action.
  • This CAS action provides the flexibility to add various types of layers, such as the input, convolutional, pooling, fully connected, residual or output as desired.
  • The specified layers are then added to the model table.
  • Each new layer has a unique identifier name associated with it.
  • This action also makes it possible to randomly crop/flip the input layer when images are given as inputs.

3. Remove layers from the model.

• • Carried out using the removeLayer() CAS action.
  • By specifying the necessary layer name, layers can be removed from the model table.

4. Perform hyperparameter autotuning.

• • dlTune() helps tune the optimization parameters needed for training the model.
  • Some of the tunable parameters include learning rate, dropout, mini batch size, gradient noise, etc.
  • For tuning, we must specify the lower and the upper bound range of the parameters within where we think the optimized value would lie.
  • An initial model weights table needs to be specified (in the form of a CASTable), which will initialize the model.
  • An exhaustive searching through the specified weights table is then performed on the same data multiple times to determine the optimized parameter values.
  • The resulting model weights with the best validation fit error is stored in a CAS table object.

5. Train the neural net model.

• • The dlTrain() action trains the specified deep learning model for classification or regression tasks.
  • By allowing the user to input the initial model table that was built, the best model weights table that was stored by performing hyper-parameter tuning and the predictor and response variables, we train the necessary neural net model.
  • Trained models such as DNNs can be stored as an ASTORE binary object to be deployed in the SAS Event Stream Processing engine for real-time online scoring of data.

6. Score the model.

• • The dlScore() action uses the trained model to score new data sets.
  • The model is scored using the trained model information from the ASTORE binary object and predicting against the new data set.

7. Export the model.

• • The dlExportModel() exports the trained neural net models to other formats.
  • ASTORE is the current binary format supported by CAS.

8. Import the model weights table.

• • dlImportModelWeights() imports the model weights information (that are initially specified as CAS table object) from external sources.
  • The currently supported format is HDF5.

As advances in deep learning are made, SAS will also continue to advance its deepLearn CAS action set.

This blog post is based on a SAS white paper, "How to Do Deep Learning With SAS: An introduction to deep learning neural networks and a guide to building deep learning models using SAS."