# Saved Model

A SavedModel contains a complete TensorFlow program, including weights and computation. It does not require the original model building code to run, which makes it useful for sharing or deploying (with TFLite, TensorFlow.js, TensorFlow Serving, or TensorFlow Hub).

If you have code for a model in R and merely want to load weights into it, see the guide to training checkpoints.

## Creating a SavedModel from Keras

For a quick introduction, this section exports a pre-trained Keras model and serves image classification requests with it. The rest of the guide will fill in details and discuss other ways to create SavedModels.

library(keras)
library(tensorflow)

physical_devices <- tf$config$experimental$list_physical_devices('GPU') if (length(physical_devices) != 0) tf$config$experimental$set_memory_growth(physical_devices[[1]], TRUE)

We’ll use an image of Grace Hopper as a running example, and a Keras pre-trained image classification model since it’s easy to use. Custom models work too, and are covered in detail later.

file <- get_file(
"grace_hopper.jpg",
library(magick)
#> Linking to ImageMagick 6.9.9.39
#> Enabled features: cairo, fontconfig, freetype, lcms, pango, rsvg, webp
#> Disabled features: fftw, ghostscript, x11
img <- image_draw(img)
img

x <- image_load(file, target_size = list(224, 224)) %>%
image_to_array() %>%
mobilenet_preprocess_input
dim(x) <- c(1, dim(x))
pretrained_model <- tf$keras$applications$MobileNet() result_before_save <- pretrained_model(x) preds <- imagenet_decode_predictions(as.matrix(result_before_save)) tf$saved_model$save(pretrained_model, "/tmp/mobilenet/1/") The save-path follows a convention used by TensorFlow Serving where the last path component (1/ here) is a version number for your model - it allows tools like Tensorflow Serving to reason about the relative freshness. SavedModels have named functions called signatures. Keras models export their forward pass under the serving_default signature key. The SavedModel command line interface (see below) is useful for inspecting SavedModels on disk: saved_model_cli show --dir /tmp/mobilenet/1 --tag_set serve --signature_def serving_default We can load the SavedModel back into Python with tf.saved_model.load and see how Admiral Hopper’s image is classified. loaded <- tf$saved_model$load("/tmp/mobilenet/1/") loaded$signatures$keys() # ["serving_default"] #> KeysView(_SignatureMap({'serving_default': <tensorflow.python.saved_model.load._WrapperFunction>})) Imported signatures always return dictionaries. infer <- loaded$signatures["serving_default"]
infer$structured_outputs #>$act_softmax
#> TensorSpec(shape=(None, 1000), dtype=tf.float32, name='act_softmax')

Running inference from the SavedModel gives the same result as the original model.

labeling <- infer(tf$constant(x, dtype = tf$float32))[pretrained_model$output_names[1]] imagenet_decode_predictions(as.matrix(labeling[[1]])) #> [[1]] #> class_name class_description score #> 1 n03763968 military_uniform 0.701242745 #> 2 n02883205 bow_tie 0.186902404 #> 3 n04350905 suit 0.026838219 #> 4 n02817516 bearskin 0.021798020 #> 5 n03929855 pickelhaube 0.008993604 ## The SavedModel format on disk A SavedModel is a directory containing serialized signatures and the state needed to run them, including variable values and vocabularies. ls /tmp/mobilenet/1 The saved_model.pb file stores the actual TensorFlow program, or model, and a set of named signatures, each identifying a function that accepts tensor inputs and produces tensor outputs. SavedModels may contain multiple variants of the model (multiple v1$MetaGraphDefs, identified with the –tag_set flag to saved_model_cli), but this is rare.

saved_model_cli show --dir /tmp/mobilenet/1 --tag_set serve

The variables directory contains a standard training checkpoint (see the guide to training checkpoints).

ls /tmp/mobilenet/1/variables

The assets directory contains files used by the TensorFlow graph, for example text files used to initialize vocabulary tables. It is unused in this example.

### Overview of commands

The SavedModel CLI supports the following two commands on a SavedModel:

• show, which shows the computations available from a SavedModel.
• run, which runs a computation from a SavedModel.

#### show command

A SavedModel contains one or more model variants (technically, v1.MetaGraphDefs), identified by their tag-sets. To serve a model, you might wonder what kind of SignatureDefs are in each model variant, and what are their inputs and outputs. The show command let you examine the contents of the SavedModel in hierarchical order. Here’s the syntax:

usage: saved_model_cli show [-h] --dir DIR [--all]
[--tag_set TAG_SET] [--signature_def SIGNATURE_DEF_KEY]

For example, the following command shows all available tag-sets in the SavedModel:

saved_model_cli show --dir /tmp/saved_model_dir

The following command shows all available SignatureDef keys for a tag set:

saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve

If there are multiple tags in the tag-set, you must specify all tags, each tag separated by a comma. For example:

$saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve,gpu To show all inputs and outputs TensorInfo for a specific SignatureDef, pass in the SignatureDef key to signature_def option. This is very useful when you want to know the tensor key value, dtype and shape of the input tensors for executing the computation graph later. For example: saved_model_cli show --dir \ /tmp/saved_model_dir --tag_set serve --signature_def serving_default To show all available information in the SavedModel, use the –all option. For example: saved_model_cli show --dir /tmp/saved_model_dir --all #### run command Invoke the run command to run a graph computation, passing inputs and then displaying (and optionally saving) the outputs. Here’s the syntax: usage: saved_model_cli run [-h] --dir DIR --tag_set TAG_SET --signature_def SIGNATURE_DEF_KEY [--inputs INPUTS] [--input_exprs INPUT_EXPRS] [--input_examples INPUT_EXAMPLES] [--outdir OUTDIR] [--overwrite] [--tf_debug] The run command provides the following three ways to pass inputs to the model: • --inputs option enables you to pass numpy ndarray in files. • --input_exprs option enables you to pass Python expressions. • --input_examples option enables you to pass tf$train$Example. ##### --inputs To pass input data in files, specify the --inputs option, which takes the following general format: --inputs <INPUTS> where INPUTS is either of the following formats: • <input_key>=<filename> • <input_key>=<filename>[<variable_name>] You may pass multiple INPUTS. If you do pass multiple inputs, use a semicolon to separate each of the INPUTS. saved_model_cli uses numpy.load to load the filename. The filename may be in any of the following formats: • .npy • .npz • pickle format A .npy file always contains a numpy ndarray. Therefore, when loading from a .npy file, the content will be directly assigned to the specified input tensor. If you specify a variable_name with that .npy file, the variable_name will be ignored and a warning will be issued. When loading from a .npz (zip) file, you may optionally specify a variable_name to identify the variable within the zip file to load for the input tensor key. If you don’t specify a variable_name, the SavedModel CLI will check that only one file is included in the zip file and load it for the specified input tensor key. When loading from a pickle file, if no variable_name is specified in the square brackets, whatever that is inside the pickle file will be passed to the specified input tensor key. Otherwise, the SavedModel CLI will assume a dictionary is stored in the pickle file and the value corresponding to the variable_name will be used. ##### --input_exprs To pass inputs through Python expressions, specify the --input_exprs option. This can be useful for when you don’t have data files lying around, but still want to sanity check the model with some simple inputs that match the dtype and shape of the model’s SignatureDefs. For example: <input_key>=[[1],[2],[3]] In addition to Python expressions, you may also pass numpy functions. For example: <input_key>=np.ones((32,32,3)) (Note that the numpy module is already available to you as np.) ##### --input_examples To pass tf$train$Example as inputs, specify the --input_examples option. For each input key, it takes a list of dictionary, where each dictionary is an instance of tf$train\$Example. The dictionary keys are the features and the values are the value lists for each feature. For example:

<input_key>=[{"age":[22,24],"education":["BS","MS"]}]

#### Save output

By default, the SavedModel CLI writes output to stdout. If a directory is passed to --outdir option, the outputs will be saved as .npy files named after output tensor keys under the given directory.

Use --overwrite to overwrite existing output files.