SavedModel contains a complete TensorFlow program, including weights and computation. It does not require the original model building code to run, which makes it useful for sharing or deploying (with TFLite, TensorFlow.js, TensorFlow Serving, or TensorFlow Hub).
If you have code for a model in R and merely want to load weights into it, see the guide to training checkpoints.
SavedModel from Keras
For a quick introduction, this section exports a pre-trained Keras model and serves image classification requests with it. The rest of the guide will fill in details and discuss other ways to create
We’ll use an image of Grace Hopper as a running example, and a Keras pre-trained image classification model since it’s easy to use. Custom models work too, and are covered in detail later.
file <- get_file( "grace_hopper.jpg", "https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg") library(magick) #> Linking to ImageMagick 220.127.116.11 #> Enabled features: cairo, fontconfig, freetype, lcms, pango, rsvg, webp #> Disabled features: fftw, ghostscript, x11 img <- image_read(file) img <- image_draw(img) img
The save-path follows a convention used by TensorFlow Serving where the last path component (1/ here) is a version number for your model - it allows tools like Tensorflow Serving to reason about the relative freshness.
SavedModels have named functions called signatures. Keras models export their forward pass under the serving_default signature key. The
SavedModel command line interface (see below) is useful for inspecting
SavedModels on disk:
We can load the
SavedModel back into Python with tf.saved_model.load and see how Admiral Hopper’s image is classified.
Imported signatures always return dictionaries.
Running inference from the
SavedModel gives the same result as the original model.
labeling <- infer(tf$constant(x, dtype = tf$float32))[pretrained_model$output_names] imagenet_decode_predictions(as.matrix(labeling[])) #> [] #> class_name class_description score #> 1 n03763968 military_uniform 0.701242745 #> 2 n02883205 bow_tie 0.186902404 #> 3 n04350905 suit 0.026838219 #> 4 n02817516 bearskin 0.021798020 #> 5 n03929855 pickelhaube 0.008993604
SavedModel format on disk
SavedModel is a directory containing serialized signatures and the state needed to run them, including variable values and vocabularies.
saved_model.pb file stores the actual TensorFlow program, or model, and a set of named signatures, each identifying a function that accepts tensor inputs and produces tensor outputs.
SavedModels may contain multiple variants of the model (multiple
v1$MetaGraphDefs, identified with the –tag_set flag to saved_model_cli), but this is rare.
variables directory contains a standard training checkpoint (see the guide to training checkpoints).
assets directory contains files used by the TensorFlow graph, for example text files used to initialize vocabulary tables. It is unused in this example.
SavedModels may have an
assets$extra directory for any files not used by the TensorFlow graph, for example information for consumers about what to do with the
SavedModel. TensorFlow itself does not use this directory.
SavedModel in C++
The C++ version of the
SavedModel loader provides an API to load a
SavedModel from a path, while allowing SessionOptions and RunOptions. You have to specify the tags associated with the graph to be loaded. The loaded version of
SavedModel is referred to as
SavedModelBundle and contains the MetaGraphDef and the session within which it is loaded.
Details of the
SavedModel command line interface
You can use the
SavedModel Command Line Interface (CLI) to inspect and execute a
SavedModel. For example, you can use the CLI to inspect the model’s SignatureDefs. The CLI enables you to quickly confirm that the input Tensor dtype and shape match the model. Moreover, if you want to test your model, you can use the CLI to do a sanity check by passing in sample inputs in various formats (for example, Python expressions) and then fetching the output.
Broadly speaking, you can install TensorFlow in either of the following two ways:
- By installing a pre-built TensorFlow binary.
- By building TensorFlow from source code.
If you installed TensorFlow through a pre-built TensorFlow binary, then the
SavedModel CLI is already installed on your system at pathname
If you built TensorFlow from source code, you must run the following additional command to build saved_model_cli:
Overview of commands
SavedModel CLI supports the following two commands on a
show, which shows the computations available from a
run, which runs a computation from a
SavedModel contains one or more model variants (technically,
v1.MetaGraphDefs), identified by their tag-sets. To serve a model, you might wonder what kind of
SignatureDefs are in each model variant, and what are their inputs and outputs. The show command let you examine the contents of the
SavedModel in hierarchical order. Here’s the syntax:
For example, the following command shows all available tag-sets in the
The following command shows all available SignatureDef keys for a tag set:
If there are multiple tags in the tag-set, you must specify all tags, each tag separated by a comma. For example:
To show all inputs and outputs
TensorInfo for a specific
SignatureDef, pass in the
SignatureDef key to signature_def option. This is very useful when you want to know the tensor key value, dtype and shape of the input tensors for executing the computation graph later. For example:
To show all available information in the
SavedModel, use the –all option. For example:
Invoke the run command to run a graph computation, passing inputs and then displaying (and optionally saving) the outputs. Here’s the syntax:
The run command provides the following three ways to pass inputs to the model:
--inputsoption enables you to pass numpy
--input_exprsoption enables you to pass Python expressions.
--input_examplesoption enables you to pass
To pass input data in files, specify the
--inputs option, which takes the following general format:
where INPUTS is either of the following formats:
You may pass multiple INPUTS. If you do pass multiple inputs, use a semicolon to separate each of the INPUTS.
saved_model_cli uses numpy.load to load the filename. The filename may be in any of the following formats:
- pickle format
.npy file always contains a numpy
ndarray. Therefore, when loading from a
.npy file, the content will be directly assigned to the specified input tensor. If you specify a variable_name with that
.npy file, the variable_name will be ignored and a warning will be issued.
When loading from a
.npz (zip) file, you may optionally specify a variable_name to identify the variable within the zip file to load for the input tensor key. If you don’t specify a variable_name, the
SavedModel CLI will check that only one file is included in the zip file and load it for the specified input tensor key.
When loading from a pickle file, if no variable_name is specified in the square brackets, whatever that is inside the pickle file will be passed to the specified input tensor key. Otherwise, the
SavedModel CLI will assume a dictionary is stored in the pickle file and the value corresponding to the
variable_name will be used.
To pass inputs through Python expressions, specify the
--input_exprs option. This can be useful for when you don’t have data files lying around, but still want to sanity check the model with some simple inputs that match the dtype and shape of the model’s
SignatureDefs. For example:
In addition to Python expressions, you may also pass numpy functions. For example:
(Note that the
numpy module is already available to you as
tf$train$Example as inputs, specify the
--input_examples option. For each input key, it takes a list of dictionary, where each dictionary is an instance of
tf$train$Example. The dictionary keys are the features and the values are the value lists for each feature. For example:
By default, the
SavedModel CLI writes output to stdout. If a directory is passed to
--outdir option, the outputs will be saved as
.npy files named after output tensor keys under the given directory.
--overwrite to overwrite existing output files.