model %>%fit(x = mnist$train$x, y = mnist$train$y,batch_size =32,epochs =5,validation_sample =0.2,verbose =2 )
When we are happy with our model accuracy in the validation dataset we can evaluate the results on the test dataset with:
model %>%evaluate(x = mnist$test$x, y = mnist$test$y)
loss accuracy
0.04058329 0.98740000
OK, we have 99% accuracy on the test dataset and we want to deploy that model. First, let’s save the model in the SavedModel format using:
save_model_tf(model, "cnn-mnist")
With the model built and saved we can now start building our plumber API file.
Running locally
You can run the tensorflow/serving Docker image locally using the great stevedore package. For example:
docker <- stevedore::docker_client()container <- docker$container$run(image ="tensorflow/serving", # name of the image# host port and docker port - if you set 4000:8501, the API # will be accecible in localhost:4000port ="8501:8501", # a string path/to/the/saved/model/locally:models/modelname/version# you must put the model file in the /models/ folder.volume =paste0(normalizePath("cnn-mnist"), ":/models/model/1"), # the name of the model - it's the name of the folder inside `/models`# above.env =c("MODEL_NAME"="model"),# to run the container detacheddetach =TRUE)
Now we have initialized the container serving the model. You can see the container logs with:
container$logs()
Now you can make POST requests no the following endpoint : http://localhost:8501/v1/models/model/versions/1:predict. The input data must be passed in a special format 0 - see the format definition here, which may seem unnatural for R users. Here is an example:
This is how you can serve TensorFlow models with TF serving locally. Additionaly, we can deploy this to multiple clouds. In the next section we will show how it can be deployed to Google Cloud.
When done, you can stop the container with:
container$stop()
Deploying to Google Cloud Run
THe first thing you need to do is to follow the section Before you begin in this page.
Now let’s create a Dockerfile that will copy the SavedModel to the container image. We assume in this section some experience with Docker.
Here’s an example - create a file called Dockerfile in the same root folder as your SavedModel and paste the following:
FROM tensorflow/serving
COPY cnn-mnist /models/model/1
ENTRYPOINT ["/usr/bin/tf_serving_entrypoint.sh", "--rest_api_port=8080"]
We need to run the rest service in the 8080 port. The only that is open by Google Cloud Run. Now you can build this image and send it to gcr.io. Run the following in your terminal:
Next, we can create the service in Google Cloud Run using:
gcloud run deploy --image gcr.io/rstudio-162821/cnn-mnist --platform managed
You will be prompted to select a region, a name for the service and wether you allow unauthorized requests. If everything works correctly you will get a url like https://cnn-mnist-ld4lzfalyq-ue.a.run.app which you can now use to make requests to your model. For example:
req <- httr::POST("https://cnn-mnist-ld4lzfalyq-ue.a.run.app/v1/models/model/versions/1:predict", body = instances, encode ="json")httr::content(req)
Note that in this case, all pre-processing must be done in R before sending the data to the API.