Using Pre-Trained Models
Keras Applications are deep learning models that are made available alongside pre-trained weights. These models can be used for prediction, feature extraction, and fine-tuning.
Weights are downloaded automatically when instantiating a model. They are stored at
The following image classification models (with weights trained on ImageNet) are available:
Classify ImageNet classes with ResNet50
# instantiate the model model <- application_resnet50(weights = 'imagenet') # load the image img_path <- "elephant.jpg" img <- image_load(img_path, target_size = c(224,224)) x <- image_to_array(img) # ensure we have a 4d tensor with single element in the batch dimension, # the preprocess the input for prediction using resnet50 dim(x) <- c(1, dim(x)) x <- imagenet_preprocess_input(x) # make predictions then decode and print them preds <- model %>% predict(x) imagenet_decode_predictions(preds, top = 3)[]
class_name class_description score 1 n02504013 Indian_elephant 0.90117526 2 n01871265 tusker 0.08774310 3 n02504458 African_elephant 0.01046011
Extract features with VGG16
model <- application_vgg16(weights = 'imagenet', include_top = FALSE) img_path <- "elephant.jpg" img <- image_load(img_path, target_size = c(224,224)) x <- image_to_array(img) dim(x) <- c(1, dim(x)) x <- imagenet_preprocess_input(x) features <- model %>% predict(x)
Extract features from an arbitrary intermediate layer with VGG19
base_model <- application_vgg19(weights = 'imagenet') model <- keras_model(inputs = base_model$input, outputs = get_layer(base_model, 'block4_pool')$output) img_path <- "elephant.jpg" img <- image_load(img_path, target_size = c(224,224)) x <- image_to_array(img) dim(x) <- c(1, dim(x)) x <- imagenet_preprocess_input(x) block4_pool_features <- model %>% predict(x)
Fine-tune InceptionV3 on a new set of classes
# create the base pre-trained model base_model <- application_inception_v3(weights = 'imagenet', include_top = FALSE) # add our custom layers predictions <- base_model$output %>% layer_global_average_pooling_2d() %>% layer_dense(units = 1024, activation = 'relu') %>% layer_dense(units = 200, activation = 'softmax') # this is the model we will train model <- keras_model(inputs = base_model$input, outputs = predictions) # first: train only the top layers (which were randomly initialized) # i.e. freeze all convolutional InceptionV3 layers for (layer in base_model$layers) layer$trainable <- FALSE # compile the model (should be done *after* setting layers to non-trainable) model %>% compile(optimizer = 'rmsprop', loss = 'categorical_crossentropy') # train the model on the new data for a few epochs model %>% fit_generator(...) # at this point, the top layers are well trained and we can start fine-tuning # convolutional layers from inception V3. We will freeze the bottom N layers # and train the remaining top layers. # let's visualize layer names and layer indices to see how many layers # we should freeze: layers <- base_model$layers for (i in 1:length(layers)) cat(i, layers[[i]]$name, "\n") # we chose to train the top 2 inception blocks, i.e. we will freeze # the first 172 layers and unfreeze the rest: for (i in 1:172) layers[[i]]$trainable <- FALSE for (i in 173:length(layers)) layers[[i]]$trainable <- TRUE # we need to recompile the model for these modifications to take effect # we use SGD with a low learning rate model %>% compile( optimizer = optimizer_sgd(lr = 0.0001, momentum = 0.9), loss = 'categorical_crossentropy' ) # we train our model again (this time fine-tuning the top 2 inception blocks # alongside the top Dense layers model %>% fit_generator(...)
Build InceptionV3 over a custom input tensor
# this could also be the output a different Keras model or layer input_tensor <- layer_input(shape = c(224, 224, 3)) model <- application_inception_V3(input_tensor = input_tensor, weights='imagenet', include_top = TRUE)
The VGG16 model is the basis for the Deep dream Keras example script.