All about TensorFlow Serving with Docker for Model Deployment

Introduction to Tensorflow Serving

TensorFlow Serving is used for production environments as scalable, flexible, high-performance serving systems for machine learning models. TensorFlow Serving makes it easy to deploy models and it is very much best suited to use it with docker.

In a simple run, our models will be used as an endpoint for other services.

Advantages of TensorFlow Serving while using Multiple Models

I will explain this with an example

Let us suppose we have three models

1. Cat Vs Dog classifier

2. MNIST data classifier

3. Flowers classifier

• We want all three models should respond and needs to be active at the same time.

• Tensorflow Serving will help you and give API for these three models very easily.

• This is the most optimized method in the world of production.

Docker Concepts while using it

1. TensorFlow Serving Image

The image can be found on the docker hub that only the docker run command as it is already docker build and pushed to the docker hub.

The docker run command with this image will automatically pull the image from the docker hub like this

docker pull tensorflow/serving

TensorFlow Serving Image

2. Mounting Volumes

Mounting volumes in docker is very helpful when your local folder that is being mounted can be updated and with every update, you do not need to re-deploy the model it will be automatically be reflected inside the container. The local directory that will be mounted can have multiple models being explained and they will be run altogether.

Using volumes

3. Port Mapping

For your TensorFlow Serving model service inside a container's ports needs to be mapped for using it locally for other services and one recommendation will be to use network also.

How networking works from the container's point of view

Everything at one place

Here is the labeled image of the docker run that needs to be required

Thanks For Reading!