Mini-project: Training a DNN in Python and exporting it to the ONNX format to run its predictions in a C++ micro-service

Python is nowadays the favourite platform for many machine learning scientists. It is an easy to learn language that provides a vast number of nice data science and AI tools perfect for rapid prototyping.

Models trained using Python DNN libraries like PyTorch and Tensorflow usually perform well enough to be used for production runs, but there are some situations that require the predictions to be run in C++ i.e., when best performance is required or when the model needs to be integrated with an existing C++ codebase.

ONNX (Open Neural Network Exchange) is an AI framework designed to allow interoperability between ML/DL frameworks. It allows, for example, models trained in scikit-learn, PyTorch, TensorFlow and other popular frameworks to be converted to the "standard" ONNX format for later use in any programming language with an existing ONNX runtime.

In this mini-project we trained a "dummy" DNN network (single task target predictor) in Python with PyTorch, exported it to the ONNX format and set up a C++ REST micro-service that uses the previously trained model to compute the predictions.

Since the trickiest part in this mini-project is gluing all the components together, we'll leave you with the repository link so you can explore it yourselves :)
In the repository you will find:

- the Python script with its conda environment file and the data to reproduce the training + the model already trained and exported (both in ONNX and torchscript formats).
- a Dockerfile to set up the C++ REST micro-service that runs the predictions using the previously trained model.
- the Python script to prove that the prediction results of the PyTorch model and of the ONNX exported model running in both Python and C++ are the same.

Link to the repository