How to Deploy a Scikit-Learn Model as a FastAPI Microservice

Your model performs well on the holdout set. That is not the same as being useful. Nobody on your team wants to open a notebook, install your environment, and run three cells to get a churn probability. They want a URL they can call from another service. FastAPI plus Docker gets you there with a modest amount of code, and the path is short: save the model correctly, wrap it in an API that rejects bad input, then package the whole thing so it behaves identically on your laptop and on the server.

Serialize the Model and Its Assumptions

The first mistake is pickling a bare estimator that expects a DataFrame someone else has already cleaned. Preprocessing logic that lives in a notebook is not part of the artifact, so the service will drift from training in a week. Put imputers, encoders and the estimator into a single sklearn Pipeline and fit it once. Then the artifact carries its own preprocessing.

Use joblib.dump for anything NumPy-heavy — it handles large arrays more efficiently than the standard pickle module. Save to a versioned filename such as model_v3.joblib rather than model.joblib, so a rollback is a config change and not a rebuild.

Alongside the pipeline, persist the small pieces of context that are easy to forget:

• The exact list of feature names, in the order the pipeline expects.
• The mapping from encoded output back to human-readable labels.
• The decision threshold, if you tuned it away from 0.5.
• The library versions used at training time, as a plain text file.

That last one matters more than people expect. A pickle can fail to load across incompatible scikit-learn versions, and debugging that at 2 a.m. is unpleasant. Record the versions, pin them in your requirements, and keep custom transformer classes in an importable module that ships with the service.

Build the Smallest Useful API

Create a package with a main.py that defines the app. Load the model once, at startup, inside a lifespan handler, and keep it in a module-level variable. Loading inside the request handler is the most common performance bug in model APIs: deserialization can take hundreds of milliseconds and allocates memory on every call.

A GET /health endpoint that returns a small JSON object is worth adding on day one. Container orchestrators and load balancers need something cheap to poll, and it gives you a quick way to confirm the model loaded without running a prediction.

FastAPI runs ordinary def endpoints in a threadpool, so calling a blocking pipeline predict inside one is fine. Reserve async def for endpoints that await network or database calls. Mixing the two carelessly is how you accidentally block the event loop.

Validate Inputs with Pydantic

Define a Pydantic model describing one record. Be explicit about types and constrain the ranges you actually know about — Field(ge=0, le=120) on age, a maximum length on categorical strings, a regex where a value has a fixed format. FastAPI returns a 422 with a readable error body automatically when validation fails, which saves you writing that logic.

Two details separate a demo from something usable. First, resist the urge to give every field a default. A missing value that silently becomes 0 or "unknown" produces a confident, wrong prediction, and silent wrongness is worse than an error. Make the required fields required. Second, cap batch size. If you accept a list of records, set a limit such as 1000 and reject anything larger before it reaches the model.

Keep the schema aligned with the feature list you saved. When someone adds a column to training, the schema is the thing that should fail loudly.

Write the Prediction Endpoint

A single POST /predict endpoint is enough to start. Convert the validated body into a dictionary, then into a DataFrame with columns in the exact saved order, call predict, and return a response model rather than a raw dictionary so the output shape stays stable for callers.

Return the prediction plus, where it exists, a probability. Do not echo back the full input; it clutters responses and, if the data contains personal information, it spreads that information into logs of every caller.

Handle failure honestly

Wrap the prediction call in a try/except. Log the full exception and the model version, then return a generic 500 message to the client. If the output shape is unexpected — a pipeline returning two columns where you expect one — treat that as an error, not something to squeeze into a response. And log inputs carefully: hashing or truncating identifiers keeps debugging possible without turning your log store into a copy of the customer database.

Test Locally Before You Containerize

Run uvicorn against the app and open the interactive docs page FastAPI generates at /docs. It is the fastest way to see the schema your API actually accepts. Then write a few tests with the test client so you do not need a live server in CI:

1. A known record produces the same prediction the pipeline gives you directly in Python. This catches column-order bugs, which are the most common silent failure.
2. A request missing a required field returns 422.
3. A string where a number belongs returns 422.
4. An oversized batch is rejected.
5. /health returns success after startup.

That first test is the one that earns its keep. It ties the service to the training artifact instead of to whatever the notebook happened to produce.

Containerize with Docker

Start from a slim Python base image, copy requirements.txt first, install, then copy the application code. Docker caches layers, so this ordering keeps rebuilds fast when you only change a handler. Create a non-root user, expose port 8000, and start the server with host 0.0.0.0 — binding to the default localhost makes the container unreachable from outside.

Trim what you install. Notebook tooling, plotting libraries and test frameworks have no place in the runtime image, and they inflate both the build time and the attack surface. Add a .dockerignore for .git, virtual environments and local data files. Pin versions in requirements.txt so the image that loads your pickle uses the same scikit-learn line as training.

Build, run with a port mapping, and repeat the same checks from the previous section against the container. If the local tests pass and the container fails, the problem is almost always a missing dependency or a file that never got copied.

A Short Deployment Checklist

Before you point traffic at it, confirm the operational basics. Most of them take minutes and prevent the incidents that take hours.

• Memory limit set to comfortably more than the model's size in RAM, times the number of workers.
• Image tagged with a version, not latest, so a deploy can be rolled back precisely.
• The model file either baked into the image or mounted read-only from a known path — not fetched from an untracked location at boot.
• Structured logs including model version, latency and response status.
• A retraining route: who rebuilds the artifact, how it is tested, and how the new version reaches the service.

Ship the first version with a single worker and a small instance. Watch latency and error rates for a few days, then tune. A working endpoint that returns slightly slow predictions is far more useful than a perfectly tuned service that never leaves your machine.

Photo: geralt / Pixabay