From Prototypes to MLOps

Productionizing ML requires three things a notebook does not give you: reproducibility, automation, and monitoring. The model is rarely the hard part. The hard part is that a notebook encodes a result, and production needs a process that can regenerate that result on demand, six months later, on a machine nobody has logged into yet.

1. Package the training code

Move the code out of cells and into modules with an entry point. Pin dependencies — exact versions, lockfile committed. Capture configuration as data (YAML, not scattered constants), because every hyperparameter that only exists in a cell is a result you cannot reproduce.

The test: can a colleague clone the repo and reproduce your metric to three decimal places? If not, nothing downstream is trustworthy.

2. Automate the pipeline

Wire the stages together so they run end to end without a human: data → train → eval → deploy. Each stage takes explicit inputs and writes explicit outputs. Each is individually runnable and individually cacheable.

The payoff is that retraining stops being a project. It becomes a command.

3. Track datasets, models, metrics, and lineage

For every model in production, you should be able to answer instantly: which data trained it, which code version produced it, what it scored, and who shipped it. Without lineage, debugging a regression means archaeology; with it, it's a diff.

4. Ship with rollbacks and canaries

Deploy behind a traffic split. Send a small percentage to the new model, compare against the incumbent on live traffic, then ramp. Keep the previous version warm and make rollback a single command — not a redeploy.

Offline evaluation tells you the model is better on your test set. Only live traffic tells you it is better for your users.

5. Monitor drift, performance, and cost

Track input distribution drift, prediction distribution drift, and — whenever labels eventually arrive — actual accuracy. Watch latency and cost per prediction alongside quality, since a model that quietly doubles inference spend is a real regression too.

Set thresholds that trigger retraining automatically, and schedule a periodic retrain regardless. Models do not decay on a convenient schedule, and the world does not file a ticket when it changes.