“ML practitioner” is used to describe all these different roles throughout the ML lifecycle.

Now, let’s look at the challenges these ML practitioners face when they operationalize and make their models available for production.

These challenges include managing and keeping track of complex details, such as data, model architectures, hyperparameters, and experiments.

As for specific pain points, we hear it can be challenging to:

Keep track of different versions of the models and their codes, different training procedure parameters, hyperparameter values in each trial, and performance metrics.

Control the experiment space to advance.

In every iteration, these practitioners need to monitor what changes are being made, which ideas are being tried, which ideas work, and which ideas don’t.

Pinpoint the best performing model when the models are benchmarked against each other.

The best model here refers to the one that delivers the ideal result for your specific use case.

Collaborate with

• data scientists,

• data engineers,

• ML engineers,

• application developers,

• site reliability engineers,

• business analysts, and

• business users

in operationalizing the ML models.

And then there is the matter of reproducibility.

Deploying a model to a production environment is difficult unless it can be reproduced.

In fact, bypassing the reproducibility of the model is often discouraged or disallowed by policies or regulations.

Reproducibility can be a major concern for ML practitioners, because they want to be able to rerun the best model with a more comprehensive parameter sweep.

When a team successfully trains and makes a model ready for production in a streamlined fashion, performance and agility are considerably improved.

Even if there’s a manual review step in the pipeline, automation ensures that each job is configured and executed in a repeatable manner, which reduces the risk of errors.

Also, for a production application, the model needs to be updated regularly as new data comes in.

Therefore, traceability becomes paramount.