Finding your way out of the pipeline jungle

"For those not knee-deep in the ModelOps process, the process may seem simple," says Ankit Sinha, Director of Product Management at Experian: "You build the model, deploy the model and reap the benefits."

But the process starts to become very complex when you're using multiple database systems and data sources to build the model. Complexity also comes with missing data sets, hosting, converting, testing and validating the model.

In fact, Sinha says, a recent Experian survey revealed that it takes 15 months on average to build and deploy a new model – and 50 percent of models never make it into production. Additional challenges revealed in the survey include:

50% of data sources are not model ready.
64% of businesses run into problems with the model ops process.
25% of respondents only check for model drift once a month.

Navigating the pipeline jungle

Speaking at SAS Innovate in Orlando, Sinha puts a name to this complex modeling scenario, calling it "the pipeline jungle." The jungle includes the integration of multiple data sources and systems, building and recording new models, and moving models into production.

"How do we solve for this pipeline jungle?" asks Sinha. "We need a platform that is a highly scalable, one-stop shop for unlocking the powers of advanced analytics." This platform includes:

Feature and model development – including persistent pinning and linking across data sources, read-to-use code templates, and easily integrated into any technology stack.
Feature and model operations – including containerized deployments, streamlined testing and proactive monitoring of models.
Production runtime services – including data orchestration, real-time capabilities and model inventory management.

Containing the pipeline jungle

If it takes 15 months for a model to move from development to deployment, ten of those months are spent on deployment, says Sinha. To speed up this process, Experian developed Ascend Technology Platform™ using SAS^® Viya^® and SAS^® Container Runtime, with a goal to give smaller and mid-market lenders access to the most advanced analytics tools.

"We created a pathway where a model – regardless of how it was originally coded – has a pathway to be containerized and put into production," says Sinha. "If you invest in SAS Viya, you create freeways or interstates so you can run that model quickly."

According to Sinha, containerized deployment reduces deployment times by at least 1-2 weeks. Other benefits include:

Portability: Code once, deploy anywhere.
Efficiency: No recoding required.
Agility: The model can update itself, and any rollbacks or updates do not impact the production system.
Flexibility: Integrates with all modeling algorithms and is compatible with all compute instances.
Governance: Containers prevent information loss and ensure consistency in algorithms.

Today, the system is working to deploy and adjust models quickly even during disruptions. For example, Sinha says, if the minimum wage changes or if inflation outpaces wage growth, detecting potential drifts in models as a result of these outside factors happens quickly.

Regardless of the situation, Sinha is confident that the system monitors performance shifts and model drift. He says consistency and robustness are two of the biggest benefits of the system.

Blogs