Should data scientists buy into the DevOps philosophy?


DevOps is enjoying a lot of attention at the moment, mostly because of its promise to increase software quality at lower cost. DevOps is a hybrid term linking development and operations. It's a philosophy that focuses on improving efficiencies in how we build and deliver software, solutions and the supporting environments.

DevOps is about collaboration across business units

The term describes an agile relationship between business units managing development and IT operations. DevOps aims to improve the relationship by supporting better communication and collaboration between the two. The approach is now being used by a number of organisations to help them save money and deliver better quality software more quickly.

While software companies bought into this approach some time ago, it's now becoming more widely applicable as more companies start to build and develop systems to generate and analyse data. As more organisations modernise to a data-driven approach, they're transforming into software companies, and that means they need to care about their development processes.

A niche activity

The current approach to analytics is fairly fragmented in most companies. Many have invested in high-quality data scientists and are willing, in principle, to use data and analytics to achieve insights. They want to be data-driven.

But they also lack a coherent approach to developing and testing systems and models. They have no guarantee that a new model will always be available to replace an outdated one, or that systems become part of the standard infrastructure rather than just used for an individual project. The result is that analytics remains a niche activity, and also something of a one-off to address a particular problem. The sense that it's a core part of the business is not yet present.

DevOps may be the answer for a broad range of companies. For companies that do not wish to see themselves as software developers, it may be worth thinking of this as a kind of AnalyticsOps approach. This could bring several benefits.

Adopting an AnalyticsOps approach

The key in DevOps has been to reduce the gap between developers and users. The developers provide the software, and the business users provide the feedback, creating a continuous loop of feedback–new developments–feedback.

Similarly, an AnalyticsOps approach could bring together individuals from different disciplines and business units to work towards bridging the gap between experiments and market-ready products. Data scientists, together with business users and operational teams, could work together to bring the results of data science to the business. The result? The business really does become data-driven, achieving higher-quality, more relevant insights.

In turn, the main benefits of this approach are likely to be faster time to market and reduced cost of delivery.  Improved communication across units and better, more structured feedback can lead to issues being caught earlier in the development lifecycle, reducing the cost associated with reworking and delivering updates sooner. At the same time, it will be possible to achieve better quality products, because of improved insights into what customers want and need.

Getting started with DevOps

Most organisations don't immediately jump in at the deep end and try to implement a DevOps approach in one go. Instead, they start by dipping their toes in the water, focusing on one or two areas of priority.

To do so, companies need to adopt technologies and techniques that support specific areas. For example, a recent study by Puppet found that the top two focus areas for companies new to DevOps were deployment automation and infrastructure automation. Automation of consistent, standardised development environments and the supporting infrastructure is a great place to start.

What these things have in common is a focus on the quality of the development process, and ultimately the products. Automation of those processes, via a structured platform, is key to providing consistency. Validation ensures that regular testing, and therefore correction, is built into development processes, and should be happening on an ongoing basis. Monitoring of the entire platform ensures operations teams have the necessary visibility to support the development processes.

Making it part of the company’s DNA

These three aspects together formalise and standardise the development process. In other words, it becomes part of the company’s DNA, and helps to make it more of a consistent work methodology, rather than a one-off initiative. For more help with this, download the white paper: Getting the Right People on the Big Data Bus.


About Author

Greg Willis

Business Solutions Manager

Greg works with global organisations across all industries, from finance to communications and retail to utilities, to help increase value from their analytical environments by mapping business requirements to the correct skills, technology, resources, data and processes. Greg acts as a trusted advisor when working with organisations at all levels, bridging the gap between senior management, IT and business lines in order to define solutions. His responsibilities cover the entire SAS platform, with a focus on Enterprise Architecture, High Performance Analytics, Big Data, Hadoop, Event Stream Processing, Cloud and Internet of Things.

Comments are closed.

Back to Top