What analytics architecture is needed to support an accelerator? I have recently been working with my colleague Jussi Martikka to extract lessons from the City of Helsinki’s AI Experimentation Accelerator work.

We used the steps of the accelerator process to define the analytics architecture needed. This was a useful way of making sure that nothing was missed – similar to using the analytics life cycle. We defined these steps as gathering and qualifying ideas, developing a minimum viable product and putting the pilot into production. This last stage is important because otherwise the time and money put into the pilot will be wasted.

Idea gathering and qualifying

Accelerators need ideas. One way to do this is to give people access to data and analytics on a broad scale. For example, you might widely share data with all employees in a company. However, for this to work, the analytics system must be both easy to use and secure.

Why easy? Because to enable large numbers of people to play around with data and analytics, it is important that the tools can be used by both experienced programmers and people who have never written a line of code in their lives.

To enable large numbers of people to play around with data and analytics, it is important that the tools can be used by both experienced programmers and people who have never written a line of code in their lives.

Sharing data with a large group of people means that the system needs to be extremely secure. You need to be able to protect individual privacy and sensitive corporate information, and remain compliant with data protection regulations – but still allow data to be shared.

The qualification stage involves going through the ideas to choose which ones should enter the accelerator. There is a balance between complexity, feasibility and value generated. At first glance, this step might not seem to have any requirements for an analytics platform. However, evaluation of complexity will depend on the capabilities and efficiency of the analytics platform. Different use cases also require different amounts of data and complexity in calculation. Scalability is therefore essential.

Developing a minimum viable product

This phase involves creating and building the minimum viable product for the use case, with the production version in mind. The minimum viable product may be production-ready, or it may be necessary to create a blueprint for how the product will be put into production.

During this stage, there are two main requirements for the analytics platform: efficiency and scalability, and a full-stack platform. This means that the analytics platform should allow and support data ingestion/transformation, visualization and advanced analytics/machine learning – as well as putting the model into production. This makes development easier and puts the focus on developing use cases instead of integrating different components for different purposes.

Putting the results into production

The final step is putting the results from the accelerator into production as a completed product. Having the same architecture for both development and production makes this process seamless. This is true even if the two stages take place in separate environments. At this stage, there are four main requirements: scalability, security, governance and availability.

Scalability is important on the production side, as well as the initial stages. When you are creating customer-facing applications, it is important that they work with any number of customers. Security is also important: being able to protect privacy is essential. This is an important reason for good governance, but you also need to be sure that you can monitor models, and retrain them if necessary. Finally, the service needs to be up and running when customers expect it. With customer-facing applications, this usually means that the platform needs high availability and disaster recovery capabilities.

Bringing it all together

There are therefore some common themes in the requirements for all stages of the platform. These give you the most important requirements for the analytics architecture. They are:

• Scalability. The system must scale to facilitate any requirements for data volume and speed.
• Security. It must deliver data privacy so that people can only see and access the data they should.
• Efficiency of use. The efficiency of use enabled by the platform will affect the time to market and the number of models or services that you can put into production.
• Governance. Control mechanisms must be in place to ensure that you can trust the models and data and reduce the risk of machine learning services malfunctioning.
• Availability. Analytics capacity should be available where and when you need it.
• Ease of use. The platform should be usable by people with different skill levels, including programmers, data analysts and managers.

All these requirements need to be met – but you also need to look at the context of the individual organisation or accelerator and make sure that your solution fits that. One size very definitely does not fit all.