The distinction between specialists and generalists has been around for a long time, though not necessarily referring to people.

For example, did you know that there is a species of birds that only lives in the Marlborough Sounds in New Zealand? The king shag is only known to breed at four sites, all rocky and coastal. They are one of the ultimate specialists. Giant pandas are another, feeding only on bamboo.

Urban foxes, however, are serious generalists. They feed on a wide variety of food, including small mammals, frogs, worms, and food from human dustbins, and are found across a wide geographical area. Raccoons are similarly flexible.

In evolutionary terms, both approaches have advantages. Specialists often face less competition for their particular niche, or place in the ecosystem. However, when things change, and their niche disappears, they are in trouble. Generalists may face more competition, but they are also more likely to survive when times change.

Generalists and specialists in organisations

In organisations, there is a similar distinction. Generalists are people with a wide range of skills and knowledge, all relatively shallow (unless of course, you find that rare breed of individual – someone who knows a lot, about a lot of things). They can turn their hand to a lot of tasks, and know a little about a lot of things, but do not have deep expertise in any particular subject. Specialists, by contrast, have deep expertise, but in a defined area.

Research also suggests that specialists and generalists in organisations thrive under different conditions. Generalists are more productive in slower-moving fields, where there are more opportunities to apply knowledge from elsewhere – as well as being adept at juggling multiple tasks or projects. When the pace of change speeds up, specialists can become more useful because their deep understanding of a particular area allows them to rapidly develop new solutions.

The distinction is becoming more important

In my world of solution engineering and technical account management, we have traditionally needed people who knew something about a lot of things – generalists, in other words. As AI applications become more and more prevalent, however, this is changing. The pendulum is not, though, simply swinging to specialists. Instead, we are seeing a new breed of specialist emerging, who combine deep knowledge of a particular area, such as data mining or statistics, with broader knowledge about the business in which they operate.

The best way to make you extremely valuable in a team is to understand everything, but be a master of something.

Why is this happening? For AI to affect business outcomes, it needs some inputs (data), something to happen to the data (some kind of analysis and exploration) and then some outputs (something which can be acted on and deployed). Data scientists have traditionally been involved in the middle part of that: the data discovery element. It is, however, no longer enough for them simply to be able to manipulate and analyse data. They now need to understand more about both the context, or the business problem that they are trying to solve, and the consequence, the value that may emerge as a result of their work.

In other words, they still need to have the same specialist data science knowledge, but they also need to understand the business. They need the communication skills to translate their own specialty for others. Similarly, business users – the generalists in this situation – need to understand a little more about the mechanics of data science. In other words, they need to become more specialist, and deepen their skills and knowledge in particular areas.

The silver lining

The bad news is that both these situations are becoming even more of a challenge with the rate of change in the data science field. New applications are emerging rapidly, particularly with AI introducing techniques like convolutional and recurrent neural networks, and finding people who understand what they are and can articulate their applications is difficult.

The good news, however, is that paradoxically, this should make it easier to recruit and retain good people. There is a huge skills shortage in data science. Data scientists are at a premium, and can pick and choose their employer. Let them get bored, and they will jump ship to a more exciting climate.

Fortunately, the variety and pace of projects, and the increasing range of problems that need data science skills, are likely to keep teams engaged and interested. The opportunity to develop new skills and knowledge, for business teams and data scientists alike, means that staff are less likely to leave. As they feed off each other and develop professionally, the depth and breadth of knowledge and skills increases in both groups, and so the organisation also gains.

AI is expected to change the world of work. Experience suggests that it is already starting to do so by breaking down the distinction between generalists and specialists. We all, it seems, need both skills now: wide knowledge, but mastery of a particular area.

From the present to the future, AI is changing our world for better. Find out more from these expert insights.