I’m a big fan of hackathons, the innovative, collaborative and short-term programming events that use data and APIs to solve real-world problems. Those that are particularly data-focused are often called datathons, although the distinction is a bit of a moot point. But whether datathon or hackathon, they are about coming together to look at problems in a different way, using data and technology to find solutions.

I enjoy attending these events, and I also enjoy winning them! They are great fun for all those who participate. I particularly like the contrast between the beginning of the hackathon, when you are a group of individuals with lots of ideas for addressing the problem, and the end, when you are a team with a minimum viable product (MVP).

But although hackathons are great fun, are they really useful and usable in the real world, and should data scientists be encouraged to attend them? I think so. They build vital skills for today’s data scientists, who need to work as teams to solve problems, often in a short space of time and with limited resources. Does this sound familiar?

The benefits of hackathons

First of all, for attendees, hackathons are a great opportunity to try things out, whether technology or techniques. The atmosphere is all about experimentation and doing new things, and you can learn from all the other attendees about their experience with those technologies. The networking opportunities are almost worthwhile by themselves, especially at the big events.

The learning opportunities are also enormous. It is hard to overestimate the benefits of having concentrated thinking time to look at a particular problem. The time pressure also encourages quick thinking and rapid prototyping. The idea of there not being a ‘later’ to come back to something is very persuasive.

But the benefits are not just for attendees. Companies running internal hackathons, or using hackathons as a recruitment opportunity, have realised that their awesome potential. Forget team-building away days with artificial exercises, it could be much better to send your data science team off to a hackathon for the weekend. They will be using real-world skills in a way that mirrors their day-to-day work, and learning to work together under pressure.

There is another way that hackathons are proving themselves to be very useful to companies: they are a great way to show students what it is like to work in data science. The experience of one group of students in Australia has led to hackathons being introduced to several data science courses at the University of Technology Sydney. One student noted that hackathons require compromise, because of the constraints on time and data sources. This is much more like working in the real world than doing projects as a student, and helps students to be much more ‘work-ready’.

Companies sponsoring hackathons and providing data also get benefits. First and foremost, they get their ‘problem’ examined differently. Hackathons are about nothing if they are not about innovation. The whole premise behind hackathons is to look at existing problems in new ways, partly because attendees are often not familiar with the detail and ‘what we’ve always done’. The side benefit is that the sponsor company is in a very good position to look at those attending, and spot potential hires.

The future of hackathons

Hackathons already come in all shapes and sizes, from the very big, with over 1000 participants, to much smaller and more local events. Some are aimed specifically at students, and others at the data science community more generally. Some are more prestigious, attracting experienced competitors, and others are aimed at newer or less experienced participants. I think this diversity is part of the strength of hackathons: there is one for everyone, whatever their interest or experience.

I think there may be potential to expand them, however. In particular, I would like to see teams including citizen data scientists as well. This would be particularly useful for internal hackathons, run by a particular organisation for its own staff. Bringing together business expertise and data scientists to spend a concentrated amount of time on working together to solve a particular problem could really help to understand and address the problem. It will also be a very good way to build cooperative working relationships between business and data science teams for the future.

It can be a challenge to find events that are both fun for participants, and useful to their employers and/or the event sponsors. But hackathons really do tick all the boxes.