Why we need more experimentation in innovation and growth policy

Why we need more experimentation in innovation and growth policy

Lou-Davina Stouffs, programme manager at Nesta, and Teo Firpo, researcher at the charity’s Innovation Growth Lab (IGL), report from the innovation and growth policy global conference.

Despite the developments in technology and business models that we have witnessed over the past decades, our support programmes and policies for innovation, entrepreneurship and growth have changed surprisingly little. And, more importantly, we don’t really know whether these programmes and policies are really working. 

To discuss this, over 200 senior policymakers, practitioners and researchers working in innovation, entrepreneurship and growth policy gathered in late May at the IGL Global Conference, ‘Making Innovation & Growth Policy Work’. The audience represented more than 25 different countries, each with their own set of challenges in innovation and growth support. But all of them agreed on one thing: there was more room for experimentation in this field. 

One of the first sessions brought together leaders from fields where experimentation and use of evidence has become an integral part of the culture, such as education and development. Sir Kevan Collins, CEO at the Education Endowment Foundation, reminded us that policies and programmes cannot be made based on whims and fancies – they should be made building on what we know. 

Testing new initiatives before roll-out 

Often governments choose to introduce large-scale programmes and policies working on the assumption that these will achieve exactly what they set out to do. However, examples from other fields have shown that in some cases, we may even be achieving the opposite effect. Instead, setting up pilot programmes to test new initiatives at a small scale allows for amendments to the programme when they do not seem to achieve the desired impact, before wider scaling up. 

In order for this approach to work, however, the right systems need to be put in place to measure the impact of each new initiative, so the effective ones can be separated from the ones that don’t work. This point was reinforced by Iqbal Dhaliwal, deputy director at MIT’s ‘The Abdul Latif Jameel Poverty Action Lab’, who argued that the best way to know whether a programme is effective is to have a counterfactual – which is most effectively achieved by using randomised controlled trials (RCTs). 

Both Collins and Dhaliwal emphasised the importance of disseminating evidence of what works – but also what does not work. This way, we can stop funding going to programmes and policies that do not work, or are even harmful, and focus government spending on those that are useful. 

The conference also offered a great opportunity to learn about how such an experimental approach might be put into practice, learning from work in other fields. 

Lucy Kimbell, director of the Innovation Insights Hub at University of the Arts London, argued that design-thinking and user-centred approaches can be used to shape programmes that are fit for purpose from the start. This is already being done in the UK by the Policy Lab, a Cabinet Office initiative that brings together designers and data scientists. 

Using RCTs to evaluate policies 

As for rigorous evaluations, the event also confirmed that the UK is already at the forefront of using RCTs to evaluate policies. In the field of innovation, entrepreneurship and growth, these advances have led to some of the first RCTs in the field, such as the one currently ongoing to test the Growth Vouchers programme, presented by Mark Sayers, deputy director for Business Support Delivery at the Department for Business, Innovation and Skills. Nevertheless, there is a long way to go to catch up with other fields. 

Throughout the conference, it became quite clear that most attendees saw benefit in making innovation and growth policy more experimental. However, most reported institutional status quo as an obstacle. In this respect, we can learn from other fields – like education and development – where organisations that fund research and translate findings to make them accessible for policymakers have led the way. 

IGL’s aim is to be the organisation that ensures this is the case for innovation and growth policy. The conference was a first step in achieving this. We plan to make the event annual, so that we can continue to build a strong community of researchers and policymakers committed to making innovation and growth policy more experimental and evidence-based.

Tell us what you think – have your say below or email [email protected]