The UK is suffering from a data scientist skills gap, and despite the government announcing the National Data Strategy to help alleviate this, we need innovation right now.

Almost half of businesses (48%) are recruiting for roles that require hard data skills but just under half (46%) have struggled to recruit for these roles over the last two years, according to the recent policy paper Quantifying the UK Data Skills Gap published by the Department for Digital, Culture, Media & Sport last year.

Data scientists are now in the bizarre position of being highly sought after for their skill-sets, highly paid and yet wasting the majority of their working lives on the grunt work – inefficiently searching for and preparing useful data rather than carrying out sophisticated analysis.

This is where we are stepping in to help modern organisations accelerate insight and increase productivity.

Consumable data

Analysts and data scientists are continually looking for valuable data that can be combined with their own data to improve outcomes. More than 1,500 official bodies publish vast amounts of useful, GDPR-compliant data but it is difficult to know what data is available. Sometimes it is hard to find, and almost impossible to use.

Many data companies are good at analysing or visualising data but not at engineering it. Data scientists have been spending many hours researching, scraping public data scattered across the internet, and then repeating the process several times yearly to refresh it to get the latest information. Data is often used in ad-hoc projects, and after putting in a lot of work and effort, the same scientists have to re-learn all those skills and re-find the data many months later.

Never mind “big data”, which is not a phrase we often hear these days. The current thinking is about consumable, focused data. From information available, the key to rapid results is zooming in on data which implies or can be used to derive what you need to know. 

We have met this challenge by offering cloud-based data feeds that data scientists can tap into on demand. Our data is refreshed in alignment with the official publishers, and can be easily interrogated online or ingested and combined internally. Organisations do not have to source and prepare the data themselves. Instead, they can immediately capitalise on its value.

By making data consumable, we want to create a world where companies of all sizes can innovate and develop their own IP at pace.

We are experiencing a big appetite for our unique data solution. PricewaterhouseCoopers’ UK Deals Insights and Analytics team recently selected Doorda to augment their analytics capabilities.

Data analytics are at the heart of the PwC business, and the Deals Insights and Analytics team is increasingly incorporating external data sources into its analysis on behalf of its private equity, hedge fund and corporate clients. The team has now decided to incorporate data curated by Doorda to increase productivity, answer new business questions and accelerate time to insight.

We are delighted to be supporting the data and analytics experts in the Deals Insights and Analytics Team at PwC by giving them immediate access to the wealth of Open Data and unique features within Doorda Data Products to help improve outcomes and shorten time to insight.

The future: refinement

The next step in the evolution of data access and analysis will be greater data refinement – drilling down further to offer ever-deeper consumable and reliable data within a joined-up data landscape. If the government succeeds in its drive to plug the data scientist skills gap, this, combined with newly available cloud-based data engineering services, could significantly increase productivity across multiple UK industries and rapidly boost Britain’s economy.