Answering the Data Scientist Shortage with Intelligent Process Automation

October 8, 2019 / Business, Data Science, Intelligent Process Automation

Easing the complexity of deep learning so that non-technical folks can use it too


Companies are adopting artificial intelligence technology at a furious rate, consequently driving up demand and salaries for skilled data scientists who understand AI and intelligent automation technology. It’s a dynamic that has the industry crying out for simplified AI solutions that don’t require specialized expertise to implement.

AI-related job postings on the jobs site Indeed increased 29.1% from May 2018 to May 2019. That jump follows two years of even larger increases: 57.9% from May 2017 to 2018 and 136.2% between May 2016 and 2017, according to VentureBeat.

At the same time, searches for AI-related jobs appear to be leveling off, with the number of searches decreasing by 14.5% from May 2018 to 2019 as compared to the year before. That suggests there could be more open jobs than qualified workers to fill them, Indeed said.

In fact, demand significantly outstrips supply and will continue to do so in the medium term, according to “The State of AI: Divergence 2019,” a report by the venture capital firm MMC Ventures (PDF). Citing Indeed data, the report says 2.3 roles are available for every suitable AI candidate.

That, of course, drives up salaries for data scientists who have the skills to implement AI applications including cognitive, robotic and intelligent process automation. Almost all data scientists reported an increase in pay in the past three years, while nearly half (45%) saw their salary grow by 20% or more, according to the MMC report (citing data from Kaggle). At the 20 highest-paying companies – from Uber to Veritas Technologies –  salaries for AI engineers averaged $224,000, the report said, citing Paysa.

Buy vs. build artificial intelligence 

With experienced data scientists being both expensive and in short supply, AI vendor companies are trying to fill the void by building AI solutions that are more or less off-the-shelf, ready to deploy.

Increasingly, that’s what buyers want. Nearly half of companies in a 2018 Gartner survey said they prefer buying AI solutions from third parties while only a third intend to build their own custom solutions, the MMC report says. Only one in 10 companies are content to wait for AI to be embedded in their favorite software products.

In Europe alone 1,600 startups offer AI-focused products, each targeting a particular industry or business function, MMC says. 

No data scientists, no problem

The idea is to build in AI intelligence, but put it behind the curtain, so to speak, so users get all the AI benefit without having to actually understand or deal with the technology.

For example, a white paper from the Everest Group says an intelligent process automation (IPA) solution should include technologies such as: 

  • Machine learning (ML) and deep learning models to classify and extract information from documents. 
  • Optical character recognition (OCR), to convert document images into machine-coded text, using ML and deep learning algorithms to train for increased accuracy.
  • Natural Language Processing (NLP), to analyze text in documents, understand surrounding context, consolidate extracted data, and map the extracted fields to a defined taxonomy. NLP also enables IPA solutions to recognize sentiment in text, such as in emails and other unstructured data, and classify them into different categories.

That’s exactly the approach we take at Indico with with our IPA solutions. Our platform is simple enough to be used by the business people who are on the front lines, the folks who best understand the processes that need to be automated. And it’s smart enough to be able to deal with unstructured data such as emails, PDFs, images and the like – thanks to the behind-the-scenes use of OCR, ML and NLP.

While data scientists are certainly welcome to use the Indico platform and will likewise benefit from its ease of use, business folks can build working models by themselves in about an hour. We recommend getting in touch with our team here to identify potential use cases.

Don't Miss a Post

Get our best content on Intelligent Process Automation sent to your inbox weekly.