AI was very popular 30 years ago, then disappeared, and is now making a big come back because of new robotic technologies: driver-less cars, automated diagnostic, IoT (including vacuum cleaning and other household robots), automated companies with zero employee, soldier robots, and much more.
Will AI replace data scientists? I think so, though data scientists will be initially replaced by “low intelligence” yet extremely stable and robust systems. There has been a lot of discussions about the automated statistician. I am myself developing data science techniques such as Jackknife regression that are simple, robust, suitable for black-box, machine-to-machine communications or other automated use, and easy to understand and pilot by the layman, just like a Google driver-less car can be “driven” by an 8 years old kid.
My approach to automating data science and data cleaning / EDA (exploratory data analysis) is not really AI: it’s just a starting point, but not a permanent solution. In the long term, it is possible that AI will handle complex regression models, far more complex than my Jackknife regression: after all, all the steps of linear or logistics regression modeling, currently handled by human beings spending several days or weeks on the problem, involve extremely repetitive, boring, predictable tasks, and thus it is a good candidate for an AI implementation entirely managed by robots.
As machine learning (ML) more and more involves AI, and the blending of ML and AI is referred to as deep learning, I can see data science evolving to deep data science (DDS) or automated data science (ADS), where AI, robots, or automation at large, take a more prominent role.
True AI systems can even predict travel time in real time based on expected traffic bottlenecks and road closures
Which jobs are threatened by AI?
Just like data science will take years to get a high level automation, where as much as 50% of human tasks are replaced by robots, I believe that these professions are at risk, but the erosion will be modest and slow, taking a lot of time to materialize:
AI and automation has already replaced many data science tasks long ago
Many people talk about the threat of AI, but as of today, many jobs have already been automated, some more than 30 years ago. For instance, during my PhD years, a lot of data transited through tapes between big computer systems, and involved trips to the computer center, interacting with a number of people taking care of the data flow. This has entirely disappeared.
We used to have shared secretaries to write research papers (they could write LaTeX documents), I think this has all but disappeared.
One of the applications that I developed in the eighties was a remote sensing software that could perform image segmentation and clustering, for instance to compute the proportion of various crops in a specific area based on satellite images, without human interactions – thus eliminating all the expensive jobs that were previously performed by humans to accomplish this task.
Those who automate data science are still data scientists. Just like those developing robots to automate brain surgery work in a team, with many members being brain surgeons. it’s just shifting the nature of the job rather than eliminating it.