As 2017 is coming to its end, we’re already predicting what could be happening for big data in 2018; both in terms of its impact and its developments.

How will big data innovate businesses and control how consumers use their products or services? And what will the future of big data look like for workers? These ten big data experts share their thoughts.


Lillian Pierson - future of big data

Lillian Pierson is a Data Scientist that specialises in advising, training and coaching through her boutique data science consultancy Data-Mania.

Though it seems much of the western world has moved onto the next set of data buzzwords (#AI, #IoT, #DeepLearning), Lillian believes that “Big Data” still persists, and these new buzzwords are a result of the big data industry’s speciation as it advances.

“Outside the modern world, however, in places like the Middle East and Asia, modern data technologies and methods have been adopted much more slowly,” says Lillian. “Even coming into 2018, their profound impacts are just beginning to be felt.”

As a trainer to corporate executives and technical professionals in these regions of the world, Lillian describes it as immensely rewarding to introduce such powerful methods and tools to the people who can use them to create massive efficiencies for their employers.

What’s more, “I’ve begun to recognise data skills as ‘the great equalizer’,” says Lillian. “No matter a person’s nation of origin, citizenship, gender or religion, people that are highly skilled in data science and/or engineering continue to enjoy open invitations to work on major projects in the most prestigious places on Earth. What could be more motivating and inspiring!”


Gregory Piatetsky-Shapiro - Future of Big Data

Gregory Piatetsky-Shapiro, PhD is the President of KDnuggets, a leading site for Analytics, Big Data, Data Science, Data Mining, and Machine Learning.

He is a co-founder of KDD (Knowledge Discovery and Data mining conferences), a co-founder and past chair of ACM SIGKDD, a professional association for Data Mining and Data Science, and a well-known expert in the field.

Based on the amazing result of AlphaGo Zero, Gregory believes the Google DeepMind team will follow up and achieve another superhuman performance on a task “which only a few years ago, many people thought it was impossible to do by a computer.”

Gregory also predicts that we will see more self-driving car (and truck) progress, and also first-time problems (such as the Las Vegas self-driving shuttle that couldn’t move out of the way) will be solved to ensure self-driving vehicles will do as they should have in the first place.

Gregory’s final predictions will be that the AI bubble will see signs of growth in the near future, but he thinks there will be signs of “shake-out and consolidation.”

More information on the artificial ‘artificial intelligence’ bubble and the future of cybersecurity can be read here.


Kirk Borne PhD is the Principal Data Scientist at Booz Allen Hamilton and a top 10 Data Science and Big Data influencer.

Kirk’s biggest prediction is that that we will see much more focus on the ROI of Big Data and Machine Learning projects in 2018. “The marketing hype on these topics has been intense for a few years, and I believe that the data community (and its observers) have developed ‘hype fatigue’,” says Kirk.

It’s time to demonstrate value, and Kirk believes the most important “V” of big data is “Value“. “That refers to value creation and innovation across all data and information assets,” he says. “Our stakeholders will demand to see more discussion, demonstration, and proofs of value and ROI from big data in the coming year.”


Ronald Van Loon - Future of Big Data

Ronald Van Loon is the Director of Adversitement, which helps data-driven companies generate business value, and a top 10 Big Data, Data Science, IoT, & AI Influencer.

With the number of connected devices around the world set to reach 30 billion by 2020, Ronald predicts that the use of Edge Analytics will continue to climb. “Many devices are generating vast amounts of data that cannot be analysed,” he says, “and Edge Analytics is providing the solution.”

Ronald explains that its use will continue to increase in the coming years, corresponding with the increase in IoT enabled sensors that give organisations the ability to perform real-time analysis at any point where data is generated, whether it’s a network switch, sensor, or connected device.

“Edge Analytics will help companies address challenges related to centrally analysed data generated from so many connected devices,” he says.

With huge volumes of streaming data from so many connected devices, Ronald says this can pose data management difficulties for businesses, “mainly, delayed analysis as a result of overtaxed central systems, and slow or restricted network availability.”

“Edge Analytics is an effective and efficient form of analysis that can be utilised across all industries,” Ronald says, “supporting the development of definitive outcomes for businesses via faster decision-making capabilities and responsiveness.”


Lisa-Christina Winter - Future of Big Data

Lisa-Christina Winter, PhD is an expert in statistical and empirical research methods, with an emphasis on evaluation methods and experimental design, making use of various statistical software tools and technologies.

Following her strong interest in Data Science, Lisa now works at GateB as a Digital Marketing and Data Science Specialist.

“I see a bright future in the field of big data for 2018,” says Lisa. “The big data movement will not cease to surprise us with amazing new applications and results anytime soon.”

“While some years ago, analytics was a ‘nice to have’, by now they are a ‘must have’”, Lisa says. “If your company wants to keep up with the others, you won’t be able to bypass analytics anymore. That was already the case in 2017, and it will be even more so in 2018.”


Ben Rogojan - Future of Big Data

Ben Rogojan has spent his career focused on healthcare data, developing algorithms to detect fraud and reduce patient readmission, and redesigning insurance provider policy to help reduce the overall cost of healthcare. He also helps develop analytics for marketing and IT operations in order to optimize limited resources such as employees and budget.

“If you can get past the buzzwords, 2018 has the potential to bring a lot of what has occurred in the big data and data science field in 2017 and 2016 to a head,” says Ben.

In 2017, Booz Allen and Kaggle had a competition geared towards reducing the false positive rate of lung cancer by utilising CT scans. “The winners of this project are now working on actually integrating their solutions to help reduce the number of time doctors spend looking at charts,” Ben says.

With the rise of healthcare costs, Ben thinks healthcare companies will take on these projects. “With the massive amount of digital medical images available, data scientists and big data professionals have the ability to make doctors work more efficient,” he explains.

The ability to not just store massive amounts of data but to analyse it is what Ben is most excited about in 2018.

“With all the data gathered in the last few years and all the advancements made in streaming analytics, there is an amazing opportunity to start providing insight in real time,” he continues. “And not just in healthcare, but in millions of self-guided drones delivering packages, and smartwatches, for example.”


David Talby - Future of Big Data

David Talby is the Chief Technology Officer at Pacific AI, specializing in applying Big Data and Data Science in Healthcare.

With the avalanche of research showing AI outperforming human specialists in clinical diagnosis, patient risk prediction, pathway recommendation, and imaging, David thinks live monitoring and similar tasks will start translating into well-funded startups and real-world proof of concept projects.

“This will be small-scale and will hit the realities of getting patients and providers to change behaviour,” says David. “But this will provide great learning opportunities nonetheless.”

David also raises concerns that large provider and payer systems will continue to struggle with core data integration, patient 360, and classic BI and reporting projects that they need to operationalise at scale.

“As more systems have the core data warehouses in place, they will start working in the next level of challenges – common data models, building a data science platform, and integrating more data-driven systems within their transactional EMR,” he says.

David believes that many of the core ‘big data’ technologies like cloud, data lakes, machine learning – which has now matured in other verticals – are now being adopted in healthcare more quickly.

“Newer techniques like reinforcement learning and streaming data pipelines will not reach healthcare until compliance, cybersecurity, scalability, and support models are proven elsewhere,” says David.


Daniel Yarmoluk - Future of Big Data

Daniel Yarmoluk is the Director of Business Development, IoT and Analytics at ATEK Access Technologie, produces the “All Things Data Podcast” and is a contributor to VertiAI.

Daniel believes big data is a large and careful orchestration of multiple languages in a complicated environment.

“In order for us to truly deal with speed and velocity, we need to look at the process of querying big data with proper metadata management tools to extract value,” says Daniel, “meaning the orchestration tools need to simplify the process of managing complex datasets needed to speed up productionising BI and analysts tasks through process improvements.”

There are clear winners of the organisations that can manage big data (Amazon, Facebook, Google, and Apple), and Daniel says that “it would be interesting to see other leverage data into meaningful strategic assets and business models.”


EduarEduardo Siman - Future of Big Data

Eduardo Siman is a technology and future-obsessed IT leader with a passion for VR/AR, data visualization, Blockchain, Machine Learning, Big Data, Internet of Things and going to Mars. He is an Angel Investor and a member of the Board of Advisors of Virtualitics; the first platform to merge Artificial Intelligence, Big Data and Virtual/Augmented Reality. Eduardo also serves as the Co-President of the Miami Chapter of the Virtual Reality Augmented Reality Association (VRARA).

Eduardo believes the future of big data analytics will focus on 3D data generated by virtual and augmented reality experiences. “As e-commerce migrates to V-commerce, consumers will feel more and more comfortable interacting with virtual goods,” he says. “This will present a bonanza of data for brand owners and manufacturers that was never before available.”

He predicts that this torrent of 3D consumer analytics data will require big data systems that are built for graphics-intensive tasks such as computer vision. “The Hadoop and Spark clusters that dominate big data processing today will be replaced with clusters of GPU intensive servers that can process consumer insight data tied to specific vertices in a 3D mapping,” he says.

Eduardo continues to explain that if a consumer is placing an augmented reality furniture piece in their house using iOS11, all of the data related to the floors and walls of that person’s room will be used to analyse possible up-sell opportunities.

“Brand owners and retailers will want a big data and machine learning system that can recommend the particular space in the consumer’s home, where they should place the next piece of furniture,” he says.

“Although this 3d version of recommendation systems might seem unattainable given current technology, the speed of GPU acceleration and evolution of machine learning will make this type of analysis possible much faster than previously imagined.”


Marcus Borba - Future of Big Data

Marcus Borba is a global influencer in Big Data and the founder of Borba Consulting, which helps organizations solve complex challenges with data, develop strategies to plan and implement solutions using analytics, big data, machine learning and data science.

In today’s complex business environment, Marcus explains that organisations are generating more and more data, and are increasingly concerned with using that information to make better decisions. “Nowadays data is not only a major asset of companies; how you use the data have become the most important driver of a company’s success,” says Marcus.

Marcus believes ‘big data’ is no longer a buzzword, currently being replaced by AI, but despite the hype, Artificial Intelligence is real and is here to stay. “AI will be one of the major drivers of technology in the coming years,” he says.

Marcus also believes the main concerns of AI are with security, privacy and ethics, which in his opinion, should be widely discussed by society as a whole. “Machine learning has made huge progress in recent years, and its capabilities will be increasingly incorporated into many types of software and platforms, enabling professionals to take advantage of them without knowing how they work,” he says.

“The convergence of big data, cloud, machine learning and IoT are creating new opportunities for self-service analytics.”

Lastly, Marcus believes companies will increase adoption of a cloud-first strategy for big data analytics. “IoT platforms will offer more and more machine learning capabilities that allow users to analyse sensor data, look for correlations, and determine what response to take,” he says.


So what is the future of big data analytics? Let us know your predictions for big data in 2018 in the comments below.

1 thought on “Big data in 2018: the experts have their say

  1. Neha Goyal says:

    Is there any future of Scrum master, I am planning for certification course for the same from Xebia Academy Global.

Leave a Reply

Your email address will not be published. Required fields are marked *