Impact of IoT on Big Data Landscape

By Kaushik Pal, (Techalpine).

What are the Impacts of IoT on Big Data Landscape?

The Internet of Things (IoT) has been a major influence on the Big Data landscape. The main idea behind the IoT revolution is that almost every object or device will be having an IP address and will be connected to each other. Now, considering the fact that millions of devices will be connected and will be generating enormous volumes of data, the efficiency of data collection mechanism is going to be challenged.

First, companies need to employ highly efficient data collection mechanisms.

Second, companies are going to face unprecedented security issues which are probably not going to be addressed with traditional security mechanisms.

Third, not all data generated by the devices will be useful. Companies need to distinguish between useful and redundant data. So, they face huge challenges to improve their data and analysis capabilities. In this context, tools like Hadoop are going to receive a lot of attention.

Last, IoT Big Data is going to change our day-to-day lives at a fundamental level.

Below we examine areas of Big Data landscape impacted by the IoT.

Collection of IoT Big Data

Companies need to collect all the data that is relevant to their business and that is a seriously challenging task because they need to filter out redundant data and also protect the data from getting attacked. This requires highly efficient mechanism that includes software and protocols.

The most common data collection tool is the sensor-fitted devices. IoT data collection also requires custom protocols. Message Queue Telemetry Transport (MQTT) and Data Distribution Service (DDS) are two of the most comprehensive protocols. Both protocols can help thousands of sensor-fitted devices connect with real-time machine-to-machine networks. MQTT collects data from multiple devices and puts the data through the IT infrastructure. On the other hand, DDS distributes data across devices.

Generally, devices collect and transmit data over the network to a central server. For example, one of the largest shipping companies in the world, UPS, uses sensors in its vehicles to improve delivery performance and cut costs. The sensors monitor miles per gallon, mileage, number of stops, speed, and engine health and capture more than 200 data points everyday for each of its vehicles. The data helps UPS to reduce fuel consumption, harmful emissions and idling time. Virgin Atlantic has been using IoT on its Boeing 787 aircrafts. According to David Bulman, the Virgin Atlantic IT Director, the data collected can help in improving flight performance and fuel efficiency and predicting maintenance requirements.

Data Security Issues

The IoT has thrown new security challenges that cannot be addressed by traditional security mechanisms. Facing IoT security issues require a paradigm shift. For example, how do you deal with a situation when the refrigerator and coffee maker at your home are fitted with hidden Wi-Fi access and spambots?

In a trendsetting move, researchers have hacked the building control system of Google’s office in Australia. According to experts, security issues have two aspects: hacking and confidentiality. For example, confidential data such as credit card details could be hacked with sophisticated methods and the owner would not even know.

Confidentiality means enterprises and even the government snooping about people’s private lives to collect information in an unauthorized manner.  IoT offers hackers and other cyber criminals a veritable gold mine of data. It has opened up new windows of accessing confidential data which were not available before. With just about everything in our lives about to get connected, data is going to flow out from everywhere.

However, data security mechanisms have not been keeping pace with these developments. So cyber criminals have an unfettered entry. Privacy and confidentiality are going to take a hit because of companies callously handling customer data and the governments in many countries collecting information about our private lives on the sly.

The images below show how confidentiality compromise is going to impact our lives in different areas.

Retail

Healthcare

Financial Services

Identifying Redundant Data

Not all of the Big Data from devices provide useful insights. At any time, companies are going to pull redundant data from the sources and it is not easy at the moment to filter out such data. A survey conducted by ParStream shows that 96% of the organizations surveyed are struggling to filter out redundant Big Data from the devices. The following points emerged from the survey:

• More than 86% of the respondents recognize that efficient data collection is beneficial for their business.
• Few of the respondents are able to collect data efficiently and they realize that.

The image below highlights the challenges organizations face with data collection efficiency.

It appears that data collection mechanisms are still at a nascent stage and so, teething problems are expected. The main reasons companies have not been able to employ better data collection methods are:

• 33% of the respondents do not track their IoT projects with quantifiable metrics.
• 29% of the respondents have documented goals for their IoT projects but do not measure the progress.

The statistics below show the various difficulties companies are facing with data collection activities. The percentage figure indicates the percentage of the respondents to the ParStream survey facing the difficulty.

• Data collection difficulties – 36%
• Data is not captured reliably – 25%
• Slowness of data capture – 19%
• Too much data to analyze correctly – 44%
• Data analyzing and processing capabilities are not mature enough – 50%
• Existing business processes are not flexible to allow efficient collection  – 24%

Impact on Daily Lives

IoT Big Data is going to redefine our lives. Our lives are not going to be the same again. To keep things simple, let us consider a few examples of just three areas of our lives: work, home and health.

At work, your keyboard or mouse could be fitted with sensors to find out how much time you spend at your desk. The coffee vending machine could measure from your ID card how much time you are spending in front of the vending machine. Obviously, your employer would want this information to find out how productive you are.

At home, smart devices could save a lot of power and money by automatically switching off electrical devices when you leave home. You do not need to remember to switch them off. The music system in your car could automatically play just your favorite songs based on the data of your preferences.

IoT will also help you handle medical emergencies efficiently and quickly. If someone at home is seriously ill, a smartwatch or even a pacemaker could identify an emergency situation by analyzing the data such as heart beat, and blood pressure real time and send an emergency notification to the server of the nearest clinic.

Summary

IoT is definitely an exciting prospect from the Big Data perspective but we need to quickly improve our whole setup to handle the impact of IoT on the Big Data landscape. There are a few areas of concern and security and privacy and data collection efficiency are probably the most pressing problems we are facing. Security compromise and inefficiencies in data collection mechanisms result in a loss of reputation, time, effort and money. But there is hope because both the IoT and the infrastructure to manage it are at a nascent stage and there will be improvements.

Bio: Kaushik Pal (www.techalpine.com)  Has 16 years of experience as a technical architect and software consultant in enterprise application and product development. He has interest in new technology and innovation area along with technical writing. His main focuses are on web architecture, web technologies, java/j2ee, Open source, big data and semantic technologies.

Related:

• Taming the Internet of Things - KNIME Case Study
• From Big Data to Big Profits: A Lesson from Google’s Nest -
• Data Science and Mobile Devices: Joined at the Hip