Interview: Satyam Priyadarshy, Halliburton on Big Data Challenges in Oil & Gas Industry

We discuss Analytics at Halliburton, Big Data challenges unique to Oil & Gas industry, and the 7 V’s of Big Data.

satyam-priyadarshyDr. Satyam Priyadarshy is a Scientist, Technologist, Author, Coach, Faculty, Speaker, and is a pioneer in the fields of data science, big data, analytics, and emerging technologies. He is currently chief data scientist at Halliburton’s Land-mark PSL, leading expansion of integrated workflow capabilities and other data development initiatives.

Dr. Priyadarshy has appeared as speaker at several international conferences, and has written, co-authored, presented and published numerous research papers in peer-reviewed journals and magazines. He has held various leadership positions in AOL, Network Solutions, Acxiom Corporation prior to joining Halliburton. He was recently named to The Financial Times’ list of potential board candidates with emerging technology and analytics expertise. Dr. Priyadarshy is an adjunct faculty at Georgetown University. He is currently senior fellow at the International Cyber Security Center at George Mason University. He is advisory board member at multiple organizations including Big Data Summit, Virginia Tech’s MBA Board, etc.

Dr. Priyadarshy holds a Ph.D. from Indian Institute of Technology in Bombay, and an MBA from Virginia Tech.

Here is my interview with him:

Anmol Rajpurohit: Q1. What role does Analytics play at Halliburton? What are the typical problems that you work on?

halliburtonDr. Satyam Priyadarshy: Halliburton, founded in 1919, is one of the world’s largest providers of products and services to the energy industry. The company serves the upstream oil and gas industry throughout the lifecycle of the reservoir. The life cycle phases include locating hydrocarbons, managing geological data, drilling and formation evaluation, well construction, well completion, and optimizing production through the life of the oil field.

For each of these phases during the lifecycle of oil field, data and analytics play a significant role. However, based on which phase were talking about, some leverage a significant amount of knowledge creation through scientific and 1st principle models, augmented with data analytics, while others rely on data analytics more. The product service line that I am in is called Landmark. Landmark is a leader in providing Analytics platform for the upstream oil and gas industry that is used internally by Halliburton as well as by majority of the oil and gas companies.

AR: Q2. How has the rapid advancement of emerging technology devices (such as sensors, IOTs, etc.) impacted Analytics in Oil and Gas industry?

fast-analyticsSP: Yes, the deployment of IOTs, Sensors and M2M communication impacts the analytics in oil and gas in-dustry. For example, earlier it was fine to get the data from the remote sites and process in 24 hours or more, but as the new communication networks enable faster movement of data, the expectation for turnaround analytics based information reduces to near real-time.

AR: Q3. What are the Big Data challenges that are unique to the Oil and Gas industry?

data-silosSP: The first challenge as I see is ‘Data in Silos’. Silos by definition restrict innovation. As we know the com-panies that have successfully created significant value from their Data, is mainly due to the ‘Data democratization’. The data democratization does not mean that one has let go data governance, privacy, and security requirements. Oil and gas industry is very slow in moving towards opening their silos for analytics purposes. One cannot find new patterns in the data if limited set of data is available to you.

The second challenge as I see is the agility in adapting emerging technologies for creating value from the data that is available. For example, the world of scientific visualization and information visualization is fast moving into ‘browser/client’ with minimal round trips to servers, using new technologies, but Oil and Gas industry is still embedded with desktop version of the software, which has great degree of limitations.

The third challenge as I see is that the unlike other industry verticals, Big Data in oil and gas needs to work with multivariate, multidimensional, multi-variant, stochastic and scientific data. Again, based on which phase of oil field lifecycle we are discussing the variability and value from the data is going to be different.

AR: Q4. Big Data is typically described in terms of 3 V's - Volume, Variety, and Velocity. However, you suggest that Big Data should be characterized with 7 V's? What are the other 4 V's and why are they important? big-data

SP: Traditionally, the Big Data has been described as Velocity, Volume and Variety (3Vs). Later on Veracity was added by various organizations and that made it 4Vs. However, I feel this confuses people, as it is incomplete way to describe Big Data.

So, after few iterations, I have added three more V’s = Value, Virtual and Variable. Let me explain. In the era of data explosion, we all generate data and consume the data ourselves. However, lot of data that we generate has no value for us, as in case of smart phones, but the same data has significant value for the phone service provider, the device manufacturer, and the app creators. Hence, Big Data must have Value.

The next V, I added is Virtual, because the old thinking of moving data and putting in the silos is outdated. As long as we have the ability to connect through data pipelining the data one can leverage the data sets anywhere within the enterprise for big data analytics.

Now, the last one is Variable - it refers to the fact that for different use cases there is variability in the combination of these 6 Vs - Volume, Velocity, Variety, Veracity, Value and Virtual. If one compares the seismic, drilling phases of oil field lifecycle, the volume of data is much different in these two phases, to an extent quite of real-time analytics is needed in drilling phase compared to seismic so velocity is different in these two phases, but the value in seismic is large compared to drilling, because any predictability with seismic data leads to the beginning of the oil field.

The second part of the interview