Interview: Hobson Lane, SHARP Labs on How Analytics can Show You “All the Light You Cannot See”

We discuss the impact of rapid growth in magnitude of data, programming skills for data science, major trends, advice, data science skills, and more.

hoson-laneHobson Lane is Principal Data Scientist at Sharp Labs of America. He has designed, simulated, patented, and built terrestrial and space robotic systems often stretching his optical, thermal, mechanical, chemical, and electrical engineering knowledge. He has chased his unreliable software across a field and into a street sign (1st and 2nd DARPA autonomous vehicle Grand Challenge competitor). He has also renovated and skippered a fiberglass sailboat halfway around the world with his wife. He can't wait to see what automation technology will make possible next year.

First part of interview

Here is second part of my interview with him:

Anmol Rajpurohit: Q6. How has the rapid growth in data impacted Predictive Analytics? What are the most underrated challenges of working with Big Data?

Hobson Lane:

The recent acceleration in data generation and openness has overwhelmed the skill base required for managing that data. The tedious, thankless work of Data Engineering is consistently sidelined and shortchanged in favor of crowd-pleasing visualization and analysis projects. Data validation is time consuming and undervalued, often leaving managers to make decisions based on incomplete or inaccurate data.

AR: Q7. How important are programming skills for a data scientist?

HL: That depends. Most data science problems can be solved by a data or software engineer (not scientist) using off-the-shelf libraries and services combined with an undergraduate understanding of statistics. Machine learning technology is proliferating so quickly, and data is being generated at such a rapid pace, there is rarely a project where available data has been fully mined with existing tools. Rarely is new software or deep experience required.

But when a company does need a data scientist, they'll want her to have software development skill and an understanding of algorithms so she can customize and test new algorithms to solve their difficult problems. A strong roster of these "programming scientists" makes sense in highly competitive industries where predictive analytics is your core business, like Finance, Information Security, Banking, Web Analytics, and Web Search.

AR: Q8. What key trends will drive the growth of Predictive Analytics for the next 2-3 years? deep-learning

HL: Fully autonomous deep learning services will rapidly catch up with the flood of data. These services will improve business efficiency, trim waste, supplant industry incumbents, reduce payrolls, and increase job satisfaction for those that survive the "neck down". Businesses that build or employ these services will thrive.

AR: Q9. What is the best advice you have got in your career? ideas-and-execution

HL: A brilliant idea is worth little compared to diligent execution. Many will have the same excellent idea. Few will have the humility to share it with others or work diligently to build and employ it. Dreaming is easy. Building something valuable to others is hard work.

AR: Q10. When and how did you get motivated to work in Analytics?

HL: Since I was a child I dreamt of machines that could make decisions better than I and help me see "all the light we cannot see." I've not been disappointed or bored by machines yet.

AR: Q11. What key qualities do you look for when interviewing for Data Science related positions on your team?

HL: Humility. If you're unwilling to have your theories disproven, you will have a hard time letting the data and your teammates speak the truth to you. Almost all theories are eventually disproved or refined. Humility helps you advance down that path more rapidly. all-the-light

AR: Q12. What was the last book that you read and liked? What do you like to do when you are not working?

HL: All the Light We Cannot See by Anthony Doerr.

I like to rock climb, tinker with toy robots in the basement, and teach science.