From Insight-as-a-Service to Insightful Applications

To achieve their objectives, insightful applications combine big data management with artificial intelligence concepts and systems. In particular, insightful applications include:

1. Flexible data management systems that store the big data on which they operate and learn from.
2. Rich knowledge representation capabilities in order to encode domain knowledge as well as learned knowledge.
3. Reasoning capabilities through which they reason over the represented knowledge.
4. Machine learning, including deep learning, systems to automatically extract patterns and relations.
5. Planning systems that can synthesize sets of related actions to generate an outcome that is associated with an insight.

The complexity and cost of developing insightful applications has been decreasing significantly and this trend is expected to continue for the foreseeable future. This is because artificial intelligence systems having become more readily available and better understood, particularly with the release of open source packages from Google, Facebook, Microsoft, and other companies. Additionally, cheaper storage, abundant and cheap processing power and networking bandwidth, and cloud-enabled separation of storage and computing have helped drive the development of insightful applications.

Despite our improving understanding of insight generation, the technology advances we have made, and the growing number of insightful applications currently under development, we are not completely out of the woods. There are four major areas in particular where we need to make progress:

1. Domain knowledge acquisition, representation, enhancement, and maintenance—the so-called “ontology development”—remains a big issue, particularly from data sources such as image, video, and also spoken language. For example, IBM is facing this issue as it continues to develop the Watson Oncology Assistant. It has tried to address it in a variety of ways, including the acquisition of other companies. Corporations such as Google, Apple, Facebook, Amazon, Microsoft, and others are aggressively acquiring startups with the right know-how and IP in this area.
2. Sensor technology (e.g., size, energy usage, local processing, amount and type of sensing each sensor can accomplish) is important for data acquisition in many of areas, such as autonomous driving and health care. For example, see the recent acquisition of Cruise Technologies by GM, which was driven by the development of lower cost sensors and corresponding algorithms that enable the conversion of existing cars into self-driving vehicles.
3. Extracting valuable relations and associated actions in hyper-dimensional domains (e.g., areas where each event may be characterized by millions of features, such as cancer or the many situations an autonomous vehicle has to address) remains difficult. Deep learningapproaches may prove particularly useful in this area, but we are still in the very early stages of applying these approaches to large, real-world problems.
4. Self-learning systems that are able to automatically improve their performance based on previously established KPIs without relying on data scientists are still in their infancy.

For the reasons we have previously described, while insightful applications present an exciting investment opportunity, they are complex and difficult to develop. As a result, we don't foresee the development of such applications leading to the complete elimination of connectors and data scientists.

In the second post of this series, we noted that over the past few years, venture investors have been investing in three types of big data applications: shallow applications that use general-purpose analytic tools, applications that process big data but that do not use predictive or prescriptive analytics, and applications that use embedded predictive analytics. More recently, as they have recognized the importance and economic value of successful insightful applications, a few venture investors are starting to invest in startups that develop insightful applications. Because I have come to recognize how critical it will be for corporations to utilize big data through insightful applications in their effort to innovate, particularly using my startup-driven innovationmethodology, I am focusing my new venture fund on startups that develop such applications.

We anticipate that insightful applications will be developed over a few different generations with the ultimate goal of the application completing 70% of the process and humans—including data scientists, connectors, and business users—completing the remaining 30%. Today's first generation insightful applications are able to assist connectors and data scientists. The next generation’s applications will be better able to understand situations automatically. IBM, for example, is in the process of equipping Watson with sophisticated natural language processing technologies that can automatically understand and encode domain knowledge from a variety of sources, such as journal articles as well as spoken problem descriptions. The generation after that will be able to make decisions more autonomously, matching libraries of insights and action plans to descriptions of new problems. The final generation of insightful applications will be able to discover new insights and action plans on their own, with limited guidance from expert users.

Insightful applications are the key to effectively providing big data-driven solutions to many important problems while simultaneously controlling the costs of such solutions and dealing with the shortage of the necessary specialized personnel. Because of their complexity, the development of such applications will be neither simple nor quick. Patience will be necessary, as we anticipate that the promise of insightful applications will be realized in several generations of increasingly sophisticated and increasingly automated applications. Recognizing the opportunity afforded by such applications, a few corporations and venture investors have started aggressively investing in their development—the initial results are already impressive and fill us with excitement about what will be possible in the near future.

Bio: Evangelos Simoudis is a seasoned venture investor and senior advisor to global corporations. His investing career started 15 years ago at Apax Partners and continued with Trident Capital. Recently Evangelos co-founded Synapse Partners, a venture capital and corporate advisory firm, where he is a managing director and invests in early-stage companies developing big data applications for the enterprise.

Original. Reposted with permission.

Related:

• Amazon Top 20 Books in AI & Machine Learning
• What Is Machine Intelligence Vs. Machine Learning Vs. Deep Learning Vs. Artificial Intelligence (AI)?
• Top 10 Deep Learning Projects on Github