Interview: George Corugedo, CTO, RedPoint on YARN and Customer Analytics

We discuss significance of YARN for Hadoop 2.0 platform, unique benefits of RedPoint Convergent Marketing Platform and Master Key Management for Customer Analytics.

George CorugedoA mathematician and seasoned technology executive, George Corugedo has over 20 years of business and technical expertise. As co-founder and CTO of RedPoint Global, George is responsible for leading the development of the RedPoint Convergent Marketing Platform™. A former math professor, George left academia to co-found Accenture’s Customer Insight Practice, which specialized in strategic data utilization, analytics and customer strategy. Previous positions included director of client delivery at ClarityBlue, Inc., a provider of hosted customer intelligence solutions to enterprise commercial entities, and COO/CIO of Riscuity, a receivables management company specializing in the utilization of analytics to drive collections. George holds a BS in geology and a BA in mathematics from the University of Miami, and an MS in applied mathematics from the University of Arizona.

George recently delivered a talk at Big Data Innovation Summit 2014 held in Santa Clara on “Gain Traction for Your Big Data Initiatives: Harness the Power of Hadoop Quickly and Cost-Effectively”.

Here is my interview with him:

Anmol Rajpurohit: Q1. What does the emergence of YARN for the Hadoop 2.0 platform mean to the Big Data industry? How does it impact the levers of competitive advantage?

George Corugedo: YARN’s impact on the Big Data industry cannot be overstated.
Simply, if YARN was not developed, Big Data and associated technologies would be at risk of never getting into the mainstream of business technologies.
Code-based solutions, such as those built on MapReduce, are impossible for enterprise IT organizations to adopt and manage on a large scale. In addition, IT organizations want consistency in their solutions and code-based solutions are notorious for being inconsistent.

Already we are seeing Big Data projects failing due to unrealistic expectations foisted on immature technologies. What “pure YARN” enables is the porting of mature, enterprise-class technologies directly onto the Hadoop platform, allowing them to work directly at the HDFS level within Hadoop. Without YARN, technologies would be constrained to using or generating MapReduce code with all of its associated limitations and skill requirements. On the other hand, “pure YARN” applications will enable enterprises to leverage their existing ETL/DBA/Analyst resources with minimal retraining thus enabling easier enterprise-wide adoption.

“Pure YARN” applications, in contrast to “MapReduce/YARN” applications, push their functional binaries directly into YARN to manage, distribute and process directly on HDFS. Only “pure YARN” applications will enable direct porting of mature technologies onto the Hadoop computation platform. “MapReduce/YARN” applications still utilize MapReduce to define a process and are subject to the limitations of MapReduce. Users should ask vendors which type of application they are providing. The vast majority of 3rd party Hadoop related technologies are still “MapReduce/YARN.”

As with all technologies, there are both risks and benefits to being the first technology of a kind in the market.

Success is often tied to how well one can educate a market on the new technology’s capabilities and benefits. Being the first can give you a significant first mover advantage if the messaging is effective.

The same applies in this case. Based on the level of investment going on in the market (like the $900 Million round Cloudera just closed), I predict that in the next 18-24 months there will be a wave of mature platforms leveraging YARN for everything from security to analytics. This is the wave that Hadoop will ride into the mainstream of business IT.

AR: Q2. What are the unique benefits of the RedPoint Convergent Marketing Platform? Who do you consider as your major competitors?

GC: The RedPoint Convergent Marketing Platform (CMP) solves three core problems facing most marketers today.

First and foremost, the Convergent Marketing Platform solves the marketing data problem by providing a robust set of ETL/ELT, Data Quality, Data Capture, Web Service, Automation and Master Key Management. At its foundation is RedPoint Data Management, which is ranked independently for Data Quality and received numerous #1 rankings in customer satisfaction surveys, including ease of use, speed and performance. In addition, we released RedPoint Data Management for Hadoop™ -- one of the first “pure YARN” applications to run natively on Hadoop. This level of power and robustness stands in stark contrast to the largely nonexistent capabilities of our competitors.
RedPoint CMP Second, we help marketers “defragment” their ecosystem. Marketers are continually bombarded with messages that say, “Sign up with me, upload your data and we will do everything you need.” Only later do they find out that it’s not true. What this proliferation of execution platforms does is exacerbate the fragmented data, processes and strategy problem.

Through RedPoint’s bi-directional orchestration layer, we create a centralized point of operational control that becomes the connective tissue in their marketing ecosystem. All operations, content, rules and data reside inside the RedPoint platform; execution channels (e.g., ESP, SMS, mobile push, social) become commoditized plumbing that delivers messaging and can easily be switched or replaced. This allows marketers to create and maintain a centralized and current 360° view of their customers, coordinate their operations centrally, easily unify their strategy and operate in unprecedented timeframes.

Third and most importantly is the “credibility gap.” We actually deliver what we promise and the system works just as advertised. This may seem like a strange point but marketing technology providers are notorious for creating hype around their technologies. The fact is that most of the marketing technologies out there don’t work as advertised, they don’t “do everything” and are built largely on antiquated architectures that have been re-skinned or subsumed into a larger suite of non-integrated marketing applications. No one is worse at this hype than the so called Marketing Cloud vendors who have acquired numerous technologies so they can claim to have everything marketers need. Their strategy is to become the final piece to the enterprise software puzzle.

The problem is that none of these companies have a track record of actually integrating any of the technologies they acquire…anyone remember what a Siebel implementation was like? Consequently, implementations take months, if not years and they rarely succeed. In contrast, RedPoint can typically build a 360° view of the customer and implement the full platform, with training, in about a quarter of the time of our nearest competitor. Some of our largest implementations have been done in weeks. That is the proof that the system works as advertised.

AR: Q3. Please explain the term "Master Key Management". What value does it add to Customer Analytics?

Master key management GC:
Master Key Management (MKM) is a take on Master Data Management (MDM) that is more focused on the resolution of identities and persistent linkage of individual person data across sources and over time. While MDM is more broadly IT-focused, MKM is focused on the specific needs of organizations dealing with customer or party data.

RedPoint sees this as the core of building and maintaining a 360° longitudinal view of the customer. Using these capabilities, we can link people, their relationships, addresses, ecommerce profiles, social profiles, mobile profiles and their devices over time. This kind of linkage and breadth of data supercharges analytics and enables modeling and prediction at a level that is otherwise unattainable. Insight like this is critical to developing the level of customer intimacy that most consumer brands are trying to achieve with their customers.

Second and final part: George on Big Data Trends and Important Skills