Follow Gregory Piatetsky, No. 1 on LinkedIn Top Voices in Data Science & Analytics

KDnuggets Home » News » 2015 » Jan » Opinions, Interviews, Reports » Shining Light on Dark Data ( 15:n03 )

Shining Light on Dark Data


Dark Data is the ever-present, relatively unknown, and unmanaged volumes of data that exist in every corner of one’s business. Here’s what to do with it.



By Tom Fountain, CTO of Pneuron.

Dark DataIn early 2014, Mark Gibbs authored several fascinating blog posts around the topic of “Dark Data”. Dark Data as one may recall, is the ever present, relatively unknown, and certainly unmanaged volumes of data that exist in every corner of one’s business. This data may lie in spreadsheets and files, but is often a crucial component of running a business as it represents highly localized and valuable knowledge managed directly by users. Outside of the fears of possibly losing such crucial intellectual property, just imagine the potential if a business was able to catalogue and distribute such knowledge to others in the firm.

Gibbs outlined the many barriers to harnessing such content – not the least of which involved the protectionism of its local owners, who fear disruption to the often elaborate stitching together of corporate and local assets to solve their unique needs.If corporate IT or any other organization wants to access, catalogue, and potentially re-distribute such intelligence, they will have to ensure at least three conditions are met:
(1) any attempts to secure, catalogue, extract, or re-distribute information will not disrupt current usage patterns on the part of current owners;
(2) there will be no or very few new constraints placed on current owners who wish to continue to evolve their creation in pursuit of higher levels of value; and
(3) there will not be excess administrative burdens placed on local owners requiring them to better “document” their creations.

This is a tall order - gaining access, cataloguing, and extracting critical elements of an existing “information infrastructure” without causing disruption or new barriers to the organic evolution of new forms of value. To fully capitalize on this opportunity, organizations are advised to evaluate a new breed of technology which non-invasively connects and extracts targeted elements of an information infrastructure.

Such technology must thrive (read “be highly accessible”) in the hands in both subject matter experts who solve real problems and business analysts who understand information management and systems. Their collective ability to access and use these pockets of dark, yet valuable, information and knowledge will directly drive reuse and creation of new composite value streams. In the best case, these users will be able to rapidly experiment with these new pockets of insight by seamlessly integrating not only data but all other assets (applications, data services, websites, etc.) within their “digital” reach within a single interoperable fabric. This fabric promotes short cycle trial and error, low cost of success (and failure), yet retains the high degree of agility needed to smoothly swap in or out these newly discovered pockets of value.

At the end of the day, firms who create a lasting ability to rapidly innovate and bring new ideas to market that capitalize on their collective strengths are those who win. “Dark Data” represents far too much potential value to be left in the shadows, and firms with a strategy to full leverage these assets will be well positioned to compete.

Bio: Tom Fountain Tom Fountain is the Chief Technology Officer of Pneuron Corporation. In this role, Fountain provides leadership of marketing, pre-sales and product management with focus on product strategy and positioning in the Enterprise market and working closely with customers to develop innovative solutions to their most pressing and complex business needs.

Related:

Sign Up