The first observation of a new insight is often published in a lab notebook or personal journal or as notes from a meeting that captured an anecdote. Self-publishing often marks the first generation of new knowledge because “new” is often hard to comprehend and not accepted by the status quo or established publication channels. If entrepreneurs want to spot trends early this is where they must look.
Self-Publishing Often Marks The First Generation of New Knowledge
The first time I heard the word samizdat was at an ACT UP meeting. The term was applied to the vast stacks of photocopies that every member picked up on the way into the Monday-night meeting: treatment guidelines, drug studies, bureaucratic analyses, action plans, contact lists, and announcements of events ranging from performances and gallery openings to house parties and memorial services. This collection was never referred to as anything other than “the table” (even though it usually spread over two or three), a twelve- or eighteen-foot-long banquet of paper down both sides of which several hundred gay men and lesbians, nearly indistinguishable in their Doc Martens and Levi’s and sloganed T-shirts, bent their spiky or shaved heads and served themselves and one another with the ordered geniality of an Amish wedding. I was an intellectually pretentious but under-educated twenty-two-year-old who didn’t want to admit he was unfamiliar with a term that had the clear, clannish peal of jargon, the ignorance of which marked him out as neophyte or, worse, interloper. In fact I heard the word as “same-as-that,” which led me to think of it as an assertion of status: though these stapled stacks of paper, most written by people with no political background or scientific or journalistic training, lacked the credentials and durability of proper books, they were nevertheless the real library of AIDS, and the bound books that trickled out of traditional publishing houses were the table’s supplements rather than the other way around.
I spent the morning at the Pycon 2012 poster session. There were projects or products related to weather forecasting, data driven journalism, drug discovery, non-SQL databases, robots, data driven activism, open source volunteer enlistment and management, gridbeam, etc.. Running in the same convention hall as a job fair for Python programmers. Python is an enabling technology, it and related products and open source projects are being applied to engineering and scientific problems in a variety of industries and disciplines: defense, oil and gas, animation and movie making, financial analysis, pharma, and engineering. These poster sessions and talks represent new knowledge in a variety of fields.
Coupled with my experiences at Strata and Big Data Camp that I detailed in “A Picture is Worth a Thousand CPU Hours” it’s clear to me that high performance computing–databases, parallel algorithms and infrastructure, and visualization all in service of substantially more complex analysis–is in a very productive ferment. I think this goes well beyond the Hadoop centric cloud computing models that gained early prominence that I have focused on for the last four years o so:
- “Hadoop Summit 2009–Quick Impressions“
- “A briefing on Cloud Computing Paradigms at IEEE-CNSV May 19, 2009″
- “Deep Roots are Not Reached by the Frost (2008)“
- “Notes from July 19, 2008 IEEE Cloud Computing Event”
- “Structure08: The Whole is Less than the Sum of the Parts“
- “June 17, 2008 Cloud Computing Panel at VLAB“
Insight As a Service: Company and Usage Data Enables Proprietary Distillation of New Knowledge
The closest I have come to identifying a second waypoint–after Hadoop–for navigating this rapidly evolving landscape is Evangelos Simoudis‘ “Insight as a Service” formulation:
- Insight as a Service (Part 1: Oct-4-2010)
- Insight as a Service Part 2″ (Feb-1-2012)
- Insight as a Service Part 3” (Mar-9-2012)
These are tools that extract rough insights from company data, usage data, and syndicated or third party data. The first two represent substantial opportunities for new knowledge that is not immediately available to your competitors.