Sumit Sharma

Archive for November 2009

What will drive adoption of the various web data overlays

leave a comment »


I’ve talked about metadata and how it will help aggregation of information in a relevant manner…however I’ve never really directly addresses when/how this will happen. This topic of adoption is something that John Hagel has encouraged me to think about…and after putting in some thought here is what I’ve come up with…

Users and Providers will drive adoption

We will begin to see two forces, from opposite ends that will drive adoption of the semantic web. Over time, as users experience this crowding out and their frustrations reach a tipping point because their needs won’t be met, they will begin to demand for more effectively aggregated data inflows that fit their preferences and immediate requirements. To complement this trend, the edge data and application providers will also demand for better more effective aggregation since they will be frustrated at their lack of discoverability as well as not being able to effectively pinpoint who their customers are.

User Adoption

There are a few factors that will drive adoption. First of all, users will continually become more aware and demand what they want to imbibe – relevant information at the right time, at the right place. This will be accelerated by the cyber children or the web generation – those that have grown up in the digital age, and rightly so. Because of their heightened access to information and data, this new generation have a better idea of the world we live in, and with technology and knowledge at their fingertips we’re seeing that they are able to decipher through what they want and don’t want. There is a level of sophistication in this generation’s ability to choose what activities they engage in, both online and in the physical world, and this will further drive this demand for well aggregated information. Eventually, the many disparate sources of information will consolidate into fewer sources – in fact, we’re already seeing this in motion. People are decreasing the numbers of sources of information, as evidenced by the Yahoo!, Google and MSN homepages where they are able to integrate news, facebook and other widgets (or apps) into a single point of reference.

In summary, we’d be seeing a Boomerang effect: starting from the old era of specialization, silo’d awareness and high attention spans, and then evolving towards diverse awareness, breadth of knowledge and coming back towards a state of heightened focus, but at a much wider and varied degree across the spectrum. In other words, instead of the 8-10 major career foci one had 20-30 years ago, that will change to 200-300 career foci points.

Where does the power lie at this current point in time? Google has information on random search queries. Facebook has information on our social graph. LinkedIn has information on our professional graph and like this there are several other highly powerful powerhouses of data which if federated their APIs across each other could enable this to happen.

This will be highly disruptive to the world as we know it today: media, services and products’ industries will all need to re-think their operations and strategies because at their current models, they will to provide increasingly irrelevant information to users.

Provider adoption

A second major force pushing towards the adoption of the semantic web will be driven by the edge providers of information and services, and paramount to them will be the issue of discoverability as well as the ability to effectively target towards the appropriate user base. We’ve already used the Music Genome Project as an example of Providers catching on to using metadata to better describe their products and services.

Discoverability at the Edge

The issue of discoverability for the user’s immediate needs in a relevant manner will be of prime importance. Today, there are certain mainstream apps, such as that generic ESPN sports ticker or that Amazon.com widget that will be very popular and easy to find however just because information is more popular doesn’t mean it is more relevant to you or me. The discoverability and relevance issues won’t relate to mainstream data and applications, but rather with the tail end of applications and data – or what we can refer to as the edge information and services. The proliferation of this obscure set of edge applications and information will increasingly elongate the tail and this data will crowd each other out and make it tougher to discover. Simply look no further than the iPhone app store and under each category there will be 1000s of apps that would take a long time to sift through. This crowding out is detrimental to each and every app that needs to make itself known. Given that there is already a rating system in place to help users spot most relevant apps, it can be argued that this subjective rating mechanism may not be the most relevant way to filter so unless a user is “in the know”, he/she may not get exposed to such interesting apps and data that may have been very relevant to their needs. So in order to accurate correlate user characteristics/activity with the right app, there will need to be a sophisticated algorithm that employs the usage of semantic metadata in doing do.

On the flip side, the ability for a provider to discover motive and intent as data and information grows will be important. There are many constraints, as described in the user meta data profile, that go into defining user characteristics, and without them it is tough to accurately measure motive.

Written by Sumit

November 23, 2009 at 7:14 am

Wheres the value: data or analytics?

leave a comment »

I want to mention share something that John Seely Brown mentioned the other day during one of our meetings at the Deloitte Center for the Edge: The future doesn’t lie in IT, but in analytics….here are my thoughts on that…

As users’ demands grow, less value will be put on data, and more on analytics. Today we see companies protecting whatever data they possess and use it as their source of competitive advantage. However, an argument can be made that going forward, unless something is done with the data, it will be of little or no meaningful use. This, if approached with the right strategy, will be in the best interest of the large incumbent companies because by opening up their data and APIs, they’ll be able to extract expertise and knowledge from outside their boundaries. Thus, the barriers to get whatever data Google has about its customers for example will decrease over time.

Written by Sumit

November 20, 2009 at 7:31 am

My rant on privacy

leave a comment »

A cultural shift is occurring, and we’ve witnessed it in recent history. Look at human society and its history of social networking. Look at the evolution of human social behavior in the recent century. A century ago, there were conservative values – we only mingled within our families, had a very filtered group of friends and married within our tight micro communities. Eventually, with evolution of modes of transport came the broadening of our networks which eventually broke down the filters and notions we lived by. Our friends were from diverse backgrounds, and we began to see inter cultural mixing. With the internet came along a further breaking down of barriers in the form of online dating, chats and online social networking. The cultural shift has happened and it will continue to happen – the common thread has been a gradual breakdown of cultural norms and we are beginning to see a further breaking down of the wall of privacy. Today we are able to browse each others Facebook profiles and see each others personal information to a very large extent – this is becoming to way of doing business – through maintaining and fostering trust based relationships. It is inevitably going to be the case where we further let our guard down and begin to share our activities

There has been an exaggeration of the negative effects of sharing your lives with others, and notably in certain movies this has been creatively portrayed to negatively infiltrate the degree of control we will have on our lives. Those that are threatened or scared by such a notion of the new webs of data are probably now aware of parallel initiatives that mirror similar behavior and in addition, may not fully understand the power of the power and benefits of sharing personal data.

With the privacy concern being a legitimate one, we should be aware that it such infiltrations of privacy are already occurring in the examples of various products: GM onStar, Remote burglary monitoring systems, Airlines reservations systems, Credit Card companies’ logs of our activities and so on. There already is a big brother out there so there is no stopping it, and of course there can always be a negative twist to anything in life, but with the appropriate regulation, control and security, the benefits far outweigh the negatives.

It is like getting scared of a box of matches (for creating fire) because now we won’t be able to spend 4hrs lighting up a fire through rubbing sticks. Why do we need to rub sticks when we can easily be more productive and valuable by enjoying the fire created and putting our minds to greater use? That’s all that is being done here, we’ll have a means for a higher quality of data, appropriate to us at the appropriate moment, which will be complementary to our nature, lifestyle and thoughts – we’ll get what we’re truly looking for, in the most efficient and effective manner. Privacy won’t be an issue in the long term. Initially sure it may appear intrusive or annoying to have irrelevant information or recommendations enter your life but as soon as these chinks are get ironed out – as intelligent systems learn what we like and don’t like they will surely adapt to providing us appropriate and relevant information at the right time at the right place. Having said that though, there is always the option to opt out of such means of consuming data and that will always be a choice.

Written by Sumit

November 18, 2009 at 7:14 am

Privacy and metadata capture

leave a comment »

I will now pen some thoughts down on THE issue I’m sure is eating all of you as I’ve talked about meta data and all that over the last few posts…

Capturing of metadata for users and providers will undoubtedly ruffle many feathers, especially regarding the so-called infringement of privacy. There is no doubt that this information can be used in a negative fashion, as with all other information captured in this world.

Addressing Privacy through a Governance framework

There are layers of privacy that need to be defined and once put into use, should be dynamically configurable. The frameworks in place should transparently, and conspicuously address metadata being captured. Ideally, for this to be possible, we should have a personal dashboard illustrating whatever metadata related to our lives is being consumed and with the flip of a switch we can have the ability to discontinue. If all users had a single integrated profile in the same way one’s SSN defines your tax profile to the IRS describing our activities in all planes it is conceivable that one day we shall have the ability to control what data about us gets consumed and not.

This will be one of the drivers of a single consolidated identity, and will drive the major website providers to form alliances and share APIs transparently amongst each other.

Of course this solution outlined about is wishful thinking, especially given that today incumbent providers hesitate to share data with each other since their business models depend on their possession of proprietary data. Short term solutions would be for websites and the physical entities that do capture metadata about our activities to send daily or weekly itemized summaries highlighting what information about us is publicly available. This would also be an excellent opportunity to slip in a few questions on what data we wouldn’t mind sharing, if its not already captured in metadata.

Written by Sumit

November 17, 2009 at 7:29 am

The aggregators role in bringing us closer to what we don’t know!

leave a comment »

Influencing Serendipity

Another challenge to be solved by the aggregators would be to answer the question, “Do we know always know what we need?”. This questions arises because not only will we be figuring out how to make the information in front of us out there relevant to us in the present moment, but ensuring that we are privy to any other information that isn’t necessarily in front of us! In other words, being able to frame the web of unknown data and information to our benefit. This could usher in a new era of productivity and innovation where ever expanding web of data will enable us to live a dynamic life in which serendipitous moments frequently occur, productive friction and tinkering are accepted styles of working all leading to a heighten degree of innovation. There are many disruptions possible if information was aggregated in an intelligent fashion and nearly impossible to fathom the magnitude of impact this could have on our social lives, on how businesses operate, medical research and learning to name a few areas.

Written by Sumit

November 14, 2009 at 7:11 am

Aggregators and Advertisers: they’ll need to behave

leave a comment »

It should be noted that there are plenty of ways to abuse the system that infringe on rules of ethics. Especially in the case of aggregators who make their money from advertising, they will need to be cautious with providing credible information (including advertising). For example, Yahoo displayed some interesting headline links on their page, and it was, intentionally or not, looking like it was part of the Yahoo page though there were in reality some advertisements. This is counterproductive for several reasons. Firstly, the advertising company is just being dishonest in its presentation and has the potential to alienate and frustrate users. Its one thing to advertise next to a legitimate article but totally different to mask a link as an article when its really an advertisement. Second, because this product’s page will go down into my history of pages visited, it will become a part of my virtual persona and might further frustrate a user through spam mail or more similar advertisements – It contributes towards a falsification of who I am.

The control migrated from the user to the provider in such instances– which brings up the interesting argument of when can we trust the providers/aggregators to provide us with meaningful information. We need to distinguish between advertising that can be potentially predatory in nature and meaningful information. This point shouldn’t be taken lightly because until aggregators find a source of income other then advertising, Internet advertising will only become a mainstay and these aggregators will continue to be incented to include ads on their pages in any little space possible.

Such advertising can result in a snowballing effect of making the internet into a worse place, clogged with toxic information such as predatory advertisements. Take the case of online gaming where players, typically teenagers without cash, will click on an offer is a CPA advertisement that users complete in exchange for in-game virtual currency. This in turn may lead to additional advertising revenues which will get plugged right back into the ecosystem, as noted by Michael Arrington, editor of TechCrunch, and in such cases, it only fosters an ecosystem of low quality advertisers. This was evidenced by Mark Pincus, fonder of Zynga, who admitted that he “did everything he could to get revenues” in the initial stages when Zynga was starting up and desperately needed funds to survive.

Written by Sumit

November 10, 2009 at 7:09 am

Incumbent aggregators should beware

leave a comment »

The barriers to compete against the large incumbent powerhouse aggregators such as Google, Facebook and Amazon of today are not as high as the technology community may make it out to be. The key will lie in being able to effectively aggregate and provide relevant data – and in the next few years as we are drowned in the influx of information, our loyalty to Google, Facebook will be quite fickle in the same way that our loyalty to Yahoo was when Google came, or to Friendster when Facebook came in.

This will be quite disruptive to the monetization models of aggregators today, where it is primarily based on advertising, and the companies with the biggest advertising budgets win however going forward this disruption allows for companies with the highest relevance to win. For example, if you Google searched for “buy Wilson tennis racket”, the sponsored links appearing at the top may be from bigger companies (that may or may not stock Wilson and there is low chances of the aggregator filtering through that information as a criteria) however, in a world where businesses are semantically contextualized, perhaps the business geographically closest to you will appear on the web, or the businesses that only specialize in Wilson rackets will appear on the top – that will question the integrity of aggregators portraying sponsored links before more relevant search results.

Written by Sumit

November 8, 2009 at 7:08 am