I like to use Google search as an example since I was involved in building a competing search engine roughly at the same time as Google, so I happen to know a thing or two about the difficulty of that task.
Everyone and their dog thinks Google won solely because of PageRank, but anyone who has ever worked on web scale search engine can tell you that long before you can start worrying about ranking, you have to solve a bunch of problems that are really hard. Like copying a sufficiently large portion of the web onto your servers and processing that copy. Or even removing all the duplicates you will get (which is certainly much harder than I had imagined when I was tasked with this problem).
Google did all of these things as well as, or better than the competition. And on top of that they put together a cocktail of ranking methods that they were able to apply to information amounts for which there was effectively no practically oriented literature to turn to for advice. (Sure you could find lots of neat algorithms for this and that, but very few people had any experience with these at web scale).
Oh, and Google were not alone in having PageRank, or algorithms similar to PageRank. All web search engines I have any inside knowledge of (2-3) had this within six months of Google popularizing the idea.
Ask any one of the dozen or so web search engine companies that did web search if any of this was "easy".
What Google did in the late 90s was indeed heavy lifting. What pointcast did was relatively trivial and easy to copy in a few months.
Yet no other search engine suffered from these problems. Excite, AltaVista, dogpile...everyone else managed to successfully spider, index, and filter the web. I'm not seeing how infrastructure or data management was any particular value of Google's, and have seen absolutely nothing declaring otherwise. Indexing/deduplicating/managing data didn't seem to be a problem for anyone.
And FWIW, Google's real strength -- I guess I'm not everyone and their dog? -- was that they made a business case for the individual searcher. In the collapse of the .COM bubble an individual searching was seen as a close to worthless commodity, which is why every once-vigorous search engine had largely abandoned engineering advances. Google at the time first saw being the premiere search engine as a way of selling search appliances to business, and then with their text ads reinvigorating web ads after, again, the .COM collapse.
Alas, herein we see the magic of the survivorship bias: Google won, therefore Google must have done everything right. Anyone who lost must have done everything wrong. This is cargo cult thinking that does nobody any good. How you so casually write off the engineering challenges of Pointcast (an incredibly popular, multimedia-rich product, including pooled and queued subscription for millions of people, when bandwidth and computing power were very limited. A whole industry of Pointcast caching appliances appeared).
It's also interesting how you create the caricature of executives decrying the Google interface, as if it were the vindicated underdog after all of those years. Yet in those early years, the simplistic interface of Google was overwhelmingly the number 1 lauded feature of the site. Even before it was featured for good and quick search results, it was heralded for its simplicity.
Look at who is successful, note some random characteristics of these individuals or businesses and then come up with your own theory of how success is deliberately attained. The share it over the web.
What's even more amusing is when someone sings Google's praises as a company that "did things differently" and then proceeds to dismiss any future innovators (who choose to do things differently) in the markets that Google is in.
I have often thought Google is a grossly misunderstood company, despite many facts being readily available. I'm still waiting for someone to show me otherwise.
Some years back I watched a presentation where a representative of one of the internet's most respected infrastructure companies claimed that the secret of Google's success was diversification and not relying on one product (search). He suggested the winning strategy was to release numerous products and cited non-search Google's products as examples. However, every example was a product released by another company (who, go figure, focused exclusively on one product) that Google had acquired. Some folks do know the facts about Google, but I see an inordinately large number of folks who you think would know the facts, but sadly do not. Instead they have their theories and are eager to share.
> I'm not seeing how infrastructure or data management
> was any particular value of Google's, and have seen
> absolutely nothing declaring otherwise.
> Indexing/deduplicating/managing data didn't
> seem to be a problem for anyone.
Well, I'd say otherwise. It was hard and there were big differences in how efficient the various platforms were. And sadly, not all of them could keep up with the development pace and the growth of the web.
(I used to work for two of Google's competitors in the late 90s and early 2000s, I had some knowledge of the internals of a third competitor and I eventually ended up working at Google for a few years).
> Alas, herein we see the magic of the survivorship bias: Google
> won, therefore Google must have done everything right.
That was not what I said. Please do not pretend that I said that. I said that what Google did represented heavy lifting and what Pointcast did wasn't. I didn't touch on why Google succeeded at all. In fact, Google's success is entirely irrelevant.
>That was not what I said. Please do not pretend that I said that. I said that what Google did represented heavy lifting and what Pointcast did wasn't[....]In fact, Google's success is entirely irrelevant.
At this point I have absolutely no idea what it is you are trying to say, then, as you apparently want to have your argument work in any way way that you think has some sort of lesson.
Google is successful today because they made business bets after the .COM crash that no one else was making. They brought a new approach that earned them attention and customers, and the rest is history. They did good engineering, but so did countless companies that failed or continue in obscurity. I suppose that is the "heavy lifting"?
Pointcast obviously was doing good engineering (they had an engineering scale that was pretty much unprecedented at the time), but their business case completely fell apart and the organization was dissolved. There were literally zero issues with the engineering of Pointcast, nor were they replaced by competitors (despite that apparently being your angle?). It simply wasn't a viable business at the time, and they couldn't find a way to make money.
I like to use Google search as an example since I was involved in building a competing search engine roughly at the same time as Google, so I happen to know a thing or two about the difficulty of that task.
Everyone and their dog thinks Google won solely because of PageRank, but anyone who has ever worked on web scale search engine can tell you that long before you can start worrying about ranking, you have to solve a bunch of problems that are really hard. Like copying a sufficiently large portion of the web onto your servers and processing that copy. Or even removing all the duplicates you will get (which is certainly much harder than I had imagined when I was tasked with this problem).
Google did all of these things as well as, or better than the competition. And on top of that they put together a cocktail of ranking methods that they were able to apply to information amounts for which there was effectively no practically oriented literature to turn to for advice. (Sure you could find lots of neat algorithms for this and that, but very few people had any experience with these at web scale).
Oh, and Google were not alone in having PageRank, or algorithms similar to PageRank. All web search engines I have any inside knowledge of (2-3) had this within six months of Google popularizing the idea.
Ask any one of the dozen or so web search engine companies that did web search if any of this was "easy".
What Google did in the late 90s was indeed heavy lifting. What pointcast did was relatively trivial and easy to copy in a few months.
edit: fixed typo