Google Search Appliance and Microsoft Sharepoint Search Side-by-Side

I recently spoke on Tuning Enterprise Search at the Gilbane SF show (you can view the tuning enterprise search slides here). In fine-tuning my slide deck, I found myself comparing the tools that the Google Search Appliance and Microsoft’s Search Server Express provide for tuning search results.

I’ll be posting my observations in three parts this week:

• This one looks at how these two systems allow you to address the most popular searches.
• The second looks at how these search solutions allow you to tilt the landscape of search results to meet the needs of your specific content ecosystem
• The final post discusses how you can broaden search queries

As a general observation, Microsoft provides more power – particularly programmatic power – while Google opts for a simpler but less comprehensive approach.

Addressing the Short Tail of Search

I’ve written before about the shape of search and the leverage you can obtain by focusing on the most frequent search phrases - the short tail of search.

Both the Google Search Appliance and Microsoft Search provide simple interfaces for overriding algorithmically produced search results, and selecting a result – or group of results – to appear in first, second or third position (etc). Google’s KeyMatch interface allows you to select a search phrase and define a URL (and its title) to be returned. You have the option of returning the result if someone searches for the exact phrase you’ve defined, or a partial match.

Microsoft Search Server also lets you define “Best Bets” to achieve the same outcome, which provides a few additional capabilities

• Defining synonyms to the keyword phrase
• Defining the text to appear below the link
• Specifying start and end dates for the Best Bet

If you do nothing else to improve the effectiveness of your intranet search engine, use these tools:

• Review the 100 most frequent searches this month
• Work with subject matter experts to define the “best” results
• Do it again next month (in other words, be consistent)

In a very short period, you will be delivering much more effective results to at least one-third of your searchers.

Earlier this week, I described my observations about the Gilbane San Francisco conference, where I presented on Tuning Enterprise Search. As promised, here is more on Udi Manber’s keynote address.

Udi Manber (VP of Engineering, Search at Google) opened the conference with an enlightening keynote speech. He made a number of observations that I thought were useful — some about Google, some about the Internet as a whole. I’ve paraphrased some of his comments below, in bold.

The 20th Century was about conquering nature – building rockets, damns, roads, airplanes. The 21st Century is about understanding people – health care, education…. I hope he’s right. It’s nothing if not optimistic.

Google made 450 changes to its ranking algorithm last year. That’s more than eight changes each week. No wonder search engine optimization is such a challenging field.

Google removes friction for innovation. Engineers have an idea, and they experiment with changes to the algorithm. Analysts evaluate the success of the change. At weekly meetings, results are reviewed and plans are made for successful changes to be implemented more broadly. Wow. That explains the rate of change to the algorithm, and the extent to which Google embraces risk and eschews traditional approval processes.

Areas of Focus for Google

Cross-language information retrieval: breaking the language barrier. Google is actively working on 70 languages and it’s clear that automated translation has become critical. He demonstrated real-time translation of a search query, displaying the search results and the linked page.

Custom search is now Google site search. Individual sites can use Google site search to allow visitors to search their site. The service is free if you let Google run adverts; there’s a nominal cost if you don’t. Interesting side note: Google has created a secondary index specifically dedicated to Google Site Search. This ensures all the pages on sites using Google Site Search are indexed, but it’s separate and distinct from Google’s primary index. In other words, you can’t use this as a backdoor to get your pages indexed and ranked by Google.com.

Universal Search was launched about a year ago and the number of non-text assets such as videos, images and blogs has grown dramatically. This probably doesn’t surprise frequent users of Google – it has become difficult to craft a search that doesn’t return some non-text results.

Audience Questions (again, my paraphrasing of both Q and A)

Q. How does Google Scholar compare to Google.com?

A. Google Scholar is a specialty search program – they’re not including it in Universal Search. Citations play an important role in ranking within Google Scholar.

Q. Is Google exploring the knowledge embedded in social networks, or is this largely hidden from Google?

A. One way of approaching search is to think about how to signal to people that specific pieces of content will prove valuable to them. The fact that their friends all do something is a signal. Is it important for search? We don’t know. Google’s not using it today, but in the future? Possibly.

As many of you know, NLC has been very active in the area of social computing, enterprise 2.0 and the whole social network building within the enterprise.   Lately we’ve been translating enterprise 2.0 prophesies into actionable strategy and waxing poetic about the Cluetrain.  While there is little debate around the buzz, the intrinsic value and merit of social computing and enterprise 2.0 are still hotly debated.  In fact,  it is pretty easy to argue that social computing is the next major battle ground for the titans of software (Microsoft and IBM) to tangle.

But let’s deal with that issue in another post.  Here I would like to address what I think is the beginning of a very long list.  Loosely titled “stuff not to forget as you craft your Enterprise 2.0 strategy”, I am sure there are many more to add.  Here are five:

1. People: Seems rather basic but I can’t tell you the number of meeting I have been in where folks have wanted to:
   

   “lets just build it and see what happens” (my personal favourite) or

   “we are launching Monday, someone should send an email to the team to tell’em its up”

   You are basically issuing your death writ if these are the types of discussions your team is having.  People need a compelling reason to do almost anything and “collaborating” in most people’s minds means “more meetings”.

2. Trust: This is certainly an amorphous topic, but one that can’t be overlooked.  People trust people they can connect with on various multi-layered levels.  People don’t instinctively trust databases, computers, or lists returning the folks that proclaim to have found the “smart people”.
3. Content: The first person into a network is a very brave soul – very alone, but very brave.  Why would anyone want to join a network that is empty?  You need to think (hard) about what content is going to drive users there again and again (see reason 1 for the justification).  Seed the content stores in wiki’s and blog’s, have personal profiles already completed for a core group. Nurture and captivate the “publishers with something to say” in your organization and then promote the hell out of them.
4. Search: This is a huge, huge neglected area in most Social Computing strategies.  If you do your job correctly, you will end up with a massive amount of asynchronous, decentralized and ad hoc content.  In there, scattered among your employee’s pictures of dogs, and cats and cottages, will be the tacit knowledge of your organization.  If you didn’t think about it before hand, I have two words for you: “good luck”.
5. Ease of Use: This should be a no-brainer.  The meteoric rise of social computing is partly due to the ease of use, self conforming nature of the toolset.  However, I am thinking more around instructions on how to start a wiki being unclear, or a vague or ambiguous policy and governance around content creation that cause your folks to pause and hesitate before contributing comments or a post.  (agreed, sometimes a good thing).

Okay, that’s a quick list, from me, on a plane, again.  What are some others?

I presented at the Gilbane San Francisco show last week (my presentation on enterprise search tuning, with voiceover is available here).

It was an interesting experience. Here are some of my observations about the learning sessions and the conference itself (in no particular order).

The Google keynote address on the first day was illuminating. I’ll blog more about this tomorrow, but we learned information that I’ve not seen Google disclose previously. More on this tomorrow.

The vendor floor was full, and most of the CMS booths were indistinguishable. It’s clear the marketing message for all vendors has coalesced around a core set of CMS attributes. This lack of differentiation means that life for purchasers of a CMS is only going to become more difficult .

The analysts universally agreed that customers should “try before they buy” when it comes to content management solutions, and they encouraged customers to walk away from any vendor that wouldn’t cooperate. This sounds like a fine idea. In practice, we’ve found that few customers have the time/bandwidth/motivation to learn to use multiple CMS products and undertake real-world evaluations. Perhaps this “date before you get married” motto promoted by the analysts will catch on, but I rather suspect it won’t.

The conference was filled with questions about Enterprise 2.0 and the role of web content management platforms. Answers were in much shorter supply.

Microsoft Sharepoint 2007 was the white elephant in the room. No one – neither vendors nor analysts – spent significant time talking about the ground shaking impact Sharepoint is having on the CMS marketplace. The closest they came was when Forrester Research analyst Rob Koplowitz first described MOSS as a set of festering boils and then shifted gears and suggested MOSS is fundamentally changing the way businesses get work done.

Three comments worth noting (and I’m paraphrasing here):

• Tony Byrne of CMSWatch:
  Most enterprise 2.0 success stories we are hearing about involve huge companies that benefit from the network effect, or knowledge-intensive organizations where knowledge is the key asset.
• Cairo Walker of Step Two:
  Technology can only support what is already happening (the implication being that technology can’t change how you work, just support how you work).
• Rachel Happe of IDC:
  Content is exploding – we’ll see a six-fold increase in content volume between 2006 and 2010. And 70% of this content will be user-generated.

Finally, participants, vendors and analysts all commented on the weirdness of having Gilbane SF and WebContent 2008 happening at the same time. What? There aren’t 52 weeks in a year? The overlap meant that both attendees and speakers were diluted. I’d have very much liked to hear Lisa Welchman’s keynote on Web 2.0 and Web Operations at WebContent 2008.

Just returned from the Microsoft Canada Leadership Summit.

I would add one more Clue Train quote that seems to be resonating quite clearly (although it scares the bejeezus out of them) with some CIOs here at command central:

“The intranet revolution is bottom-up. There’s no going back. If a company doesn’t recognize this, the top-down intranet it puts in can breed the type of cynicism that results in ugly bathroom graffiti and mysterious golfing cart accidents. The intranets under the radar screen — and the rest of the Net panoply, including e-mail, mailing lists, and discussion groups — ignore the corporate blather and ass-covering pronouncements. Instead, these new Web conversations are actually being used to get some work done”.

The trickiest part of the building from the bottom up is gaining critical mass of your intranet 2.0.  There are certainly some very prudent steps to take (policies, procedures, codes of ethics, etc) but at the end of the day, it is going to be your corporate culture that  is going to dictate the success you will have initially on your social networking initiatives.

As the expression goes – “Enterprise 2.0” will need “Management 2.0” before you can expect success.  As someone said at the conference: “culture is what people are doing when nobody is telling them what to do”.  Where is your company… Are you ready?