Crawl Troubleshooting

Every once in a while SharePoint Crawler will behave unexpectedly on web sites that you wish to crawl. You may get error messages that you can understand and help you troubleshoot the problem or you will get just one error that is not helpful at all. Either way, I have found that you can troubleshoot the crawl by using the following technique with Fiddler Web Debugging Proxy (http://www.fiddler2.com). Using Fiddler we are going to configure SharePoint Search to crawl through Fiddler as a Proxy so we can watch the traffic.

Troubleshooting SharePoint Search

When I first started working on SharePoint Search I was troubled by how hard it was to figure out what was actually happening when the crawl was running. The crawler/indexer is a black box that I could not see into. This made it very challenging when I would open the crawl logs and see a cryptic error message. (Often the cryptic message is not the fault of SharePoint Search, the log is just bubbling the error up, often from the iFilter.) Here are two approaches that I use to troubleshoot specific errors. The first is a tool for troubleshooting iFilters the second for watching the actual crawl activity.

I recently received a question from a client who has been working on improving the quality of their search results. They are crawling their non-Microsoft CMS with SharePoint and wanted to ensure that the Keywords that they entered in the CMS and were finally rendered in the Web pages, would be findable in the search center. The keywords were emitted as META tags in the page.

<meta name="keywords" content="Elephant,Giraffe,Lion" /> 

This is a demo I often do when asked about how I perform Search Center XSL changes to improve the SharePoint Search Center Search Core Results Web part display. This information is buried in a few of my other posts so I thought I’d call it out all by itself for clarity. I usually perform these steps right after I create a new Search Results page and corresponding tab in the process of customizing the tabbed Search Center as I explain in the post Getting Started with Search Results. I perform these steps for many reasons that I detail in my No Code Search Center Customizations lecture. The primary reasons are:

As I have taught, before a document property can be used in search results, it must be promoted to a Managed Property. Many properties are already managed properties out of the box. On several recent SharePoint 2010 Search projects I have used third party iFilters to solve specific document indexing challenges and needed to work extensively with Crawled Properties. For example:

• I want to extract XMP data from images like Latitude and Longitude
• I need to promote a particular field for use as Document Keywords
• I need to troubleshoot a specific iFilter

Time and time again I have evangelized the use of supported third party iFilters like the iFilter Shop and their great selection of products to solve indexing challenges (rather than free crap). Likewise, I LOVE the Foxit Software PDF iFilter. In tests the multithreaded Foxit iFilter 2.0 CRUSHED the Adobe iFilter by indexing files 39 times faster than Adobe (processing a scorching 29 files per second). Read the full review. But what happens when you decide to use these tools and the results are not what you expect, or you want to take the plunge and extract more metadata, like the XMP file information from an image? The reality is that the SharePoint Crawled Property interface are not very “user friendly”. I hope to change that with the help of our friends at the iFilter Shop and some recent experience.

Reposted from AC’s Blog: Next week brings the Microsoft SharePoint Conference 2011 in CA. The instructors at Critical Path Training will be in full force as we’re presenting over 14 sessions all week.

We are also giving away over 120 SharePoint books at our booth with hourly book drawings and eight book signings throughout the week!!!

But wait, that’s not all… we’re also giving away over $25,000 in SharePoint training to lucky winners throughout the week.

Some of my favorite demonstrations are about how to enhance the Search Results pages in SharePoint. I feel that presenting “actionable” search results can make the difference between a good search center and a great search experience. I am working on some new demos and decided that I need to commit the fundamentals to my blog so that my audience has the necessary resources to apply the guidance from my presentations.

Microsoft unveiled the SharePoint 2010 and Windows Phone 7 Training Course! If you are looking to create Windows Phone 7 mobile applications that integrate with SharePoint this is the kit for you. This kit was developed by Darrin Bishop, [Todd Baginski](https://mvp.support.microsoft.com/profile/Todd, and myself. We also worked very closely with Paul Stubbs, Dave Pae, and several engineers at Microsoft throughout the project. Without their help, guidance and willingness to dive in on problems we never would have finished!!

Do you have tough technical questions regarding SharePoint for which you’re seeking answers? Do you want to tap into the deep knowledge of the talented Microsoft Most Valuable Professionals? The SharePoint MVPs are the same people you see in the technical community as authors, speakers, user group leaders and answerers in the MSDN and TechNet forums.

By popular demand, we have brought these experts together as a collective group to answer your questions live. So please join us and bring on the questions! This chat will cover WSS 3.0, MOSS, SharePoint Foundation 2010 and the SharePoint Server 2010. Topics include setup and administration, design, development and general question.

A long time ago I pulled together a bunch of my SharePoint Search XSL samples for SharePoint 2007. I spent the morning updating the samples so that they work in SharePoint 2010. I also added a new sample for rendering links directly from a links list (rather than landing on the DispForm.aspx page).

Included in this release:

Open Document Library

Renders a link next to document result items that opens a new window to the containing library or folder.