Geeks With Blogs

News These postings are provided "AS IS" with no warranties, and confers no rights.
Locations of visitors to this page virtusa corp sharepoint shehan peruma

Shehan Peruma

As part of setting up Enterprise Search for a major IT company, I had to create content sources for a number of web sites hosted on the company’s intranet. Pretty easy you say, well yes. But the problems arose when MOSS began crawling the websites.

There were more Crawl Errors in the crawl log than Crawl Successes. Seems that the web applications MOSS was crawling were not being maintained properly and there were loads of broken links. Naturally, I pointed this fact out to the administrators and naturally they asked me for the list of broken links, etc. Well, if there were only a couple of errors I could easily copy and paste the offending  URLs, but these sites had loads and I couldn’t find an “export log”  feature, drat!!!

Right, so the only option was to cook up a little C# winform application to export the crawl log. Easier said than done :) Other than for MSDN I couldn’t find anything else online to get me started, so, had to use Reflector quite a bit to figure out the internal workings of the assemblies, but managed to get something going….

I’ve uploaded the complied application on ProjectDistributor and looking to soon put it up on CodePlex.

As of now the log can be exported to only CSV format and there is no threading, so don’t freak out when the app starts hanging.  I’ll be refactoring the code and adding more export options once I get the time…

Get the application from here:

Posted on Thursday, February 21, 2008 11:44 AM MOSS , Search , Tools | Back to top

Comments on this post: Crawl Log Exporter

# re: Crawl Log Exporter
Requesting Gravatar...
I can not agree with you more because I remember I was also exposed to the crawl errors and I tell you it is frustrating and especially if you have no remedy. When you tell the administrators that MOSS application was crawling they ask for the broken links, which just sucks!
Left by Jordan on Nov 15, 2010 2:07 PM

# re: Crawl Log Exporter
Requesting Gravatar...
I’ve been intrigued by the reviews I have read, all of which buying the book. I forgot the name of it, I'll try to find those comments.

free printable baby shower invitations
Left by wedding invitations templates on Feb 17, 2011 9:40 AM

# re: Crawl Log Exporter
Requesting Gravatar...
I had been reading a few of the articles right here but everyone provide great information and I needed to leave just a little remark to support a person as well as wish you a good continuation. Wishing you the best of luck for those your own blogging efforts.I truly appreciate the time you absorbed writing this article.
Left by home office furniture san diego on Nov 14, 2011 8:03 AM

Your comment:
 (will show your gravatar)

Copyright © shehan | Powered by: