Crawlers Identification Problem

One of the problems rarely mentioned by developers of log-analyzers is identifying a crawler. Search bots (chiefly Google bot and MSN crawler) can generate more than half of the overall traffic. So the problem is overstating the overall statistics. A less obvious, but more important problem is a significant distortion of the navigation paths due to bots' random walking through the site. This makes off-the-shelf log-analyzers useless for the comprehensive analysis of the navigation paths.