Re: [SLUG] htdig or other searchers

[David <david@xxxxxxxxxxxxx>]

On Tue, Jul 05, 2005 at 07:55:20AM +1000, telford@xxxxxxxxxxxxxxxxxxxxx wrote:
> On Sun, Jul 03, 2005 at 04:58:10PM +1000, David wrote:
> 
> > Does htdig go to the server (thereby parsing includes?). The site 
> > navigates perfectly from a browser.
> 
> Yes, in my experience it does, there are entries in the access_log.
> Beware if you are running virtual servers because usually it connects
> to 127.0.0.1 (but that's configurable). Also it has filters based on
> file type, usually it avoids all CGI references.

thanks for your reply, but as usual it was my ignorance that was the 
problem.. in htdig.conf I had put two start url's on seperate lines, 
assuming that \n was counted as white space... silly me.

I'm replying to my own question in case someone else makes the same 
mistake. It took me three days to figure it out.