- To: Peter Rundle <slug@xxxxxxxxxxxxxxxxxx>
- Subject: Re: [SLUG] Spider a website
- From: Ycros <waikurosu@xxxxxxxxx>
- Date: Tue, 3 Jun 2008 14:25:56 +1000
- Cc: SLUG <slug@xxxxxxxxxxx>
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; bh=reLN5M0LmtRMbsqLRrk2bK879wFD0PjYTYxGqc3LSFg=; b=fhEl8CkP9FVOcOlxmXaeyayVUSeslNZ3KWlfChbta3nDfoe1QOLulEIU4qP65nKCGsiaxC0fBcJtBGp+HsJN0j7zvaV/qzoA+aR9y5vrYoXrKwTEQuzFHc3e8qVnxS0zX8/KBXNDKrnEmbK49qaVpRIW3wAp8Zsfs9DRFz11Udo=
- Domainkey-signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type:content-transfer-encoding:mime-version:subject:date:references:x-mailer; b=L33YGVg6Aw9YyyAlINpMB4wRbQ9xuzKoW/4J05FxPxa3T3CXE2I5qIkz19TxGC+2wUv7dM+07yPE2cBYf8O/t09vZiqjejhDm5Sn62xMlDNpEStTaQcH4t6MtWZR9G75UNn25PCHcobbZMLeS596JAsobdfKKlQcBcR6osMvnH8=
You could use wget to do this, it's installed on most distributions by
default.
Usually you'd run it like this: wget --mirror -np http://some.url/
(the -np tells it not to recurse up to the parent, which is useful if
you only want to mirror a subdirectory. I add it on out of habit.)
It's not always perfect however, as it can sometimes mess the URLs up,
but it's worth a try anyway.
On 03/06/2008, at 2:20 PM, Peter Rundle wrote:
I'm looking for some recommendations for a *simple* Linux based tool
to spider a web site and pull the content back into plain html
files, images, js, css etc.
I have a site written in PHP which needs to be hosted temporarily on
a server which is incapable (read only does static content). This is
not a problem from a temp presentation point of view as the default
values for each page will suffice. So I'm just looking for a tool
which will quickly pull the real site (on my home php capable
server) into a directory that I can zip and send to the internet
addressable server.
I know there's a lot of code out there, I'm asking for
recommendations.
TIA's
Pete
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
SLUG Mailing List Archives