Skip to main content

Wget: Mirroring Website

WGET is Ultimate Downloader. While you can use it simply to retrieve a single file from a server, it is much more powerful than that and offers many more features.One of its features is mirroring website.

Mirroring simply means downloading the entire contents of a Web site and uploading it somewhere else. So, we can make local copy of a website using wget.

The basic command to mirror website is

$ wget -m http://www.example.com

This command will allow you to download website but it can't make a complete local copy of a website. This command only download the page but the links still point to the real URLs including the images and stylesheets.

We have to add another option to make links not point to real URLs. Add option -k after -m

$ wget -m -k http://www.example.com

Now, the links will point to next page in our local copy website.

To delay your download, you can add option -w and how much time that you want to delay it. Because if we delay the mirroring, it can reduce the load of bandwidth of the site.

$ wget -m -k -w 25 http://www.example.com

From command above, -w 25 means we delay it for 25 seconds between requests. We can replace the number depend on our need. We can add suffix for the number like m for minutes, h for ours and d for days.

Now if you want to download your favorite website you can use wget to do that. This wget can be use on Linux, *Nix and Ms Windows


ref : http://fosswire.com

Popular posts from this blog

Free Stuff

I have free stuff that can i share with you. The free stuff is free magazine, free domain, free hosting, free rapidshare link generator and free money. 1. Free Magazine. You can get free magazine for you to read. There are many topics that can you choose. These are the topics that can you choose : Agri c ulture Automotive Biotechnology & Pharmaceuticals Computers Construction Education Engineering Executive & Management Finance Food & Beverage Government Healthcare Human Re sources Industrial & Manufacturing Internet Information Technology IT - Data Management IT - Security IT - Software & Development IT - Storage Insurance Mechanical / Machine Meetings & Travel Multimedia Network / Communications Purchasing & Procurement Retail Sales & Marketing Small & Medium Business Telecom & Wireless Trade/Professional Services Transportation & Logistics Utility & Energy Many topics...

Centos 5 Live CD, Rescue CD or Workstation

Have you ever try CENTos 5 Live CD? I had tried it. This Centos 5 Live Cd can be use as rescue cd and workstation cause it based on CentOS 5.0 i386 distribution. The following software that include with it : OpenOffice.org 2.0.4 Firefox 1.5.0.10 Thunderbird 1.5.0.10 Gaim-2.0.0 Scribus-1.3.3 xchat-2.6.6 k3b-0.12.17 Gimp-2.2.13 And for rescue, there is another software or tool : Full set of LVM and RAID command line tools QTParted Nmap and NMapFE Graphical Traceroute samba-3.0.23c with cifs kernel support to connect to Windows file shares System Log Viewer GUI Hardware Device Manager Unfortunately, in first release, this Centos Live CD doesn't hvae installer like Fedora Live CD.  And i have a problem when using it as workstation, i can connected to network just for a while. But, when i restart network service it worked but for a while and then it was drop. You have a same problem or solution ? Please tell me.  I will be appreciated it. Last, if you want to use root account, use thi...

Installation Of SMS Server Tools 3 And PlaySMS On Debian Server

This tutorial will show you how you can set up an SMS server on Debian with playsms and smstools. In my example I have used an old Nokia 5140i with a USB datacable. What we need before we start is apache2, mysql server, php5, lynx, wget, curl. Now we install updates, and install dependencies: apt-get update apt-get upgrade apt-get install apache2 mysql-server php5 php5-cli php5-cgi php-pear php-db phpmyadmin make gcc sendmail lynx wget curl Now create a password for mysql root user, and remember it. Installing playsms adduser playsms mkdir -p /var/www/playsms mkdir -p /var/spool/playsms mkdir -p /var/log/playsms chown -R www-data /var/www/playsms chown -R www-data /var/spool/playsms chown -R www-data /var/log/playsms And next download latest version of playSMS, find latest here, or grab my url: http://sourceforge.net/projects/playsms/ wget http://downloads.sourceforge.net/project/playsms/playsms/Version%200.9.5.2/playsms-0.9.5.2.tar.gz This will extract playsms to /usr/local/src. tar ...