Skip to main content

Wget: Mirroring Website

WGET is Ultimate Downloader. While you can use it simply to retrieve a single file from a server, it is much more powerful than that and offers many more features.One of its features is mirroring website.

Mirroring simply means downloading the entire contents of a Web site and uploading it somewhere else. So, we can make local copy of a website using wget.

The basic command to mirror website is

$ wget -m http://www.example.com

This command will allow you to download website but it can't make a complete local copy of a website. This command only download the page but the links still point to the real URLs including the images and stylesheets.

We have to add another option to make links not point to real URLs. Add option -k after -m

$ wget -m -k http://www.example.com

Now, the links will point to next page in our local copy website.

To delay your download, you can add option -w and how much time that you want to delay it. Because if we delay the mirroring, it can reduce the load of bandwidth of the site.

$ wget -m -k -w 25 http://www.example.com

From command above, -w 25 means we delay it for 25 seconds between requests. We can replace the number depend on our need. We can add suffix for the number like m for minutes, h for ours and d for days.

Now if you want to download your favorite website you can use wget to do that. This wget can be use on Linux, *Nix and Ms Windows


ref : http://fosswire.com

Popular posts from this blog

Privacy Policy

Another Pratama Privacy Statement What follows is the Privacy Statement for all Another Pratama websites (a.k.a. blogs) including all the websites run under the computer.pratama.us domain. Please read this statement regarding our blogs. If you have questions please ask us via our contact form. Email Addresses You may choose to add your email address to our contact list via the forms on our websites. We agree that we will never share you email with any third party and that we will remove your email at your request. We don’t currently send advertising via email, but in the future our email may contain advertisements and we may send dedicated email messages from our advertisers without revealing your email addresses to them. If you have any problem removing your email address please contact us via our contact form. Ownership of Information Another Pratama is the sole owner of any information collected on our websites. Comments/Message Boards Most Another Pratama websites contain com...

Extend a LVM partition after increasing its virtual disk on Virtualbox

No Linux machine at work? the easy way could be to simply install Virtualbox in one of the PC, create a VDI and install Ubuntu 14.04 in it. But the day will come when you need more space! Here is how to resize it: Resize Virtualbox VDI Open Virtualbox, make sure to shutdown your virtual machine. Then open a terminal (here windows): 1 2 3 4 cd “ C : \ Program Files \ Oracle \ VirtualBox ” VBoxManage list vms VBoxManage clonehd uuid 0000000000000000 backup . vdi VBoxManage modifyhd uuid 0000000000000000 -- resize 204800 This is the Virtualbox official manual  of available commands. And  this is a website  to convert Gb into Mb (–resize takes Mb as input, 200Gb = 204800Mb). Boot Gparted to resize the partition After a default install of Ubuntu Server on one physical disk, you will have a SWAP partition, and then an extended partition in which you’ll have your LVM partition (So 3 partitions).  Download Gparted  .iso,...

DirectX Standard Multimedia Development on Windows

In the past, game programmers create their own plot between device and the software. Because Computer Architecture is complicated, so it hard to obtain the desired speed for real animation, manufactures of sound cards and display adapter had to write drive for various display adaptor, it made the development of game is really hard. And if you know, most computer games were written for MS-DOS. Maybe you remember game Mario Bross, F1, Prince of Persia and others. Then Microsoft released MS Windows, so the game developer made game for its OS. And for the easier of programmer made their game, Microsoft give an idea that made programmer can make a standard plot to connect with hardware. This idea introduce in 1995 and at last it become a standard for develop multimedia on Windows platform named DirectX.