Skip to main content

Wget: Mirroring Website

WGET is Ultimate Downloader. While you can use it simply to retrieve a single file from a server, it is much more powerful than that and offers many more features.One of its features is mirroring website.

Mirroring simply means downloading the entire contents of a Web site and uploading it somewhere else. So, we can make local copy of a website using wget.

The basic command to mirror website is

$ wget -m http://www.example.com

This command will allow you to download website but it can't make a complete local copy of a website. This command only download the page but the links still point to the real URLs including the images and stylesheets.

We have to add another option to make links not point to real URLs. Add option -k after -m

$ wget -m -k http://www.example.com

Now, the links will point to next page in our local copy website.

To delay your download, you can add option -w and how much time that you want to delay it. Because if we delay the mirroring, it can reduce the load of bandwidth of the site.

$ wget -m -k -w 25 http://www.example.com

From command above, -w 25 means we delay it for 25 seconds between requests. We can replace the number depend on our need. We can add suffix for the number like m for minutes, h for ours and d for days.

Now if you want to download your favorite website you can use wget to do that. This wget can be use on Linux, *Nix and Ms Windows


ref : http://fosswire.com

Popular posts from this blog

Install ffmpeg on Centos

I want to install youtube-clone to my server (Centos 5) and it need some files. The important files that i have to install is ffmpeg. Then ask to uncle google where can i find how to install ffmpeg. I found one website that give tutorial how to install ffmpeg on Centos 5. The installation is begin from download the files. First, ffmpeg in rpm and others in tar.gz. Rpm files easy to install. But another have to do some config. You can read the tutorial install ffmpeg on Centos 5 here and here (Centos 4) . Another Link about Centos Guide For CentOS Centos 5 Live CD Password For root

Rapidshare Premium Link Generator

Rapidshare Premium LINK Generator . Rapidshare.com is one of the biggest site that allow you to upload and share files with vary sizes. From rapidshare, you also can download files that another person upload the re. There are 2 kinds of user can download from rapidshare. First, free user, this user have limit features from rapidshare. Limit bandwidth, limit connection, etc. Second, premium user. Premium user have many features that free user doesn't have. If you want to be a premium user, you have to buy it from the web. You can use this link to buy premium account or from reseller that near you. But if you have not enough money to buy it, you can use Rapidshare Premium Link Generator . This is a website that give free service for anyone to generate rapidshare link from free to premium, so anyone can download with premium feature. <br><br>I...

15 Linux lsof Command Examples (Identify Open Files)

lsof stands for List Open Files. It is easy to remember lsof command if you think of it as "ls + of", where ls stands for list, and of stands for open files. It is a command line utility which is used to list the information about the files that are opened by various processes. In unix, everything is a file, ( pipes, sockets, directories, devices, etc.). So by using lsof, you can get the information about any opened files. 1. Introduction to lsof Simply typing lsof will provide a list of all open files belonging to all active processes. # lsof COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME init 1 root cwd DIR 8,1 4096 2 / init 1 root txt REG 8,1 124704 917562 /sbin/init init 1 root 0u CHR 1,3 0t0 4369 /dev/null init 1 root 1u CHR 1,3 0t0 4369 /dev/null init 1 root 2u CHR ...