Skip to main content

Wget: Mirroring Website

WGET is Ultimate Downloader. While you can use it simply to retrieve a single file from a server, it is much more powerful than that and offers many more features.One of its features is mirroring website.

Mirroring simply means downloading the entire contents of a Web site and uploading it somewhere else. So, we can make local copy of a website using wget.

The basic command to mirror website is

$ wget -m http://www.example.com

This command will allow you to download website but it can't make a complete local copy of a website. This command only download the page but the links still point to the real URLs including the images and stylesheets.

We have to add another option to make links not point to real URLs. Add option -k after -m

$ wget -m -k http://www.example.com

Now, the links will point to next page in our local copy website.

To delay your download, you can add option -w and how much time that you want to delay it. Because if we delay the mirroring, it can reduce the load of bandwidth of the site.

$ wget -m -k -w 25 http://www.example.com

From command above, -w 25 means we delay it for 25 seconds between requests. We can replace the number depend on our need. We can add suffix for the number like m for minutes, h for ours and d for days.

Now if you want to download your favorite website you can use wget to do that. This wget can be use on Linux, *Nix and Ms Windows


ref : http://fosswire.com

Popular posts from this blog

How To Setup FTP server on centos 7 ( VSFTP )

From :  http://www.krizna.com/centos/setup-ftp-server-centos-7-vsftp/ FTP server is used to exchange files between computers over network . This guide helps you to setup ftp server on centos 7 . This guide contains configuration steps for both FTP and SFTP as well as user creation . Here i've used VSFTP package which is secure and less vulnerable . 1.  FTP Server 2.  SFTP Server 3.  User creation Setup FTP server on centos 7 Step 1 »  Update your repository and install VSFTPD package . [root@krizna ~]# yum check-update [root@krizna ~]# yum -y install vsftpd Step 2 »  After installation you can find  /etc/vsftpd/vsftpd.conf  file which is the main configuration file for VSFTP. Take a backup copy before making changes . [root@krizna ~]# mv /etc/vsftpd/vsftpd.conf /etc/vsftpd/ vsftpd.conf.org Now open the file and make changes as below [root@krizna ~]# nano /etc/vsftpd/vsftpd.conf Find this line  anonymous_enable=YES  ( Line no : 12 ) and change value to NO to disable anonymous FTP ac...

15 Linux lsof Command Examples (Identify Open Files)

lsof stands for List Open Files. It is easy to remember lsof command if you think of it as "ls + of", where ls stands for list, and of stands for open files. It is a command line utility which is used to list the information about the files that are opened by various processes. In unix, everything is a file, ( pipes, sockets, directories, devices, etc.). So by using lsof, you can get the information about any opened files. 1. Introduction to lsof Simply typing lsof will provide a list of all open files belonging to all active processes. # lsof COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME init 1 root cwd DIR 8,1 4096 2 / init 1 root txt REG 8,1 124704 917562 /sbin/init init 1 root 0u CHR 1,3 0t0 4369 /dev/null init 1 root 1u CHR 1,3 0t0 4369 /dev/null init 1 root 2u CHR ...

Rapidshare Premium Link Generator

Rapidshare Premium LINK Generator . Rapidshare.com is one of the biggest site that allow you to upload and share files with vary sizes. From rapidshare, you also can download files that another person upload the re. There are 2 kinds of user can download from rapidshare. First, free user, this user have limit features from rapidshare. Limit bandwidth, limit connection, etc. Second, premium user. Premium user have many features that free user doesn't have. If you want to be a premium user, you have to buy it from the web. You can use this link to buy premium account or from reseller that near you. But if you have not enough money to buy it, you can use Rapidshare Premium Link Generator . This is a website that give free service for anyone to generate rapidshare link from free to premium, so anyone can download with premium feature. <br><br>I...