Skip to main content

Wget: Mirroring Website

WGET is Ultimate Downloader. While you can use it simply to retrieve a single file from a server, it is much more powerful than that and offers many more features.One of its features is mirroring website.

Mirroring simply means downloading the entire contents of a Web site and uploading it somewhere else. So, we can make local copy of a website using wget.

The basic command to mirror website is

$ wget -m http://www.example.com

This command will allow you to download website but it can't make a complete local copy of a website. This command only download the page but the links still point to the real URLs including the images and stylesheets.

We have to add another option to make links not point to real URLs. Add option -k after -m

$ wget -m -k http://www.example.com

Now, the links will point to next page in our local copy website.

To delay your download, you can add option -w and how much time that you want to delay it. Because if we delay the mirroring, it can reduce the load of bandwidth of the site.

$ wget -m -k -w 25 http://www.example.com

From command above, -w 25 means we delay it for 25 seconds between requests. We can replace the number depend on our need. We can add suffix for the number like m for minutes, h for ours and d for days.

Now if you want to download your favorite website you can use wget to do that. This wget can be use on Linux, *Nix and Ms Windows


ref : http://fosswire.com

Popular posts from this blog

How To Setup FTP server on centos 7 ( VSFTP )

From :  http://www.krizna.com/centos/setup-ftp-server-centos-7-vsftp/ FTP server is used to exchange files between computers over network . This guide helps you to setup ftp server on centos 7 . This guide contains configuration steps for both FTP and SFTP as well as user creation . Here i've used VSFTP package which is secure and less vulnerable . 1.  FTP Server 2.  SFTP Server 3.  User creation Setup FTP server on centos 7 Step 1 »  Update your repository and install VSFTPD package . [root@krizna ~]# yum check-update [root@krizna ~]# yum -y install vsftpd Step 2 »  After installation you can find  /etc/vsftpd/vsftpd.conf  file which is the main configuration file for VSFTP. Take a backup copy before making changes . [root@krizna ~]# mv /etc/vsftpd/vsftpd.conf /etc/vsftpd/ vsftpd.conf.org Now open the file and make changes as below [root@krizna ~]# nano /etc/vsftpd/vsftpd.conf Find this line  anonymous_enable=YES  ( Line no : 12 ) and change value to NO to disable anonymous FTP ac...

15 Linux lsof Command Examples (Identify Open Files)

lsof stands for List Open Files. It is easy to remember lsof command if you think of it as "ls + of", where ls stands for list, and of stands for open files. It is a command line utility which is used to list the information about the files that are opened by various processes. In unix, everything is a file, ( pipes, sockets, directories, devices, etc.). So by using lsof, you can get the information about any opened files. 1. Introduction to lsof Simply typing lsof will provide a list of all open files belonging to all active processes. # lsof COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME init 1 root cwd DIR 8,1 4096 2 / init 1 root txt REG 8,1 124704 917562 /sbin/init init 1 root 0u CHR 1,3 0t0 4369 /dev/null init 1 root 1u CHR 1,3 0t0 4369 /dev/null init 1 root 2u CHR ...

Privacy Policy

Another Pratama Privacy Statement What follows is the Privacy Statement for all Another Pratama websites (a.k.a. blogs) including all the websites run under the computer.pratama.us domain. Please read this statement regarding our blogs. If you have questions please ask us via our contact form. Email Addresses You may choose to add your email address to our contact list via the forms on our websites. We agree that we will never share you email with any third party and that we will remove your email at your request. We don’t currently send advertising via email, but in the future our email may contain advertisements and we may send dedicated email messages from our advertisers without revealing your email addresses to them. If you have any problem removing your email address please contact us via our contact form. Ownership of Information Another Pratama is the sole owner of any information collected on our websites. Comments/Message Boards Most Another Pratama websites contain com...