Have you ever use or heard W-get? If you are not, you missing a great free software. GNU W-get is a software for retrieving files using HTTP, HTTPS and FTP, the most widely-used Internet protocols.
W-get is a non-interactive command line tools without user interface. W-get on the first hand was dedicated as download tool for Unix and Linux platform, because of its popularity, W-get has been ported to many other operating system including Windows and Mac OS.
Some of its main feature is recursive download, conversion links for offline viewing of HTML, proxy support and resume download support. Here is w-get full feature list taken directly from GNU website.
- Can resume aborted downloads, using REST and RANGE
- Can use filename wild cards and recursively mirror directories
- NLS-based message files for many different languages
- Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
- Runs on most UNIX-like operating systems as well as Microsoft Windows
- Supports HTTP and SOCKS proxies
- Supports HTTP cookies
- Supports persistent HTTP connections
- Unattended / background operation
- Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring
As for me, my main use is to make local copy of website for offline viewing and recursive download from no index website folder. Check out my example how to use w-get.
PS: If you want to know why I write w-get instead of normal word without the dash, read my next post 
