pyBot: How far can you go without typing?

Description Launching Usage Download
Description
This is pyBot, a webcrawler. The concept is simple, just enter a domain, and watch as the program finds every domain that site links to and every site those sites link to. So basically it tells you how far you can go by just clicking links. This program is very hard to test. I mean it works to the best of my knowledge, but it might just break for you, I wrote it in one sitting, and thought the idea was kinda cool. I might rewrite it better in the future. But for now, this software is Pre-Alpha, use at your own risk and such. This system was designed to work like this, you give it a page, it finds links to other websites (If the start page is Abeisgreat.com, and it finds a link to Abeisgreat.com/about.php, it will not travel that path, it only goes to new sites) because of this, the results could be different if someone actually replecated this, because they could click on a link to another page on the same site, and get many more results. Pybot was written to do this because it prevents getting trapped in search engines, or other sites that have hundreds of pages of links.

Launching
To launch the pyBot in linux (assuming your using a GUI) go to the folder which the .zip file was downloaded to, extract the contents. Then open the terminal and type
cd /path/to/pyBot/folder
then type
python pybot.py
Please note you need python installed in order to run this script. And if your having troubles look at www.python.org.

Usage
pyBot really does everything for you, except pick a domain, open the script in a text editor and change
urlList = ['http://www.dcemu.co.uk']
to
urlList = ['yourURL']
(obviously replace yourURL with the domain name you want to do) run it and watch it go, once you get a normal BASH prompt again you know the script has finished. If the script hangs up on a site, like if the site's server is down, hit ctrl-c and the script will skip it


Download
Download Version 0.1