When you visit a website with a web browser or a web scraping tool, such as Visual Web Ripper, the website owner can record your IP address and may be able to use this information to identify you or block your access to the website.
If you do not want a website owner to be able to identify you while you visit a website, you can use a proxy server to hide your IP address. When you use a proxy server, you do not visit the target website directly, but instead request that the proxy server visit the website for you.
There are many different types of proxy server, but Visual Web Ripper supports only HTTP proxy servers. It does not support other types of proxy servers, such as SOCKS proxies.
Many companies specialize in providing proxy server access for a fee. Proxy servers are also freely available on the web, but they are often slow and unreliable.
We offer a FREE account at Private Proxy Switch for all our customers and trial users. The free account includes 500MB of traffic per month and you can upgrade your account at anytime if you need more traffic.
Private Proxy Switch is a high performance proxy server with a large pool of IP addresses. An IP address is randomly assigned to you when you make a request for a new webpage. This makes your computer completely hidden and makes it very difficult to detect and block your web scraping activity.
Private Proxy Switch rotates 20 proxies every 12 hours. If you get the 20 proxies blocked, you will need to wait up to 12 hours and try again. Please care for the proxies and set random page load delays when at all possible. Treat them well and they will treat you well.
Follow these steps to set up your free Private Proxy Switch account:
After you have purchased proxy server access or found freely available proxy servers on the web, you will receive one or more proxy server IP addresses and possibly a username and password to access the proxies. You need to enter this information into the data extraction project by opening the Project Options screen and selecting the Proxies tab.
This screen allows you select the Proxy Source and Proxy Rotation settings. You can select one of the following proxy sources:
The Proxy rotation option specifies how to rotate between multiple proxies while extracting data.
The Proxy List screen is used to specify a list of proxies and set the proxy verification options.
The Proxy Address & Port must be specified in the following format:
In the example above, 126.96.36.199 is the IP address of the proxy server and 60099 is the port number.
If you are using a large number of proxy servers, it can be tedious to add them all to each project. You can use the Import Proxies button to import a list of proxies from a CSV file. The CSV file must have the following format:
Proxy Address, Username, Password
Username and Password are optional columns.
CSV Example 1:
CSV Example 2:
proxy, username, password
188.8.131.52:8800, user1, pass1
184.108.40.206:8800, user1, pass1
220.127.116.11:8800, user1, pass1
TOR is a free network of proxies that you can use to hide your IP address. This network works well with web scraping, because it switches proxies automatically approximately every 15 minutes. A data extraction project can run for hours and use many different proxy servers automatically. Unfortunately, the TOR network is generally very slow and not suitable for professional use.
To use TOR with Visual Web Ripper, you need to install the TOR software bundle. You can find it at www.torproject.org. When you install the software bundle, make sure you choose to install TOR, Polipo and Vidalia.
Now you can connect to the TOR network by running Vidalia. After connecting to the TOR network, you can configure Visual Web Ripper to use the TOR network by using this proxy address:
You can also configure Internet Explorer to use the TOR network by setting the proxy server to localhost:8118. If you have already configured Internet Explorer to use TOR, Visual Web Ripper will use the TOR network in WebBrowser mode automatically.