The problem: I have to compile a list (EXCEL or CSV) of all public purchases of the Brazilian Government from July 2012 to July 2013. This is public info but the process to get all the data manually might take me months. I need to automate it.
The base URL is:
- First step: This URL is already filtering the type of docs I need. However, I can only list docs published 15 days at a time (see the start and end dates in the URL above). So Visual Web Ripper will have to query 15 days at a time, automatically;
- Second step: Each listing of 15 days will redirect us to an unknown number of pages, with 10 items in each page. Visual Web Ripper must click each button labeled "Items and Download", for all 10 items in all pages. There's a button named "Avançar" which means "Next" by the end of the page;
- Third step: For each "Items and download" button we clicked in the previous item, we go to another page, in which we must click the button named "Download" (it's by the end of the page).
- Fourth step: And once we do click this "download" button, a popup asking for a captcha appears. I'd like to send that to DeathByCaptcha.com.
- Last Step: FInally, once we solve the captcha, a download of doc I need starts.
We must do this over and over to get all files, 15 days at a time, from July 2012 to July 2013.
I've been told Visual Web Ripper can easily do that in a snap and am willing to buy it fast (I neeed this data ASAP, my job is on the line).
Can you help me create this project?