My company is looking to utilize a web scraping service for a project we’re working on, and we’re exploring possible vendors for this service. We were wondering if your service might be able to meet our project’s requirements.
What we are looking to do is web-scrape the FEMA.gov website, the Disaster Declarations section of the website in particular (http://www.fema.gov/disasters).
We want to scrape every hyperlink to the Disaster Declarations, and then scrap information on a 2nd hyperlink within each of those disaster declaration hyperlinks, with the 2nd hyperlink titled “Designated Counties”, to scrape the following disaster information (name, dates, counties designated for Public Assistance, counties designated for Individual Assistance).
The primary challenge we are trying to work through is we don’t want to pull a report of all that information every day.
What we want to pull every day, is only disasters where a change has occurred on the page, that change being primarily new counties being added/designated for FEMA Assistance on the disaster declaration page. We don’t want to pull disasters where there has been no change in the designated counties between yesterday and today.
If possible, we’d like the scrape to produce the change results in Excel format if you have that capability, however if Excel format isn’t possible, we might be able to utilize other formats.