I'm having a problem navigating the site I'm trying to scrape. I managed to create a list of all countries in all continents but there are differences in the structure of each country:
Example 1: http://www.wannasurf.com/spot/Africa/Algeria/
- This is a country directly with a list of spots to scrape from.(which is not problematic)
Example 2: http://www.wannasurf.com/spot/North_America/USA/
- Here we have a list of zones to select from.(which is not problematic too)
Example 3: http://www.wannasurf.com/spot/North_America/USA/California/index.html
- Here we have a list of Zones and a list of Spots and the way I have created the lists VWR doesn't make any difference between the 2 lists. It opens the spots with the Zones List, which is unacceptable for me.
Please give me some instructions how to repair this. I want the "Spot list" to select only Spots and the "Zones list" to select only Zones.
Thank You in advance.
PS: Is there a way for me to navigate ,to a link that I select, from a list already created and not the first link in the list?