Latest Inquiries - Data Extraction Software

How to add function that changes page number in the link

Submitted: 6/20/2017

Hello,


I would like Visual Web Ripper to navigate to the next link in the project by editing the link each time and changing the page number to the next page number.


For example: https://www.bbb.org/en/us/search?inputText=Electricians+-+Residential&locationLatLng=36.135923%2C-115.213814&locationText=las+vegas&page=4

How do I set up the Page Navigation link so that it navigates to the next page like that each time? I would then be able to use WebCrawler to quickly extract data.


Thank you,


Alex

test 2 06192017.rip

Replied: 6/22/2017 7:10:22 AM

Hi Alex,

I created the script in the link transformation in page navigation, however, the website doesn't allow viewing it in web crawler mode as you need javascript to view the data. You need web browser mode to scrape the data.

Best regards,



test 2 06192017.rip