Latest Inquiries - Data Extraction Software

Troublesome dynamic site

Submitted: 6/26/2017

This site has two issues I can't seem to get around:

  1. It times out while loading the first template, and I can't select the links interactively, I have to use an xpath.
  2. It errors when trying to load the next page, and I can't figure out how to get the details page to open in a new browser.  I need this because it uses the annoying "scroll for more entries", and I need to get all the entries to scrape.

Can you please look at this one and advise?

Thank you,


Replied: 6/27/2017 7:37:16 AM

Hi Scott,

The only possible way to accomplish it is to use a back template or use the button to go back to the list results and put a scroll link after that template because you have to ensure that all list is loaded again before it executes the page area again.

Or alternatively, you can use the API link,

for page 1,2 and three respectively.

You can get the pagination code by finding the "id="wd-ViewPage" in the web page element.

Best regards,