Latest Inquiries - Data Extraction Software

techdata auto scrolling browsers

Submitted: 1/5/2016


You already addressed this issue, but I can't seem to find the log file describing how you created a group and scroll link object to cause the product list page to scroll more and show more products before grabbing them.

I was running the VWR project you helped fix and it sort of works.  But I need to change it a little so it will scroll more, like 5 to 10 pages so I can get more than 30 products. 

tried to review your scroll element and check its settings but I don't see how it works.  It just looks like a link that selects the entire page with product list.  Can you explain where I change the setting to make it scroll more times before data extraction on the products begins?


attached is the project file.

Replied: 1/6/2016 2:01:38 AM

Please check the attached new project.

Actually, you don't have to make any new configuration for templates, instead, I only move some templates to certain place such as the 'product list' template should be put underneath 'group' template, so after 'group' template and its sub-template 'scroll' , page navigation template loads more products , then 'product list' template starts to collect through all products.

When you edit 'scroll' link template, 'windowscroll' action has been added in Advanced tab > Action section (at righ hand side), and setting to 'Repeated AJAX calls ' = true, this will ensure that page can be scroll down automatically , moreover, you should be noted that 'Repeated AJAX calls max. count' = 0 by default, that means it will auto-matically scroll page down until there is a failure (no more pages), but sometimes, if there is too much pages being loaded then it will raise a 'out of memory ' issue, in case you should set a fixed max. count, also, you should set a fixed page count with page navigation template for the limitation of pages.

You still can refer to the below topic link for understanding: