Latest Inquiries - Data Extraction Software

Template takes too long

Submitted: 12/17/2015
The template we had you build for a particualr site template (ARI Responsive) is taking 4-5 hours when other templates of similar size take 20-30 minutes. What can we do to speed up the process. Attached is an example file.

Replied: 12/18/2015 3:02:23 AM

The website couldn't work in web crawler agent, maybe the product link can work with web crawler agent as you configured, moreover, for speeding up, you can try to disable loading iframe / images / activex / flash in project > project options > browser tab.


Be noted that the web site used responsive design, for matching your templates, so please you run the project on a bigger hidden browser window and setting 'Uses sized window' = true, 'Browser width / height' = 2000 as better (visible window won't apply the width / height option, unless you manually resize the browser window at run-time).

Likewise you should setup a fixed page count for page navigation template - 'Next' , this will avoid performance issue and the 'out of memory' issue if there is existing a huge list of products. attached new project has set to max. 10 , you might try to increase it more (e.g, 100) according to how many you expect to extract.