Latest Inquiries - Data Extraction Software

Not enough memory...

Submitted: 3/3/2016

Hello

clickking through the project works perfect. When I start the scraper I got a memory error. The time when this happens is different. Sometimes I get the stop after 80 articles, sometimes after 150 articles. I tried all the browser emulations VWR offers with no difference.

BTW: This happens only with the attached rip. I did over hundred and they all work fine.


Any idea?


11:39 Processing PageNavigation template pagination (page 7)

11:39 Processing PageArea template pageraea2 (1 of 20)

11:39 Processing Link template openlink detail pages (1 of 1). Text: Los 121. URL: javascript:

11:39 Switching to new web browser instance.

11:39 *** Not enough memory (952 MB available). Restarting web browser... ***

11:39 Error activating link: javascript:

11:39 Restarting web browser...

11:39 Waiting for memory leaks to clear...

11:40 *** Not enough memory (1010 MB available). ***

11:40 Starting export...

11:40 Generating export data...

11:40 Data extraction partly completed

Venator-hanstein.de.ABSTURZ - Kopie.rip

Replied: 3/4/2016 2:01:52 AM

Have you read the topic as below link?

http://manual.visualwebripper.com/default.aspx?manual_id=675

To resolve 'Out of memory' issue, the better way is setting 'web crawler agent' for whole project or individual template(s).

For your project , it 's not able to run whole project  or individual templates in web crawler agent, so it will restart browser to clear memory then trying to carry on last broken point, the final link template - 'openlink detail pages' you have set as AJAX action, therefore, it won't be able to carrry on last broken detail page,

For carry on last broken page, 'openlink detail pages' will need to be opened as direct link, attached new project has revised xpath for 'pageraea2' page area template and followiing 'openlink detail pages' link template and setting link transformation regex scripts to get direct link for 'openlink detail pages' link template ,it seems to carry on next detail page when restarting browser.

Attached new project also has just remained 'Execute javascript's option for increasing performance.

Venator-hanstein.de.ABSTURZ - Kopie.rip