Latest Inquiries - Data Extraction Software

Agent restart

Submitted: 4/4/2017

Hi, 


I am scraping the data from the website "https://www.11880.com/suche/Pflegedienst/deutschland" . After scraping few pages i.e 34th pages I am getting an error "*** Maximum memory usage reached (1027 MB). Restarting web browser... *** " and the agent is getting restarted. Again the page is starting from page2, page 3, etc. Once it comes to 35th-page the same error pops out and its keep on looping. 

I have attached the log file and RIP file for your reference. Please help me to find the best way to scrape all those 400 pages data. 


Thanks in Advance!

11880.rip
11880_info_17_04_04.log

Replied: 4/7/2017 8:46:14 AM

Hi,

The website leaks a massive amount of memory in IE. You can try in a standard version of IE and you'll have the same problem, so you will not be able to process this website in VWR.

You could try with our premium software Content Grabber (https://contentgrabber.com). Content Grabber uses the Chromium engine which may behave better.

Replied: 4/5/2017 8:26:25 AM

Hi,

Go to Project > Project Options > Advance tab > Browser section and increase the "Max memory usage"

Capture.PNG

Replied: 4/5/2017 8:59:38 AM
I am getting the same issue after increasing the memory.