Latest Inquiries - Data Extraction Software

more results

Submitted: 8/7/2017

Hey,

i have no idea how to extract all object (7362) out of this website http://i.imgur.com/CZzmKNF.png inkl. the tour links.

I tried windowscroll, pagenavigation in pagenavigation but all without any result and get max. 25 objects without links.

Any idea how to solve?


regrads

Dieter

Outdooractive.rip

Replied: 8/8/2017 1:52:41 PM

Are you using the latest version of VWR?

Replied: 8/8/2017 7:45:34 AM

Hi Brain,

thank you but i got an error loading agent http://i.imgur.com/DK1ODBF.png

Regarding Tour url: idea to get them out?


best regards

Dieter

Replied: 8/9/2017 3:57:09 AM

Try attached agent. It works well on me.


Outdooractive-V2.rip
Capture.JPG

Replied: 8/9/2017 11:26:49 AM

Hi Brain, deinstalled vwr and install from scretch new - works without errors ;-)

1. How can i reduce/optimize the memory consumption? http://i.imgur.com/aVheLkb.png without limiting total results?

2. unlimited run - i tried this start url https://www.outdooractive.com/de/touren/#area=1039950&bm=oa_map&cat=*&wt=Baden-W%C3%BCrttemberg%20(Bundesland)&zc=7,9.0033,48.62565 with 2992 results http://i.imgur.com/pV9bQ9N.png  but the agent newer ends. Idea why and how to limit maybe manually?

3. One issue - i found no way to get out the tour links http://i.imgur.com/h8Yh4vf.png likes this https://www.outdooractive.com/de/r/3329180

best regards

Dieter



Replied: 8/11/2017 6:20:55 AM

Hi Dieter,

  1. I already set the Maximum Memory for the agent (4GB) because VWR is a 32-bit software and it can only utilize that limited memory. Try to divide the search and reducing the number of results. Or if you badly need it, you can use Content Grabber scraper. It is built in 64-bit system and it can use more than 4GB or unlimited memory, depending on the capacity of your machine. You can download a trial and test it or you can request a free agent demo.
  2. In "more" link template, I set the XPath to ignore the More result button when it disappears from the page using this, //DIV[@class='oax_pointer icons-furtherResults oax_marg_bottom_30 oax-list-further-btn'][@style!="display: none;"]/SPAN[1] and so if it is not stopped, try to run it with view browser and see what causing the problem. Most likely, when the agent clicks the More result button, the website did not load the results and make the button frozen
  3. I reselect the page area to capture the links you want, see attached,

Best regards,


Outdooractive-V2.rip

Replied: 8/8/2017 2:35:48 PM

Yes latest Version http://i.imgur.com/9GwBNaa.png

Replied: 8/21/2017 5:32:20 AM

Hi,

Glad to know it works. Can you give me the agents with that problem?

Best regards,


Replied: 8/8/2017 3:24:55 AM

Hi Dieter,

Try repeat parent. 

Best regards,


Outdooractive.rip

Replied: 8/15/2017 2:18:18 PM

Hi Brain,


works great - thank your for your efforts!

One small problem using list http://i.imgur.com/k2BYAcg.png , now way to get them run, after first entry all other ignored.

Best regards

Dieter