Latest Inquiries - Data Extraction Software

Extracting data from dynamically changing javascript dropdown-box

Submitted: 10/30/2017
Dear Visual Web Ripper experts,

I am attempting to extract the resulting data from this web form, in which the user can select 3 input values: product, insurer and year.

I'm running into trouble while creating a template for the product selection, since the list of products is dynamically adjusted (I believe by javascript) when scrolling through it.

Could you provide me with some tips on how to best handle this in Visual Web Ripper?

Many thanks in advance!

-Koen 
Replied: 11/6/2017 11:06:41 PM
Thanks again!

There is just one thing I keep struggling with: the first dropdown-box needs to be scrolled down to load more than 20 elements. I'm trying to figure this out with a javascript scroll function, but I haven't managed yet.. Help is appreciated!

Also, if you have any tips on how to speed up the extraction, that would be very welcome!


171103 KM Consumentenbond v3.rip

Replied: 11/14/2017 7:44:44 AM

Hi Koen,

I created a Link template to click that result and put some Action Events in the Advance tab.

Best regards,

171109 KM Consumentenbond v5.rip
Capture.PNG

Replied: 10/31/2017 2:46:13 PM

Hi Koen,

We can't a normal formfield command since the selection is not an ordinary web form. Ordinary webform uses select and option tags but in this case, the website used div and ul instead. And so we should use the link template instead. See attached agent.

Best regards,

Consumentenbond.rip

Replied: 11/13/2017 5:07:27 AM

Hi Koen,

Is the agent you attached a correct one? It seems it is an agent for TripAdvisor.


Replied: 11/3/2017 7:17:47 AM

Hi,

Please check the attached agent.

Best regards,

Consumentenbond.rip

Replied: 11/10/2017 2:01:33 PM
Ah that's a good idea!

I've almost gotten it to work. The only thing I'm struggling with now is closing the dropdownbox after entering the csv-input.

Any thoughts on that?
171109 KM Tripadvisor rippen v5.rip

Replied: 11/13/2017 8:44:43 AM
My bad! I was working on two projects at the same time.

Apologies, attached the correct agent.
171109 KM Consumentenbond v5.rip

Replied: 11/1/2017 10:16:38 PM
Thanks a lot!

Very impressive and I believe this is exactly what I needed.

However when I'm moving to the second data selection I run into the error 'PageArea JaarArea contained less elements after navigating back to the PageArea. '

I believe this is due to the fact that the dropdownbox is no longer expanded and that we need another 'clickJaar' to load. Presumably this is done in the full page load/AJAX load options. Could you further assist me on this?

Replied: 11/7/2017 5:31:17 AM

Hi,

The best way is to use the number code as search and click the first result. That means collecting the numbers and use them as input in formfield command. We can see the collection here, https://zkv.openstate.eu/static/js/zkv.js

Best regards,