Latest Inquiries - Data Extraction Software

Problem Running Multiple Instances Of The Same Project

Submitted: 4/11/2017
Hi,
I'm trying to run the attached project in multiple instances on the same PC.
All instances gets fired up however the data from all instances gets mixed up.

I used session Id as this article suggests:
http://manual.visualwebripper.com/default.aspx?manual_id=115053

each instance should add a product to the shopping cart and extract it's price and shipping info. Then the product is removed from cart and the agent continues to the next product on the list
However, when running multiple instances the products in the cart gets all mixed up with product from another instance.
Why is it happenning and how can i fix it?

Thanks,
Matan


UpdateShippingDetails-iHerb.rip

Replied: 5/9/2017 3:38:39 PM

Hi,

I am replicating your problem on our side. I will give you a feedback shortly.

Best regards,


Replied: 5/22/2017 3:24:36 PM

Hi,

Sorry for late reply. I sent your request to the development team. I will give you an update once they response.

Best regards

Replied: 5/17/2017 2:08:40 PM
Hi,
Any progress?


Replied: 4/13/2017 7:49:19 PM
I followe exctly what the document says. I activated the instances through the api and set a unique GUID as a session id for each instance
with the following code:

project.SessionId = MyGUID;

I'm attaching 3 xml outputs. as you can see in the "itemsInCart" attribute, each time I have more than 1 product in the cart.  when I look at the browser while the agents are running I see products from different instances in the smae cart. 
UpdateShippingDetails-iHerb_e3c0fa46-942d-4c38-ba2d-42316f7dd029.xml
UpdateShippingDetails-iHerb_948a2b9b-f5de-4a3b-b337-c4d18dabf4bd.xml
UpdateShippingDetails-iHerb_96f4f249-bf16-409f-ad04-bfc47910dc51.xml

Replied: 5/22/2017 3:57:47 PM

Hi,

We found the problem. This is something to do with the cart. Session data applies only to the internal database working VWR. When 4 separate agents running on the same time, they add products to the cart. So when it tries to visit the cart page, it had all the four products. But because 4 agents is adding and deleting the products, a mixed up will happen: the first agent add the product, the second agent delete the product on the cart after the fist agent add it, then the third one add the other products, so when the first agent go to the cart, it grabs the product data added by the third agent. This happens before agent clear its cookies. 

Best regards,


Replied: 5/21/2017 11:04:09 AM
Hi,
It's been 5 weeks since I opened this bug. I need to get this resolved, please make the check on your end and get back to me.
Replied: 4/24/2017 3:54:04 PM
Hi, I tried running the project from console but I get the following response:

"RunProject.exe is not recognized as an internal or exteranal command..."

Am I doing something wrong?
I also reinstalled VWR 
Capture.PNG

Replied: 4/13/2017 4:32:24 PM
I followe exctly what the document says. I activated the instances through the api and set a unique GUID as a session id for each instance
with the following code:

project.SessionId = MyGUID;

I'm attaching 3 xml outputs. as you can see in the "itemsInCart" attribute, each time I have more than 1 product in the cart.  when I look at the browser while the agents are running I see products from different instances in the smae cart. 
UpdateShippingDetails-iHerb_96f4f249-bf16-409f-ad04-bfc47910dc51.xml
UpdateShippingDetails-iHerb_948a2b9b-f5de-4a3b-b337-c4d18dabf4bd.xml
UpdateShippingDetails-iHerb_e3c0fa46-942d-4c38-ba2d-42316f7dd029.xml

Replied: 4/19/2017 4:30:52 PM
Sure,
I will be able to update you with the results after the weekend
Replied: 4/18/2017 3:07:50 PM

Hi,

Try to do it using the command line to run multiple agents so we can determine if the problem is in the agent or in your MyGUID. 

Replied: 5/1/2017 2:44:20 AM

Hi,

Try to navigate first to the Visual Web Ripper Program directory

See screenshot.

Capture2.PNG

Replied: 5/1/2017 9:50:11 AM
Thanks,
I ran the robot in 5 different threads in the console. 
Each with a different session id.
The threads are affected from one another.
Adding a product to the cart in one thread affects the number of product in the cart in all other threads.
Also  removing a product from the cart affects other threads
You can follow the "ItemsInCart" element in all outputs. it should allways be "1" but somtimes it's "2" or null.
UpdateShippingDetails-iHerb_4.xml
UpdateShippingDetails-iHerb_2.xml
UpdateShippingDetails-iHerb_5.xml
UpdateShippingDetails-iHerb_1.xml

Replied: 4/30/2017 10:53:43 AM
??
Replied: 5/24/2017 3:19:13 PM

Hi,

Unfortunately, this is not possible because there is no mechanism in VWR to separate the web browser session as this is not supported in Internet Explorer. VWR uses Internet explorer as web browser emulation in scraping. If you badly need to scrape this data, I suggest using our premium scraping software, the Content Grabber. It uses Chromium, an open source web browser from which Google Chrome draws it source code and there is no difference between the two in terms of browsing functions. 

Content Grabber has a separate web browser session when you run multiple instances of the same agent that fits your need. It is advance scraping software compared to VWR. You can download the trial of Content Grabber here to test it out. 

Best regards,

Replied: 4/13/2017 8:21:27 AM

Hi Matan,

Have you followed exactly what it says in the manual like supplying the session id and not using two IDs at the same time? Can you attached your export data here both XML and excel if possible so we can examine them.

Best regards,

Replied: 5/22/2017 5:53:08 PM
Thanks Brian,
I already figured out this was the case.
The question is what can I try to bypass this issue?
Any ideas?
Replied: 5/9/2017 10:13:13 AM
??