Latest Inquiries - Data Extraction Software

Ecommerce site

Submitted: 12/16/2013

Hi,

I need to get online catelogue data from www.flipkart.com. Its an ecommere site. If I can get a project which scraps online catelogue of just 1 category say men clothing or samsung mobile phones or kids clothing etc then I can learn from it to develop my own projects for other categories. Please feel free to chose any one category of the site & build the projet.

Pls reply back if you need any more info

Thanks & Regards,

Mitul

 

Replied: 12/30/2013 3:50:48 AM

Hi Joongki, Thanks for your guidance and the project that you provided (Flipkart.rip). There is something I couldnt follow on how it was done. On the first page when you have selected catgories : Mobiles ,Laptops, Cameras etc.

When I select 1(eg Mobile) then I right click on another one (eg Personal Appliance) & create list then it just selects 3-4 categoris only not all the 13 categories avaiable. From the List menu I can see you have used the Manual (edit XPath) option then I can also see some code on the Set Manual XPath option. Pls guide on how can I select all the categories in the List ?? What do I write in Set Manual XPath ??

Again when you have set the Page Area I can see that the Set Xpath Manually being used & some code has been put.

I want to learn this of using Set Xpath Manually & using small code since I would be using List quite often.

Also is there any offline manual which I can download since at times its tough to browse.

Replied: 12/31/2013 8:59:55 PM

Hi Joongki,

First I would like to thank you for all your support. I could follow your above response, however stuck on a last thing. I have searched all the online manual but couldnt get the answer. Pls help on this one.

 I am attaching a project (Flipkat_TShirts.rip). I have used the Ajax & the windowscroll function. However after 20 Ajax loads the page converts loading into a clickable link instead of auto loading thru Ajax. The link comes as "Show more results". Pls visit the page in any browser & keep scrolling till 20 pages then you can see the Ajax changes to a clickable link "Show more results".

http://www.flipkart.com/mens-clothing/t-shirts/pr?p%5B%5D=facets.type%255B%255D%3DRound%2BNeck&p%5B%5D=sort%3Dpopularity&sid=2oq%2Cs9b%2Cj9y&facetOrder%5B%5D=type&otracker=ch_vn_tshirts_me_filter_Categories_Round%20Neck

 This is where in my project Ajax + windowscroll function stop working & I just get data from 20 loads which came frm AJAX. I think we need to add some template or some other mechanism after the page stops the AJAX & changes to clickable link. Pls guide me on what do we use to load the complete page. You can use the attached (Flipkart_TShirts.rip) project to add the required changes which can help us load the entire content & send me back for my reference.

Thanks & Wishing you a very Happy New Year !!

 

Flipkart_TShirts.rip

Replied: 12/31/2013 9:58:46 PM
Have you already read the topic as follow link that I ever mentioned also?


You got the new project, let me attach it again, 

After you open the TShirt_Cat link template, you will see where's an new element called 'trans' ,you edit the element then go to right side panel > Transformation tab.
Flipkart_TShirts.rip

Replied: 12/17/2013 4:27:03 PM
When you open the project, you can see the templates and contents. You can just click to open templates and access to next page.
This is the sample project and you need to put it into Visual Web Ripper Default folder.
Please check the attached files.

Flipkart.rip
Flipkart.xls

Replied: 12/31/2013 6:30:37 PM
Please check the attached new project.

There is existing a hidden 'next' page link, you can utilize the 'next' link to navigate through all pages instead of ajax scrolling , but you have to first enable the visible of the 'next' link using Page Transformation script.
Flipkart_TShirts.rip

Replied: 12/31/2013 3:39:48 AM

Hi Joongki,

First I would like to thank you for all your support. I could follow your above response, however stuck on a last thing. I have searched all the online manual but couldnt get the answer. Pls help on this one.

 I am attaching a project (Flipkat_TShirts.rip). I have used the Ajax & the windowscroll function. However after 20 Ajax loads the page converts loading into a clickable link instead of auto loading thru Ajax. The link comes as "Show more results". Pls visit the page in any browser & keep scrolling till 20 pages then you can see the Ajax changes to a clickable link "Show more results".

http://www.flipkart.com/mens-clothing/t-shirts/pr?p%5B%5D=facets.type%255B%255D%3DRound%2BNeck&p%5B%5D=sort%3Dpopularity&sid=2oq%2Cs9b%2Cj9y&facetOrder%5B%5D=type&otracker=ch_vn_tshirts_me_filter_Categories_Round%20Neck

 This is where in my project Ajax + windowscroll function stop working & I just get data from 20 loads which came frm AJAX. I think we need to add some template or some other mechanism after the page stops the AJAX & changes to clickable link. Pls guide me on what do we use to load the complete page. You can use the attached (Flipkart_TShirts.rip) project to add the required changes which can help us load the entire content & send me back for my reference.

Thanks & Wishing you a very Happy New Year !!

 

Flipkart_TShirts.rip

Replied: 12/30/2013 5:16:06 PM
When I select 1(eg Mobile) then I right click on another one (eg Personal Appliance) & create list then it just selects 3-4 categoris only not all the 13 categories avaiable.
==============================================
Can you please attach a screenshot to guide me this? I don't seem to see this case.

F.Y.I:

Selection techniques

We don't provide offline manual to download since VWR is often updated and the online manual will be sync. also, that will always give you the best new guidance for the latest VWR.
Replied: 12/31/2013 9:48:53 PM

Hi Simon,

Thanks for replying, however in the new project that you attached (Flipkart_TShirts.rip) , I couldnt find the page transformation script enabled etc. Pls send the example project that can help me understand the concept of Page Transformation. Pls add the Page Transformation which can enable the hidden link which we can use to load entire page instead of using AJAX & windowscroll.

Appreciate all the help. Pls send the new project file with Page Transformation.

Thanks & Regards,

Replied: 12/31/2013 10:50:23 PM

Thanks Simon for your quick response. I have gone through the link that you have given. However the new project that you have attached still does not have a new element 'trans'. I am attaching the screenshots of what I see in the project that you have attached. Its just has templates as Flipkart_TShirts > TShirt_Cat > Scroll_1 > More Results. Under TShirt_Cat I dont get any element as 'trans'.

Pls have a look at the screenshots.

Thanks again for your quick help. Pls chk the project & resend.

Screenshots.docx

Replied: 1/1/2014 2:31:46 PM
Please check the attached screen shot that where you can find out the 'Tran' content and how to make it enables.
TranScrren.png
TranScreenShot.png

Replied: 12/17/2013 6:09:15 AM
Thanks Simon. I could see just the main page of site (www.flipkart.com) in the project. What guidance I am looking was a specific category.
Like all the mobile phone data (image , price , details, link etc) for the mobile category of flipkart site - http://www.flipkart.com/mobiles/samsung~brand/pr?sid=tyy,4io&otracker=ch_vn_mobile_filter_Mobile%20Brands_Samsung

Thanks
Mitul Mehra



Replied: 12/31/2013 10:00:03 PM

Hi Simon,

Thanks for replying, however in the new project that you attached (Flipkart_TShirts.rip) , I couldnt find the page transformation script enabled etc. Pls send the example project that can help me understand the concept of Page Transformation. Pls add the Page Transformation which can enable the hidden link which we can use to load entire page instead of using AJAX & windowscroll.

Appreciate all the help. Pls send the new project file with Page Transformation.

Thanks & Regards,

Replied: 12/31/2013 10:19:23 PM

Thanks Simon for your quick response. I have gone through the link that you have given. However the new project that you have attached still does not have a new element 'trans'. I am attaching the screenshots of what I see in the project that you have attached. Its just has templates as Flipkart_TShirts > TShirt_Cat > Scroll_1 > More Results. Under TShirt_Cat I dont get any element as 'trans'.

Pls have a look at the screenshots.

Thanks again for your quick help. Pls chk the project & resend.

Screenshots.docx

Replied: 1/8/2014 11:36:25 AM
Hi,
I learned to transform the page. However the challenge is that in many sites they dont use the pagination they just use the scroll. Pls find enclosed a project of website www.jabong.com. I am unable to load the full page since it does not use pagination. Pls help on ripping all data for a single category eg: Women Clothing. I tried to rip for a sub-category under Women Clothing but got stuck on how to load the full page.
Pls find enclosed the project.

If you find that the project which I have attached is not upto the mark then pls create a fresh project of ripping entire data for a single category Women Clothing (Elements : Product URL, Image URL, Product Name, Brand Name, Price, Color etc)
Jabong123.rip

Replied: 1/8/2014 4:28:16 PM
I've added a 'scroll' link template to iterate through all pages first, then it calls the page area template to iterate through all products.

F.Y.I:

Content loading on scroll

Jabong123.rip

Replied: 12/31/2013 9:03:09 PM
I'm Simon, have you got my last reply and new project?

Please check the attached new project.

There is existing a hidden 'next' page link, you can utilize the 'next' link to navigate through all pages instead of ajax scrolling , but you have to first enable the visible of the 'next' link using Page Transformation script.

I attach it again.
Flipkart_TShirts.rip

Replied: 12/16/2013 4:50:36 PM
Please check the attached demo project.

You need to put the project file in the default projects folder, then running the project.
Flipkart.rip