Latest Inquiries - Data Extraction Software

Time consumption and Loading issue

Submitted: 2/4/2016


Time Consumption:

      In my below project,for 1000jobs the scraping is taken for 2 hrs.10000+ jobs are there in site Where that jobs are to be scraped quickly.Now it is taking too time.So i need time consumption for running those jobs.Kindly advise us ASAP.

Loading Issue

     For the same project. since the site is taking too much to load,so the element in the ripper are not taken.The element have to wait until the site is loaded properly.Please advise us ASAP.


Replied: 2/5/2016 6:36:58 AM

You mean that you want to enhance the performance to scrape those job links? 

Linkin website doesn't seem to be able to run in web crawler agent, it needs to perform javascript then loading content properly.

I try to optimize your project as follow:

1) Disable 'Load image / iframes / activex / flash' in Project > Project options > Browser tab.

2) The 'Size' link template seems to exist issue, you should select <a> link rather than <span> in xpath, 

I'm not sure exactly what element you have to wait , what specific issue are you encountering for it ? can you please clarify this issue again?

I guess that's the 'PA_Listing' page area template, if so, you should try to set 'Delay after completed action' = 3000 (or more a little bit) in Project > Project options  > Advanced tab > Action section, this will make sure that start page has been loaded completely.

See the attached new project .