Latest Inquiries - Data Extraction Software

Cannot find Link and Page Area after page navigation

Submitted: 7/13/2019
I want to collect online reviews for the hotel in TripAdvisor. The project can successfully collect the first page reviews (first 5 reviews), but after the page navigation it always fails to (1) find the link to expand the review (by clicking the "Read More" link) and (2) find the page areas to collect the defined content elements.

Please find the project in the attachment.

Appreciate your advice. Thank you.

Replied: 7/16/2019 2:32:21 PM
I am sorry that the provided agent does not work for me in that the PageArea template still cannot be found on page 2 onwards.

Also, I want to extract complete review details from each reviewer, do I need to have a link template to click "Read More" links?
I don't understand the purpose of

By the way, does it make any difference to use WebBrower vs. WebCrawler as the default data extractor?

Thank you again.

Replied: 7/16/2019 2:31:25 AM
Hi, sorry for unclear information provided earlier.

I actually want to collect customer review comments from a list of specific properties (i.e. hotels) that I have already known. To be specific, I know which hotels and their URLs in Tripadvisor website.

For example, for the Westin Austin Downtown (, I can extract the hotel basic information (name, address, telephone number) as well as the first 5 customer reviews (including the reviewer's name, rating, detail comments, date of stay etc), but my problem is how to extract all reviews after the first page of 5 reviews.

I have set up a page navigation template with "Next page link". From page 2 onwards, however, my project always reports an error that "it cannot find the page areas and links that I set up as the templates).

Attached is the project I created.

Look forward to your advice and help. Thanks.


Replied: 7/15/2019 3:29:20 PM


Pardon me, I'm actually confused about what you are trying to achieve. To make it clear, do you want to get the first 5 results, then click the property page (in all first 5) and get the data? Also, can I know what data you want to extract in the property page?

Best regards,

Replied: 7/16/2019 10:00:28 AM


Please try the attached agent. I have moved the PageArea template to first to select all the "Reviews" available. And then made the Page Navigation which navigates to the "next pages".

Best Regards,


Replied: 7/17/2019 8:24:39 AM

Please check the attached agent. I made a link template to click all the Reviews and grab from it.

And Webcrawler mode doesn't hold good for all the Web pages and has few limitations as it doesn't execute javascript and doesn't display the correct result in such cases. For more information regarding Web Crawler mode, please check out the manual on "Web Crawler Agent".

Find the data Excel.

Best Regards,

Hotel Review_20190717_124739.xml.xml