Latest Inquiries - Data Extraction Software

how to design content grabbing templates after the url feed

Submitted: 5/27/2017

hi,

attached is my recent project, which is making use of the url data feed from data table input.  The idea is to first log in so I have access to the account, and then cycle through the simple .csv table that contains the full URL path to each product keyword for which I'm using to land on the product page.  But this feature is not a template, but just an enabled function of the project itself, so it does not have place holder with respect to the other content templates I want to create after the URL trigger loads from the input data table.


So how do I content elements to grab the data from the page that is produced after the URL loads and brings it up?

It seems like a loop but not sure if it also behaves like a group template, where any content elements designed are after in sequence to the url feed.


please explain this or make a few examples of content elements on the landed pages.


thanks


ken

Arli-PN-Extractor-by-URL.rip
arli-code-test.csv

Replied: 6/18/2017 3:30:03 AM

awww, so you're saying vwr can't be smart enough to get thru this site? i really think the site was modified to fool web scrapers.  i'm probably not the only one to do this on this popular national vendor.


i had content grabber on my pc, only problem is that the license: XGD84-CW5FD-UNZYD-7RXNR-W8AJF   is now expired and i am forced to buy.

how can i reset trial period to give it some time to see if it works out for me?


ken

Replied: 6/21/2017 1:50:19 AM

Hi Ken,

I have extended your existing Content Grabber trial by 2 weeks.

Best regards


Tony

Replied: 6/1/2017 7:46:19 PM

While I await your answer to the above support questions regarding the technique of using start URL's from an input data source, I have attached a simpler project file which uses the more common approach of grabing a produce code from a data table and inputting into the search form to produce the product detail page results for each product code.


However, I'm getting a long URL that is embedding into the search form on the site as well as possibly the product code search keyword.

I went to the Product -> Options dialog box and see a radio button that necessarily selects one of the three features for handling URL's.

A.  Single Start URL (radio button enabled).

B.  List of Start URL's

C.  Feed URL's from Datasource.


I believe I had tried option C before and this was getting me nowhere.  I fail to understand how to handle this feature in a data extraction project.  But why now is this feature still enabled on THIS project when it should not be. 


Please examine the attached .rip file and accompanying test data table and let me know how to DISABLE any URL grabbing.


thanks

ken

arli-sku-lookup.rip
arli-code-test.csv

Replied: 6/2/2017 5:55:18 AM

Hi Ken,

I think there is something wrong with the website's behavior. Let me see if there's something I can do.

Best regards,

Replied: 6/1/2017 1:00:38 PM

Hi,

I am struggling with this and I don't know how to make it work.

Can you please inspect the uploaded .rip file.  The concept of using this type of data table of URL links and the redirect template does not work well with the data extraction elements I've put in there.  I'm not getting the data.


I've reviewed the help documentation on the subject of sing this construct, but it does nto explain program flow and how it is all put togther to work properly.

I hope you can make it work, send it back to me, so that I can see what you did and why I was not able to figure it out.


thanks

ken

You might have to set the database to the short test list of URL's, because the longer data table of URL's causes the program run to take longer initially.


Arli-PN-Extractor-by-URL.rip
arli-code-test.csv

Replied: 5/30/2017 12:35:45 PM

Hi,

Use redirect template and use Get URL from the database.

Best regards,


Arli-PN-Extractor-by-URL.rip
Capture.PNG

Replied: 6/10/2017 3:29:02 AM

Hi,

just checking to see if you were able to manipulate or make better, the last script I sent up to you.  You last said there was something "wrong with the website's behavior".


Your expertise in working this out is much appreciated.


ken

Replied: 6/20/2017 4:12:21 PM

Hi Ken, 

I will ask the sales team if they will allow your trial license to extend the period.



Replied: 6/6/2017 7:07:27 PM

How is it coming?  Is it possible to use an input data table of sku's to access the product detail pages either by:


A.  Using the input sku in the website's search field  -or-

B.  Embedding the sku in a URL that directly goes to the product detail page.


The second option, is what I can't quite figure out because I first need a log in template that goes before the URL so that I have full site access.

But I think when I enable the URL links, it just starts automatically at the get go.


Let me know if you have any solution to what I'm trying to do.


ken

Replied: 6/12/2017 12:59:12 PM

Hi Ken,

I tried several methods, including separating the sign in and feed the link as Start URLs but still not working due to the site's behavior is not that stable. When feeding the URLs after login, the site did not behave as it is. See attached agent.

We suggest using our premium software, Content Grabber. Instead of Internet Explorer, it uses Chromium, an open source web browser from which Google Chrome draws its source code plus it has a lot of advanced features that could work in this particular problem. You can download the trial version and test whether if it works there. You can purchase a license for a one-time payment, or make a subscription if you intended to use it in limited time. If you want to purchase a license, please let me know so we can give you a $100 discount link and I can ask the sales team if they have an available discount.

Best regards,



arli-sku-lookup (1).rip