Latest Inquiries - Data Extraction Software

trying to grab first search result product as a link. doesn't always work

Submitted: 1/23/2016

hi,

the attached .rip project uses a simple spreadsheet .csv file to pull product codes as a search form keyword.

The resulting search from that keyword pulls up more than one prouct, but the first product is the best match to the keyword being used in the search.


I created a link to only the first product showing on the list results.

Then I link over to that products detail page and create an image element with a regex transformation script to modify the

URL image link to grab the larger version of that image (you showed me how to do that in a prior project).


The issue here, is that it only works for the first two searches and thereafter ignores the product link that chooses just the top product.

So i do not get images after the 2nd image grabbed.


please provide assistance on what might be going on here.  thanks.


ken

stockpart-1-1999.csv
Arli-equipment-photos.rip

Replied: 1/26/2016 2:42:33 AM
What do you mean by 'You might have to do a little scrolling around the screen shot below ..'? I haven't seen any file you attached, I'm unable to understand your issue as well, can you please attach the screenshot for clarify again?
Replied: 1/25/2016 7:54:03 AM

See the attached new project.

The main reason is that xpath of 'itm_numlink10095' link template hasn't been adapted commonly for any results that you do query through, I've revised the xpath, it works through after 2nd queires more.

//TD[startswith(@name, 'itm_numlink')]/A[1]

Then I've added a new 'group' template feeding input data source instead of global one, it will avoid to navigate back re-signing in for each time, 'itm_numlink10095' link template should be working on Javascript + Async. action. 

Arli-equipment-photos.xls
Arli-equipment-photos.rip

Replied: 1/26/2016 3:59:20 PM

I am switching to a new input database having different keywords.  Why is the design time value reverting or staying on the first keyword of the prior table I was using instead of taking the first keyword from the new database table I'm now using?


Replied: 1/26/2016 2:40:48 PM

Ahh.  Thanks for that reference on the topic of "group template'.  Now I know how it works.  I could not figure out why the input field for the search key was NOT receiving more than 2 new keyword (product code) data from the datasource table.  The Group, I guess, contains the iteration (a fancy word for a repeating or recurring process) of database data retrieval into the input search field. 

I am still unclear on the "navigate back" feature.  My question about it is, if a content (not a template) is set to 'navigate back', does this control work from that template, causing the previous template to open?  Or does the control cause the next template in line to revert back to the template that is set to navigate back [to]  ?

Another scenario:  If I by mistake set the log in template to 'navigate back', but set the next 3 templates not to navigate back, will the program flow still jump back to the log in template that was set to 'navigate back' even though the project flow is skipping back two prior templates which had not had the 'navigate back' setting set?

ken



Replied: 1/25/2016 6:42:30 PM

I apologize for my difficulty in implementing your fixed version of my .rip file.  You might have to do a little scrolling around the screen shot below but there is an error message that appears below which I cannot understand.  I copied the spreadsheet into the same relative location as you had saved it while you were working with the project file, and I disabled the global database datasource settings under the Project -> Datasource tab -- i think that's what you instructed me to do.  Other than that, I left alone the Group setting in the Source Template Field that you created.  What do you suppose is the issue here?

Replied: 1/26/2016 4:13:02 PM

Yes. Something happened and keywords are not feeding into the search form field anymore.  I am getting this error report after I quit the process:

Processing Link template itm_numlink10095. No links found

07:07 Processing Group template group. (4 of 606)

07:07 Setting form element keyword (Value: ACFAC100, Text: )

07:07 Processing form template nav-submit-button.

07:07 Switching to new web browser instance.

07:07 Processing Link template itm_numlink10095. No links found

07:07 Processing Group template group. (5 of 606)

07:07 Setting form element keyword (Value: ACFAC100, Text: )

07:07 Processing form template nav-submit-button.

07:07 Switching to new web browser instance.

07:07 Processing Link template itm_numlink10095. No links found

07:07 Starting export...

07:07 Generating export data...

07:07 Error exporting data. The process cannot access the file 'C:\Users\ken\Documents\Visual Web Ripper\Output\Arli-equipment-photos\Arli-equipment-photos.xls' because it is being used by another process.



ACTUALLY, UPON FURTHER INSPECTION, i LOGGED ONTO THE SITE MYSELF AND SEARCHED 'ACFAC100' AND NO PRODUCT WAS FOUND.  IS THIS WHY I'M GETTING THE ERRORS?  PROBABLY.

Replied: 1/27/2016 6:51:03 AM

Ahh.  Thanks for that reference on the topic of "group template'.  Now I know how it works.  I could not figure out why the input field for the search key was NOT receiving more than 2 new keyword (product code) data from the datasource table.  The Group, I guess, contains the iteration (a fancy word for a repeating or recurring process) of database data retrieval into the input search field. 

=====================================================================

This iteration issue wasn't related to form field or 'group' template, I explained "The main reason is that xpath of 'itm_numlink10095' link template hasn't been adapted commonly for any results that you do query through, I've revised the xpath, it works through after 2nd queires more."

I am still unclear on the "navigate back" feature.  My question about it is, if a content (not a template) is set to 'navigate back', does this control work from that template, causing the previous template to open?  Or does the control cause the next template in line to revert back to the template that is set to navigate back [to]  ?

=====================================================================

Content element doesn't have option 'Navigate back' in Misc tab, usually, a template has default 'Navigate back' option, its aim is to auto-navigate back to last level - parent template what you saw page previously. 

Another scenario:  If I by mistake set the log in template to 'navigate back', but set the next 3 templates not to navigate back, will the program flow still jump back to the log in template that was set to 'navigate back' even though the project flow is skipping back two prior templates which had not had the 'navigate back' setting set?

======================================================================

As mentioned above, 'navigate back' option for current temlates aims to its parent template, therefoere, in your case it won't be able to navigate back to 'login' tempalte (i.e, the parent template of  those 3 sub-templates), hope you make sense.

I am switching to a new input database having different keywords.  Why is the design time value reverting or staying on the first keyword of the prior table I was using instead of taking the first keyword from the new database table I'm now using?

=======================================================================

Have you set the new input data source in 'group' template > advanced option > input data ? if you did , that's not possible, it will use new one keyword as my thought.


I've only opened the database table without renaming it and saved new values into it and resaved under same file name. same location.

Additionally, I went back to check the database input settings to make sure it was still pointing to the same database (called codefile.csv).

==============================================================================================

I'm guessing that you want to change the input data source for 'group' template in advanced tab > input data section, after you setup the new input data source , you close the option dialog, then you click 'save' button for 'group' template, then it will actually save the new change.

For your last reply, if there is no results with specific query keyword, VWR won't be able to reach also, if you thought that's really existing a result but VWR doesn't extract it, that should be a config issue .., then please you attach current project and give me the exact sample query where you cannot extract results.

Replied: 1/26/2016 4:05:30 PM

As a matter of fact, the group template and the keyword template are selecting only this design time value, as though it is no longer recognizing the database input table of new values.

I've only opened the database table without renaming it and saved new values into it and resaved under same file name. same location.

Additionally, I went back to check the database input settings to make sure it was still pointing to the same database (called codefile.csv).

It's kind of buggy, how settings get lost and the thing just defaults to some other procedure instead of remember what database it was set to read from and what key search terms it is supposed to obtain from the datafile.

in the Group template, I checked under the advanced tab, the input database setting and verified the codefile.csv file is selected.

What am I missing here now that for no apparent reason, the project is no longer running as it did just a minute ago?  Maybe I'll find the issue.