Latest Inquiries - Data Extraction Software

topsy

Submitted: 9/18/2013
I need to scrape the topsy.com historical twitter data.
I'd like to feed the scraper links like these:
...

and
as a result get a table with all of the results, like the one below:

name || twitter handle || Tweet || twitter link

e.g.
mariachong || @mariachong || I gave Congress +K in Stalemate and Ineffectiveness on @klout. #angry || http://twitter.com/mariachong/status/286363150146748416

Kelley. Jo. || @kelleyjo_ || Like not everyone can watch shows when they air so people can stop ruining it! Eff you. #angry || http://twitter.com/kelleyjo_/status/293903842238664706

#JenFact/Hussy©™ || @jiggyjen00 || Can't get my motherfucking kindle to download a motherfucking book. #angry instagr.am/p/UMV23zQULk/ || http://twitter.com/jiggyjen00/status/288353632913616897
Replied: 9/27/2013 6:27:10 PM
I ran the project again, I don't seen any error like yours.

Can you please try to configure proxy in this project, then running this demo project , see if it works?

if not , please you check "View browser" & "Debugging" option when running the project, then attaching the info.log file at here, the log file is placed in default log folder.
Replied: 9/18/2013 5:47:25 PM
Please check the attached demo project.

You need to place the project file in default projects folder, then run this project in VWR program.

If you want to feed more start urls in this project, you can manually add them by yourself, or utilize script to generate them , as I saw, the parameter "offset" in each start url is variable.

F.Y.I:

Feeding start urls
Topsy.rip

Replied: 9/27/2013 12:48:46 AM
hi, for some the scrape doesn't work, attached is the error message.
also, I've purchased the full version of visual web-ripper (if that gets me any priority in the support queue)
112.png