Welcome to the Invelos forums. Please read the forum rules before posting.

Read access to our public forums is open to everyone. To post messages, a free registration is required.

If you have an Invelos account, sign in to post.

    Invelos Forums->DVD Profiler: Desktop Technical Support Page: 1  Previous   Next
A way to data mine web site for profile data ?
Author Message
DVD Profiler Unlimited RegistrantSparkss
Registered: May 27, 2009
Posts: 50
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Is there a way to data mine/scrape the data from a "known" web site, like imdb, blu-ray or amazon (all .com) ? I am finding myself manually entering quite alot of profiles for our DVDs and having to cut and paste each individual element is getting old fast. Even if there was a way to parse and pull in the actors, runtime, etc.. the basic data, that would be great. I have looked at the plugins and poked around the program quite a bit, but just am not seeing anything that will do what I need. I have less than 150 out of the 800 discs we have (combination BD and DVD) and already I am frustrated . Any assistance or direction would be greatly appreciated.
DVD Profiler Unlimited RegistrantStar ContributorLewis_Prothero
Strength Through Unity
Registered: May 19, 2007
Reputation: Superior Rating
Germany Posts: 6,730
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
None that I would be aware of, and this is for a very good (legal!) reason.
Most of this "Third Party"-material has copyrights (especially data form IMDb and Amazon).
Another reason is that these sources a known to be inaccurate.
The ultimate valid source is still the DVD itself. All the data needed for a profile can be found on it.

Usually (if entered via UPC / EAN) you should get quite pleasing results for your collection from our database too. Just go to DVDProfiler menu "DVD -> Add to Collection... -> Add by UPC". Except for some rarities or very new releases you will find that most profiles are already there with a (mostly) high accuracy in the quality of the data.

EDIT: If entering the UPCs manually takes too long (800 DVDs will usually take about 5 hours of typing an downloadtime) you might want to consider a barcode scanner.
It all seems so stupid, it makes me want to give up!
But why should I give up, when it all seems so stupid?


Registrant since 05/22/2003
 Last edited: by Lewis_Prothero
DVD Profiler Unlimited RegistrantStar Contributortweeter
I aim to misbehave
Registered: June 12, 2007
Reputation: High Rating
United States Posts: 2,665
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
You mentioned in another thread your DVD boxes are stored in the attic.  It might be worth pulling them down to enter the UPCs.  If not then do what the goblin suggests except make it "DVD -> Add to Collection... -> Add by Title".

I don't know where you live (region/locality) but you should find a few options for each film, regardless (said the potentially naive guy living in the U.S.).  More tedious than adding by UPC but will eliminate a lot of the need to go hunting for data.
Bad movie?  You're soaking in it!
DVD Profiler Unlimited RegistrantStar ContributorWinston Smith
Don't be discommodious
Registered: March 13, 2007
United States Posts: 21,610
Posted:
PM this userEmail this userView this user's DVD collectionDirect link to this postReply with quote
Quoting Sparkss:
Quote:
Is there a way to data mine/scrape the data from a "known" web site, like imdb, blu-ray or amazon (all .com) ? I am finding myself manually entering quite alot of profiles for our DVDs and having to cut and paste each individual element is getting old fast. Even if there was a way to parse and pull in the actors, runtime, etc.. the basic data, that would be great. I have looked at the plugins and poked around the program quite a bit, but just am not seeing anything that will do what I need. I have less than 150 out of the 800 discs we have (combination BD and DVD) and already I am frustrated . Any assistance or direction would be greatly appreciated.

Well, Sparkss. Number one the vast majority of websites, particularly IMDb don't permit data scraping, it represents a violation of their Copyright. The second point is are you referring to your own local data or are yyou wishing to contribute. IF you wish to contribute read and follow the Rules, which means no data scraping from ANYWHERE and no data from elsewhere. our rules spell out where we are to get the data from for Contribution purposes. Yes it is time consuming to do it correctly, it took me three years to get through my collection. But and again I stress that for Contribution purposes there are no useful shortcuts.

Skip
ASSUME NOTHING!!!!!!
CBE, MBE, MoA and proud of it.
Outta here

Billy Video
DVD Profiler Unlimited RegistrantW0m6at
You're in for it now Tony
Registered: April 17, 2007
Australia Posts: 1,091
Posted:
PM this userVisit this user's homepageView this user's DVD collectionDirect link to this postReply with quote
Quoting Dr Pavlov:
Quote:
But and again I stress that for Contribution purposes there are no useful shortcuts.
Not true. Cloning profiles and confirming them against the credits is a shortcut, particularly if you find the credits are the same.
Adelaide Movie Buffs (info on special screenings, contests, bargains, etc. relevant to Adelaideans... and contests/bargains for other Aussies too!)
DVD Profiler Desktop and Mobile RegistrantStar ContributorTaro
Registered: February 23, 2009
Reputation: High Rating
Belgium Posts: 1,580
Posted:
PM this userView this user's DVD collectionDirect link to this postReply with quote
Quoting W0m6at:
Quote:
Quoting Dr Pavlov:
Quote:
But and again I stress that for Contribution purposes there are no useful shortcuts.
Not true. Cloning profiles and confirming them against the credits is a shortcut, particularly if you find the credits are the same.

Agreed. I do it often with Blu-ray submission where I take the DVD data, check if the credits remained unchanged and copy-paste. It saves the time of having to re-enter the same data. I do the same also for Blu's from a different regionality, and adapt the credits where necessary (if necessary).
Blu-ray collection
DVD collection
My Games
My Trophies
    Invelos Forums->DVD Profiler: Desktop Technical Support Page: 1  Previous   Next