Author Topic: Problems with the IMDB People plugin  (Read 16642 times)

0 Members and 4 Guests are viewing this topic.

Offline ronaldomm

  • Member
  • *
  • Posts: 11
    • View Profile
Problems with the IMDB People plugin
« on: November 08, 2009, 05:13:35 pm »
The layout of IMDB was changed again. The filmography hasn´t been imported by IMDB People plugin. Need to update. Tks

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: Problems with the IMDB People plugin
« Reply #1 on: November 08, 2009, 06:02:24 pm »
Thx for the info, i will take a look at it
Gentlemen, you can’t fight in here! This is the War Room!

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #2 on: November 08, 2009, 07:11:20 pm »
Quote
The filmography hasn´t been imported by IMDB People plugin.

It seems to be working fine for me. Can you be more specific about what's happening? Post a log if it shows anything meaningful.

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #3 on: November 10, 2009, 06:29:47 am »
Now I'm getting the same behaviour. No filmography is downloaded, and if Merge filmography is not set, the existing filmography is deleted. FWIW, a log...

Code: [Select]
(11/09/2009 09:16:03 PM) POST: http://www.imdb.com/register/login?
(11/09/2009 09:16:03 PM) Redirected to: https://secure.imdb.com/register-imdb/login
(11/09/2009 09:16:03 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:03 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:03 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:03 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:03 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:04 PM) [1 of 6] Searching person information for: A. Bonzzoni
(11/09/2009 09:16:04 PM) GET: http://imdb.com/name/nm0095307
(11/09/2009 09:16:04 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0095307
(11/09/2009 09:16:04 PM) Redirected to: http://www.imdb.com/name/nm0095307
(11/09/2009 09:16:04 PM) Redirected to: http://www.imdb.com/name/nm0095307/
(11/09/2009 09:16:06 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:06 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:06 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:06 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:06 PM) Person -> LoadImages -> 0ms
(11/09/2009 09:16:06 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:06 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:06 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:06 PM) Person -> LoadAwards -> 0ms
(11/09/2009 09:16:06 PM) Person -> LoadImages -> 0ms
(11/09/2009 09:16:06 PM) [2 of 6] Searching person information for: Aldo Sambrell
(11/09/2009 09:16:06 PM) GET: http://imdb.com/name/nm0759757
(11/09/2009 09:16:07 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0759757
(11/09/2009 09:16:07 PM) Redirected to: http://www.imdb.com/name/nm0759757
(11/09/2009 09:16:07 PM) Redirected to: http://www.imdb.com/name/nm0759757/
(11/09/2009 09:16:11 PM) Person -> LoadStatic -> 0ms
(11/09/2009 09:16:11 PM) Person -> LoadMultivalues -> 0ms
(11/09/2009 09:16:11 PM) Person -> LoadFilms -> 10ms
(11/09/2009 09:16:11 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:11 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:11 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:11 PM) Person -> LoadMultivalues -> 0ms
(11/09/2009 09:16:11 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:12 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:12 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:12 PM) [3 of 6] Searching person information for: Alfredo Fraile
(11/09/2009 09:16:12 PM) GET: http://imdb.com/name/nm0005708
(11/09/2009 09:16:12 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0005708
(11/09/2009 09:16:13 PM) Redirected to: http://www.imdb.com/name/nm0005708
(11/09/2009 09:16:13 PM) Redirected to: http://www.imdb.com/name/nm0005708/
(11/09/2009 09:16:17 PM) Person -> LoadStatic -> 0ms
(11/09/2009 09:16:17 PM) Person -> LoadMultivalues -> 0ms
(11/09/2009 09:16:17 PM) Person -> LoadFilms -> 10ms
(11/09/2009 09:16:17 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:17 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:17 PM) Person -> LoadStatic -> 0ms
(11/09/2009 09:16:17 PM) Person -> LoadMultivalues -> 0ms
(11/09/2009 09:16:17 PM) Person -> LoadFilms -> 10ms
(11/09/2009 09:16:17 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:17 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:17 PM) [4 of 6] Searching person information for: Antonio Prieto
(11/09/2009 09:16:17 PM) GET: http://imdb.com/name/nm0697391
(11/09/2009 09:16:18 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0697391
(11/09/2009 09:16:18 PM) Redirected to: http://www.imdb.com/name/nm0697391
(11/09/2009 09:16:19 PM) Redirected to: http://www.imdb.com/name/nm0697391/
(11/09/2009 09:16:21 PM) Person -> LoadStatic -> 0ms
(11/09/2009 09:16:21 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadFilms -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:21 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:22 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:22 PM) Person -> LoadImages -> 0ms
(11/09/2009 09:16:22 PM) [5 of 6] Searching person information for: Arrigo Colombo
(11/09/2009 09:16:22 PM) GET: http://imdb.com/name/nm0173031
(11/09/2009 09:16:22 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0173031
(11/09/2009 09:16:22 PM) Redirected to: http://www.imdb.com/name/nm0173031
(11/09/2009 09:16:22 PM) Redirected to: http://www.imdb.com/name/nm0173031/
(11/09/2009 09:16:24 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:24 PM) Person -> LoadAwards -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadStatic -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadMultivalues -> 10ms
(11/09/2009 09:16:24 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:24 PM) Person -> LoadAwards -> 0ms
(11/09/2009 09:16:25 PM) Person -> LoadImages -> 10ms
(11/09/2009 09:16:25 PM) [6 of 6] Searching person information for: Arturo González
(11/09/2009 09:16:25 PM) GET: http://imdb.com/name/nm0328062
(11/09/2009 09:16:25 PM) Redirected to: http://www.imdb.com/rg/parked-domains/imdb.com/name/nm0328062
(11/09/2009 09:16:25 PM) Redirected to: http://www.imdb.com/name/nm0328062
(11/09/2009 09:16:26 PM) Redirected to: http://www.imdb.com/name/nm0328062/
(11/09/2009 09:16:28 PM) Person -> LoadStatic -> 0ms
(11/09/2009 09:16:28 PM) Person -> LoadMultivalues -> 0ms
(11/09/2009 09:16:28 PM) Person -> LoadFilms -> 0ms
(11/09/2009 09:16:28 PM) Person -> LoadAwards -> 40ms
(11/09/2009 09:16:28 PM) Person -> LoadImages -> 11ms

Offline AimHere

  • Older Power User
  • *****
  • Posts: 213
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #4 on: November 10, 2009, 09:52:48 pm »
Heh, I just noticed the IMDB page layout changes myself. Definitely breaks the Filmography import. Damn website designers, always changing things!

Aimhere

Edit: Well, this is wierd... sometimes it works, sometimes it doesn't. Sometimes it works on the first try for a given actor, sometimes I have to try two or three times, or do an import for a different actor before returning to the first one...
« Last Edit: November 10, 2009, 10:22:09 pm by AimHere »

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: Problems with the IMDB People plugin
« Reply #5 on: November 11, 2009, 03:11:41 am »
Please try this updated plugin http://www.videodb.info/bin/plugins/imdb_people.zip

P.S. I will update the download page a bit later
Gentlemen, you can’t fight in here! This is the War Room!

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #6 on: November 11, 2009, 05:52:45 am »
A few problems...

1. Updating http://imdb.com/name/nm0520859 changes his name to "<small>Filmography by Genre for </small><a class="main" href="/name/nm0520859/">Livio Lorenzon</a>" and adds http://www.imdb.com/name/nm0520859/filmogenre to the URL. I've only seen this once in limited testing, so I see no pattern.

2. Character names in filmographies that are linked to character pages are not handled correctly. The first part of the link (<a onclick=") instead of the character name is being retrieved.

3. Last modified is not updated for all kinds of changes. Changing a photo or updating a filmography, for example, does not update the time.

Offline AimHere

  • Older Power User
  • *****
  • Posts: 213
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #7 on: November 12, 2009, 02:32:09 am »
Hi again,

Since the IMDB changes that broke the IMDB People plugin, I've noticed other odd behavior from the plugin besides that which has been mentioned previously.

I've been using IMDB to import filmographies for all the people in my database for a long time now. But recently, when I run the plugin on a person who already has a filmography (with the option "Merge filmographies" checked in Preferences) in order to update them, the plugin creates a duplicate entry in the filmography for a few movies; the titles, original titles, and release years all match exactly, too. Usually these movies are not in my master Movies list, so as an experiment I've clicked on them in People view to make them visible in Movies view. And I've found that of the two identical movies, one has a correctly-formatted IMDB URL ... but the other has a bizzare, malformed URL with some extra stuff tacked onto the end, something like this:

Code: [Select]
http://imdb.com/title/tt0076759/" onclick="(new Image()).src='/rg/filmo/title-title/images/b.gif'
What's more, the two movie entries often have lists of actors associated with them, but different lists (some people present in one are missing from the other).

Also, I've been occasionally finding Movie View entries (for movies I've actively added to my database) with similarly-malformed IMDB URLs in their URL fields... in addition to a correctly formatted IMDB URL!

In further testing, after clicking the duplicates in the Person filmography to make them both visible in Movies view, I then CTRL-Delete them both, to completely remove them from the database... then switch back to People view, run the IMDB People plugin import again, re-adding the movie to the filmography... and click on it to view it in Movies view. Lo and behold, the URL is correct!

So now, I'm wondering whether the "old" filmography entries (the ones existing prior to my recent attempts at updating them, not the new duplicates) actually had malformed URLs all along, and I simply didn't know about it because they were hidden in Movies view? And maybe at some point the IMDB plugin was malfunctioning, or simply receiving bad data from IMDB itself???

I don't know how the malformed URLs were getting into my "real" Movie View entries (the ones I added myself, not clicked on in People View)... unless, maybe, the IMDB People plugin was at some point trying to merge imported filmographies, saw an existing Movie entry, and added the malformed URL to it (even though it already had an IMDB URL associated with it) rather than creating a duplicate movie record? Bizzare...

Aim(my brain hurts)here

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: Problems with the IMDB People plugin
« Reply #8 on: November 12, 2009, 03:00:16 am »
It seems that getting links for some entries in filmographies was already corrupt for some time, but nobody knew :( The only thing you can do is to rewrite all filmographies all over again.

P.S. The imd people plugin is now updated to 0.2.2.2 fixed the 2 issues found by Rick
Gentlemen, you can’t fight in here! This is the War Room!

Offline nostra

  • Administrator
  • *****
  • Posts: 2852
    • View Profile
    • Personal Video Database
Re: Problems with the IMDB People plugin
« Reply #9 on: November 12, 2009, 03:00:39 am »
Quote
3. Last modified is not updated for all kinds of changes. Changing a photo or updating a filmography, for example, does not update the time.

I need to fix this in the program itself.
Gentlemen, you can’t fight in here! This is the War Room!

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #10 on: November 12, 2009, 04:21:26 am »
Quote
my brain hurts

I'm not surprised. ;)

I've noticed strange things too, but haven't commented because I haven't been able to figure out what's happening. You may be on to something with the malformed URL's. It seems plausible that if an incorrect URL is retrieved, the program is going to treat it as a separate record—thereby creating a duplicate.

I've noticed something similar involving series. A person's involvement with a series has a year associated with it (the most recent). This year is often different from that of the series (the year it started). As a result, updating the person's record causes a different item to added to their filmography than the original series record. This causes a duplicate series record to be created, and the person is not included in the original series record.

I think the plugin/program logic needs to recognize the year associated with a series in a filmography is not the same as the year associated with the series. If a person's filmography is updated via the update of a series, then the year in the filmography may necessarily be set to that of the series (rather than the actual year they last appeared). When the person's record is updated directly by the people info plugin, the existing entry for the series should be updated rather than adding a separate record with the later year.

Offline AimHere

  • Older Power User
  • *****
  • Posts: 213
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #11 on: November 12, 2009, 09:09:52 pm »
It seems that getting links for some entries in filmographies was already corrupt for some time, but nobody knew :( The only thing you can do is to rewrite all filmographies all over again.

sigh... guess I'll just have to fix the bad entries as they crop up, because I am NOT going to totally rebuild all these filmographies.

Oh well... thanks for the updates anyway, Nostra.

Aimhere

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #12 on: November 13, 2009, 12:37:49 am »
I think the plugin/program logic needs to recognize the year associated with a series in a filmography is not the same as the year associated with the series...

Thinking about this some more, I remind myself the issue is somewhat larger than this. The database design is based on the idea of visible vs. invisible movie records. If a movie is part of one's collection, it's visible. If it only exists because the movie is in a person's filmography, it contains only the information necessary for that purpose, and it's invisible. Theoretically, any one movie in the database should be in one state or another. If an invisible movie is added to one's collection, the existing invisible record should be made visible. In other words, there should be no duplicates.

In practice, however, this is not what's happening. I've already pointed out the problem with series records and years. Another problematic situation is where one is using the option to include only "main page" actors (the top 15, in credits order). BTW, I find this option essential for restricting the number people in the database to a manageable level. When people records are updated, their full filmography is added. For actors, that filmography is likely to include movies in the collection for which the person is not one of the top 15 credited. Exactly what happens in this case, I'm not sure. Based on what I see in my database, it's either of (1) the actor is added to the existing movie record, even though they are not one of the top 15; or (2) a duplicate invisible record is created to record the movie detail as presented in the filmography. In case (1), updating the movie will remove the actor from the movie credits, and the movie is removed from their filmography—a "no win" situation where neither a people or a movie update will fix the problem. In case (2), it seems the next time this happens with a different actor, the same duplicate invisible record is not necessarily used to record the filmography data.

I thought it would be interesting to see what impact this situation has had on my database. So I attempted to view my "invisible" movie records. Unfortunately, my tired old computer has difficulty handling a list with 270,000 (!!) items. I understand that number is going to be large, but I have to wonder if it's that large because of this issue. I was able to examine one interesting case before my computer choked. The X Files series had 48 duplicate invisible records. The visible record showed the top three actors (for reasons I don't understand, these are the only ones on the IMDb main page) followed by six unknown actors who had appeared in one to three episodes. Updating the record, as expected, removed these six actors.

I wasn't able to see if movies are similarly affected, but I suspect they are. When I look at filmographies, it's very common to see movies that I know are in my collection, but which are not highlighted as such. Ctrl-clicking such a link confirms it to be an invisible duplicate. This, of course, is expected. What I'm unsure of is whether there are multiple invisible duplicates—as there are for series.

I'm not sure what the solution is. I suspect it necessarily involves a significant change to the database structure, along with how the program handles these relationships. In other words, something to consider for the next major version, not a bug fix. Perhaps movie records for filmographies need to be completely distinct from "visible" records, and the two kept in sync in terms of credits.

Offline rick.ca

  • Global Moderator
  • *****
  • Posts: 3241
  • "I'm willing to shoot you!"
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #13 on: November 13, 2009, 01:46:05 am »
I don't know how the malformed URLs were getting into my "real" Movie View entries (the ones I added myself, not clicked on in People View)... unless, maybe, the IMDB People plugin was at some point trying to merge imported filmographies, saw an existing Movie entry, and added the malformed URL to it (even though it already had an IMDB URL associated with it) rather than creating a duplicate movie record? Bizzare...

The significance of this observation didn't register with me until I discovered the consequences in my own database today. 278 movie records had the malformed URL added. Although the original URL is still there, the plugin attempts to use the bad one and fails. I've had to restore my database from backup predating my first noticing the problem on November 9.

Hopefully, not too many users have been affected, since probably few run mass people updates on a regular basis. Ironically, there wouldn't have been so many records affected in my database if I hadn't been investigating the problem. It would be a good idea, however, to issue a program update so users get fair warning of the problem before corrupting their databases.

To check for records affected by this, do an Advanced Search of URL containing onclick.

And always read AimHere's posts carefully. ;)

Offline AimHere

  • Older Power User
  • *****
  • Posts: 213
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #14 on: November 21, 2009, 01:06:33 am »
And always read AimHere's posts carefully. ;)

Well, that goes without saying, doesn't it?  ;D

Offline Anson

  • User
  • ***
  • Posts: 46
    • View Profile
Re: Problems with the IMDB People plugin
« Reply #15 on: November 30, 2009, 01:46:40 am »
I suspect it necessarily involves a significant change to the database structure, along with how the program handles these relationships. In other words, something to consider for the next major version, not a bug fix.

*IF* there are major changes to the database in some future version anyway, here is a small reminder what else could be changed :-)
the handling of series and their episodes could be improved to allow for better handling of the episodes' detail info, and better linking. for example: currently, the "connection" info (mostly "spoofs", "references" and similar) may lead to episodes with specially formatted titles to indicate series/season/episode which are stored as "main movies" in PVD instead of pointing to the proper episode in a serie's tree, thus generating duplicate entries and those special titles later (when downloading IMDB data) often are changed to the proper episode title and/or a localized title, losing any "first sight" info that they are episodes and no movies.

But as rick already said: something to consider for the next major version, not a bug fix
and a more detailed discussion (if any) should go to the suggestions forum ...