IMPawards download error

[Paul T]

Hi All,
      whenever I try and download poster information using the IMPawards plugin 0.1.4.0 using the latest PVd (0.9.9.14) I get " HTTP/1.1 503 Service Unavailable" . I am sure it worked before on V 0.9.9.11 but do not have a copy of my database or program for that that version to try.

Great Program all the same.

Paul T

[rick.ca]

Does it happen after downloading multiple posters in batch mode? Does it work again after 24 hours or so? See this topic. I don't know what, if anything, can be done about it.

[Anson]

... downloading multiple posters in batch mode ... again after 24 hours or so ...

This question started me to think about a general problem on all imports/plugins:

when people do an update of large amounts of data, eg selecting all movies and running a plugin on them, would it be nice to do such actions more slowly by having an option to pause between queries for each movie, or even between queries for several pages/files which belong together for a single movie?

when running such an update AFK, i wouldn't care whether it takes 1 or 12 hours overnight, but it would avoid flooding a website with requests for several movies (dozens of pages and even more files) per minute, a speed which no normal user could achieve and which even for big sites might cause some burden if too many people start doing similar ...

[Paul T]

Hi All working again. 

At the time I was recovering from a major computer crash so had to reload approx 120 files. This must have blocked my IP as suggested in linked post.
Sorry did not find anything when I searched for the problem and thought it was linked to the new release.

All the best

Paul T

[rick.ca]

when people do an update of large amounts of data, eg selecting all movies and running a plugin on them, would it be nice to do such actions more slowly by having an option to pause between queries for each movie, or even between queries for several pages/files which belong together for a single movie?

That would be fine—if it were effective. In this case, I don't think we know for sure what is triggering this, or if slowing down down the process would avoid it. If nostra knows, he hasn't said, and the plugin hasn't been updated since this was first reported.

In the case of IMDb, I sometimes get the "take a rest" message when using the site manually, but I'm not aware of them having blocked or restricted the plugin.

[Anson]

In the case of IMDb, I sometimes get the "take a rest" message when using the site manually, but I'm not aware of them having blocked or restricted the plugin.

I never got such "take a rest", but after using the IMDB plugin to get episode info on 8 seasons of a series without problems, the next few seasons always skipped one or two episodes with an error "page not found". After waiting a few minutes, i got the next few seasons again without problems. The same happened again at some other time of day on another series.
btw: the errorbox for "page not found" had only one button for confirmation. would it be possible to add a second button for the option to retry ?

Since the number of pages multiplies if someone also gets people info for each episode's actors, i thought it a possibly good idea to restrict oneself a bit before they implement stricter limitations, and not flood a site with requests when not necessary. Since we don't want to mirror the entire site, a little slowdown shouldn't hurt us too much.
The method for the slowdown (wait after each movie, wait after each page, wait after each file, wait some time after a burst of one minute, etc) remains to be discussed later if such kind of feature would be in the planning state.

[rick.ca]

Are you using the -debug switch so a log is created? Anecdotal observations about such things don't help much. If you capture problem events in your log, that information might suggest what's wrong. I don't think nostra is going to change anything because we speculate a change might help. Keep in mind too a plugin is just scraping a website. It's going to run into all the same random glitches you do when you're browsing.

The topic here, however, is a special case. IMPAwards is using Google to do the search, and it's Google that's implementing the ban. My last comment on the matter was...

I'm now experiencing the "HTTP1.1 503 service unavailable" error with the IMPAwards plugin, as originally reported here. This is a result of being banned by Google. If previous experience holds, this means I will be unable to use the IMPAwards plugin for about 24 hours.

The plugin executes the following search...

http://www.google.com/search?q=allintitle:+TITLE+Poster+site:www.impawards.com&filter=0

I note I can still do the equivalent Web Search...

http://www.impawards.com/googlesearch.html?cx=partner-pub-6811780361519631%3A48v46vdqqnk&cof=FORID%3A9&ie=ISO-8859-1&q=TITLE

This results in an embedded Google search which is seems unaffected by the ban. (It passed my crude test of being fed about 30 queries in about 30 seconds.) I wonder if this suggests a way the plugin might be modified to resolve this problem.

...but nostra has not commented on the suggestion. It seems to make sense, but I really don't know if it has any merit. I do know nostra doesn't have time to be fixing plugins that work most of the time.