Split information gathering and user interaction...

[raldo]

I'm now in the process of adding several new movies to PVD.

This process is very slow because PVD mixes information gathering with user interaction.

My suggestion is to store ambiguities to be resolved while a large list of movies is processed. Then a dialog is presented where ambiguities are resolved.

After all ambiguities are resolved by the user, PVD uploads/scans info from the net...

(This goes for both movie metadata and posters)

[rick.ca]

Much the same functionality already exists in the form of Silent mode. When adding a large number of movies (i.e., any number where you really don't want it to be interactive), turn on Silent mode. Then turn it off, and run the same operation on the movies that did not update. Since all of these will require some kind of interaction, the fact that information will be downloaded again will make no practical difference to the process. Once an ambiguity is resolved, there's no reason not to download the information—while the user goes on to the next ambiguity. Besides, ambiguities are often resolved by a "best guess," in which case it's necessary to see the results immediately.

[raldo]

Much the same functionality already exists in the form of Silent mode.

I don't see how silent mode solves the particular challenge of separating user interaction from downloading.

It does for a subset of files, but the rest of the files need to be resolved in the manner I described in my original post...

[rick.ca]

You don't see, or you disagree? I think my original response was perfectly clear.

[raldo]

I'm just saying that silent mode doesn't solve the problem of separating user interaction from datamining (other than for the small subset of file/poster lookups that return only one result).

After one run of silent mode, I still have to do a rerun without silent mode. For each item that is looked up, it takes a long time between each dialog popup, which I find awkward...

[rick.ca]

I'm sorry, I just don't buy this as a valid problem definition. If 80% of items are updated in Silent Mode, then your concern only applies to situations where hundreds of items are being updated at a time. For most users, this is only encountered when initially building a database. It's not worth the development effort to add a feature that's merely a one-time convenience for most users. Especially if would be at odds with the normal, everyday use of the program. 

If the problem originates in the fact nowhere near 80% of items are being updated in Silent Mode, then there must be something else wrong. Maybe the file scanner is not configured properly, or further refinements to that function are required. Maybe there should be a option to download the top x posters so you don't have to choose (instead, you could view the actual posters, select the one to display, and/or delete the ones you don't want). And then there's the situation I mentioned before—where the user can't resolve the ambiguity either. Deferring the action risks obscuring the issue from the user. For example, it there's no poster, there's no poster. I would rather know this immediately, so I have the option of finding one by other means while the update continues to run, or bookmark it for doing so later. These are things I may not be able to do while running a routine such as you describe. And even if I could, that would just make the whole process more problematic and confusing.

Can you elaborate on the conditions that result in you having an intolerable number of items requiring user interaction?

[raldo]

I'm sorry, I just don't buy this as a valid problem definition. If 80% of items are updated in Silent Mode, then your concern only applies to situations where hundreds of items are being updated at a time.

I'd say the definition of the problem is valid because it holds as long as more than one file needs to be disambiguated, no matter how many files there are altogether.

For most users, this is only encountered when initially building a database. It's not worth the development effort to add a feature that's merely a one-time convenience for most users. Especially if would be at odds with the normal, everyday use of the program. 

It's not at odds with any use of the program.

If the problem originates in the fact nowhere near 80% of items are being updated in Silent Mode, then there must be something else wrong. Maybe the file scanner is not configured properly, or further refinements to that function are required. Maybe there should be a option to download the top x posters so you don't have to choose (instead, you could view the actual posters, select the one to display, and/or delete the ones you don't want). And then there's the situation I mentioned before—where the user can't resolve the ambiguity either. Deferring the action risks obscuring the issue from the user. For example, it there's no poster, there's no poster. I would rather know this immediately, so I have the option of finding one by other means while the update continues to run, or bookmark it for doing so later. These are things I may not be able to do while running a routine such as you describe. And even if I could, that would just make the whole process more problematic and confusing.

I'm not suggesting auto selection of posters or whatnot.

Can you elaborate on the conditions that result in you having an intolerable number of items requiring user interaction?

I described this in the first post.

If I am a new user and select all my 500 movies for update, I'd rather do the selections with as little as possible timegap inbetween each movie.

If I am an experienced user and I add, say, 10 movies, I'd prefer not to wait between, say, 2 disambiguations.

[rick.ca]

It's not at odds with any use of the program.

It is if adds unnecessary complexity, invites confusion about it's proper use, and/or increases the risk of error.

I'm not suggesting auto selection of posters or whatnot.

I know you're not—that's my point. There may be other improvements that would be of wider benefit and substantially reduce the need for this.

If I am a new user and select all my 500 movies for update...

Use Silent Mode.

If I am an experienced user and I add, say, 10 movies, I'd prefer not to wait between, say, 2 disambiguations.

This actually makes more sense to me than your original case, which involved "a large list of movies." Turning on Silent Mode for a handful of movies is pointless. But deferring those that require user interaction to the end of the batch might be helpful. I assume your idea includes saving whatever has been downloaded (e.g., search results page, post thumbnails, etc.) so that doesn't have to be done again. Once in this interactive mode, however, I see no point in deferring a download which is normally takes only seconds anyway.

[raldo]

It is if adds unnecessary complexity, invites confusion about it's proper use, and/or increases the risk of error.

Why would it do any of these things?

I'm just saying that when not using silent mode, the code shall wait until after alternatives have been mined, pose all the questions to the user, and then do the data download.

This shouldn't necessarily change the code structure much.

For me, such a change in progress would be gold, both for a few files and for many files. I think this would be good for new users to.

[rick.ca]

Why would it do any of these things?

This conversation suggests it has the potential to be all these things.

If this were simply a matter of deferring necessary user interaction to the end of the batch, with no deferral of the download once an ambiguity is resolved by the user, I wouldn't be too concerned about such things. But I still have questions. How would the program handle error conditions? Something like the site not being available should obviously result in the user being able to abort the batch immediately. But what about error conditions that might be difficult to distinguish from the kind of user interaction you're thinking about. I don't think any user would be happy about waiting for the whole batch to be processed, only to find (due to some problem they were not aware of) that every item is requires interaction.

I really don't think this would be appropriate for large batches. I doesn't make sense to run an overnight update of 500 with the idea the user will be willing and able to resolve an interminable number of ambiguities in the morning. It makes more sense to evaluate the results and then decide what the next step should be and how to best carry it out.

[raldo]

If this were simply a matter of deferring necessary user interaction to the end of the batch, with no deferral of the download once an ambiguity is resolved by the user, I wouldn't be too concerned about such things. But I still have questions. How would the program handle error conditions?
[...]
I really don't think this would be appropriate for large batches. I doesn't make sense to run an overnight update of 500 with the idea the user will be willing and able to resolve an interminable number of ambiguities in the morning.

Yeah, well, I'm sure there are some issues that I haven't thought of but I, and I am sure many others, would very much want such functionality.

[rick.ca]

I sure everyone is in favour of complete, trouble-free updates with a minimum of user-interaction. I don't think we're likely to get any closer to that goal—unless all the relevant issues are properly considered.

I wonder if a "unified" approach might be feasible. By that, I mean one update routine that adapts itself to whatever the circumstances. Based on what we've discussed so far, it might be naturally "silent" by deferring items needing user interaction to the end of the batch. It's unclear to me, however, how different error conditions can be properly identified and then handled in the most productive manner (i.e., abort batch, skip item, or defer and ask user at end?). Once the routine is in "interactive mode," should the user be asked (once an ambiguity or whatever has been resolved) whether to download now, queue download, skip, abort, etc.? Maybe there just needs to be one dialog at the end of the first pass. Something like, "x items cannot be processed without your input... Proceed, Quit, Bookmark and quit?"

[raldo]

I don't really know, because I haven't seen errors like these...

What about some kind of error log?
Edit:
Errors resulting from http problems (server glitches, file no longer exists, record no longer exists) -> error log?
Option to bookmark unresolved items vs. disambiguation dialogs -> seems like a good idea.

[rick.ca]

Not being able to make errors happen, I have to rely on my faulty memory on what kind of errors can occur and their frequency. Maybe all possible errors can be cleanly divided into two types—those that should result in the routine being aborted (e.g., site not accessible), and those that should result in the item being deferred for another attempt, user input, or error reporting. But I wonder about errors that might fit both definitions. For example, if every 10th item fails, it would make sense to defer those items At worst, the routine will be 90% successful. But what if the very same error happens with every record? I suppose that possibility might be taken care by something like aborting after three consecutive errors, rather than the first of any particular type.

Everything is logged if the program is run with the -debug switch. In some cases, I suppose it would be helpful to present captured error messages in the user interaction stage. At the end of the routine, it might be handy to see a list of items not updated (which might be due to error, or the user's choice to skip rather than disambigulate), along with an option to bookmark these items for follow-up.

Another complication in this is the fact plugin/scripts can be run in batches. I suppose this idea would have to applied to each source separately. Problems resolved with the first source may avoid problems with subsequent sources.