Generating log files after import/update

[afrocuban]

Instead of a need to press "OK" every single time it happened (and we don't know that easy for which movie it happened), wouldn't be appropriate if PVD could generate log file after mass updating/importing data from the internet (or any other mass/single updating/importing), for which movies data weren't updated/imported for a specific reason (no movies found, socket errors, etc), but to allow PVD continuous updating/importing. Log file with the specific error, as an option in Preferences, would be popped up after importing.

During updating/importing, dialogue boxes about errors would have time limited user response options to be chosen: "Continue" as the default action after 30 secs for example and "Cancel" to cancel the download.

Right now, we have to write down every single failed import during this process, which is not convenient.

[rick.ca]

To update a batch of new (i.e., not yet imported) movies, run the plugin in Silent Mode. Then turn Silent Mode off, and run the plugin again on the movies that were not updated the first time. This time, there will be a prompt to answer for each movie (usually to choose from search results) with little waiting in between. If you believe a log will be helpful, run the program with the -debug switch and display the log using the command on the Help menu.

[afrocuban]

Rick, sorry but I was lost in translation somewhere.

Can you, please, clarify following:

Let's say there are 500 movies in my PVD database. I was going to update 100 of them.

How to get to know which movies weren't updated the first time in "Silent Mode on", of the subset of 100 movies I was updating?

[rick.ca]

How to get to know which movies weren't updated the first time...

If necessary to separate the 100 from others not yet updated, bookmark them. After updating, sort or group by a field that would have data (e.g., [Genre]) if it had been updated. Or do an Advanced Search for [Genre] IS NULL.

[afrocuban]

Oh, not NULL again!  Didn't we found back then that we never could confirmed for sure that data IS NUL, or empty (blank), or just zero...

However, please let's get back to the topic. First, thanks for an answer rick.

But...

I already have a list of movies, keeping bookmarked and updated for other reason. Nevertheless, I'm not sure if I was clear. I already have movie data in PVD for those 100 movies. I just want to update their info, like you wanted to update IMDb's Top 250 the other day. How you're safe your top 250 were updated for sure? Before answering to this, please read the following.

After silent mode on we don't know anything for sure, I think.

I believe a log as I described it, only could be helpful. Why?

Because, on the contrary, using actual -debug log isn't much of a help. We don't get almost any parsing reason:

- No movie's found on the site (there is no one, or there is for sure, but my title is wrong)
- No new/changed data found on the site
- Was the movie updated with new/changed original data from the site

The only thing we can get from log file is if the movie was parsed because of a socket error, no internet connection.

Now, imagine combining scripts. Imagine testing scripts and continuous testing updates in a row. That's where I'm in now, and I have to be assured that every PVD entry is properly processed. And that's how I conclude that only the log file could be helpful. Please, try to put yourself into this perspective.

Or, just tel me about how you processed your "Top 250" movie list.

[rick.ca]

Or, just tel me about how you processed your "Top 250" movie list.

That would be easier...

• Identify and bookmark currently ranked movies.
• Use the editor to remove the rankings.
• Update the rankings using Ivek's script.
• Identify those with no rankings and remove their bookmarks (i.e., assume they're no longer on the list).
• Sort the remaining movies by rank and review the sequence...
• For those missing from the sequence, look up the movie on the Top 250 chart.
• If necessary, add such movies to the database and update the rank.

This sounds tedious, but if done regularly there won't be many movies missing from the sequence, and most of those are at the end of the list. If the script fails to update a few, you're going to find them anyway. If the script fails to update many, then something's wrong with the script or the internet connection. The existing log may help identify the cause, but no log is going to provide a practical solution for updating the rankings. I've never found the need, but the current rankings are easily backed-up by exporting an Excel or CSV file. If getting the rankings for the first time and your database is missing many of the movies in the list, it would make more sense to parse the list into [Title], [URL] and [Rank] and import that. Doing so will update existing movies with the rank, and add any movies not currently in the database. This, of course, can be an alternate method used on an ongoing basis. Parsing the list takes some time, but the import does effectively replace steps 3 to 7.

I have to be assured that every PVD entry is properly processed.

I don't think you do. You can't be absolutely assured the data at the source is 100% correct, so there's no basis for wishing your database will be. The log is never going to provide field level changes (i.e., "[field] changed from XXX to YYY"). If it did, the information would be too overwhelming to be of much use. It may tell you something is wrong, but nothing more about the cause than you can already deduce from reviewing the results. The database itself is a more useful tool determining whether data is missing or makes sense. The existing log will normally provide clues about systemic problems. And you're probably going to prefer those be fixed rather than use a log to determine what's missing or incorrect.

I'm not even sure what you expect from the log you're imagining. There's very little chance data is going to get into a log and not be updated to the database. If a plugin or script fails to get data, the log is going to tell you there is none. A plugin doesn't know what's in the database. It only can get what it's designed or configured to get from the source. It can't tell you what it didn't find or whether what it did find is something new.

[afrocuban]

Wow! Thanks a lot for an answer. It sure did take some time to made it, and I really appreciate it.

I read your post thoroughly, and I see your point. And I agree with your scenario in a lack of better one. And I think you might not thought that there were some "plot holes" in this scenario. The biggest one for sure would be under 4.

But I'll sum it up in a simple sentence: A simple additional messages in a log file: "Movie "Neprijatelj" not found! on xxx.com" and "Movie "Neprijatelj" not found! on xxx.com because of socket error/no internet connection/no such host", would be more of a help and it'd effectively replace steps 2-7 (prior to step 3, replaces step 1).

Meanwhile, as always while having discussion with you, I got an idea. This time it is to "mimic" your scenario. Because my "Bookmark" command is not available for other purposes, I'll create custom field "Mass Update/Import", fill it in with something for all the movies I'm going to update, then to all movies I'll clear some common data for all sites (Title, for example, but I have to test this), I'll update info, then (but for sure not by using any NULL search ) search ones with no "Title" imported and revise them manually. At the end, of course, to clear custom field of data.

Thanks for your time, once again.

[afrocuban]

Btw, I found this piece of code in Ivek's script. Don't know if it's in the matter...

Code: [Select]

if R > 0 then
LogMessage(' Rating: ' + IntToStr(R) + '/10')
else
LogMessage(' Rating: NR or error');
Anyway, to relax a bit, I also found this piece of code:

Code: [Select]

if R > 0 then
if GET_RATING then
AddFieldValue(mfRating, IntToStr(R));
if R > 0 then
AddCustomFieldValueByName('AllMovie rating', IntToStr(R));

Speaking of NULLs will this work against 'em?

[rick.ca]

But I'll sum it up in a simple sentence: A simple additional messages in a log file: "Movie "Neprijatelj" not found! on xxx.com" and "Movie "Neprijatelj" not found! on xxx.com because of socket error/no internet connection/no such host", would be more of a help and it'd effectively replace steps 2-7 (prior to step 3, replaces step 1).

If what you mean is error messages the program displays (if Silent Mode is off) don't appear in the log, it may be because the plugin (which is creating the log) knows nothing about the error—it simply stops working and has nothing to report. In any case, as I've explained before, the recommended process is to ensure all the movies have valid URLs first. This eliminates search failures from the process. There's nothing the program can do about an Internet or server problem. If those sorts of things are happening, you'll get error messages when not using Silent Mode and/or it will be evident from the log some such problem is occurring. The most common solutions are to wait for the issue to fix itself, or to wait until nostra fixes the plugin.

Because my "Bookmark" command is not available for other purposes...

Had you asked again, I would have explained you can use custom fields for any number of flag-like purpose—as you have deduced. I don't know what you're using Bookmarks for now, but the fact it seems to be permanent suggests the purpose might be better served with a custom field. Bookmarks are most useful for using on-the-fly for temporary purposes (like flagging files for further processing). There are assigned shortcuts and toolbar buttons for adding and clearing them, which are really needed for that sort of use.

Btw, I found this piece of code in Ivek's script. Don't know if it's in the matter...

Sorry, I don't understand what you're getting at.

[afrocuban]

I don't know what you're using Bookmarks for now, but the fact it seems to be permanent suggests the purpose might be better served with a custom field. Bookmarks are most useful for using on-the-fly for temporary purposes (like flagging files for further processing). There are assigned shortcuts and toolbar buttons for adding and clearing them, which are really needed for that sort of use.

Being a visual type, I use "Bookmark" to color my "1001 movies you must see before you die" movies collection which I haven't completed yet. So, it is on-the-fly use, but the flight is long distance one . This way, I can easily track and spot them while browsing my movie list for any purpose: what to watch for example, without the need to multiple filter the movie list. I don't want to filter the list, actually: I want to decide on the fly while browsing being aware of the specifity of the movie . Also, I use it as a control field: are the number of bookmarked movies equal to a number of movies that are assigned custom field's "Collections" value: "1001 Movies...". And I use every other single available color for their original purpose, so browsing through list is much more comfortable and faster, and without much prior filtering.

Custom color assigning would be more of a help, and would be a nice new feature, I admit. Further more, imagine filtering by color then? Nice, isn't it?

If what you mean is error messages the program displays (if Silent Mode is off) don't appear in the log, it may be because the plugin (which is creating the log) knows nothing about the error—it simply stops working and has nothing to report.

Excactly that was my point: if PVD (or who ever do) knows and generates the error "No movie's found " in silent mode off, why is not possible PVD to collect all those messages in silent mode on in a single log file? That's all I need!

Regarding Ivek's script code, I thought maybe log messages are generating, but not sterd somewhere. I don't know anything about scripting, programming, etc...