Post reply

Warning: this topic has not been posted in for at least 120 days.
Unless you're sure you want to reply, please consider starting a new topic.

Note: this post will not display until it's been approved by a moderator.

Name:
Email:
Subject:
Message icon:

Attach:
(Clear Attachment)
(more attachments)
Allowed file types: gif, jpg, png, txt, tpl, lng, ini, pvd, psf, ini, cfg, csv, zip, xml, pas, 7z
Restrictions: 4 per post, maximum total size 1024KB, maximum individual size 1024KB
Note that any files attached will not be displayed until approved by a moderator.
Verification:
Type the letters shown in the picture
Listen to the letters / Request another image

Type the letters shown in the picture:
What is the fifth word in this sentence?:

shortcuts: hit alt+s to submit/post or alt+p to preview


Topic Summary

Posted by: rick.ca
« on: October 05, 2009, 07:48:28 pm »

Thanks. I look forward to your post or wiki entry on the matter.
Posted by: patch
« on: October 05, 2009, 10:19:35 am »

I don't think it's a good idea to suggest there are "default" or "supported" formats by way of example.

np
I will leave it to your judgement then how best to direct none technical users to naming conventions which work with PVD
Posted by: rick.ca
« on: October 05, 2009, 09:13:25 am »

I don't think it's a good idea to suggest there are "default" or "supported" formats by way of example. The regex are the most concise definition of what is supported. True, most people don't understand regex. But the best way to deal with that is to provide an example for each expression and "walk through" exactly how the pattern recognition works. Then it would be clear what file names will be recognized, how much flexibility there is, and possibly what modifications would be required to handle different or multiple patterns.
Posted by: patch
« on: October 05, 2009, 06:26:06 am »

Quote
We could probably do that by listing (default) supported formats in the wiki.
Currently this aspect is avoided

That would just encourage the misconception the program is intended to handle any file naming convention one cares to use.
Not if it was listed as

PVD is designed to support...
I can in fact extract the required information if you use other naming conventions but you may need to manually edit the title during "scan folders for new movies / changed file paths" or modify the regex expressions in "Tools -> preferences -> File scanner". See ... for further details


My intention is we would document the default / preferred naming. Illustrating some of the flexibility in punctuation & capitalisation, but not covering every possibility as that is best done by reading the regex search terms.
Sorry I didn't make that clear.
Posted by: rick.ca
« on: October 05, 2009, 04:17:26 am »

Quote
We could probably do that by listing (default) supported formats in the wiki.
Currently this aspect is avoided

That would just encourage the misconception the program is intended to handle any filenaming convention one cares to use. I'd rather just say a consistent filenaming convention like Title (Year).ext for movies and SeriesTitle S00 E00 [EpisodeTitle].ext for series will be foolproof. The regex provided will handle many forms in addition to these. The regex can even handle a somewhat inconsistent filenaming convention, but that gets complicated very quickly. Considerable care has to be taken to ensure the regex do not conflict with one another, and are applied in the correct order.
 
In fact, a list of "supported formats" would be misleading unless it also enumerated all the possible conflicts. This recent post is a good example of the difficulty. You can name movie files Year - Title [(Director]).ext if you like, but then you have to provide a regex that will recognize this pattern (including the Director part, whether or not it exists) and remove (or reorder) any regex that would misinterpret the year as an episode number.
Posted by: patch
« on: October 05, 2009, 01:08:58 am »

The point is there just needs to be a consistent structure to the filenames so the regex can work.
We could probably do that by listing (default) supported formats in the wiki.
Currently this aspect is avoided
http://www.nimidia.com/pvd_wiki/tiki-index.php?page=Organize-files-by-episode
http://www.nimidia.com/pvd_wiki/tiki-index.php?page=Adding-a-New-Movie&structure=PVD-Manual

But I may have missed it as I'm having problems using the search function in the wiki.
Posted by: rick.ca
« on: October 04, 2009, 06:24:55 pm »

The periods are irrelevant. They could also be underscores, spaces or pipes. The point is there just needs to be a consistent structure to the filenames so the regex can work.
Posted by: nostra
« on: October 04, 2009, 04:29:03 pm »

What system are you using that would not let you use periods in file names???
Posted by: Nebol
« on: October 04, 2009, 03:42:05 pm »

In the case of episodes, there are so many different ways they may be organized and named, it's probably easier for all if you decided how it should be done. I like Dadeo's ..\Title\Title.S02E01.episodename.avi, especially if, with that format, your routine could consistently identify titles—whether the episode files are in a separate sub-folders or not.

Problem is, that would not work at all for me, since, in my system, I definitely can not have periods in the file names, only before the suffix. I use underscores. Title_S02E01_whatever.suffix
Also, using periods like that is damn ugly.
Posted by: rick.ca
« on: June 17, 2009, 04:17:15 am »

Quote
3. I am very curious about your 1.0 feature suggestions, Rick

I haven't thought about it much, other than in this conversation. But to summarize:

1. The scan results dialog should allow the user to make changes...

a) For records already in the database, forcing the match in cases where the scanner "missed" and thinks a file is new. This would include forcing a match of an episode to an existing series when the scanner has failed to recognize it as an episode.

b) For records not in the database, changes to title and year, to increase the likelihood of a successful web search.

2. As noted here, the scanner should not identify files as series unless a season and episode is determined by the regex.

3. The introduction of a regex variable for storing an online database code from which a URL can be constructed from base URL mask saved in the user's configuration. For example, an IMDb mask could be http://www.imdb.com/title/tt[URL code]/, where [URL code] is the 7-digit IMDb title code.

4. A file path renaming utility which uses relevant database fields as variables. This sort of thing can be rather dangerous unless the user can see the results before committing—it needs some kind of "old path ~ new path" preview, or perhaps some kind of integration with a reintroduced Gridview (where a preview can be shown in a column beside the existing file path). Examples of what should be possible:

[parent]\[title] ([year]) [rating] [url code].[ext]
[parent]\[title] ([year])\{DVD rip files}
[parent]\[series] S[s#] E[e#].[ext]
[parent]\[series]\Season [s#]\[e#] [episode title].[ext]

5. Since some (like patch) will use a renaming utility to ensure other users/databases will be able to get online information with 100% accuracy, it would be nice to also save the current poster using the same name—at the same time the renaming is done (i.e., as an option, "Save poster with same pathname?").

I think a file renamer should have the effect of making users much more accepting of the scanning process. They only need to change new filenames to get results that are "good enough" (that depends on the user—how good their regex are and how willing they are to make corrections in the results dialog). Once the "processing" is done, they will be able to use the renamer to make the file pathnames "perfect."
Posted by: nostra
« on: June 17, 2009, 12:45:45 am »

OK, I will be laconic:

1. I do not think there will be some built-in support for tagging. I only think of realizing this functionality with scripts/plugins

2. There will be file rename feature, but only to organize files for existing movies in database

3. I am very curious about your 1.0 feature suggestions, Rick
Posted by: rick.ca
« on: June 15, 2009, 09:30:23 am »

Quote
The reason I would prefer it as part of PVD is I like the way PVD file scanner just shows me the new or changed files when I point it a at directory trees. I also think a dedicated tool like PVD could do a better job of managing video files.

I don't think there's any advantage in trying to turn PVD into a file manager. Don't be mislead by the fact I use regex in my file manager. Those regex can do no more or less than those in the scanner configuration. The difference is, if there anything that needs to be done manually, the file manager is the appropriate environment in which to do it. If I had a good eye and didn't care about the file names, I could just make the bare minimum of changes necessary so the scanner regex would work. As a practical matter, renaming the files in the file manager not only "cleans" them, but makes it obvious which ones need manual adjustment so that the scanner regex will work.

The function of the scan results dialog should be restricted to correcting associations for movies already in the database, or enabling a more accurate search. In other words, it's for moving forward to the next step, not for fixing errors from the previous step. In a well designed work flow, this dialog should more often than not just be a simple confirmation.

Quote
Sorry I didn't explain myself, I was thinking how I would like a PVD file renamer to work, not how Directory Opus works

A PVD renamer would be rename files based on user-configurable database variables like Title, Year, IMDb#, etc. To use such "other information," an additional regex variable would have to be provided that you could use to save the data in a custom field. Then, of course, the renamer would have to allow variables for custom fields. But allowing any custom field to be used in a file name doesn't seem like a good idea. Maybe the Features field could be used for this purpose.
Posted by: patch
« on: June 15, 2009, 04:14:27 am »

Quote
You do of course realise there is a certain amount of similarity between what you are doing with your file manager and what I (& I assume others) are trying to do with PVD file scanner.

Yes, I thought about that and concluded that just because a similar function is required at two different stages of my workflow does it mean it has to be restricted to one or the other.

mmm
PVD already has regex, it already stores user defined regex expressions for the file scanner.
As a wish list item I would like to combine the essential features of rick.ca file renamer into PVD. To explain:-
The "Scan results" dialogue should show the fields it will potentially use to search the online databases & set up the record. For movies these are
Title
Original title
Year
url
Path

To that I would like added the ability to run user defined regex (via buttons or menu selection). Typically they would take the "path" as the input string & output to the above fields as appropriate. To be general it maybe worth having tha ability to start from other fields. I imagine some may prefer to run a regex on the existing Title to clean it.
It would also be useful to make the url active so the linked page could be viewed in the users default browser.

For series a similar thing could be done, but more fields are required
Season
Episode
Episode title

The user needs also to be able to correct the scanner if it gets it wrong. A possible solution is to have a series check box, which when ticked indicates the video is part of a series and the extra fields are shown (Unless it is felt the scanner would always be correct). I have not thought through how to incorporate the add to series functionality, but that too would be applicable here.

The reason I would prefer it as part of PVD is I like the way PVD file scanner just shows me the new or changed files when I point it a at directory trees. I also think a dedicated tool like PVD could do a better job of managing video files.

The actual file name updating could then be done as a second step, useful for those who look at the file names or want to ensure subsequent accurate re-scanning.

Quote
I would like at least the option to append the old file name ending to the new cleaned name, maybe Title (Year) IMDb# [old descriptors].ext as I'm not sure I would really like to loose some of that information (CAM, TS, dvdrip, HDTV, BluRay)

Just specify the regex that will do so. Instead of skipping over them (or, for example, dropping everything after finding the title and year), put those parts in variables and append them as [old descriptors].

Sorry I didn't explain myself, I was thinking how I would like a PVD file renamer to work, not how Directory Opus works

Quote
Not sure if it would cope with graphics data but given the often reasonable image PVD can pull from imdb I still think it is a good approach.

Don't forget it's now very easy to export posters to images "beside" the media file. So if a lot of effort has gone into getting better quality posters from elsewhere, this is a reasonable thing to do—to make those available to other PVD databases and/or just to be a visual reference in the file system.

A nice companion to a file renaming feature would be the option to save the poster beside the file at the same time.

Good ideas
Posted by: rick.ca
« on: June 14, 2009, 08:11:36 pm »

Quote
Does the file manager you use cope with multi-part files appropriately? Which one do you use?

Directory Opus. It's renamer only does what the regex I specify does (it also has other modes, like find & replace, and script—where file attributes and tags could be used). I haven't run into problems with multi-part files. When I do, maybe I'll be inspired to modify the regex.

Quote
You do of course realise there is a certain amount of similarity between what you are doing with your file manager and what I (& I assume others) are trying to do with PVD file scanner.

Yes, I thought about that and concluded that just because a similar function is required at two different stages of my workflow does it mean it has to be restricted to one or the other. It does mean that something has to be done at the first stage. You don't have to be very sophisticated at the file renaming stage—just change it so the essential variables (title, year, season, episode, disc#) are in a fixed pattern. Then the second stage regex can't miss—they're just pulling out those variables from a fixed pattern. Also, I should emphasize the renaming stage is not automatic, but under my supervision. So I'm also using my own judgment as to what is required. That may range from tweaking the regex and running it on 100 new files, to deciding the one file name I'm processing right now is easier to edit manually.

BTW, all this is explained in 0.9.9.x File Scanner and Regular Expressions. This post includes useful regex references and a link to a good freeware renamer.

Quote
True if I knew the Exact imdb title & year the text search would work well, but often I do not know this prior to looking it up on imdb.

Torrent file names often include a year (which is easy to identify with regex). It seems to me that even if the title is inexact, including the year dramatically improves the IMDb search result.

Quote
I would like at least the option to append the old file name ending to the new cleaned name, maybe Title (Year) IMDb# [old descriptors].ext as I'm not sure I would really like to loose some of that information (CAM, TS, dvdrip, HDTV, BluRay)

Just specify the regex that will do so. Instead of skipping over them (or, for example, dropping everything after finding the title and year), put those parts in variables and append them as [old descriptors].

Quote
Not sure if it would cope with graphics data but given the often reasonable image PVD can pull from imdb I still think it is a good approach.

Don't forget it's now very easy to export posters to images "beside" the media file. So if a lot of effort has gone into getting better quality posters from elsewhere, this is a reasonable thing to do—to make those available to other PVD databases and/or just to be a visual reference in the file system.

A nice companion to a file renaming feature would be the option to save the poster beside the file at the same time.

Nostra, I'm warming up to the idea of starting a 1.0 wish list. It seems I now have a number of feature requests that could go with my recommendation for Scanners (1981). ;)
Posted by: patch
« on: June 14, 2009, 02:56:49 pm »

My file manager includes a regex file renamer. One button cleans movies, another episodes. Maybe I'll configure a third to deal with those that have a more accurate title in the containing folder. Even with some requiring manual adjustment, it still works out to seconds per file. And then the filenames are accurate and meaningful in my file system as well as in PVD. The same thing can be done with a number of excellent freeware file renamers.

That sound like a good system.
I got sick of doing it by hand. The issue I found was often multi-part files were used and for replay all files names needed to match (sub-titles, multiple discs etc). As such it was easier to leave the cryptic but matching set of file names. Typically the folder names are more human readable, hence my approach.

Does the file manager you use cope with multi-part files appropriately? Which one do you use?
You do of course realise there is a certain amount of similarity between what you are doing with your file manager and what I (& I assume others) are trying to do with PVD file scanner.

But only 50% of those have NFOs

imo PVD being able to link directly to the exact imdb reference would be a very useful feature. Currently I do this by hand which is much less efficient.
True if I knew the Exact imdb title & year the text search would work well, but often I do not know this prior to looking it up on imdb.

Maybe what would be much more useful would be the ability to rename files. Files could be renamed to Title (Year) IMDb#.ext and a variable added so regex could capture the IMDb# and add the URL.

That is a neat idea.
We would need to ensure it worked with multi-part files.
I would like at least the option to append the old file name ending to the new cleaned name, maybe Title (Year) IMDb# [old descriptors].ext as I'm not sure I would really like to loose some of that information (CAM, TS, dvdrip, HDTV, BluRay)
Not sure if it would cope with graphics data but given the often reasonable image PVD can pull from imdb I still think it is a good approach.
Posted by: rick.ca
« on: June 14, 2009, 12:05:42 pm »

Quote
Note I have spent time making file names look pretty in the past, but given I now access my movies from PVD the actual file names and paths are largely irrelevant. If I need to change them to import into PVD then that indicates the efficiency of operating PVD could be improved.

Obviously, the actual file names and paths are relevant. A lot of work went into a sophisticated file scanner that can accurately extract a title and year—which in turn will very likely result in a successful IMDb search. All that is required it that there be reasonable degree of consistency in the file path names. The fact that you're unwilling to use it as designed doesn't mean it's efficiency needs to be improved.

But fine. You're right, there are probably others who expected PVD to accurately process inconsistent, meaningless and misleading file path names used by many torrents. Most of the torrents that provide reliable NFOs are also more likely to use reasonable names, but let me be generous: Let's say that 50% of these torrent file names are so bad you really need a URL from an NFO. But only 50% of those have NFOs. So that means the you're still going to get an unacceptably high error rate.

I use "clean" file names, and the file scanner performs flawlessly. My file manager includes a regex file renamer. One button cleans movies, another episodes. Maybe I'll configure a third to deal with those that have a more accurate title in the containing folder. Even with some requiring manual adjustment, it still works out to seconds per file. And then the filenames are accurate and meaningful in my file system as well as in PVD. The same thing can be done with a number of excellent freeware file renamers.

Quote
The aim of using tags is to further improve the file scanner performance.

Maybe now that the scanner works so well with clean filenames, we don't need tags. Maybe what would be much more useful would be the ability to rename files. Files could be renamed to Title (Year) IMDb#.ext and a variable added so regex could capture the IMDb# and add the URL.
Posted by: patch
« on: June 14, 2009, 09:37:20 am »

Quote
The only enhancement I can think of is sometimes the enclosing directory has a better name, so being able to easily select that would be handy.

You should be able to handle that with regex as well, unless the pattern is hopelessly inconsistent. What is the source of the directory and file name?

Media file names are often very cryptic eg "vmt-agmihtf-xvid.avi" but the enclosing directory is more informative "A Good Man Is Hard to Find". Maybe others don't have this experience, but I find it is rather common. Currently I cut & paste in the "Scan results" dialogue to make the name more informative for the imdb lookup. This is a wish list suggestion to improve to the "Scan results" dialogue to make this process more efficient. Regex would not make the manual editing more efficient as the very reason it is a manual process is the naming is generated by humans so is inconsistent.

Quote
Some media files have an associated .nfo file containing a link to an internet database (most often imdb).

I don't see much point in this. Sure, it there's a reliable URL consistently available in a consistent format, then this may help, but the primary approach is supposed to be to get the filename (and maybe year) and use that to search IMDb.

URL are in a consistent format (they are a valid url). If one is available it will always point to the correct imdb record with greater accuracy than a search (duplicate records in imdb, similar movie titles etc). So if it is available it is worth using, if not then falling back to file naming & database search is of course required. Currently I can "Open enclosing folder" open the .nfo file, cut & paste from there. Importing into a custom field would provide similar functionality but not address why I suggested it ie the improve scanner performance.

Note I have spent time making file names look pretty in the past, but given I now access my movies from PVD the actual file names and paths are largely irrelevant. If I need to change them to import into PVD then that indicates the efficiency of operating PVD could be improved.

.nfo are supplied with at least 50% downloaded content and most have imdb link, so I suspect I'm not the only one who would benefit from this functionality, hence it's suggestion as a scanner enhancement wish list item

Quote
Tag export/import (like ID3 for mp3) has been marked as already added. This is good news if that is the case. Looking at 0.9.9.10 I can't see where it is implemented.

I marked that off because I did not interpret the feature request to necessarily be an integrated tag management system, and I thought most of the pieces were in place for this to be done on an ad hoc basis.

PVD is run on an open database with various import & export capabilities. It has always been possible to import movie information from various sources in an adhoc fashion. The aim of using tags is to further improve the file scanner performance. To do so it becomes part of the file scanner. Maybe PVD has all the tagging capability nostra wanted to implement, if so then a some what adhoc solution must be the answer.

BTW
The above suggestion are in-order of solution specificity, from most general (but least exact) to most exact / potentially automatic.
Posted by: rick.ca
« on: June 14, 2009, 07:22:20 am »

Quote
Tag export/import (like ID3 for mp3) has been marked as already added. This is good news if that is the case. Looking at 0.9.9.10 I can't see where it is implemented.

I marked that off because I did not interpret the feature request to necessarily be an integrated tag management system, and I thought most of the pieces were in place for this to be done on an ad hoc basis. Perhaps I'm wrong, but it doesn't seem you've actually tried to do anything with the tools that have been provided. I'm not trying to give you a hard time, but since I don't really have much interest in this, I was expecting you would give it a shot.

Is there not an template command for exporting text file beside a media file? That could be used to export a text file with the URL. I assume the Text File plugin can import that. Those two could export/import any text, but, as I understand it, the URL is your primary concern—with that, another instance of PVD would not have to search IMDb to find the movie. I believe there's also a template command for exporting posters beside media files. And there's Image by ID to reassociate them with a new database. That would work only for new databases, so I do understand your interest in Reset's plugin.

Quote
The only enhancement I can think of is sometimes the enclosing directory has a better name, so being able to easily select that would be handy.

You should be able to handle that with regex as well, unless the pattern is hopelessly inconsistent. What is the source of the directory and file name?

Quote
Some media files have an associated .nfo file containing a link to an internet database (most often imdb).

I don't see much point in this. Sure, it there's a reliable URL consistently available in a consistent format, then this may help, but the primary approach is supposed to be to get the filename (and maybe year) and use that to search IMDb. In normal circumstances, that works fairly well. What if the URL is wrong? In that case, a mess would have to be undone, and the normal method repeated—after removing the offending NFO. Maybe what would be more practical would be to import the entire contents of any NFO into a custom field. Then, if any of the information was needed, it would be readily accessible.
Posted by: patch
« on: June 14, 2009, 01:30:46 am »

Tag export/import (like ID3 for mp3) has been marked as already added. This is good news if that is the case. Looking at 0.9.9.10 I can't see where it is implemented.
Further to discussions here

Actually I'm not sure what is going to be the most efficient path to getting movie tagging functionality.
Importing & exporting, text & image data near a movie file is part of the functionality required.
However I suspect working reliably with the file scanners name parsing and multi-part file identification could well be more challenging.

To explain
For a single file movie the tag information would probably go in .jpeg and .txt (or .xml or .pvd?) files with identical path and base file name.
For multi-file movies the base file name is less obvious as it would involve stripping out the disc number or using the first file name or exporting for all component files
For DVD images (ie with .vob files) the tag files could go within the directory containing the .vob files or 1 level above; with a generic file name or directory name (if 1 level above .vob files)
For TV series similar issues arise however then there is the need to identify the episode, season, series, as well as grouping with other episodes in the same series.

As such it wouldn't surprise me if nostra intentionally left it till other parts of the file scanner were implemented & stable.

mmm
Reading through this again, it looks as if I'm making it all too hard so perhaps it is time to stand back & think what it is all about.
My aim is for PVD to import media with maximum reliability and minimum user intervention.
The problem is PVD needs to make a connection between the video data I have & various internet databases but PVD can't watch movies and it is often ambiguous which record in and internet database is preferred. So an indirect link needs to be used. The best way of doing this is going to vary with the information available to PVD, for example:

1) Analyse file (& path) name to deduce movie name, then use that to look up internet databases. PVD does this very well (& flexibly with regex). The only enhancement I can think of is sometimes the enclosing directory has a better name, so being able to easily select that would be handy.

2) Some media files have an associated .nfo file containing a link to an internet database (most often imdb). It would be good if PVD could automatically extract the reference. I'm actually not sure if it is doing this already as it has been discussed. Knowing what .nfo file to associate with what media file could be a problem as they are generated by humans, thus done inconsistently. The way to start is support the easy cases ie if have .nfo with same base file name as media or folder with one .nfo & one media file then PVD could assume they belong together. Ideally PVD would display some how that it found the reference in the "Scan results" dialogue box. Perhaps the action could show "Add movie" imdb link. This would save the user worrying about the information in the file name.

3) If the movie has already been imported into PVD, and links to internet databases checked, then it would be good if that information could be written back to the movie file, so if it was moved to another system, it could then be imported into PVD there with minimal user intervention. Writing data back to the movie file is also likely to assist communication between media programs (is it does for music with mp3 tags). For PVD though the value is to provide assistance to the file scanner, so the writing protocol needs to be linked to the file scanner reading protocol. Again just doing the easiest case (single movie file) would be useful (provided it cleanly ignored the more complicated case and its functionality was documented eg in change log)
Posted by: nostra
« on: December 08, 2008, 01:03:21 pm »

The export plugin will be improved with an extra option to save files near video files in the next version.
I am also planning an export template and some additional import functionality (including file scanner) using predefined XML files near video files.
anything