Creating own scripts

[nostra]

Information in this topic is outdated. Please refer to this document for most current state of the script engine plug-in: http://www.videodb.info/help/hlp_scripting.html

Beginning with version 0.9.8.11 there is a possibility to create scripts using Pascal syntax to download movie and person information from the internet.

Scripts were added to make it possible for users to add additional functionality to the program without deep programming knowledge.

There are already some scripts available, so the only thing you will need to do in most cases is to set some values for constants  and write code to parse HTML pages. Mostly it is just about finding unique texts around information needed.

I have added multiple useful functions to make creating scripts easier.


Scripts should be saved in Scripts directory and have a .psf extension. All the scripts communicate with the main program with help of a special Script Engine plugin scriptint.dll.
Scripts can not work without this plugin.

PVD treats the scripts just like plugins, so you can configure them, combine in bacth files etc.

The following information can be set using scripts:

• All movie information fields, including custom fields
• All person information fields
• Posters

Please take into account that only a small subset of information fields can be set up in the program to be overwritten or not. All other fields are always overwritten.

[nostra]

1.
Open the Scripts directory and make a copy of an existing script with another name.

2.
Open the file in a text editor (I recommend Notepad++)


3. Constants
I the beginning of the file there are usually constants defined for ease of setting up the script and making it better readable. You do not have to use constants, but it is recommended. There must always be a const operator before constants definitions.

Constants after the line //Script data describe the script:

SCRIPT_VERSION	Version of the script
SCRIPT_NAME	Short script name
SCRIPT_DESC	Long script description
SCRIPT_LANG	Language of the information retrieved. Ex.: $09 - English, $19 - Russian, $07 - German. A full list can be found here: http://msdn.microsoft.com/en-us/library/ms776294(VS.85).aspx (you need the number after 0x from "Prim. lang. identifier" column). It is not very important to define the right language, but recommended for better usability.
SCRIPT_TYPE	Script type: movie information script (constant stMovies) person information script (constant stPeople) poster script (constant stPoster)
BASE_URL	The first level domain of the web-site. Ex.: http://www.amazon.com
RATING_NAME	Name of the site rating with the word "rating"
SEARCH_STR	URL to use for searching the web-site. %s is replaced with a movie title or person name
CODE_PAGE	Web-site encoding. For example if the web-site encoding is windows-1250 than this constant should have a value of 1250. You can usually find the web-page encoding in the beginning of a html file. Needed HTML tags look like this: <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>. If you do not know what encoding to use, just enter 0 to let the program auto detect it. A full list of supported values can be found here: http://msdn.microsoft.com/en-us/library/ms776446.aspx

You can also define your own constants. (For example to enable users to turn some script functions on or off).


4. Global variables
Script global variables are defined after the line //Global variables. Global variables can store any information that should be available in all parts of the script. var operator has to precede global or local variables (local variables are only accessible from a function they are defined in)

It is not obligatory to define any global variables, but I recommend to define at least one variable to track current script state. In my scripts it is defined like this: Mode : Byte;

[nostra]

5. Obligatory functions
When you are done with defining variables and constants, you can start to write some real code . The following functions have to be implemented to provide a communication interface between the script and script engine:

function GetScriptVersion : WideString;	Returns script version
function GetScriptName : WideString;	Returns script name
function GetScriptDesc : WideString;	returns script description
function GetBaseURL : String;	Returns web-site's base URL
function GetScriptLang: Cardinal;	Returns script language
function GetScriptType : Byte;	Returns script type
function GetCurrentMode : Byte;	Returns current script state: Searching movie or person (constant smSearch) Retrieving data  (any value above 0)
function GetDownloadURL : String;	Returns URL, that the main program should download. If the script is in a search mode it should return SEARCH_STR
function ParsePage(HTML : WideString; URL : String) : Cardinal;	This is the most interesting function HTML variable will contain text of the downloaded page, URL variable will contain address of the page. Your main goal is to parse this data and fill movie/person fields.

6. Not obligatory functions:

function GetRatingName : WideString;	Returns rating name
function GetCodePage : Cardinal;	Returns web-site encoding
procedure GetLoginInfo(out URL : String; out Params : String);	Returns data needed to login to the user account on the web-site. URL variable should be set to the login script url. Params variable should be set to parameters needed to login into the account (usually login and password). (POST method is used to pass parameters)

If you use an existing script as a template then you will only need to change the function ParsePage(HTML : WideString; URL : String) : Cardinal; and maybe the function GetDownloadURL : String;

[nostra]

7. function GetDownloadURL : String;

If you script just needs to find a movie/person and download one page with information you will need to write code like this:

Result := SEARCH_STR

If you need to download and parse more than one page I recommend to define extra variables or an array of URLs that need to be downloaded. Code for 2 information pages will look like this:

   
 if CreditsURL = '' then
  Result := SEARCH_STR
 else
  Result := SecondPageURL;


You can also use the Mode variable as an extra flag. This code is for 3 pages using Mode variable:


 if (Mode > smNormal) AND (PosterURL <> '') then
  Result := PosterURL
 else
 if CreditsURL <> '' then
  Result := CreditsURL
 else
  Result := SEARCH_STR;


If you download poster in a movie information script or a photo in a person information script than make sure URL to an image is passed after all other pages are retrived!

[nostra]

8. function ParsePage(HTML : WideString; URL : String) : Cardinal;

This is the most important function you need to implement.

The following actions have to be made in this function:

• a. Determine what information does this page contain (search results, movie/person information, etc.)
• b. Set the Mode variable (if script state changes)
• c. Parse HTML and set information fields
• d. Return an appropriate result

a. Determine what information does this page contain

It's pretty simple: You need to find a unique marker in HTML for each page type and do some parsing according to what you have found on this page. Ex.:
if Pos('<title>Movie</title>', HTML) > 0 then begin

then...
б. Set the Mode variable
Mode := smNormal;

then...
в. Parse HTML and set information fields
ParseMovie(URL, HTML); //This is a helper procedure to make the code better readable.

then...
г. Return an appropriate result
Result := prFinished; //script has finished it's job

The part of the code that determines the type of the web-page and does the rest of the actions should look like this:

if Pos('<title>Фильм</title>', HTML) > 0 then begin
 Mode := smNormal;
 ParseMovie(URL, HTML); //This is a helper procedure to make the code better
 Result := prFinished; //script has finished it's job
end;


I recommend you to create a helper procedure to parse each page type. Ex.:
procedure ParseSearchResults(HTML : WideString); //parse search results
procedure ParseMovie(MovieURL : WideString; HTML : WideString); //parse main movie information
procedure ParseCredits(HTML : WideString); //parse movie credits

The whole ParsePage function should look like this:
function ParsePage(HTML : WideString; URL : String) : Cardinal;
begin
 if Pos('<title>Поиск по сайту</title>', HTML) > 0 then begin
  ParseSearchResults(HTML);
  Result := prList; //search results retrieved
end else
 if Pos('Ничего не найдено по запросу', HTML) > 0 then
  Result := prError //error (movie not found)
else
 if (Pos('<title>Фильм</title>', HTML) > 0) then begin
  Mode := smNormal;
  ParseMovie(URL, HTML);
  Result := prFinished; //script has finished it's job
end else
  Result := prError; //error (unknown page retrieved)
end;

[nostra]

9. Built in parsing functions

function Pos(Substr : WideString; Str: WideString): Integer
Returns the index value of the first character in a specified substring that occurs in a given string.


function PosFrom(const SubStr, Str : WideString; FromIndex : Integer) : Integer
Returns the index value of the first character in a specified substring that occurs in a given string. Search starts from FromIndex[/b]


function LastPos(const SubStr, Str : WideString) : Integer
Returns the last index value of the first character in a specified substring that occurs in a given string.


function PrevPos(const SubStr, Str : WideString; APos : Integer) : Integer
Returns the index value of the first character in a specified substring that occurs in a given string before APos.


function RemoveTags(AText : WideString; doLineBreaks : Boolean) : WideString
Removes all HTML tags from AText. Ex.: <a href="http://test.com">Test</a> will be converted to Test. If doLineBreaks is set to True, then <br> will be converted to a line break.


function ExplodeString(AText : WideString; var Items : TWideArray; Delimiters : WideString) : Integer
Splits a string AText  into separate values using Delimiters. The values are written into a dynamic array Items.


function Copy(S: WideString; Index, Count: Integer): WideString
Returns a substring of a string.


procedure Delete(var S: WideString; Index, Count: Integer)
Removes a substring from a string.


procedure Insert(Source: WideString; var Dest: WideString; Index: Integer)
Inserts a substring into a string beginning at a specified point.


function Length(S: WideString): Integer
Returns the number of characters in a string.


function Trim(S: WideString): WideString
Trims leading and trailing spaces and control characters from a string.


function CompareText(S1, S2: WideString): Integer
Compares two strings by ordinal value without case sensitivity.
The function returns 0 if both strings are equal, a value > 0 if S1 > S2 and a value < 0 if S1 < S2.


function CompareStr(S1, S2: WideString): Integer
Compares two strings by ordinal value with case sensitivity.
The function returns 0 if both strings are equal, a value > 0 if S1 > S2 and a value < 0 if S1 < S2.


function UpperCase(S: WideString): WideString
Returns a copy of a string in uppercase.


function LowerCase(S: WideString): WideString
Returns a copy of a string in lowercase.


function StringReplace(S, OldPattern, NewPattern: WideString; ReplaceAll : Boolean; IgnoreCase : Boolean; WholeWord: Boolean): WideString 
Returns a string with occurrences of one substring replaced by another substring. If ReplaceAll is set to True, the all occurrences of OldPattern will be replaced. If IgnoreCase is set to True, then the comparison operation is case insensitive.


function StrToInt(const S: WideString): Integer
Converts a string that represents an integer (decimal or hex notation) to a number.


function IntToStr(const Value: Integer): WideString
Converts an integer number to a string.


function StrToFloat(const S: WideString): Extended
Converts a given string to a floating-point value.


  function FloatToStr(const Value: Extended): WideString
Converts a floating point value to a string.


function HTMLValues(const HTML : WideString; ABegin, AEnd, ItemBegin, ItemEnd : WideString; ValDelim : WideString; var Pos : Integer) : WideString
Gathers all values from HTML into a string using ValDelim as delimiter. The function searches for ABegin first starting from Pos, from this point it gathers values between ItemBegin and ItemEnd until AEnd is found. The last position is returned in Pos variable.


function HTMLValues2(const HTML : WideString; ABegin, AEnd, ItemBegin, ItemEnd : WideString; ValDelim : WideString; var Pos : Integer) : WideString
The same as HTMLValues, but this version searches for an end tag character > after each ItemBegin.


function TextBetween(const HTML : WideString; ABegin, AEnd : WideString; doLineBreaks : Boolean; var Pos : Integer) : WideString
Returns plain text without HTML tags from HTML, between ABegin and AEnd. The function searches for ABegin from position Pos.


function HTMLToText(const HTML : WideString) : WideString
Replaces all escaped HTML characters (like &lt;&gt;&amp;&quot;) with normal characters (like <, >, $, ")

[nostra]

10. Functions for filling fields (at last )


10.1. Functions for filling standard fields (for information import plugins)

procedure AddSearchResult(Title1, Title2, Year, URL, PreviewURL : WideString)
Adds search result. You do not need to provide information for all the variables. Most import is to assign at least Title1 or Title2 and URL. If you do not have information to assign to a certain variable just pass ''. Ex.: AddSearchResult('Terminator', '', '', 'http://www.web-site.com/terminator.html', ''); //passes Title and URL only


procedure AddFieldValue(AField: Integer; AValue : WideString)
Fill a certain standard field with (AValue). AField can have following values:
For movies:

0	URL
1	Title
2	Original Title
3	Aka
4	Year
5	Genre
6	Category
7	Country
8	Studio
9	MPAA
10	Additional rating
11	Tags
12	Tagline
13	Description
14	Duration
15	Features

For persons:

0	URL
1	Name
2	Translated name
3	Alternative names
4	Date of birth
5	Place of birth
6	Genre
7	Biography
8	Date of death

procedure AddMoviePerson(Name, TransName, Role, URL : WideString; AType : Byte)
Adds movie credits (Actor, director, etc.). AType can have one of the following values:

0	Actor
1	Director
2	Writer
3	Composer
4	Producer

procedure AddPersonMovie(Title, OrigTitle, Role, Year, URL : WideString; AType : Byte)
Adds a movie to a person's filmography. AType can have one of the following values:

0	Actor
1	Director
2	Writer
3	Composer
4	Producer

procedure AddAward(Event, Award, Category, Recipient, Year: WideString; const Won : Boolean)
Adds an award. If you do not have information to assign to a certain variable just pass ''.


procedure AddConnection(Title, OrigTitle, Category, URL, Year: WideString)
Adds movie connection. If you do not have information to assign to a certain variable just pass ''.


procedure AddEpisode(Title, OrigTitle, Description, URL, Year, Season, Episode : WideString)
Adds an episode. If you do not have information to assign to a certain variable just pass ''.


10.2. Functions for filling other fields

procedure AddFieldValueXML(AField: WideString; AValue : WideString)
This function can be used to fill all fields except custom fields and structured fields like (credits, connections, episodes, filmography). Pass field value in AValue variable. AField is a name of the field and can have one of the following values:

For movies:

num
title
origtitle
aka
year
genre
country
studio
release
mpaa
location
category
tagline
description
count
type
rating
imdbrating
orating
orname
rip
length
langs
translation
resolution
videocodec
videobitrate
audiocodec
audiobitrate
size
url
path
comment
dateadded
code
label
loan
subs
framerate
label
features
viewed
bookmark
wish
loandate
viewdate
tags

For persons:

name
transname
altnames
birthday
birthplace
bio
death

procedure AddCustomFieldValueByNumber(CustomNumber: Integer; AValue : WideString)
Sets value of a custom field by field number.
CustomNumber - number of a custom field
AValue - field value (use '-1' for True(checked) and '0' for False)


procedure AddCustomFieldValueByName(CustomName: WideString; AValue : WideString)
Sets value of a custom field by field name.
CustomName - field name
AValue - field value (use '-1' for True(checked) and '0' for False)

[nostra]

11. Other built in functions

procedure ShowMessage(const Msg, Head : WideString)
Show a message box with OK button.
Head - Window title
Msg - Message text


12. Debugging the script (check for errors)

Before you can tell the script is ready you must test it first. Start PVD with -debug command line parameter, like this: viddb.exe -debug
When you do this an additional menu item will become visible in "Help" menu called "Log".
In the log you can track following information:

• what plugins are loaded by PVD
• what scripts are compiled and executed
• what errors were found while compiling a script
• what URLs are downloaded by PVD

If a script is wrong (contains errors) it will not be loaded into the program, so it can not be selected from the Import menu or visible in Preferences -> Plugins.

If an error acquired while compiling a script, an error message apears in log with detailed information about the wrong line in script code and type of an error.
Ex.: [Error] (366:2): Semicolon (';') expected

Sometimes you can find hints produced by script compiler.
Ex.: [Hint] (368:2): Variable 'TEST' never used

If a script can be compiled but does not work as it should you could use the ShowMessage function to output variable values and check the URLs PVD downloads in program log.

The last page downloaded is save into a file page.html in program directory while in -debug mode.