Author Topic: Personal Video Database 1.0.2.7 MOD  (Read 277798 times)

0 Members and 14 Guests are viewing this topic.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #20 on: August 04, 2018, 07:08:04 am »
Really this is a correction of a first version error. The Script Manual says:

    procedure AddAward(Event, Award, Category, Recipient, Year : String; const Won : Boolean);
    Adds an award to a record. Parameters:
    Event = name of the event
    Award = award name
    Category = award category
    Recipient = for movie records this variable should contain the name of a specific person who won the award and
                     for people records the variable should contain the title of a movie for which the person won the award
    Year = release year of a movie (only applicable when adding award to a person record)
    Won = set to true if the recipient won the award and to false otherwise

This needs to be read on several occasions, but it is not yet certain that it will be clarified correctly as it was in your case.

    procedure AddAward(Event, Award, Category, Recipient, Year : String; const Won : Boolean);
    Adds an award to a record. Parameters:
.
.
.
    Recipient = for movie records this variable should contain the name of a specific person who won the award and
                     for people records the variable should contain the title of a movie for which the person won the award
    Year = release year of a movie (only applicable when adding award to a person record)
...

He also writes that movie awards have a year of prize and the name of the actor he has received.

He also writes for people awards, where the actor won the prize for a movie without recording for release year.

In the first version the movie script added the "EventYear" (normally is the next year of the movie). But now we can add awards with the people script and then the year must be the movie year so we could have the same award with different years. So, in order to respect the Script Manual, the movie scripts doesn't write the year and is the people scripts who writes the year.

In the first version, the movie script was all prailono for aswards as it should be.

BTW, I have hesitated to write the year of the movie instead of leaving it blank but I do not know the consistency of the data. What do you think?

The correct thing is that movie awards code is the same as it was in the first version of the movie script.

It is right for people awards to record movie names not to include a record for release year. I know from my own experience in testing my people awards script, where my record looks like this [Casablanca (1942)]. The correct record is only Casablanca.

In the first version the movie script added the "EventYear" (normally is the next year of the movie).

This is not always true. A good example is Casablanca (1942) and the Academy Awards, USA , where awards were only awarded in 1944.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #21 on: August 06, 2018, 10:55:59 am »
Well, I don't understand all of your arguments but I "buy" : Return to the first solution for the movie
Quote
AddAward(EventName, AwardName, AwardCategory, AwardRecipient, EventYear, AwardWon);

and the same use EventYear for the People script in order to warrant the same data (I have done the test with Casablanca and Michael Curtiz).

Ivek, thank you very much for your explanation (I attach the scripts and the screenshots).

BTW, I want to publish a new release the next Wednesday, 8/8/2018 (nice date, isn't it?) with those scripts, proxy support and other bugs corrected.

Dear users, please, tell me all you bug discovers in order to correct it fast for this next version (and the 9/8/2018 we'll can go in holidays  ;D )

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #22 on: August 07, 2018, 06:54:56 am »
IMDB_People_[EN][HTTPS]

Now that awards have been added, IMDB_People_ [EN] [HTTPS] script does not download (adds to the database) more photos of the players. It was not previously (it was a download) until the award code was added to the script. Obviously, I tested a script with a award code (photos did not add in the database) and script, when I removed the award code (photos added to the database), that the award code prevents the addition of photos to the database.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #23 on: August 08, 2018, 08:18:10 am »
Is the famous problem with downloading images in the people PVD part. I have forgotten to keep the rondabout result througth the awards part. It will be solved easily.

Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #24 on: August 08, 2018, 10:31:31 am »
Changelog:
  • MOD V.2018.08.08; VVV, Ivek23: Proxy support (if configured in PVD).
                                           IMDB Movie now retrieve Movie Technical Specs page (Ivek23) and use EventYear in awards.
                                           IMDB People now retrieve full biography page (Ivek23), use EventYear in awards and image bug with full awards solved.
                                           FilmAffinity change the ~poster~ begin string because API change.
Last version always in the first post

Offline jondak

  • User
  • ***
  • Posts: 33
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #25 on: August 13, 2018, 09:53:29 am »

Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #26 on: August 13, 2018, 05:35:30 pm »
I think that what is the problem: can you edit the PVdBScriptConfig.ini with notepad and change GET_FUL_FEATURES by GET_FULL_FEATURES with two L and try again ?
Thank you for your help.

Offline jondak

  • User
  • ***
  • Posts: 33
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #27 on: August 14, 2018, 08:06:03 am »
Yes, with that modification it worked.

Thank you.

Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #28 on: August 14, 2018, 07:54:48 pm »
 :) The problem of the hurry. It will be corrected in the next version. Thank you.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #29 on: August 19, 2018, 02:20:07 pm »
An many errors in IMDB_ [EN] [HTTPS] script.

Errors are in Rating, Top 250 IMDB Votes, Studio and Full Actors in the Full Cast and Crew section. This happened because of the change of source code on IMDB Movies web pages. I've already tested the code corrections and code corrections will be published in a few days. I beg you for patience, because I'm a little more close to my free time.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline sihyp

  • Member
  • *
  • Posts: 11
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #30 on: August 19, 2018, 02:45:18 pm »
Yes, you've been working here so hard.
Yvek, you deserve it. :)

Offline AimHere

  • Older Power User
  • *****
  • Posts: 213
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #31 on: August 20, 2018, 05:36:32 pm »
Other solutions and scripts will be ready for the near future, according to the users' wishes.

I'd like to request that the AdultDVDEmpire script be adapted for this mod. I've been sort of maintaining the ADE script in the past, but I can lo longer get PVD and Proxomitron to play nice together. From what I can glean from looking at the mod's scripting, the mod uses an external program to download all https:// web pages, then parses them locally. This is beyond my capabilities to adapt.

Thanks,
aimhere

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #32 on: August 22, 2018, 06:26:13 am »
Yes, you've been working here so hard.
Yvek, you deserve it. :)

Thanks, I'm doing my best.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #33 on: September 02, 2018, 07:58:35 am »
IMDB_ [EN] [HTTPS] script

In the tests I found several errors that I corrected and added some of the code parts for better information transfer. Added some new code sections for downloading additional information.

An many errors in IMDB_ [EN] [HTTPS] script.

Errors are in Rating, Top 250 IMDB Votes, Studio and Full Actors in the Full Cast and Crew section. This happened because of the change of source code on IMDB Movies web pages. I've already tested the code corrections and code corrections will be published in a few days. I beg you for patience, because I'm a little more close to my free time.

First of all, what I've corrected for changing the source code on IMDB pages.

Full Cast and Crew section

I've repaired pieces of code for transmitting actors information.

Quote
Function ParsePage_IMDBMovieCREDIT(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //          ~actors~ ctActors
  Var
    curPos,endPos,index:Integer;
    ItemList:String;
    Name,Role,PersonURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBMovieCREDIT BEGIN=====================||');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
  //Get ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
.
.
    //Get  to "Cast" ~actors~ ctActors
    //Go Cast list
    curPos:=Pos('<table class="cast_list">',HTML);                                     //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=curPos+Length('<table class="cast_list">');                                //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
    endPos:=PosFrom('</div>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="primary_photo">',ItemList)                               //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF; //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        //Name:=TextBetWeen(ItemList,'itemprop="name">','<',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                             //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
        //Get Role
        //Role:=TextBetWeen(ItemList,'<td class="character">','</a>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
        Role:=TextBetWeen(ItemList,'<td class="character">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC         
        LogMessage('      Parse Results Role:'+Role+'||');
      Role:=StringReplace(Role,'       (',' (',True,False,True);
      Role:=StringReplace(Role,'(uncredited)                    ','(uncredited) • ',True,False,True);
      Role:=StringReplace(Role,')                    ',') ',True,False,True);
      Role:=StringReplace(Role,' /              ',' - ',True,False,True);
      Role:=StringReplace(Role,'         / ...                    ',' • ',True,False,True);
      Role:=StringReplace(Role,'                  ',' • ',True,False,True);      
      Role:=StringReplace(Role,'      (uncredited)',' (uncredited)',True,False,True);
      LogMessage('      Parse Results Role_1:'+Role+'||');   
   
        AddMoviePerson(Name,'',Role,PersonURL,ctActors);
        LogMessage('      Get results cast:#'+IntToStr(index)+'|'+Name+'|'+Role+'|'+PersonURL+'||ctActors');                                               
        curPos:=PosFrom('<td class="primary_photo">',ItemList,curPos);                                  //String which opens the Web Result item List data. WEB_SPECIFIC
        index:=index+1;
    End;
    LogMessage('Function ParsePage_IMDBMovieCREDIT END=====================||');
  End; //BlockClose

Note:
At Actors is also part of the role code for the actors series informations.


I corrected the pieces of code for transmitting information to the director, writer, producer and music, because in certain cases it transmitted wrong information with Full Cast and Crew pages. This happened when there was no information for actors, writer, producer or music .

For example, the same name was added to the database for the director and producer, even though the producer on the Full Cast and Crew site was not mentioned. Director, writer, producer and music do not have role information in dastabase. For the correct transfer of information in the database, I added all the roles to everyone. Role and LogMessage are also for the debug log file. The role is not visible in the database information unless the transfer was in the custom field.

Part 1:
Quote
  //Get ~crew~ctDirectors,ctWriters,ctComposers,ctProducers
    //Go to "Directed by" ~crew~ctDirectors
    //curPos:=Pos('Directed by',HTML);                                                   //Strings start which opens the block content data. WEB_SPECIFIC
    curPos:=Pos('<h4 class="dataHeaderWithBorder">Directed by',HTML);                                                   //Strings start which opens the block content data. WEB_SPECIFIC   
    curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
    curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
    //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~ n the ~crew~
    endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
    ItemList:=Copy(HTML,curPos,endPos-curPos);
    //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
    curPos:=Pos('<td class="name">',ItemList)                                         //String which opens the subList data. WEB_SPECIFIC
    index:=1;
    While curPos>0 Do Begin
        If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
        //Get PersonURL (Always must it has)
        PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
        LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
        //Get Name (Always must it has)
        //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
        Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        LogMessage('      Parse Results Name:'+Name+'||');
      //Get Role NO: PVD don't save Role in crew people
      //Get Role
      Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
      LogMessage('      Parse Results Role:'+Role+'||');

        AddMoviePerson(Name,'','',PersonURL,ctDirectors);
        LogMessage('      Get results Director:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctDirectors'); 
        curPos:=PosFrom('<td class="name">',ItemList,curPos)                                            //String which opens the subList data. WEB_SPECIFIC
        index:=index+1;
    End;           
    //Go to "Writer:" or "Writers:" ~crew~ctWriters
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Writing Credits',HTML);
   If 0<curPos Then Begin   

    //curPos:=Pos('Writing',HTML);                                                       //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Writing Credits',HTML);                                                       //Strings start which opens the block content data. WEB_SPECIFIC   
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');
      curPos:=Pos('<td class="name">',ItemList)                                            //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');   
      
         AddMoviePerson(Name,'','',PersonURL,ctWriters);
         LogMessage('      Get results Writer:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctWriters'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)     //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #34 on: September 02, 2018, 08:01:34 am »
Part 2:

Quote
    //Go to "Producers" ~crew~ctProducers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Produced by',HTML);
   If 0<curPos Then Begin
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Produced by',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC   

      //curPos:=Pos('Produced',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');;
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'','',PersonURL,ctProducers);
         LogMessage('      Get results Producers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctProducers'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;   
    End;
    //Go to "Music" ~crew~ctComposers
   curPos:=Pos('<h4 class="dataHeaderWithBorder">Music by',HTML);
   If 0<curPos Then Begin

      //curPos:=Pos('Music',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC
      curPos:=Pos('<h4 class="dataHeaderWithBorder">Music by',HTML);                                                         //Strings start which opens the block content data. WEB_SPECIFIC      
      curPos:=PosFrom('</h4>',HTML,curPos);                                              //Strings end which opens the block content data.  WEB_SPECIFIC
      curPos:=curPos+Length('</h4>');                                                    //Strings end which opens the block content data.  WEB_SPECIFIC
      //Get all "raw" crew summary (in raw because we need the hidden person links) May one person or severals in the ~crew~
      endPos:=PosFrom('</table>',HTML,curPos);                                             //Strings which opens/closes the data. WEB_SPECIFIC
      ItemList:=Copy(HTML,curPos,endPos-curPos);
      //LogMessage('           Parse results ('+IntToStr(curPos)+','+IntToStr(endPos)+') complex ItemList:'+ItemList+'||');List+'||');
      curPos:=Pos('<td class="name">',ItemList)                                               //String which opens the subList data. WEB_SPECIFIC
      index:=1;
      While curPos>0 Do Begin
         If (index>PEOPLE_LIMIT) Then break;     //Limited depassed (Remember index begin in 0).
         //Get PersonURL (Always must it has)
         PersonURL:=BASE_URL_PERSON_PRE + TextBetWeen(ItemList,'<a href="/name/','/',false,curPos) + BASE_URL_SUF;     //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results PersonURL:'+PersonURL+'||');         
         //Get Name (Always must it has)
         //Name:=TextBetWeen(ItemList,'>','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC
         Name:=TextBetWeen(ItemList,'> ','</a>',false,curPos);                                 //Strings which opens/closes the data. WEB_SPECIFIC         
         LogMessage('      Parse Results Name:'+Name+'||');
         //Get Role NO: PVD don't save Role in crew people
         //Get Role
         Role:=TextBetWeen(ItemList,'<td class="credit">','</td>',false,curPos);                    //Strings which opens/closes the data. WEB_SPECIFIC
         LogMessage('      Parse Results Role:'+Role+'||');

         AddMoviePerson(Name,'',Role,PersonURL,ctComposers);
         LogMessage('      Get results Composers:#'+IntToStr(index)+'|'+Name+'|'+PersonURL+'||ctComposers'); 
         curPos:=PosFrom('<td class="name">',ItemList,curPos)                                                   //String which opens the subList data. WEB_SPECIFIC
         index:=index+1;
      End;
    End;
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #35 on: September 02, 2018, 08:34:22 am »
IMDB_ [EN] [HTTPS] script

ParsePage_IMDBSearchTitle

I corrected the pieces of code of search results for movies, where there are now missing posters.

Example for this link:
https://www.imdb.com/search/title?title=Lawrence+Of+Arabia&title_type=feature&release_date=,&view=simple
Poster link:
https://m.media-amazon.com/images/G/01/imdb/images/nopicture/140x209/film-4001654135._CB470041848_.png


Quote
Function ParsePage_IMDBSearchTitle(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prDownload; (Global var DownloadURL=IMDB page) if there is one results.
    //     Result:=prList; if there are several results. (Don't work with Preferences/Plugings/Silent Mode).
    //     Result:=prError; If not results
  Var
      curPos,ResultsNumber:Integer;
      ItemValue,ImageFile:String;
      Title,Year,MovieURL,PreviewURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBSearchTitle BEGIN======================|');
    LogMessage('Global Var-Mode|'+IntToStr(Mode)+'|');
    LogMessage('Global Var-DownloadURL|'+DownloadURL+'|');   
    If (Pos('No results',HTML)<>0) Then Begin                                           //WEB_SPECIFIC
        LogMessage('        No Movies results');
        Result:=prError;
    End Else Begin
        LogMessage('        Parsing search Movies results');
        curPos:=Pos('<div class="lister-list">',HTML);                                 //String which opens the Web Search List data. WEB_SPECIFIC
        //Get first item
        ResultsNumber:=0;
        curPos:=PosFrom('<div class="lister-item mode-simple">',HTML,curPos);            //String which opens the Web Result List data. WEB_SPECIFIC
        While curPos>0 Do Begin
            ResultsNumber:=ResultsNumber+1;
            LogMessage('    Parsing search movie results in '+IntToStr(curPos)+' ...');
            curPos:=PosFrom('<div class="lister-item-image">',HTML,curPos);             //String which opens the Web URL+Title+URL+PreviewURL data. WEB_SPECIFIC
            //Get PreviewURL
            //PreviewURL:='';
            If POSTER_IN_SEARCH Then begin
                ItemValue:=TextBetWeen(HTML,'loadlate="','"',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC
                If (Pos('https://m.media-amazon.com/images/G/01/imdb/images/',ItemValue)>0) Then Begin
                    PreviewURL:=TextBetWeenFirst(ItemValue,'https://m.media-amazon.com/images/G/01/imdb/images/','.');     //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
                    If (Length(PreviewURL)>0) then begin
                        PreviewURL:='https://m.media-amazon.com/images/G/01/imdb/images/' + PreviewURL                        //Base poster URL without '.jpg'. WEB_SPECIFIC
                        ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-SearchResult-'+IntToStr(ResultsNumber)+'-.jpg'
                        // Avoid HTTPS redirection: Download https image to file  OutPutFile=
                        If (1=DownloadImage(PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:'+PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end else if (1=DownloadImage(PreviewURL +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:' + PreviewURL + '.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end;       
                    end;
                end;
            //end;     
       
            //Get PreviewURL         
            //PreviewURL:='';
            //If POSTER_IN_SEARCH Then begin
                //ItemValue:=TextBetWeen(HTML,'loadlate="','"',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC            
                If (Pos(BASE_URL_IMAGE_PRE_TRUE,ItemValue)>0) Then Begin
                    PreviewURL:=TextBetWeenFirst(ItemValue,BASE_URL_IMAGE_PRE_TRUE,'.');     //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
                    If (Length(PreviewURL)>0) then begin
                        PreviewURL:=BASE_URL_IMAGE_PRE_TRUE + PreviewURL                        //Base poster URL without '.jpg'. WEB_SPECIFIC
                        ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-SearchResult-'+IntToStr(ResultsNumber)+'-.jpg'
                        // Avoid HTTPS redirection: Download https image to file  OutPutFile=
                        If (1=DownloadImage(PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:'+PreviewURL + '._V1_UY' + IntToStr(MAX_SEARCHLIST_HEIGTH) + '_.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end else if (1=DownloadImage(PreviewURL +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                            LogMessage('      Parse Results PreviewURL:' + PreviewURL + '.jpg'+'||');
                            PreviewURL:=ImageFile;  //Pass the downloaded file to PVD
                        end;       
                    end;
                end;
            end;
            //Get MovieURL  (Always must it has)
            MovieURL:=TextBetWeen(HTML,'<a href="/title/','/',false,curPos);               //Strings which opens/closes the data. WEB_SPECIFIC
            MovieURL:= BASE_URL_PRE+MovieURL+BASE_URL_SUF;
            LogMessage('      Parse Results URL:'+MovieURL+'||');
            //If true HTPPS link PVdB gives "Socket Error # 11001"
            //If ancient http, get redirected and give "IOHandler value is not valid
            //If invalid url, "HTTP/1.1 400 Bad Request"               
            MovieURL:=CHEAT_PREFIX_URL+MovieURL;                 //'Cheat' URL for avoid HTTPS issue in GET return.
            LogMessage('      Parse Results CHEAT URL:'+MovieURL+'||');
            //Get Title (Always must it has)
            Title:=TextBetWeen(HTML,'>','<',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            LogMessage('      Parse Results Title:'+Title+'||');
            //Get Year
            Year:=TextBetWeen(HTML,'(',')',false,curPos);                               //Strings which opens/closes the data. WEB_SPECIFIC
            Year:=StringReplace(Year,'typeof uet == '+Chr(39)+'function'+Chr(39),'',True,False,False);    //Eliminate the CHEAT_PREFIX_URL for get the true movie link           
            LogMessage('    Parse Results in Year:'+Year+'||');
            AddSearchResult(Title,'',Year,MovieURL,PreviewURL);
            //Just to check the website (Only to check the web page) not PVdB valid result.
            if CHECK_WEBSITE then AddSearchResult('                             '+#8729+' Just to check the website: '+StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'','',StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False),'');
            curPos:=PosFrom('<div class="lister-item mode-simple">',HTML,curPos)          //String which opens the Web Result item List data. WEB_SPECIFIC
        End;
        If ResultsNumber=1 Then Begin
            DownloadURL:=StringReplace(MovieURL,CHEAT_PREFIX_URL,'',True,False,False);    //Eliminate the CHEAT_PREFIX_URL for get the true movie link
            DownloadURL:=StringReplace(DownloadURL,'http:','https:',True,True,False);     //The direct url needed (no https redirection)
            Result:=prDownload; //One result in the DownloadURL.
            LogMessage('      After parsing search Movies,only one result detected');   
        End Else Begin
            Result:=prList; //Don't work with Preferences/Plugings/Silent Mode.
            LogMessage('      After parsing search Movies go to choose List Results');   
        End;
    End;
    LogMessage('Function ParsePage_IMDBSearchTitle END=====================||');
    Exit;
End; //BlockClose


ParsePage_IMDBMovieMPAA

I corrected parts of the code for ParsePage_IMDBMovieMPAA.

Quote
Function ParsePage_IMDBMovieMPAA(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError;  If żany big problem? with exit
    //Retrieve: ~MPAA~
  Var
    curPos:Integer;
    mpaaValue:String;
  Begin
.
.
    LogMessage('      MPAA.USA.Corrected:'+mpaaValue+'||');
   mpaaValue:=StringReplace(mpaaValue,'update: ','',False,False,True);                //WEB_SPECIFIC
   LogMessage('      MPAA rating: '+mpaaValue+' ||');
    //Not or bad MPAA USA then get Spain certification (if exist) and convert in USA-KODI values
.
.
    End;
 
    AddFieldValueXML('mpaa',mpaaValue);
    LogMessage('      Get result MPAA='+mpaaValue+'||');
    LogMessage('Function ParsePage_IMDBMovieMPAA END=====================||');
  End; //BlockClose


Function ParsePage

I added part of the code for Function ParsePage.

Quote
//USER FUNCTIONS AND PROCEDURES==================================================================================BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockOpen
Function ParsePage(HTML:String;URL:AnsiString):Cardinal; //BlockOpen
  Var
    MovieID,titleValue,yearValue:String;
    Date:String;
    DateParts: TWideArray;
   
  Begin
    LogMessage('Function ParsePage BEGIN======================|');
    LogMessage('Global Var-Mode|'+IntToStr(Mode)+'|');
    LogMessage('Global Var-DownloadURL|'+DownloadURL+'|');
    LogMessage('Local Var-URL|'+URL+'|');
//Parse return in mode 'Search Movie URL'--------------------------------------------------------------------------------         
.
.
//Parse with the Movie URL 'smNormal'------------------------------------------------------------------------------------
    If (Mode=smNormal) Then Begin
        Result:=prFinished;  //It will change to prError if any big problem with exit; 
        //Information in several page. Needs Provider MovieID
        MovieID:=TextBetWeenFirst(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_SUF);
        LogMessage('  ParsePage mode smNormal|'+IntToStr(Mode)+'|. Getting provider data for MovieID|'+ MovieID +'|');
        AddCustomFieldValueByName('IMDbID',MovieID);
        LogMessage('      Get result IMDbID:'+MovieID+'||');

        //Get ~url~
        AddFieldValueXML('url',StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False));
        LogMessage('      Get result url:'+StringReplace(DownloadURL,BASE_URL_PRE_TRUE,BASE_URL_PRE,True,False,False)+'||');
    //Parse principal provider page = BASE_URL_PRE_TRUE------------------------------------------------------------------
.
.         
    //Date ~Updated~ (choose simple or verbose version)
        Date:=DateToStr(CurrentDateTime);
        if Date <> '' then
         begin
            ExplodeString(Date,DateParts,'-');
            Date:=DateParts[2]+'.'+ DateParts[1]+'.'+DateParts[0];   
            AddCustomFieldValueByName('Updated',Date+' at '+TimeToStr(CurrentDateTime)+' • '+SCRIPT_NAME+' '+SCRIPT_VERSION);  // Annoying 
        End;   
      
        LogMessage('    Provider data info retreived Ok in '+DateToStr(CurrentDateTime)+' '+TimeToStr(CurrentDateTime)+'|');
        LogMessage('Function ParsePage NORMAL END======================|');
        exit;
    End;
//Parse with unknow mode-----------------------------------------------------------------------------------------------   
    Result:=prError;
    exit;
End; //BlockClose
//OBLIGATORY CALLBACK PRINCIPAL FUNCTION=========================================================================BlockClose   
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #36 on: September 02, 2018, 09:21:50 am »
IMDB_ [EN] [HTTPS] script

I added and Corrected part of the code for Function ParsePage_IMDBMovieBASE to now work properly.

Part 1:

//SCRIPT INFO

Quote
//SCRIPT INFO================================================================================================== BlockClose
//SCRIPT CONSTANTS============================================================================================= BlockOpen
Const
//Script types-----------------------------------------------------------------------------------------------------------
  stMovies        = 0;
  stPeople        = 1;
  stPoster        = 2;
  stScreenShots   = 3;
  stCover         = 4;
//Script Obligatory Infos----------------------------------------------------------------------------------------------
  SCRIPT_NAME  = 'IMDB [EN][HTTPS]_1'; //The most complete information provider
  SCRIPT_DESC  = 'IMDB download Movie info [EN]_1'; 
  SCRIPT_VERSION  = '1.1.0.0';
  SCRIPT_TYPE  = stMovies;
  SCRIPT_LANG  = $09; //English //$0a es-Spanish   $0c fr-French   $09 en-English
  RATING_NAME    = 'IMDB'; //PVD has ~imdbrating~ for this value
  CODE_PAGE      = 65001; //28591=ISO 8859-1 Latin 1; Western European (ISO). Use: 65001=Unicode (UTF-8) | 0=for Autodetect   Autodetect   
//Script Options-------------------------------------------------------------------------------------------------------
  //Retreive Data Config
  GET_MAINPAGE_ACTORS_CREDIT  = True ;  //Set to True to ensure MAINPAGE_ACTORS_CREDIT info is added. Otherwise: Set to False to ensure MAINPAGE_ACTORS_CREDIT info is not added. MAINPAGE_ACTORS_CREDIT info is in to the principal movie page.
  GET_FULL_ACTORS_CREDIT  = True ;  //Set to True to ensure FULL_ACTORS_CREDIT info is for Full Cast & Crew provider page. Otherwise: Set to False to ensure FULL_ACTORS_CREDIT info is not added.
  GET_FULL_MPAA  = True ;  //Download Certification provider page for retreive the MPAA info. Otherwise only the info of the principal movie page.
  GET_FULL_AKA  = True ;  //Download 'Also Known As' provider page for retreive the info. Otherwise only the info of the principal movie page.
  GET_FULL_CREDIT  = True ;  //Download Cast or Credit provider page for retreive the info. Otherwise only the info of the principal movie page.
  PEOPLE_LIMIT  = 25;    //Limit of number of actors (cast) or of credits (crew) retrieved.
  GET_FULL_AWARDS  = True ;  //Download Awards provider page for retreive the info. Otherwise doesn't do nothingh because no info in the principal movie page.
  EVENTS_LIMIT  = 100;   //Limit of number of events (USA Academy Awards, Golden Globes, etc) to retrive awards.
  GET_FEATURES  = True ; //Retreive the original movie features (Runtime, Sound Mix, Color, Aspect Ratio).
  GET_FULL_FEATURES  = True ; //Download Technical Specs for retreive the original movie features (Runtime, Sound Mix, Color, Aspect Ratio, etc).
  MAX_IMAGE_HEIGTH  = 1200; //Heigth limit of the stored posters.
  //Process Data Config
.
.
//Download methods-------------------------------------------------------------------------------------------------------
  dmGET           = 0;
  dmPOST          = 1;
//Movie fields----------------------------------------------------------------------------------------------------------
   mfURL      = 0;
   mfTitle      = 1;
   mfOrigTitle   = 2;
   mfAka      = 3;
   mfYear      = 4;
   mfGenre      = 5;
   mfCategory   = 6;
   mfCountry   = 7;
   mfStudio      = 8;
   mfMPAA      = 9;
   mfRating      = 10;   //This is 'Additional rating', not 'Rating'
   mfTags      = 11;
   mfTagline   = 12;
   mfDescription   = 13;
   mfDuration   = 14;
   mfFeatures   = 15;

//Credits types----------------------------------------------------------------------------------------------------------

Function ParsePage_IMDBMovieBASE

Quote
Function ParsePage_IMDBMovieBASE(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit;
    //Retrieve: ~title~, ~year~, ~origtitle~, ~poster~ / ~imdbrating~, ~IMDB_Votes~ (Custom Field) / ~TOP_250~(Custom Field) /
    //          If Not(GET_FULL_CREDIT): ~crew~ctDirectors,ctWriters,ctComposers,ctProducers(Not in base page), ctActors
    //         ~description~ / ~category~ "keywords" / ~tagline~ / ~genre~
    //         If Not(GET_FULL_MPAA) ~mpaa~
    //         ~country~ / ~rdate~ in contry provider local IP geolocation
    //         If Not(GET_FULL_AKA) ~aka~.
    //         ~budget~ / ~money~ / ~studio~ "Production Co"
   //         If GET_FEATURES ~features~   
    //         //If Not(GET_FULL_FEATURES) ~features~
  Var
      curPos,endPos,index:Integer;
      ItemValue,ItemList,ImageFile:String;
      titleValue:String;
      Name,Role,PersonURL:String;
  Begin
    LogMessage('Function ParsePage_IMDBMovieBASE BEGIN======================|');
    Result:=prFinished;  //It will change to prError if any big problem with exit;
    //Because the script don't retreives the data in the page order a token search for the first curPos position or block select is mandatory
    //Get ~title~, ~year~, ~origtitle~, ~poster~
    //Get all "raw" title summary (in raw because we need the hidden links, we avoid "complete" token in strings which opens/closes)
    //ItemList:=TextBetWeenFirst(HTML,'<div class="title_wrapper"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.
    ItemList:=TextBetWeenFirst(HTML,'<div class="title_block"','div class="plot_summary_wrapper">'); //WEB_SPECIFIC.   
    If (Length(ItemList)>0) Then Begin           
        //titleValue:=TextBetWeenFirst(ItemList,'<h1 itemprop="name" class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC
        titleValue:=TextBetWeenFirst(ItemList,'<h1 class="">','<');       //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('title',titleValue);
        LogMessage('      Get result title:'+titleValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'(','–');                                 //Strings which opens/closes the data. WEB_SPECIFIC   
        LogMessage('      Get result year_00:'+TextBetWeenFirst(ItemList,'(','–')+'||');           
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year_0:'+ItemValue+'||');   
   
        //ItemValue:=TextBetWeenFirst(ItemList,'(',')');                                 //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=TextBetWeenFirst(ItemList,'<span id="titleYear">(',')</span>            </h1>');                                 //Strings which opens/closes the data. WEB_SPECIFIC      
        AddFieldValueXML('year',ItemValue);
        LogMessage('      Get result year:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,'<div class="originalTitle">','<');          //Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)=0) Then ItemValue:=titleValue;                           //Provider hides the original title if same that title. WEB_SPECIFIC
        AddFieldValueXML('origtitle',ItemValue);
        LogMessage('      Get result origtitle:'+ItemValue+'||');
        ItemValue:=TextBetWeenFirst(ItemList,BASE_URL_IMAGE_PRE_TRUE,'.');                 //Get poster code. Strings which opens/closes the data. WEB_SPECIFIC
        If (Length(ItemValue)>0) then begin
            ItemValue:=BASE_URL_IMAGE_PRE_TRUE + ItemValue;                             //Base poster URL without '.jpg'. WEB_SPECIFIC
            ImageFile:=GetAppPath+'Scripts\'+BASE_DOWNLOAD_FILE_IMAGE_NAME+'-Poster.jpg'
            // Avoid HTTPS redirection: Download https image to file  OutPutFile=
            If (1=DownloadImage(ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg',ImageFile)) then begin  //Dowload in the selected user max size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:'+ItemValue + '._V1_UY' + IntToStr(MAX_IMAGE_HEIGTH) + '_.jpg'+'||');
            end else if (1=DownloadImage(ItemValue +'.jpg',ImageFile)) then begin  //Dowload in the web base size. WEB_SPECIFIC
                AddImageURL(itPoster,ImageFile);    //Get the photo from a file even if the next line we log the true URL.
                LogMessage('      Get result poster:' + ItemValue + '.jpg'+'||');
            end;       
        end;
    End;
    //Get ~imdbrating~, ~IMDB_Votes~
    //curPos:=Pos('<span itemprop="ratingValue">',HTML);                      //WEB_SPECIFIC.
    curPos:=Pos('<div class="ratingValue">',HTML);                      //WEB_SPECIFIC.
    If 0<curPos Then Begin                         
       //ItemValue:=TextBetWeen(HTML,'<span itemprop="ratingValue">','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=TextBetWeen(HTML,'<span>','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,',','.',True,True,False);                           //Decimal comma spanish separator to point english separator.
       AddFieldValueXML('imdbrating',ItemValue);
       LogMessage('      Get result imdbrating:'+ItemValue+'||');
       //ItemValue:=TextBetWeen(HTML,'<span class="small" itemprop="ratingCount">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=TextBetWeen(HTML,'<span class="small">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC      
       ItemValue:=StringReplace(ItemValue,'.','',True,True,False);            //Remove thousand point spanish separator.
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);            //Remove thousand comma english separator.
       AddCustomFieldValueByName('IMDB_Votes',ItemValue);
       LogMessage('      Get result IMDB_Votes:'+ItemValue+'||');
    End;
    //Get ~TOP_250~
    curPos:=Pos('Top Rated Movies #',HTML);                                 //WEB_SPECIFIC.
    If 0<curPos Then Begin
       ItemValue:=TextBetWeen(HTML,'Top Rated Movies #','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       AddCustomFieldValueByName('TOP_250',ItemValue);
       LogMessage('      Get result TOP_250:'+ItemValue+'||');
    End;
    //Get ~Bottom 100~
    curPos:=Pos('Bottom Rated Movies #',HTML);                                 //WEB_SPECIFIC.
    If 0<curPos Then Begin
       ItemValue:=TextBetWeen(HTML,'Bottom Rated Movies #','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
       AddCustomFieldValueByName('Bottom 100',ItemValue);
       LogMessage('      Get result Bottom_100:'+ItemValue+'||');
    End;
    //~Metascore:~
    curPos := PosFrom('> <div class="metacriticScore score_favorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_favorable titleReviewBarSubItem">');
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML,curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
            //AddFieldValueXML('orname',RATING_NAME1);
            //AddFieldValueXML('orating',ItemValue);   
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
    End;
    //~Metascore:~
    curPos:=PosFrom('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">',HTML,EndPos);
    if curPos>0 then begin
        curPos:=curPos+Length('> <div class="metacriticScore score_unfavorable titleReviewBarSubItem">');
        EndPos:=PosFrom('</span>',HTML,curPos);
        ItemValue:=RemoveTags(Copy(HTML, curPos,endPos-curPos+2),false);
        ItemValue:=StringReplace(ItemValue,'0</',',0',True,False,True);
        ItemValue:=StringReplace(ItemValue,'1</',',1',True,False,True);
        ItemValue:=StringReplace(ItemValue,'2</',',2',True,False,True);
        ItemValue:=StringReplace(ItemValue,'3</',',3',True,False,True);
        ItemValue:=StringReplace(ItemValue,'4</',',4',True,False,True);
        ItemValue:=StringReplace(ItemValue,'5</',',5',True,False,True);
        ItemValue:=StringReplace(ItemValue,'6</',',6',True,False,True);
        ItemValue:=StringReplace(ItemValue,'7</',',7',True,False,True);
        ItemValue:=StringReplace(ItemValue,'8</',',8',True,False,True);
        ItemValue:=StringReplace(ItemValue,'9</',',9',True,False,True);
            //AddFieldValueXML('orname',RATING_NAME1);
            //AddFieldValueXML('orating',ItemValue);   
        AddCustomFieldValueByName('Metascore',FloatToStr((StrToFloat(ItemValue) * 1)));
    End;
   
    //Get Cast. GET_FULL_CREDIT = False only the info of the principal movie page.
.
.
    //Get ~plot_summary~
    If Pos('<div class="plot_summary',HTML)>0 then begin
    EndPos:=Pos('<div class="plot_summary',HTML);
    curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<div class="summary_text">','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
          ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
          curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
          If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddFieldValueXML('comment',ItemValue);
          LogMessage('      Get result plot_summary description:'+ItemValue+'||');
       End;
    End;[/b]
.
.
 
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #37 on: September 02, 2018, 09:27:30 am »
IMDB_ [EN] [HTTPS] script

Part 2:

Function ParsePage_IMDBMovieBASE

Quote

    //Get ~plot_summary~
    If Pos('<div class="plot_summary',HTML)>0 then begin
    EndPos:=Pos('<div class="plot_summary',HTML);
    curPos:=Pos('<div class="summary_text">',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<div class="summary_text">','</div>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          ItemValue:=StringReplace(ItemValue,',            ',', ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'            ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
          ItemValue:=StringReplace(ItemValue,'Add a Plot »','',True,False,True);
          ItemValue:=StringReplace(ItemValue,'See full summary »','',True,False,True);
          curPos:=Pos('var ue_t0=ue_t0||+new Date();var ',ItemValue);
          If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddFieldValueXML('comment',ItemValue);
          LogMessage('      Get result plot_summary description:'+ItemValue+'||');
       End;
    End;

.
.
    //Get ~category~ "keywords" (field with several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Plot Keywords:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Plot Keywords:</h4>','<a href="/title/',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                                  //Change the separator to comma
       //AddFieldValueXML('category',ItemValue);
      AddFieldValueXML('tags',ItemValue);       
       LogMessage('      Get results Category:'+ItemValue+'||');
    End;
.
.
    //Get ~mpaa~. GET_FULL_MPAA = False only the info of the principal movie page.
    If Not(GET_FULL_MPAA) Then Begin
       //The text can be "Certificate:" or "Motion Picture Rating" but alway after genres.
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<span itemprop="contentRating">','</span>',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          AddFieldValueXML('mpaa',ItemValue);
          LogMessage('      Get result mpaa:'+ItemValue+'||');
       End;
    End;
    //Get ~origlang~ (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Language:</h4>',HTML);                                      //WEB_SPECIFIC.
    if 0<curPos then begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Language:</h4>','</div>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       curPos:=Pos('            |',ItemValue);
       If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);     
       AddFieldValueXML('origlang',ItemValue);
       LogMessage('      Get results Language:'+ItemValue+'||');
    End;

    //Get ~country~ (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Country:</h4>',HTML);                                      //WEB_SPECIFIC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Country:</h4>','</div>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                           //Change the separator to comma
       AddFieldValueXML('country',ItemValue);
       LogMessage('      Get results Country:'+ItemValue+'||');
    End;
    //Get ~rdate~ in contry provider local IP geolocation. See: http://sobizarre-en.blogspot.fr/2014/12/how-to-easily-defeat-imdb-geolocation.html
    curPos:=Pos('<h4 class="inline">Release Date:</h4>',HTML);                                      //WEB_SPECIFIC
    If 0<curPos Then Begin       
        ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Release Date:</h4>','(',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'January','1',true,true,true);     //Format web corrections. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'February','2',true,true,true);
        ItemValue:=StringReplace(ItemValue,'March','3',true,true,true);
        ItemValue:=StringReplace(ItemValue,'April','4',true,true,true);
        ItemValue:=StringReplace(ItemValue,'May','5',true,true,true);
        ItemValue:=StringReplace(ItemValue,'June','6',true,true,true);
        ItemValue:=StringReplace(ItemValue,'July','7',true,true,true);
        ItemValue:=StringReplace(ItemValue,'August','8',true,true,true);
        ItemValue:=StringReplace(ItemValue,'September','9',true,true,true);
        ItemValue:=StringReplace(ItemValue,' October ',' 10 ',true,true,true);
        ItemValue:=StringReplace(ItemValue,' November ',' 11 ',true,true,true);
        ItemValue:=StringReplace(ItemValue,' December ',' 12 ',true,true,true);
            //ItemValue:=StringReplace(ItemValue,' ','.',True,False,True);
        ItemValue:=StringReplace(ItemValue,' ',#46,True,False,True);
        ItemValue:=StringReplace(ItemValue,'October.','10.',True,True,True);
        ItemValue:=StringReplace(ItemValue,'November.','11.',True,True,True);       
        ItemValue:=StringReplace(ItemValue,'December.','12.',True,True,True);     //Format web corrections. WEB_SPECIFIC

        AddFieldValueXML('rdate',ItemValue);
        LogMessage('      Get results Release Date:'+ItemValue+'||');
    End;       
    //Get ~aka~. GET_FULL_AKA = False only the info of the principal movie page.       
    If Not(GET_FULL_AKA) Then Begin
       curPos:=Pos('<h4 class="inline">Also Known As:</h4>',HTML);                                      //WEB_SPECIFIC
       If 0<curPos Then Begin
          ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Also Known As:</h4>','<',false,curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
          AddFieldValueXML('aka',ItemValue);
          LogMessage('      Get result aka:'+ItemValue+'||');
       End;
    End;
    //Get ~budget~
    curPos:=Pos('<h4 class="inline">Budget:</h4>',HTML);                                      //WEB_SPECIFIC
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Budget:</h4>','<',false,curPos);      //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,#36,'',True,True,False);        //Eliminate '$' if exists 
       ItemValue:=StringReplace(ItemValue,'€','',True,True,False);        //Eliminate '€' if exists         
       ItemValue:=StringReplace(ItemValue,',','',True,True,False);        //Eliminate ',' if exists             
       AddFieldValueXML('budget',ItemValue);
       LogMessage('      Get results Budget:'+ItemValue+'||');
    End;       
    //Get ~money~ Box Office worldwide     
.
.
    //Get ~studio~ "Production Co" (several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Production Co:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Production Co:</h4>','</span>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'See more »','',True,False,True);      
       ItemValue:=StringReplace(ItemValue,'See more','',True,True,False);                           //Eliminate 'See more' if exists           
       ItemValue:=StringReplace(ItemValue,', The','',True,False,True);
       ItemValue:=StringReplace(ItemValue,'The, ','',True,False,True);
       //ItemValue:=StringReplace(ItemValue,'The ','',True,False,True);   
   
       AddFieldValueXML('studio',ItemValue);
       LogMessage('      Get results Studio/Production Co:'+ItemValue+'||');
    End;
    //Get ~features~ (multiline) GET_FULL_FEATURES = False only the info of the principal movie page.
    //If Not(GET_FULL_FEATURES) Then Begin
    If GET_FEATURES Then Begin
        ItemList:='';
        //Get "Runtime" (in minutes)
        curPos:=Pos('<h4 class="inline">Runtime:</h4>',HTML);                                      //WEB_SPECIFIC.
        If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,'M">',' min</time>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            ItemList:=ItemList+'Original Runtime: '+ItemValue+' min.<br>';
            LogMessage('      Get result Original Runtime:'+ItemValue+'||');
            AddFieldValue(mfDuration,ItemValue);
            LogMessage('      Get result lenght:'+ItemValue+'||');
        End;   
.
.
        if (Length(ItemList)>0) then begin
            AddFieldValueXML('features',ItemList);
            LogMessage('      Get result Movie Features:'+ItemList+'||');
        End;
    End;
    //Get ~User Reviews~
    If Pos('<h2>User Reviews</h2>',HTML)>0 then begin
    EndPos:=Pos('<h2>User Reviews</h2>',HTML);
    curPos:=Pos('<h2>User Reviews</h2>',HTML);                                 //WEB_SPECIFIC.
       If 0<curPos Then Begin
        curPos:=PosFrom('<p>',HTML,EndPos)+Length('<p>');
          EndPos:=PosFrom('</span>',HTML,curPos);
          ItemValue:=Copy(HTML,curPos,endPos-curPos);   //Strings which opens/closes the data. WEB_SPECIFIC
        ItemValue:=StringReplace(ItemValue,'</p>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,'</div>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,'<ul><li>','',True,False,True);            
        ItemValue:=StringReplace(ItemValue,'</li></ul>','',True,False,True);
        ItemValue:=StringReplace(ItemValue,':)','',True,False,True);   
        curPos:=Pos('Find showtimes, ',ItemValue);
        If 0<curPos then ItemValue:=Copy(ItemValue,0,curPos-1);
          AddCustomFieldValueByName('User Comments',ItemValue);
        AddCustomFieldValueByName('User Reviews',ItemValue);
          LogMessage('      Get result User Reviews: <br>'+ItemValue+'||');
       End;
    End;
      
    LogMessage('Function ParsePage_IMDBMovieBASE END=====================||');
  End; //BlockClose

//Get ~category~ "keywords"

Quote
    //Get ~category~ "keywords" (field with several values in a comma separated list)
    curPos:=Pos('<h4 class="inline">Plot Keywords:</h4>',HTML);                                      //WEB_SPECIFIC.IC.
    If 0<curPos Then Begin       
       ItemValue:=TextBetWeen(HTML,'<h4 class="inline">Plot Keywords:</h4>','<a href="/title/',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
       ItemValue:=StringReplace(ItemValue,'|',',',True,True,False);                                  //Change the separator to comma
       //AddFieldValueXML('category',ItemValue);
      AddFieldValueXML('tags',ItemValue);       
       LogMessage('      Get results Category:'+ItemValue+'||');
    End;

As for ~ category ~ "keywords" IMDB has never, explicitly repeat I never knew the ~ category ~ "keywords" as they say, have the AllMovie site. This has always been taken as ~ tags ~ "keywords" and information has always been transferred to the Tags field in dababase.

For example for 21 movie:

IMDB "keywords"

Quote
Plot Keywords: based on true story | card counting | blackjack | massachusetts institute of technology | gambling | See All (109) »
https://www.imdb.com/title/tt0478087/

AllMovie "keywords"
https://www.allmovie.com/movie/21-v386347

Quote
Keywords
betrayal, blackjack [card game], casino, college-student
AllMovie "category"
Quote
Themes
Gambling | Schemes and Ruses | Teachers and Students | Betrayal | Crime Gone Awry | Dishonor Among Thieves | Perfect Crime
https://www.allmovie.com/movie/21-v386347

It has been used since the creation of the PVD program, or it has been almost ten years since, for this I know that it was added to the database
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2706
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #38 on: September 02, 2018, 09:53:06 am »
IMDB_ [EN] [HTTPS] script

//Get ~features~ (multiline)

Quote
    //Get ~features~ (multiline) GET_FULL_FEATURES = False only the info of the principal movie page.
    //If Not(GET_FULL_FEATURES) Then Begin
    If GET_FEATURES Then Begin
        ItemList:='';
        //Get "Runtime" (in minutes)
        curPos:=Pos('<h4 class="inline">Runtime:</h4>',HTML);                                      //WEB_SPECIFIC.
        If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,'M">',' min</time>',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            ItemList:=ItemList+'Original Runtime: '+ItemValue+' min.<br>';
            LogMessage('      Get result Original Runtime:'+ItemValue+'||');
            AddFieldValue(mfDuration,ItemValue);
            LogMessage('      Get result lenght:'+ItemValue+'||');
        End;   
.
.
        if (Length(ItemList)>0) then begin
            AddFieldValueXML('features',ItemList);
            LogMessage('      Get result Movie Features:'+ItemList+'||');
        End;
    End;

Here is a change because this is a bit different information than in the Technical Specifications section.

Get ~features~

I've repaired parts of the Information Transfer Code for the Technical Specifications section.

Quote
Function ParsePage_IMDBMovieTECHNICAL(HTML:String):Cardinal; //BlockOpen
    //Returns:
    //     Result:=prFinished; Script has finished gathering data
    //     Result:=prError; If żany big problem? with exit
    //Retrieve: ~features~
  Var
    //curPos:Integer;
    curPos,endPos:Integer;
    ItemValue,ItemList:String;     
  Begin
.
.
    //Get "Runtime" info
    ItemValue:=HTMLValues2(HTML,'Runtime','</tr>','<td>','</tr>','<br><br>',EndPos);
    ItemValue:=StringReplace(ItemValue,'                                ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'                      ','<br>',True,False,True);
    ItemValue:=StringReplace(ItemValue,'          ','',True,False,True);
    LogMessage('      Get result Runtime:'+ItemValue+'||');
    if ItemValue <> '' then ItemList:=ItemList+'Runtime'+'<br>'+ItemValue+'<br><br>'; 
         curPos:=Pos('<td class="label"> Runtime </td>',HTML);                                      //WEB_SPECIFIC.
         If 0<curPos Then Begin 
            ItemValue:=TextBetWeen(HTML,' (',' min)',false,curPos);  //Strings which opens/closes the data. WEB_SPECIFIC
            AddFieldValue(mfDuration,ItemValue);
            //AddFieldValueXML('lenght',ItemValue);
            LogMessage('      Get result Original Runtime_1:'+ItemValue+'||');
         End;      
   
.
.
    //Get "Color" info
    ItemValue:=HTMLValues2(HTML,'Color','</tr>','<td>','</tr>','<br><br>',EndPos);
    ItemValue:=StringReplace(ItemValue,'                     |          ',' • ',True,False,True);
    ItemValue:=StringReplace(ItemValue,'             |         ',' • ',True,False,True);
    ItemValue:=StringReplace(ItemValue,'        ',' ',True,False,True);
   ItemValue:=StringReplace(ItemValue,'  • ',' • ',True,False,True);
    LogMessage('      Get result Color:'+ItemValue+'||');
    if ItemValue <> '' then ItemList:=ItemList+'Color'+'<br>'+ItemValue+'<br><br>';
.
.         
    if (Length(ItemList)>0) then begin
        //AddFieldValueXML('features',ItemList);   //Ivek23 CustomField ~ImdbTechSpecs~
      AddCustomFieldValueByName('ImdbTechSpecs',ItemList);
        LogMessage('      Get result Movie Features:'+ItemList+'||');
    End;
    LogMessage('Function ParsePage_IMDBMovieTECHNICAL END=====================||');
  End; //BlockClose

The Technical Specifications section information is recommended to be transferred to the ~ ImdbTechSpecs ~ custom field.

IMDB_ [EN] [HTTPS] _1 script is attached.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline fmb64

  • Member
  • *
  • Posts: 14
    • View Profile
Re: Personal Video Database 1.0.2.7 MOD
« Reply #39 on: September 08, 2018, 01:28:53 pm »
Is it normal that importing from IMDb is very slow and screen is flickering from quickly disappearing black windows... also importing does hang when I run it from C:\Program Files\... instead of a temporary directory?
PVD 1.0.2.7    Win10 x64

 

anything