Author Topic: curl - PVD to https solution  (Read 14734 times)

0 Members and 1 Guest are viewing this topic.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2713
    • View Profile
curl - PVD to https solution
« on: November 26, 2016, 12:17:13 pm »
curl - PVD to https solution

You don't need to program out of PVdB. curl.exe is a program that download a web page (even https) to a file in the command line.

Download:
https://curl.haxx.se/download.html
https://curl.haxx.se/download.html#Win32

Download the original ones:
https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z  (The needed files were in "curl-7.50.2-win32-mingw\bin\" folder)


Needed external:

1.) Download the curl-7.50.2-win32-mingw.7z file from https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z   (Thanks Viktor Szakáts https://bintray.com/vszakats/generic/curl/).

2.) Extract the three curl libraries files:
   • curl-7.50.2-win32-mingw\bin\curl.exe
   • curl-7.50.2-win32-mingw\bin\curl-ca-bundle.crt
   • curl-7.50.2-win32-mingw\bin\libcurl.dll

and copy then these files to " Program folder/Scripts ":
              curl-ca-bundle.crt,
              curl.exe,
              libcurl.dll


These three above-mentioned files are needed to get the program of the work at all.

These files can also be found on these links,
http://www.videodb.info/forum_en/index.php/topic,4069.msg19897.html#msg19897
http://www.videodb.info/forum_en/index.php/topic,4069.msg19898.html#msg19898

where you also describes how to extract the files.

Important:

There's also a Rotten Tomatoes [HTTPS] script, which does not copy of this  " Program folder/Scripts " folder. Copy it this Rottentomatoes_ [HTTPS] script on this link (http://www.videodb.info/forum_en/index.php/topic,4069.msg19948.html#msg19948), which it operates correctly (script for testing and operation of the curl program).


Needed external for full functionality:

To spread in internet, the best form to save the 4 needed files:
              Scripts/Rottentomatoes_[HTTPS].psf,
              Scripts/curl-ca-bundle.crt,
              Scripts/curl.exe,
              Scripts/libcurl.dll


If you want to download the original ones: https://bintray.com/artifact/download/vszakats/generic/curl-7.50.2-win32-mingw.7z (The needed files were in "curl-7.50.2-win32-mingw\bin\" folder).


Operation curl program:


1.) Run the program PVD

2.) Run Rottentomatoes_ [HTTPS] script.  Wait until curl.exe runs in DOS Windows and at the same time automatically creates 'downpage.htm' file, which was saved web page with search results for the current movie title, you are looking for.

3.) Now you will be opened window with your search results. Confirm the desired movie title.

4.) Wait until you curl.exe again runs in DOS Windows and at the same time automatically creates 'downpage.htm' file, which is now stored movie web page for the current movie title, you are looking for.

It is important, what should not be done to PVD does not freeze:

Once you start Rottentomatoes_ [HTTPS] script, always wait until when you opened the window with the search results, or when a window pops of the record:
Quote
Not results. Try again or make an advanced search out of the script.
Write the movie URL (http) in the PVdB manually (remember separated by spaces) and run again.

Do not interrupt the start-up operations curl.exe program because you PVD mandatory freeze and it will be necessary to re-start again. There is a risk that the loss of data. Also make sure that during this time will not switch to the next title of the film, because you will be the next title of the film mandatory overwrite on the title of the film, which information you currently transmitted.

For testing of the possible, you can use curl_try.bat file in 7z file, which is also copied to the "Program folder/Scripts" folder. There can be no PVD download the web page, so that the bat file enter the URL you want to check and run curl_try.bat.  In curl_try.bat 7z files are also downpage_search.htm, downpage_try.htm and downpage.htm, which also copy the "Program folder/Scripts" folder.

curl_try.bat 7z is attached.



For more information, read this topic curl - solution for https in the Talk section.



The final version Rottentomatoes_ [HTTPS] script found on this link:

http://www.videodb.info/forum_en/index.php/topic,4079.0.html
« Last Edit: November 28, 2016, 07:23:49 pm by Ivek23 »
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 471
    • View Profile
Re: curl - PVD to https solution
« Reply #1 on: November 27, 2016, 08:09:07 pm »
Thanks guys!

It works indeed! May I kindly ask you for a favor! Could you please explain how to set up script to work through proxy? I'm using proxy to download data, and all (working) plugins work well through proxy, but Rottentomatoes script doesn't.

It connects, I get proper search result and when I click OK to download data, I get HTTP/1.0 400 Bad request.

Thank you in advance!

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2713
    • View Profile
Re: curl - PVD to https solution
« Reply #2 on: November 28, 2016, 02:05:05 pm »
Thanks guys!

It works indeed! May I kindly ask you for a favor! Could you please explain how to set up script to work through proxy? I'm using proxy to download data, and all (working) plugins work well through proxy, but Rottentomatoes script doesn't.

It connects, I get proper search result and when I click OK to download data, I get HTTP/1.0 400 Bad request.

Thank you in advance!

Rotten Tomatoes script will not work until you have turned on the proxy. I do not know why this is happening, but also with me did not work until I had the PVD active setting for the proxy server and running Proxomitron program.

What it happens when you are not using a proxy mode.

This script has a requirement that absolutely must not are operated proxy mode if you want to get the script to work at all.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 471
    • View Profile
Re: curl - PVD to https solution
« Reply #3 on: November 28, 2016, 08:23:36 pm »
Thanks for response Ivek.

In PVD I set a proxy (127.0.0.1:4321 - my own proxy, it's not important which one and how). I can download/import data with all plugins and scripts except with Rottentomatoes [HTTPS] script. What I can do with Rottentomatoes [HTTPS] script, as I wrote above, is that I can get  search result, but when I try to download/import data for the chosen search window result, PVD returns HTTP/1.0 400 Bad request.

When I turn off proxy in PVD,  Rottentomatoes [HTTPS] script properly imports data into PVD.

I guess, probably the question is how to force cURL to download data through proxy, right?

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2713
    • View Profile
Re: curl - PVD to https solution
« Reply #4 on: November 29, 2016, 09:39:49 am »
Thanks for response Ivek.

Welcome.

When I turn off proxy in PVD,  Rottentomatoes [HTTPS] script properly imports data into PVD.

As you have already guessed, the script only works when the proxy mode is turned off.

I guess, probably the question is how to force cURL to download data through proxy, right?

Perhaps because curl program operates in normal mode as well as in proxy mode. PVD in proxy mode using port 80 and curl program in this case operates normally. Your proxy settings are apparently the cause of your problems, which means that the proxy settings change, or proxy mode then turn off when using a RottenTomatoes script. I think it will be better the second way when using a RottenTomatoes script.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline VVV_Easy_Programing

  • Older Power User
  • *****
  • Posts: 199
    • View Profile
Re: curl - PVD to https solution
« Reply #5 on: November 30, 2016, 10:07:11 pm »
Well, I don't know very much over proxy but curl has a lot of options over proxy:

Quote
PROXY
 curl supports both HTTP and SOCKS proxy servers, with optional authentication. It does not have special support for FTP proxy servers since there are no standards for those, but it can still be made to work with many of them. You can also use both HTTP and SOCKS proxies to transfer files to and from FTP servers.

You can try to make your own adaptation with the curl proxy parameters (¿--proxy?) listed in:
https://curl.haxx.se/docs/manpage.html

More info in:
https://curl.haxx.se/docs/manual.html

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2713
    • View Profile
Re: curl - PVD to https solution
« Reply #6 on: June 16, 2018, 11:24:39 am »
Happily, we have finished the "Beta" phase in the PVD MOD and move on to "Release" state.
So, we closed this topic now and we continue in the new topic:

Personal Video Database 1.0.2.7 MOD
http://www.videodb.info/forum_en/index.php/topic,4134.0.html

See you all there.  ;D
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD