Author Topic: AI and PVD  (Read 5458 times)

0 Members and 1 Guest are viewing this topic.

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
AI and PVD
« on: February 07, 2024, 10:56:00 pm »
I admit - I am terrified how wrong it will go with AI at the beginning and it already started. So, I guess we need to think how to minimize the damage starting today, to protect ourselves and our families the way people hide themselves in the basement under tornado strike.


But, I thought maybe something positive can come out of all of this. AI will be very soon able to reverse engineer PVD. Do you agree?

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2738
    • View Profile
Re: AI and PVD
« Reply #1 on: February 08, 2024, 08:41:58 am »
We will see how and what will happen in the near future.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2738
    • View Profile
Re: AI and PVD
« Reply #2 on: November 23, 2024, 08:21:02 pm »
You mentioned somewhere on the forum that you could change and fix scripts with the help of AI.

Is it possible to get some AI tool to help edit PVD scripts?

What do you use and could you share it with other PVD users?

I would also start using AI to help edit PVD scripts if I got the right information on how to use it.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #3 on: November 23, 2024, 10:39:16 pm »

You mentioned somewhere on the forum that you could change and fix scripts with the help of AI.

Is it possible to get some AI tool to help edit PVD scripts?

What do you use and could you share it with other PVD users?

I would also start using AI to help edit PVD scripts if I got the right information on how to use it.



Great to hear you are interested in!!!!!! PVD IS SAVED THEN!

I admit: I was always hesitating to ask you each time for a favor when something goes wrong, especially knowing that I am one of maybe ten users of PVD that still use it. So I decided to learn, not coding, but to recognize it and the most important, how to interact with AI. I'm still struggling with interacting, because it still struggles with itself, so today, I think I better know Pascal, then how to talk to AI, hahaha. I learned a lot in just a month or so.


At the moment, and I emphasize that I don't want to pay for this,  the best free coding AI out there (or better to say the least bad) is MS Copilot. It's free to use, but it has a lot of downsides. The biggest one, for me at least, is that the message has ~10k characters limit, so if you want to upload whole PVD script, you need 30 messages of copy pasting, and copilot will forget it after several messages. Frustrating! So at the moment, for me it's impossible to create or fix something because I don't know relations within the script and about PVD itself, so that's I asked you if you have some resources, or to share info so we could spread help. My goal is to reintegrate all the fields we once had in a single imdb script, to bring back AllMovie and Rottentomatoes scripts too, because simply there is no reason not to do it today.

The other one I tried is duckduckgo chat. It ofers 4 models:


Quote
All models are anonymously provided by DuckDuckGo
GPT-4o mini

General purpose AI with high built-in moderation

Created by OpenAI

Claude 3 Haiku

General purpose AI with high built-in moderation

Created by Anthropic

Llama 3.1 70B

Open Source
General purpose AI with medium built-in moderation

Created by Meta

Mixtral 8x7B

Open Source
General purpose AI with low built-in moderation

Created by Mistral AI

Please note: starting a new chat with any model will delete your current chat session

I tried briefly Mixtral 8x7B, and it receives ~16k characters, which is pretty good comparing to Copilot, but the total number messages per day is limited. The biggest downside for me is that it doesn't use our data to train themselves, which is brilliant for privacy, but not for outputs, hahaha. So, for someone who's not programmer, it still is extremely hard to achieve any goal.




But, as I said, the most important thing is how to instruct AI, meaning how to "talk" to it. What I learned all along the way, very hard way because I had to learn basics of Pascal and million things in parallel, is to first talk to it, to explain what you want, to ask it for examples, until you are fine with it and then to ask it for a workflow plan. That plan you save somewhere out of chat for a goal you want to achieve, so you can later just paste it. Here's the example what I got as a workflow from AI, after a talked a lot to it and once I was to check if multiple posters import is possible to PVD:


Code: [Select]

Yes, knowing that PVD cannot handle HTTPS and relies on the AHK script for downloading pages, we need to take extra caution when creating part2 of wscr3. Here are some key considerations:


1. Ensure Secure Handling of HTTPS:
Consistent Use of AHK: Ensure that all HTTPS requests are handled through the AHK script to maintain consistency and avoid errors.


Error Handling: Implement robust error handling to manage any potential issues with HTTPS downloads.


2. Avoid Direct HTTPS Calls in Part2:
No Direct HTTP Requests: Ensure that part2 does not attempt any direct HTTP requests, which PVD cannot process.


3. Manage Variable Dependencies:
Separate Variables: Use distinct variables for URLs and file paths in part2 to avoid conflicts with part1.


Consistent Paths: Ensure all paths are correctly set and accessible to both the AHK script and the main processing script.


4. Logging and Debugging:
Detailed Logs: Maintain detailed logs to trace each step and catch any issues early.


Sequential Logging: Ensure that the logs differentiate between the actions of part1 and part2 clearly.


5. Complete Initial Processing Before Starting Part2:
Clear Markers: Use clear markers to indicate the end of part1 and the beginning of part2.


Confirmation of Completion: Ensure part1 completes all tasks before initiating part2.


Example Pseudo-code for Part2:
pascal
// Ensure part1 completion
LogMessage('Function ParsePage smNormal END======================|');
Exit;  // End of initial processing


// Part 2: Start new process for multiple posters
LogMessage('Starting process to download multiple posters');


// Define new URL and file path
NewURL := 'https://www.imdb.com/title/tt3461252/mediaindex';
NewDownloadFile := 'Tmp/multipage-UTF8_NO_BOM.htm';


// Download the new page using AHK script
FileExecute(ScriptPath+'PVdBDownPage.exe', '"'+NewURL+'" "'+ScriptPath+NewDownloadFile+'"');


// Wait for the new download to finish
While Not(FileExists(ScriptPath+NewDownloadFile)) Do Begin
    wait (2000);
End;


// Process the new page for multiple posters
WebText := FileToString(ScriptPath+NewDownloadFile);
WebText := ConvertEncoding(WebText, 65001);
LogMessage('Downloaded multiple posters page');


// Extract and download multiple posters
// Add your logic here to extract poster links and download them
By taking these precautions and structuring the script carefully, we ensure that part2 integrates smoothly with part1 and reliably handles all the necessary HTTPS requests through the AHK script.


Key Implications:
Sequential Addition: Posters need to be added one by one, using the navigation buttons to move through the poster slots (e.g., pos1, pos2).


Automatic Overwrite: Adding a poster in pos1 will overwrite the existing one unless the user navigates to a new slot (pos2) before adding another poster.


Manual Actions Mimicked by Script: Since manual actions require navigation buttons to switch between slots, the script needs to mimic these actions to add multiple posters.


Approach for wscr3:
To import multiple posters programmatically while respecting the manual navigation process, follow these steps:


Extract Poster URLs:


Extract multiple poster URLs from the gallery page as discussed.


Sequential Addition with Navigation:


Use script logic to simulate the navigation and addition of each poster URL.


Ensure the script switches to the next poster slot (pos2, pos3, etc.) before adding each new poster.


Pseudo-code Example:
pascal
// Ensure part1 completion
LogMessage('Function ParsePage smNormal END======================|');
Exit;  // End of initial processing


// Part 2: Start new process for multiple posters
LogMessage('Starting process to download multiple posters');


// New URL and file path
NewURL := 'https://www.imdb.com/title/tt3461252/mediaindex';
NewDownloadFile := 'Tmp/multipage-UTF8_NO_BOM.htm';


// Download the new page using AHK script
FileExecute(ScriptPath+'PVdBDownPage.exe', '"'+NewURL+'" "'+ScriptPath+NewDownloadFile+'"');


// Wait for the new download to finish
While Not(FileExists(ScriptPath+NewDownloadFile)) Do Begin
    wait (2000);
End;


// Process the new page for multiple posters
WebText := FileToString(ScriptPath+NewDownloadFile);
WebText := ConvertEncoding(WebText, 65001);
LogMessage('Downloaded multiple posters page');


// Extract and store multiple poster URLs
PosterURLs := ExtractPosterURLs(WebText); // Pseudo-function to parse and get poster URLs


// Navigate and add each poster URL sequentially
For i := 0 to Length(PosterURLs) - 1 Do Begin
    // Simulate clicking 'nextposterbtn' to navigate to the next poster slot
    SimulateClickNextPosterButton(); // Pseudo-function for clicking the button
   
    // Add the poster URL to the current slot
    AddCustomFieldValueByName('posterURL'+IntToStr(i), PosterURLs[i]);
    AddCustomFieldValueByName('poster'+IntToStr(i), PosterURLs[i]); // Adjust as needed
End;


// Update poster count
AddCustomFieldValueByName('postercount', IntToStr(Length(PosterURLs)));
Testing:
Run wscr3: Execute the script and verify that each poster is added to a new slot correctly.


Check Navigation: Ensure that the script correctly mimics navigation actions between poster slots.


Validation: Verify that all posters appear correctly in PVD without overwriting each other.


By following these steps, you can ensure that multiple posters are imported into PVD correctly while respecting its manual navigation and addition process.


Knowing all before your last message do you think you could Mimick manual actions with part2 to switch between slots, so part2 could be able to add multiple posters?


Steps to Mimic Manual Actions:
Simulate Clicking the Next Poster Button:


Use an AHK script to simulate the click on the nextposterbtn to switch to the next poster slot (e.g., from pos1 to pos2).


Add Poster URL:


Once the new slot is active, use the script to add the poster URL to that slot.


Detailed Plan:
Extract Poster URLs:


Extract the URLs from the IMDb gallery page.


Navigate and Add Posters:


For each extracted URL, navigate to the next poster slot and add the poster.


Example Pseudo-code for Part2:
Part 2: Adding Multiple Posters
pascal
// Ensure part1 completion
LogMessage('Function ParsePage smNormal END======================|');
Exit;  // End of initial processing


// Part 2: Start new process for multiple posters
LogMessage('Starting process to download multiple posters');


// New URL and file path
NewURL := 'https://www.imdb.com/title/tt3461252/mediaindex';
NewDownloadFile := 'Tmp/multipage-UTF8_NO_BOM.htm';


// Download the new page using AHK script
FileExecute(ScriptPath+'PVdBDownPage.exe', '"'+NewURL+'" "'+ScriptPath+NewDownloadFile+'"');


// Wait for the new download to finish
While Not(FileExists(ScriptPath+NewDownloadFile)) Do Begin
    wait (2000);
End;


// Process the new page for multiple posters
WebText := FileToString(ScriptPath+NewDownloadFile);
WebText := ConvertEncoding(WebText, 65001);
LogMessage('Downloaded multiple posters page');


// Extract and store multiple poster URLs
PosterURLs := ExtractPosterURLs(WebText); // Pseudo-function to parse and get poster URLs


// Function to simulate clicking the next poster button
procedure SimulateClickNextPosterButton();
begin
    // Use AHK script to simulate the click
    FileExecute('PathToAHKScript\click_next_poster_button.ahk', '');
end;


// Navigate and add each poster URL sequentially
For i := 0 to Length(PosterURLs) - 1 Do Begin
    // Simulate clicking 'nextposterbtn' to navigate to the next poster slot
    SimulateClickNextPosterButton();
   
    // Add the poster URL to the current slot
    AddCustomFieldValueByName('posterURL'+IntToStr(i), PosterURLs[i]);
    AddCustomFieldValueByName('poster'+IntToStr(i), PosterURLs[i]); // Adjust as needed
End;


// Update poster count
AddCustomFieldValueByName('postercount', IntToStr(Length(PosterURLs)));
Steps for Creating AHK Script:
Create AHK Script:


Write an AHK script to simulate the click on the nextposterbtn.


ahk
; AHK script to simulate clicking the next poster button
ControlClick, nextposterbtn, ahk_class PVDWindowClass
return
Integrate AHK with Delphi/Pascal Script:


Ensure the Delphi/Pascal script calls the AHK script correctly to simulate the button click.


By following these steps and ensuring the AHK script correctly simulates the navigation actions, you can effectively add multiple posters to PVD as if done manually. This approach should work smoothly given the outlined process.


When and if I decide to do this, I will just paste this as my request, when I prepare the way to upload script, intermediate .html, .ahk etc.

What I achieved for now is to add my own custom field "IMDb Cast" which imports up to 15 actors from the principal page, out of PEOPLE_LIMIT in the script, and already to revise the code using AI, because meanwhile IMDb layout changed. I just pasted original snippet, gave to AI new IMDb layout, and in a second I got script corrected. I also achieved new custom "IMDb Directed by" snippet, populated as "A FILM BY MEL GIBSON" form, because I cannot position "A FILM BY " to the skin as I intend to because of the limitation of PVD, database and the nature of fields themselves. The peak of new skins (I'm creating 4 versions of it) is in the messages after this due to the limit of attachments. I'm posting my highly customized script too, if someone is interested in it, but I warn you that you have to add corresponding custom fields to PVD if you want your PVD to looks like this.

When you have further questions, just let me know. It is crucial we to share info if we want to save PVD.
« Last Edit: November 23, 2024, 11:52:45 pm by afrocuban »

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #4 on: November 23, 2024, 10:40:14 pm »
First peak of a new skin. None is finished but all of them have all the custom fields included.
« Last Edit: November 23, 2024, 10:55:14 pm by afrocuban »

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #5 on: November 23, 2024, 10:42:11 pm »
Second one
« Last Edit: November 23, 2024, 10:49:06 pm by afrocuban »

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #6 on: November 23, 2024, 10:42:42 pm »
The third one
« Last Edit: November 23, 2024, 10:50:15 pm by afrocuban »

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #7 on: November 23, 2024, 10:52:04 pm »
The last version. I still can't decide which one to keep and maintain...

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #8 on: November 24, 2024, 02:26:14 am »
Here's how I learned to talk to Copilot:


Quote
Here's a guide on how to structure your requests to make them as clear as possible. I'll use your last request as an example:


Tips for Clear Requests:
  • Be Specific: Clearly state what you want to achieve.
  • Provide Context: Give background information if necessary.
  • Use Clear Language: Avoid ambiguous terms.
  • Break Down Complex Tasks: If your request has multiple parts, list them out.
  • Include Examples: Show an example of what you're looking for if possible.
Your Last Request:

Original Request: "I need a code snippet to populate a custom field 'IMDB Studio2' with a concatenation of 'STUDIO RELEASE BY' and the parsed value of the custom field 'IMDb Studio2' in the next line."


Example Interaction:

You: "I need help modifying a code snippet. The goal is to populate a custom field named 'IMDb Studio2' with a concatenation of the text 'STUDIO RELEASE BY' followed by a line break and the parsed value of the current 'IMDb Studio2' field. Here is the code snippet I'm working with:

pascal


Code: [Select]
// Get Production Companies
curPos := Pos('<h4 class="ipl-header__content ipl-list-title">Production Companies</h4>', HTML);
If curPos > 0 Then Begin
    EndPos := curPos;
    ItemValue140 := HTMLValues(HTML, '<h4 class="ipl-header__content ipl-list-title">Production Companies</h4>', '</ul>', '<li>', '</li>', ', ', EndPos);
    ItemValue140 := StringReplace(ItemValue140, '                         ', '  ', True, False, True);
    if ItemValue140 <> '' then AddCustomFieldValueByName('Imdb Studio2', ItemValue140);             
    if ItemValue140 <> '' then LogMessage('Get results Production Companies:' + ItemValue140);
End;

Could you help me modify this to achieve the desired result?"



My Response: "Sure! Here’s the modified code snippet that will populate the custom field 'IMDb Studio2' with 'STUDIO RELEASE BY' followed by a line break and the parsed value:

pascal
Code: [Select]

// Get Production Companies
curPos := Pos('<h4 class="ipl-header__content ipl-list-title">Production Companies</h4>', HTML);
If curPos > 0 Then Begin
    EndPos := curPos;
    ItemValue140 := HTMLValues(HTML, '<h4 class="ipl-header__content ipl-list-title">Production Companies</h4>', '</ul>', '<li>', '</li>', ', ', EndPos);
    ItemValue140 := StringReplace(ItemValue140, '                         ', '  ', True, False, True);


    // Custom field: IMDb Studio2 (formatted as "STUDIO RELEASE BY\n[ItemValue140]")
    If ItemValue140 <> '' Then Begin
        ItemValue140 := 'STUDIO RELEASE BY' + #13#10 + ItemValue140; // #13#10 represents a line break
        AddCustomFieldValueByName('IMDb Studio2', ItemValue140);
        LogMessage('IMDb Studio2 field added with value: ' + ItemValue140);
    End;
End;



This code ensures the custom field 'IMDb Studio2' is populated correctly with the desired format. If you have any further adjustments or additional requirements, let me know!"

By structuring your request this way, it helps ensure I understand exactly what you need and can provide the most accurate assistance. If you have more examples or need further guidance, feel free to ask!



I think you can talk to it in your own language without a problem, too.
« Last Edit: November 24, 2024, 02:32:29 am by afrocuban »

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2738
    • View Profile
Re: AI and PVD
« Reply #9 on: November 24, 2024, 08:54:20 am »
Thanks for the extensive description of how AI can be helpful.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #10 on: November 24, 2024, 10:58:36 am »
You are welcome, Ivek.

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2738
    • View Profile
Re: AI and PVD
« Reply #11 on: November 28, 2024, 07:12:55 pm »
Unfortunately, AI assistance is not of much use to me, because I cannot access the information on IMDbPro due to payment (I do not have a profile on the IMDb people pages) so that the pages can even be opened. It is also impossible to access certain information on the IMDb pages, because this information cannot be obtained or found in the source code of the pages.
Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #12 on: November 28, 2024, 11:50:33 pm »
Unfortunately, AI assistance is not of much use to me..

Oh, sorry to hear!

... because I cannot access the information on IMDbPro due to payment (I do not have a profile on the IMDb people pages) so that the pages can even be opened.

That is expected, if I understand you well. AI cannot provide unauthorized access to sources.

It is also impossible to access certain information on the IMDb pages, because this information cannot be obtained or found in the source code of the pages.

Which, for example?

Offline Ivek23

  • Global Moderator
  • *****
  • Posts: 2738
    • View Profile
Re: AI and PVD
« Reply #13 on: November 29, 2024, 11:07:11 am »
On all IMDb pages where the limit of 5 or 6 items is visible and has the word more at the end.

For example, the following IMDb pages.

https://www.imdb.com/title/tt0147800/releaseinfo/

https://www.imdb.com/title/tt0147800/keywords/

https://www.imdb.com/title/tt0147800/locations/

https://www.imdb.com/title/tt0147800/movieconnections/

https://www.imdb.com/title/tt0147800/externalsites/

There may be some more that are not mentioned.

A special case is the Adward page.

Ivek23
Win 10 64bit (32bit)   PVD v0.9.9.21, PVD v1.0.2.7, PVD v1.0.2.7 + MOD


Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #14 on: November 29, 2024, 04:58:19 pm »
I suspected so, because I tried to get all AKA's back and that's exactly why I asked you. The problem is downpage-UTF8_NO_BOM.htm cannot download dynamic content. We could resolve it by instructing PVdBDownPage.exe to download additional, for example downpage-UTF8_NO_BOM-1.htm with dynamic content, but for that, we need to decompile PVdBDownPage.exe, but it is password protected and it can't be decompiled. There is a source for PVdBDownPage.exe, supposed to be PVdBDownPage.ahk, but that source isn't working for me, and can't create exe of it, so the issue digressed too much at the moment, so I left it there to try it later.
 

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #15 on: November 29, 2024, 06:49:57 pm »
Here's part of my interaction wth AI, trying to get AKA's, but to no avail at the moment:


Code: [Select]

Did you include this you asked for? https://www.imdb.com/tr/?ref_=ttrel-akas-seemore&pt=title&spt=releaseinfo&const=tt17505010&ht=actionOnly&pageAction=seemore
Yes, I included the URL https://www.imdb.com/tr/?ref_=ttrel-akas-seemore&pt=title&spt=releaseinfo&const=tt17505010&ht=actionOnly&pageAction=seemore in the script to fetch additional "aka" titles.


To recap, here's where the URL is used in the script:


FetchAdditionalAKAs Function:
pascal
Function FetchAdditionalAKAs(URL: String): String;
Begin
    // Use the existing DownloadPage function to fetch additional aka titles
    Result := DownloadPage(URL);
End;

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #16 on: December 08, 2024, 05:12:59 pm »
We could resolve it by instructing PVdBDownPage.exe to download additional, for example downpage-UTF8_NO_BOM-1.htm with dynamic content,.
 

I think this is achievable just by defining additional BASE_DOWNLOAD_FILE_NO_BOM_WHATEVER_HERE and instructing functions to process those pages. I will definitely try this, just to discover how to define page we need for aka's for example which is https://www.imdb.com/tr/?pt=title&spt=releaseinfo&const=tt0068646&ht=actionOnly&pageAction=see-all

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #17 on: December 08, 2024, 05:17:35 pm »
This is what I got from AI:

Quote
To download dynamic content from a webpage like the one you mentioned, you can use a headless browser or intercept the XHR/Fetch requests that the page uses to load its content. Here’s a step-by-step guide using Python and Selenium, a popular tool for browser automation:

Step 1: Install Selenium
First, you need to install Selenium if you haven't already:

bash
pip install selenium
Step 2: Download the Page
Use Selenium to open the page and download its content:

python
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time

# Path to the ChromeDriver executable
service = Service("path/to/chromedriver")

# Set up Chrome options
options = Options()
options.headless = True  # Run in headless mode (no browser window)

# Open the browser and navigate to the page
driver = webdriver.Chrome(service=service, options=options)
driver.get("https://www.imdb.com/tr/?pt=title&spt=releaseinfo&const=tt0068646&ht=actionOnly&pageAction=see-all")

# Wait for the page to load
time.sleep(5)

# Save the page content to a file
with open("Tmp\downpage-UTF8_NO_BOM_FA_Trailers.htm", "w", encoding="utf-8") as file:
    file.write(driver.page_source)

# Close the browser
driver.quit()
Step 3: Process the Downloaded Page
Now you can process the downloaded page using your existing script:

pascal
const
  BASE_DOWNLOAD_FILE_NO_BOM_TRAILERS = 'Tmp\downpage-UTF8_NO_BOM_FA_Trailers.htm';  // Additional file for trailers

// Function to process the trailers page
procedure ProcessTrailersPage;
var
  HTML: String;
begin
  HTML := LoadFromFile(BASE_DOWNLOAD_FILE_NO_BOM_TRAILERS);  // Load the additional page
  // Add your processing logic here
end;
This approach uses Selenium to automate the browser, load the dynamic content, and save it locally. You can then process the saved page with your existing script.


This is initial just initial idea...

Offline afrocuban

  • Moderator
  • *****
  • Posts: 535
    • View Profile
Re: AI and PVD
« Reply #18 on: December 08, 2024, 05:22:04 pm »


Quote

How would you set curl to download this content: https://www.imdb.com/tr/?pt=title&spt=releaseinfo&const=tt0068646&ht=actionOnly&pageAction=see-all

To download dynamic content from a page like the one you provided using curl, you need to replicate the HTTP request as closely as possible, including headers and possibly cookies. Here's a general approach you can take:
  • Inspect the Request:
    • Open the browser's Developer Tools (usually by pressing F12).
    • Go to the Network tab and reload the page.
    • Find the request for the content you want to download (it will usually be an XHR request).
    • Copy the request as curl.
  • Use curl Command:
    • Paste the copied curl command into your terminal. It should include all the necessary headers and cookies.
Example curl Command:Here’s a generic example based on typical headers. You may need to adjust based on what you see in the Developer Tools:

bash:

curl 'https://www.imdb.com/tr/?pt=title&spt=releaseinfo&const=tt0068646&ht=actionOnly&pageAction=see-all' \
  -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36' \
  -H 'Accept: application/json, text/javascript, */*; q=0.01' \
  -H 'Accept-Language: en-US,en;q=0.9' \
  -H 'Connection: keep-alive' \
  -H 'Referer: https://www.imdb.com/' \
  --compressed \
  -o Tmp/downpage-UTF8_NO_BOM_FA_Trailers.htm

Steps to Customize:
  • Inspect the Request: Use the Developer Tools to see the exact headers and cookies used by your browser.
  • Copy as curl: Right-click on the request and choose "Copy as cURL".
  • Run the Command: Paste the copied command into your terminal and adjust the output file path as needed.
This approach allows you to download the dynamic content by mimicking the browser's request as closely as possible.
« Last Edit: December 08, 2024, 05:23:58 pm by afrocuban »