Auto-crawling problems

This topic is: resolved

 

Thank you for contacting me. Please note that I live in the GMT+3 time zone - responses might be delayed by this.

This topic has 1 reply, 1 voice, and was last updated 2 months, 4 weeks ago by Szabi – CodeRevolution.

Viewing 1 reply thread
  • Author
    Posts
    • #12512


      congdol
      Participant
      Post count: 14

      It is not possible to add additional comments to the questionnaire, so I will write a new one.

      Looking into the server environment that includes the root function with the hosting company. If necessary, we will upgrade the plan and prepare it.

      I have an additional question.

      Question 1.
      I would like to bring the schedule of the golf tournament, and I would like to ask if I need to activate Puppeteer on that page as well.

      ex1 : https://www.kpga.co.kr/tours/schedule/schedule/?tourId=11
      ex2 : https://www.pgatour.com/schedule

      Question 2.
      I found news that does not activate Puppeteer, but the URL is generated in date format.

      URL : https://v.daum.net/v/20250917184314513
      To automatically scrape newly created news, including those pages

      https://v.daum.net/v/%%counter_1_2_1%%
      I’m going to write it in a format,
      In this case, I think I will scrape the news a few years ago.
      Can you write it based on that URL?

    • #12515


      Szabi – CodeRevolution
      Keymaster
      Post count: 5080

      Hello,

      1. Yes, these are also JS rendered pages, they need Puppeteer.

      2. Scraping in a paginated way will not work here, as there are gaps in the content counting. For example, this article is missing (and many others): https://v.daum.net/v/20250917184314517

      Regards.

Viewing 1 reply thread

The topic ‘Auto-crawling problems’ is closed to new replies.