It needs a lot of writing.

This topic is: resolved

 

Thank you for contacting me. Please note that I live in the GMT+3 time zone - responses might be delayed by this.

This topic has 6 replies, 2 voices, and was last updated 3 years ago by Szabi – CodeRevolution.

Viewing 6 reply threads
  • Author
    Posts

    • December 27, 2022 at 11:30 am #6485


      prbinus
      Participant
      Post count: 3

      https://24post.co.kr/people2

      I want to crawl the site.

      1 post is collected

      Collecting multiple posts is too slow.

      Is there any quick way to do it?

      The above site is posting a lot of articles in an hour.

      can’t keep up with the speed

    • December 27, 2022 at 3:34 pm #6487


      Szabi – CodeRevolution
      Keymaster
      Post count: 5084

      Hello,

      First of all, thank you for your purchase.

      Please use the below settings in the plugin for this specific source, to scrape multiple articles from them:

       

      Scraper Start (Seed) URL / Keywords
      https://24post.co.kr/people2

      Do Not Scrape Seed URL:
      Checked

      Seed Page Crawling Query Type:
      Regex Capture Group Match

      Seed Page Crawling Query String:
      #<a href=”(\/people2\/\d+)”>#

      Content Query Type
      Class

      Content Query String
      rd_body clear

       

      Let me know if this helped.

      Regards, Szabi – CodeRevolution.

    • December 27, 2022 at 5:09 pm #6489


      prbinus
      Participant
      Post count: 3

      https://gall.dcinside.com/board/lists?id=dcbest

       

      Can you check here too?

    • December 27, 2022 at 6:25 pm #6490


      Szabi – CodeRevolution
      Keymaster
      Post count: 5084

      Scraper Start (Seed) URL / Keywords
      https://gall.dcinside.com/board/lists?id=dcbest

      Do Not Scrape Seed URL:
      Checked

      Seed Page Crawling Query Type:
      Class

      Seed Page Crawling Query String:
      ub-content us-post

      Content Query Type
      ID

      Content Query String
      article_body

    • December 28, 2022 at 6:02 am #6493


      prbinus
      Participant
      Post count: 3

      thank you for the quick response

      I did as you said, but it is registered like a picture, and all the details are listed in the article list.

      is this correct?

      And I have one question.

      The criterion for collecting articles is every hour.

      Can’t it be corrected in minutes?

    • December 28, 2022 at 7:43 am #6494


      prbinus
      Participant
      Post count: 3
      This reply has been marked as private.

    • December 28, 2022 at 12:18 pm #6498


      Szabi – CodeRevolution
      Keymaster
      Post count: 5084

      Please be sure to set:

      Title Query Type
      Auto Detect

       

      Regarding minute based running, currently, the plugin is limited to a minimum of 1 hour rule running period. This limitation is to not allow users to overuse their server’s resources and contact me afterwards complaining that the plugin is not working.

      If you wish, I can add this feature to the plugin for you, for details, please contact me at email kisded@yahoo.com

      Regards.

  • Author
    Posts
Viewing 6 reply threads

The topic ‘It needs a lot of writing.’ is closed to new replies.