Szabi – CodeRevolution

Hello,

Thank you for the login credentials. I checked the site you want to scrape and indeed, scraping it is very hard, as it’s content is fully JavaScript generated (dynamic), everything is rendered on the page after it is loaded (this is why you see the lazy loading placeholders when you load the page).

I managed to scrape it automatically only using Puppeteer – this needs to be installed on your server, as shown here: https://www.youtube.com/watch?v=pRUDcSOe724 – contact hosting support and ask about this.

After it is installed, it can be used as shown here: https://www.youtube.com/watch?v=ZljpMpmi_dU

I managed to scrape the site you mentioned, using the below settings:

Scraper Start (Seed) URL / Keywords
https://m.sports.naver.com/golf/index

Content Scraping Method To Use:
Puppeteer

Headless Browser Wait Before Rendering Pages (ms):
5000

Do Not Scrape Seed URL:
Checked

Seed Page Crawling Query Type:
Class

Seed Page Crawling Query String:
grid_item

Content Query Type
Class

Content Query String
_article_content

I hope this helps.

Regards,
Szabi – CodeRevolution.

There is an alternative. Please go to the plugin’s ‘Main Settings’ -> set the ‘WordPress Feed Refresh Interval’ settings field to 3600 -> save settings.

This might also help solve this problem.

Regards.

Hello,

I checked and I am getting the same issue, please check the email I sent to you, there I show a screen recording of the issue.

Regards.

I am glad to help!

Regards.

Hello,

First of all, thank you for your purchase.

Can you send me, please, temporary admin login credentials to your WordPress install, so I can check this issue out? Send it, please, to my email address: kisded@yahoo.com.

Regards,
Szabi – CodeRevolution.

Hello,

I checked and it seems that YouTube blocked access to the resources which provided the transcripts for videos, now they are no longer publishing these in a public way. Because of this, scraping the subtitles will no longer be possible in the plugin (at least until I find a solution to make scraping work again).

I will dig into this, but it seems that they are now actively blocking access to subtitles. Others also have this issue: https://www.reddit.com/r/webscraping/comments/1gxv1ai/scraping_captions_on_youtube_is_impossible_now/ – I will try to work on this.

Regards.

I suggest you add the full manga URL for Naruto, from a Madara themed site.

Hello,

The plugin’s execution is fully stopped by your server after 20 chapters, because of this, other manga will not manage to be published also. If current hosting provider is not helpful, I suggest changing hosting provider and migrating your site to a VPS hosting.

Also, scraping and continuing from last scraped chapter is a good idea, for this, please enable the ‘Cache Items For Continuous Posting’ checkbox in rule settings.

Regards,
Szabi – CodeRevolution.