Forum Replies Created
-
Posts
-
Hello,
I am glad to hear that it works.
Tweets and videos are content which is JavaScript generated, which makes their importing possible only if you use a headless browser to render the content before scraping it in the plugin (as only headless browsers are able to render also JavaScript in the scraped content – to generate the dynamic content).
This is possible in the plugin if you install on your server Puppeteer (which is a headless browser). Please check this tutorial video on how to install Puppeteer on your server: https://www.youtube.com/watch?v=KNOIJA4pTQo – afterwards, you can set up the plugin to use Puppeteer for scraping, as shown in this video: https://www.youtube.com/watch?v=g99IlDkt_SY
If this is not possible on your server, the plugin supports also HeadlessBrowserAPI, which is a cloud based service which renders pages in Puppeteer and returns the full HTML content for them, please check details here: https://www.youtube.com/watch?v=205EinBQAoo&list=PLEiGTaa0iBIjDrfexapWc3M28iHwJI5tT&index=2 + website: https://headlessbrowserapi.com/
Regards, Szabi – CodeRevolution.
-
Hello,
First of all, thank you for your purchase.
I checked the RSS feed from where you are importing posts (https://www.telam.com.ar/rss2/sociedad.xml) and I see that it contains 20 feed items in it. This means that the plugin will be able to import from this RSS feed at any given time a maximum of 20 posts. However, the posts which will be published in the future on this site, will also be added to this RSS feed and they will be automatically imported by the plugin.
If you want to scrape also older posts from this site, I recommend you check also the Crawlomatic plugin I created, which uses direct web scraping and can get also older posts from sites: https://1.envato.market/crawlomatic
Regarding the cron functionality, I suspect that it works for you correctly, however, as posts are imported only when the feed is updated, the plugin will be able to pull in new posts by a schedule only when the RSS feed is updated with new posts – because of this, the schedule might not import anything as expected, but only when the source website publishes new content in its RSS feeds (as the plugin will not publish duplicate posts – will skip the posts which it already published on your site).
Let me know if this info helped.
Regards, Szabi – CodeRevolution.
-
Hello,
First of all, thank you for your purchase.
Can you send me, please, temporary admin login credentials to your WordPress install, so I can check the plugin registration issue out?
Send it, please, to my email address: kisded@yahoo.com
Regarding the visual selector issue, I suspect that your server’s IP address is blacklisted from the part of the https://almanhaj.or.id/ website. I suggest you use proxies in the plugin’s ‘Main Settings’ menu -> ‘Web Proxy Address List’ settings field, or alternatively, you change the current IP address of your server.
Regards, Szabi – CodeRevolution.
-
Hello,
I disabled the automatic proxy usage of HeadlessBrowserAPI, by setting in the ‘Main Settings’ of the plugin ->
Web Proxy Address List:
disabledI did this, as I saw in plugin logs that some IP addresses from the proxy range were not able to properly scrape the website. Please check if this helped.
Regarding custom updates, please contact me at my email kisded@yahoo.com and describe your ideas. Currently I am fully booked with custom work, but I might be able to help, depending on your requirements complexity.
Regards.
-
Hello,
First of all, thank you for your purchase.
Please install this free plugin and give it a try: https://wordpress.org/plugins/open-external-links-in-a-new-window/
Regards,
Szabi – CodeRevolution.
-
-
-
Hello,
I checked and this site seems to have a scraping protection mechanism active, which prevents its scraping using the conventional method.
I managed to scrape this site only using Puppeteer. It is a headless browser which needs to be installed on your server, please check this tutorial video for installation: https://www.youtube.com/watch?v=KNOIJA4pTQo
After installation, you can set in rule settings:
Content Scraping Method To Use:
PuppeteerIf installation is not possible on your server, you can also use HeadlessBrowserAPI, which is a cloud based service which renders the page using Cloud Puppeteer instances, please check details here: https://www.youtube.com/watch?v=205EinBQAoo
If you use this method, you need to set in rule settings:
Content Scraping Method To Use:
Puppeteer (HeadlessBrowserAPI)Regards.
-
Hello,
Sure, send me login info and I will check on this. Send it, please, to my email address: kisded@yahoo.com
Regards, Szabi – CodeRevolution.
-
Hello,
First of all, thank you for your purchase.
Based on your description, I think you made the right purchase, as the “URL to RSS – Custom Curated RSS Feeds” plugin is scraping other websites and creating RSS feeds from them, while the Echo RSS plugin is creating RSS feeds from the posts which appear on your own website’s blog (if I understood correctly, this is what you need).
Please check the second half of this tutorial video, which shows how to create RSS feeds using the Echo RSS plugin, which includes content from your blog: https://www.youtube.com/watch?v=d9iIUjegP0o (starting from minute: 28:42 )
Let me know if this helped.
Regards,
Szabi – CodeRevolution. -
Hello,
First of all, thank you for your purchase.
Currently running all rules in bulk is possible only if you schedule them to run each hour (Schedule parameter set for all to 1). Unfortunately this is not yet possible manually.
However, I noted this idea, will think on adding a way to manually run rules in bulk in upcoming versions of the plugin.
Regards, Szabi – CodeRevolution.
-
Hello,
First of all, thank you for your purchase.
Yes, this should be possible.
To check on this, can you send me, please, temporary admin login credentials to your WordPress install, so I can check this issue out? Send it, please, to my email address: kisded@yahoo.com
Regards, Szabi – CodeRevolution.
-
Hello,
First of all, thank you for your purchase.
Yes, this should be possible.
To check on this, can you send me, please, temporary admin login credentials to your WordPress install, so I can check this issue out? Send it, please, to my email address: kisded@yahoo.com
Regards, Szabi – CodeRevolution.
-
-
