Thank you for contacting me. Please note that I live in the GMT+3 time zone - responses might be delayed by this.
This topic has 56 replies, 2 voices, and was last updated 3 years, 1 month ago by Szabi – CodeRevolution.
-
AuthorPosts
-
-
October 8, 2021 at 5:09 pm #3906
Installed the plugin and im playing around with it but having some minor formatting issue
plugin is located here
https://groyper.com/wordpress2/wp-admin
**********(redacted)
**********(redacted)
Target website
http://www.marbellainsider.com
Try to pull all the data in
Example of target link
If these questions are available on the website let me know
1) How can pull all the images but not pull duplicates?
2) How can i pull all the posts on the target website?
-
October 8, 2021 at 7:36 pm #3907
Hello,
First of all, thank you for your purchase.
I have set up an example importing rule on your site, please check it with ID 0.
What I changed:
Scraper Start (Seed) URL -> https://www.marbellainsider.com/category/news/
Do Not Scrape Seed URL: checked
Seed Page Crawling Query Type: Visual Selector
Seed Page Crawling Query String: //*[@class=’entry-header’]
Content Query Type: Visual Selector
Content Query String: //*[@class=’entry-content’]
Lazy Loading Images HTML Tag: data-lazy-srcPlease check results.
Tutorial videos to help further set up the plugin: https://www.youtube.com/watch?v=F6vhRJgCR_M&list=PLEiGTaa0iBIgcqNzVBaoTCS4ws47vNMuQ&index=2
How to properly display lazy loaded images (like in the case of the above site): https://www.youtube.com/watch?v=BMzJWZdodlo
Regards, Szabi – CodeRevolution.
-
October 8, 2021 at 9:42 pm #3909
This looks much better. However i know see that the thumbnail image is not populating.
See here https://prnt.sc/1vckxmr
Can we fix this?
Carlos
-
October 9, 2021 at 10:45 am #3912
Hello,
I checked the ‘Auto Get Featured Image’ checkbox from importing rule settings, featured images will be set to posts.
Please check.
Regards.
-
October 9, 2021 at 12:43 pm #3913
Thanks! We are getting there…. but now it duplicates the image in the post. This was an issue with other plugin we used and we hoping we can remove this duplication.
-
October 9, 2021 at 1:05 pm #3914
Please go to the ‘Main Settings’ menu of the plugin -> check the ‘Remove Featured Image From Post Content’ checkbox -> save settings -> import new posts.
Regards.
-
October 9, 2021 at 1:11 pm #3915
Thanks again for the prompt response. Let me try this out!
-
October 9, 2021 at 1:34 pm #3916
Looking good but now its pulling additional articles and images https://prnt.sc/1vfq1jj
Also it seems the titles/content are in Spanish now 🙂 This is awesome but… not neededLove to have it as it was before but not duplicating the image
Orig.
Getting really close!
-
October 9, 2021 at 5:02 pm #3917
Hello,
Disabled translation (not sure who enabled it).
Also, added:
Strip HTML Elements by Class: entry-related
To remove the related articles from the content.
Regards.
-
October 9, 2021 at 7:46 pm #3918
It works! Great work, thank you
Now just a couple quick questions
1) Can we couple the exact URL string?Target site: https://www.marbellainsider.com/celebrate-your-love-at-the-kempinski-hotel-bahia-estepona-1282/
New Target: https://groyper.com/wordpress2/celebrate-your-love-at-the-kempinski-hotel-bahia-estepona-marbella-insider/
2) if we want to grab all do i set the number high?
3) Can we schedule to run to every 2 days?
-
October 10, 2021 at 6:04 am #3920
Hello,
Glad to hear that it works well.
1. In most cases the slug of the post from the created URL will be the same with the slug of the scraped post, but it will depend also on the title on the scraped post. If titles are scraped correctly, the slugs of scraped posts should be also the same as on source sites.
2. Yes, in this case you can increase the max scraped number. In this case, using sitemaps for scraping is also recommended: https://www.marbellainsider.com/post-sitemap.xml
Tutorial video for this: https://www.youtube.com/watch?v=xi1S1093ubo
3. Yes, to schedule runs every 2 days, please set the ‘Schedule’ parameter to 48
Regards.
-
October 10, 2021 at 8:02 pm #3922
Getting there…
If we go the post sitemap way which a lot sense will it know which categories to pull or add?
-
October 10, 2021 at 8:05 pm #3923
You need to point the plugin from where should it import categories and tags. Categories should be also selected using visual selector, like content, featured image and title.
-
October 11, 2021 at 3:18 pm #3924
Thanks. Can we remove the date or not have date on the scrape if no date is provided?
C
-
October 11, 2021 at 3:28 pm #3925
I tried running the program and im getting an yellow triangle and the post aren’t coming in
C
-
October 11, 2021 at 4:00 pm #3926
Dates are not set if you don’t point the plugin to a date from scraped pages.
Also, I ran importing on your site and for me, the plugin worked, please check below:
Regards.
-
October 11, 2021 at 4:04 pm #3927
Ah i fixed the previous issue….
But if we can resolve the adding -marbella insider
https://groyper.com/wordpress2/george-benson-to-perform-at-hotel-puente-romano-marbella-insider/#
-
October 11, 2021 at 4:06 pm #3928
Please set:
Run Regex On Title:
– Marbella Insider
-
October 11, 2021 at 4:17 pm #3929
Sorry… we want to remove the – Marbella Insider
-
October 11, 2021 at 5:01 pm #3930
Fixed the time/date issues
Think if we can fix the url and title issues we are golden
1) we would like the original urls
2) remove – marbella insider from the slug
C
-
October 11, 2021 at 5:03 pm #3931
Please check now.
Regards.
-
October 11, 2021 at 5:12 pm #3932
-
October 11, 2021 at 5:35 pm #3933
Currently the plugin cannot copy slugs of scraped posts, it can only generate the URL slugs based on scraped titles. If you wish to get this feature into the plugin, I can add it as a payed custom update. If you are interested, please contact me at my email kisded@yahoo.com
Regards.
-
October 11, 2021 at 5:40 pm #3934
ok, let me think about.
Thanks for your all your help!
-
October 11, 2021 at 8:29 pm #3935
Last question for the day. But ive set it to pull ~50 posts but it only did 6.
How can i set this to pull until it reaches the max?
-
October 12, 2021 at 6:40 am #3936
For larger number of scraped pages, I recommend setting up a sitemap as a content source, as recommended above.
-
October 12, 2021 at 1:07 pm #3937
Ok, but can we select the category if we pull all the posts?
C
-
October 12, 2021 at 1:22 pm #3938
Also lets say you didn’t have the sitemap feature…. how would you have scrape posts with 4-5 posts on them? Can tell the scrape to follow the “next” page?
-
October 12, 2021 at 1:34 pm #3939
Yes, you can also configure the plugin to follow the ‘next’ button on article listing pages. Please check this video for details: https://www.youtube.com/watch?v=WfT5R5Oi8RU
Regards.
-
October 12, 2021 at 11:22 pm #3955
Ok… getting closer but some other issues popped up
I was able to successfully scraped 63 posts in one sitting!
The only issue is that while it pulled all the data ( or it seemed)
Its not displaying correctlyfor example the feed post in the backend is
https://groyper.com/wordpress2/wp-admin/post.php?post=1643&action=editwhich looks complete but look at the front end
Target source
https://www.marbellainsider.com/this-years-marbella-4days-walking-event-packed-with-family-activities-1002/This is a issue in with most of the blogs pulled
Fill free to delete these and try a smaller amount
C
-
October 13, 2021 at 6:53 am #3956
I changed the lazy loading tag for images to data-orig-file.
It should help.
Regards.
-
October 13, 2021 at 12:04 pm #3957
Great. Let me try that.
How about doing only 10 pages at time?
C
-
October 13, 2021 at 12:16 pm #3958
-
October 13, 2021 at 1:12 pm #3959
This site seems to have an unusual image lazy loading mechanism. Changed the tag to:
data-lazy-src
-
October 13, 2021 at 1:15 pm #3960
Ok, let me try now.
Appreciate all the help
C
-
October 13, 2021 at 1:41 pm #3961
Still not working… let me know if im missing something?
C
-
October 13, 2021 at 1:59 pm #3962
Plugin updated, issue fixed, please check newly created posts.
Regards.
-
October 13, 2021 at 7:31 pm #3963
Thanks again. Seeing the same issue…
Can you check on our website and the target website?
http://www.marbellainsider.com
-
October 14, 2021 at 8:04 am #3966
Hello,
I just imported 5 new posts and images are imported correctly, please check here: https://groyper.com/wordpress2/wp-admin/edit.php?coderevolution_post_source=Crawlomatic_4&post_type=post
Please note that the posts that were imported before I made the plugin update need to be deleted and reimported, so their images get updated.
If you see newly imported posts without images, let me know.
Regards.
-
October 14, 2021 at 10:21 pm #3967
Can we still scrap the website? Looks like they put a redirect?
-
October 15, 2021 at 7:35 am #3969
I checked and it seems that the source website is down right now, they are updating their site or doing some maintenance on it. It might be back soon, not sure on this, depends on the amount of work they are investing in it.
Also, please note that it might come back with a changed HTML structure, so keep an eye out on this.
Regards.
-
October 21, 2021 at 3:49 pm #3975
Thanks again for all your help. Looks like the targeted website went dead.. 🙁
However we found another target rich blog.
https://www.myguidemarbella.com/travel-articles/adventure
Ive added it to the backend and started to crawl but having some issues.
Can you take a look for me?
-
October 21, 2021 at 6:41 pm #3976
Hello,
Please check:
‘Scraper Start (Seed) URL’ -> https://www.myguidemarbella.com/travel-articles/adventure
‘Do Not Scrape Seed URL’ -> checked
‘Seed Page Crawling Query Type’ -> XPath
‘Seed Page Crawling Query String’ -> //*[@class=’box-title summary’]
I hope this helped.
Regards.
-
October 22, 2021 at 9:01 pm #3977
Thanks!
Made the changes but im getting a yellow star. What did i do wrong 😕
-
October 22, 2021 at 10:00 pm #3978
Please check now.
Regards.
-
October 22, 2021 at 10:31 pm #3979
-
October 23, 2021 at 6:12 am #3980
Please check now.
Regards.
-
October 24, 2021 at 1:28 pm #3988
Looks to be working!!
Thanks for your help.
Carlos
-
October 24, 2021 at 1:37 pm #3989
I am glad to help.
-
November 1, 2021 at 7:04 pm #4022
Ready to now load the plugin on the main website…
Do i need to do anything else in terms of registration/license?
-
November 1, 2021 at 7:49 pm #4023
Please revoke the license on your demo site and activate the plugin afterwards, using your license, on your main site. Please check this video for details: https://www.youtube.com/watch?v=79t14bFdhy8
-
November 2, 2021 at 3:52 pm #4024
Successfully transferred the plugin and started crawling.
Almost worked 100% but the images didn’t come thru.
https://www.marbellainsider.com/wp-login.php
<div class=”pul-3-25-0-section-item pul-3-25-0-section-item–vertical pul-3-25-0-form-field” data-test-id=”admin-settings-current-login”>
<div class=”pul-3-25-0-section-item__title”>
<div class=”pul-3-25-0-form-field__label”><label>Administrator</label></div>
</div>
<div class=”pul-3-25-0-section-item__value”>
<div>admin_llg32bio</div>
<div></div>
</div>
</div>
<div class=”pul-3-25-0-section-item pul-3-25-0-section-item–vertical pul-3-25-0-form-field”>
<div class=”pul-3-25-0-section-item__title”>
<div class=”pul-3-25-0-form-field__label”><label>Current password</label></div>
</div>
<div class=”pul-3-25-0-section-item__value”>
<div><strong data-test-id=”admin-settings-current-password”>Tq8tg93W7~W$Cx~_</div>
</div>
</div>
<div></div>
<div>Also anyway we can provide a backlink to the article article?</div>
<div></div>
<div></div> -
November 2, 2021 at 4:14 pm #4025
Hello, in the example post which the plugin scrapes, I don’t see any more images than the featured image (which was imported also to your site’s post): https://www.myguidemarbella.com/travel-articles/marbella-hiking-routes
-
November 2, 2021 at 4:21 pm #4026
Sorry that post was garbled…
Login in
https://www.marbellainsider.com/wp-login.php
admin_llg32bio
Tq8tg93W7~W$Cx~_<
post is here on the target website and the featured image did get pulled but..
What setting do i need to add to display up?
Also can we give credit to the author?
-
November 2, 2021 at 4:51 pm #4027
I clicked on the link you sent, but I get error:
Sorry, you are not allowed to preview drafts.
Also, I saw that the post you mentioned has a featured image assigned: https://www.marbellainsider.com/wp-admin/post.php?post=397&action=edit
Please give more details about this issue, I don’t fully understand it yet.
Regards.
-
November 15, 2021 at 4:20 pm #4107
Thanks. Going to test and see if can get this to work.
Just curious in another one of your plugins….Echo Rss Feed Generator. Whats is the main difference between this and current plugin we are using?
Carlos
-
November 15, 2021 at 5:14 pm #4108
Hello,
The Echo RSS plugin is able to import content strictly from RSS feeds, while Crawlomatic can scrape websites and import content from them. Please check this tutorial video for details on this: https://www.youtube.com/watch?v=8CYkhs6VZyE
Regards.
-
-
AuthorPosts
The topic ‘Importing Several Pages and Images’ is closed to new replies.