Scrapper aborts without error

This topic is: resolved

 

Thank you for contacting me. Please note that I live in the GMT+3 time zone - responses might be delayed by this.

Tagged: 

This topic has 3 replies, 2 voices, and was last updated 2 years, 5 months ago by Szabi – CodeRevolution.

Viewing 3 reply threads
  • Author
    Posts

    • July 13, 2022 at 9:11 am #5512


      myls
      Participant
      Post count: 1

      Hi I have some Problems with scrapping a Woocomerce Shop (https://fontarocca.ch/produkte-2/). I want to scrapp all Products at once from this page. Sadly this doesnt work. Its working for 2 hours an after this time it breaks with no error message. I have a WordPress with woocomerce an want to import all the products. I also can’t define the post picture.

      Could you help me please.

    • July 13, 2022 at 11:04 am #5515


      Szabi – CodeRevolution
      Keymaster
      Post count: 4621

      Hello,

      First of all, thank you for your purchase.

      To make the plugin scrape all products from a single run, you have to increase the max_execution_time variable on your server, to a value greater than the current one, which seems to limit execution to 2 hours.

      To make this change, you have to edit your php.ini file on your server, as shown here: https://code.tutsplus.com/tutorials/how-to-increase-max_execution_time-in-php–cms-37017 – you should increase its value to 18000 or greater (5 hours in seconds).

      If you don’t know how to make this change or you don’t have access to this file, please contact your hosting provider for help.

      Also, to scrape products more efficiently, I recommend you use a sitemap scraping, following the settings I also used on my test site, shown below (the settings will guide you also to scrape the featured image correctly):

       

      Scraper Start (Seed) URL / Keywords
      https://fontarocca.ch/product-sitemap.xml

      Do Not Scrape Seed URL:
      checked

      Seed Page Crawling Query Type:
      Sitemap

      Seed Page Crawling Query String:
      https://fontarocca.ch/produkte/

      Featured Image Query Type:
      Class

      Featured Image Query String:
      wp-post-image

       

      Regards, Szabi – CodeRevolution.

    • July 13, 2022 at 11:57 am #5521


      myls
      Participant
      Post count: 1

      Hi thanks for your answer an the tipp. If I pick the featured Image via the class or any other option wordpress run into this error:
      Failed to load page (wp api): === — WP_Error Object ( [errors] => Array ( [http_request_failed] => Array ( [0] => Es wurde keine gültige URL übermittelt. ) ) [error_data] => Array ( ) [additional_data:protected] => Array ( ) )

      Dont now what this means.

    • July 13, 2022 at 12:34 pm #5523


      Szabi – CodeRevolution
      Keymaster
      Post count: 4621

      Hello,

      Can you send me, please, temporary admin login credentials to your WordPress install, so I can check this issue out? Send it, please, to my email address: kisded@yahoo.com

      Regards, Szabi – CodeRevolution.

  • Author
    Posts
Viewing 3 reply threads

The topic ‘Scrapper aborts without error’ is closed to new replies.