Free XML Sitemap Generator indexing problems, anyone seen this?

Author
Liam Williams Author
|
4 days ago Asked
|
16 Views
|
2 Replies
0
  • just launched my new site and using the AdsVolt 'Free XML Sitemap Generator' to help with indexing, but im hitting a snag.

  • the tool generates the sitemap fine, but when i submit it to Google Search Console, it's reporting some pages as 'discovered - currently not indexed' or sometimes even 'couldn't fetch'. it's like the sitemap isn't helping with proper indexing.

  • here's a snippet from the Search Console sitemap report:

    Sitemap: sitemap.xml
    Type: Sitemap
    Submitted: Jun 14, 2024
    Last read: N/A
    Status: Couldn't fetch
    Errors: 1
    Warnings: 0
    Pages discovered: 0
    
  • anyone else faced this? any ideas why the sitemap might not be helping with indexing or what could be causing the 'couldn't fetch' error?

  • help a brother out please...

2 Answers

0
Mariana Gonzalez
Answered 4 days ago
Hello Liam Williams, Before we dive into the indexing issues, just a quick heads-up: it's 'I'm' not 'im' when you're referring to yourself. Easy slip to make! Regarding your sitemap and indexing problems, the 'Couldn't fetch' error is a critical blocker. Google cannot even access your sitemap, let alone process it. This typically points to a server-side or configuration issue preventing Googlebot from reaching the `sitemap.xml` file. The 'discovered - currently not indexed' status, while related, is a secondary concern you'll address once the sitemap is successfully fetched. Here's a breakdown of common causes and a systematic approach to troubleshoot this:
  • Verify Sitemap Accessibility:
    • First, try to access your `sitemap.xml` directly in a web browser (e.g., `https://yourdomain.com/sitemap.xml`). If it doesn't load or shows an error, your server or file path is incorrect.
    • Use a tool like Google's `robots.txt` Tester in Search Console to fetch your `sitemap.xml` as Googlebot. This will show you exactly what Google sees.
  • Check `robots.txt` File:
    • Your `robots.txt` file (usually at `https://yourdomain.com/robots.txt`) might be blocking Googlebot from accessing your sitemap or specific directories where your sitemap resides. Ensure there's no `Disallow: /sitemap.xml` or `Disallow: /` that unintentionally prevents access.
    • Also, ensure your `robots.txt` correctly references your sitemap with a `Sitemap:` directive, e.g., `Sitemap: https://yourdomain.com/sitemap.xml`.
  • Server & Hosting Issues:
    • A 'Couldn't fetch' error can sometimes be due to temporary server downtime, slow response times, or network issues on your hosting provider's end. Check your hosting panel for any server status updates or logs.
    • Ensure your server isn't blocking Googlebot's IP ranges or user-agent string. Some firewalls or security plugins can be overly aggressive.
  • Incorrect Sitemap URL in Search Console:
    • Double-check that the URL you submitted for the sitemap in Google Search Console is absolutely correct, including `https://` and any `www` or non-`www` preference. A single character mismatch will cause a fetch error.
  • Sitemap Format & Validation:
    • While the AdsVolt tool generally produces valid XML, it's worth running your generated sitemap through an online XML sitemap validator (e.g., XML-Sitemaps.com validator). This ensures there are no hidden errors in the XML structure itself that could trip up Googlebot.
  • Addressing 'Discovered - currently not indexed':
    • Once your sitemap is successfully fetched, if pages are still not indexing, remember that a sitemap is a hint, not a command. Google uses it for discovery.
    • Content Quality: Google prioritizes high-quality, unique, and valuable content. Thin, duplicate, or low-quality pages are often "discovered" but not indexed.
    • Canonicalization: Review your `canonicalization` setup. If pages are pointing to other versions (or themselves incorrectly), it can prevent indexing.
    • Internal Linking: Strong internal linking helps Google understand the hierarchy and importance of your pages, aiding in their `crawl budget` allocation and indexing.
    • Noindex Tags: Ensure you haven't accidentally placed a `noindex` meta tag or `X-Robots-Tag` HTTP header on these pages.
  • Alternative Sitemap Generation:
    • While our free tool is great for basic needs, for dynamic sites or more advanced `technical SEO` control, consider using a dedicated SEO plugin if you're on a CMS like WordPress (e.g., Yoast SEO, Rank Math) or a server-side script that generates it dynamically. These often handle updates and technical nuances more seamlessly.
0
Liam Williams
Answered 3 days ago

So, this is a super comprehensive breakdown, Mariana. Really appreciate you laying out all the potential causes for the 'Couldn't fetch' error and then linking it to the 'not indexed' stuff. Anyone else hitting these sitemap issues should definitely check out this thread.

Your Answer

You must Log In to post an answer and earn reputation.