Skip to content
XUtilHub

Free SEO Tools – Meta Tag Checker, Robots.txt & Sitemap Generator

Practical tools for the technical side of SEO. Audit any page's title, description, canonical, Open Graph and structured data with Google and social previews, build a robots.txt (including rules for AI crawlers), test which URLs it blocks, generate an XML sitemap by crawling your site, and create meta tags from scratch.

5 tools

Meta Tag CheckerNEW

Check any page's title, meta description, canonical, robots, Open Graph, Twitter cards, headings and structured data. See Google and social previews with fixes.

Meta Tag GeneratorNEW

Generate HTML meta tags for SEO and social sharing: title, description, canonical, robots, Open Graph and X (Twitter) cards, with live Google and share previews.

Robots.txt GeneratorNEW

Create a robots.txt file in seconds: disallow paths, add your sitemap, and allow or block Googlebot, Bingbot and AI crawlers like GPTBot and ClaudeBot.

Robots.txt TesterNEW

Test whether Googlebot, Bingbot or AI crawlers can crawl your URLs. Fetch any site's robots.txt or paste your own and see the exact matching rule.

XML Sitemap GeneratorNEW

Generate an XML sitemap by crawling your website (up to 500 pages) or from a list of URLs. Respects robots.txt and noindex. Download sitemap.xml instantly.

Free technical SEO tools for bloggers and site owners

Great content can't rank if search engines can't crawl it, index it or understand it. These tools cover the technical foundations: meta tags, robots.txt and XML sitemaps. They're simple enough for bloggers and precise enough for SEO specialists.

Which tool do I need?

I want to…Use
See why a page isn't showing up well in GoogleMeta Tag Checker
Write titles, descriptions and social tagsMeta Tag Generator
Block admin pages or AI crawlersRobots.txt Generator
Check whether a URL is blocked for GooglebotRobots.txt Tester
Help Google find all my pagesXML Sitemap Generator

A quick technical SEO checklist

  1. Crawlable: robots.txt allows your important pages, and they're linked from your navigation.
  2. Indexable: no accidental noindex, and the canonical URL points to the page itself.
  3. Understandable: a unique title, a clear H1 and a helpful meta description.
  4. Discoverable: an XML sitemap submitted in Google Search Console.
  5. Shareable: Open Graph tags with a 1200 × 630 image.
  6. Rich results: valid JSON-LD structured data where it fits (articles, products, FAQs, recipes).

How the tools fetch pages

The meta tag checker, robots.txt tester and sitemap generator fetch public pages only, identifying themselves as XUtilHubBot. Nothing is stored. The sitemap crawler follows robots.txt and limits itself to a few pages at a time. For private or local pages, paste the HTML or robots.txt directly.

More tools for content work

Count words and keyword density with the Word Counter and Word Frequency Counter, check readability with the Readability Checker, and make clean URLs with the Slug Generator.

Frequently asked questions

Do these tools store the pages I check?

No. Pages and robots.txt files are fetched on demand to analyze them and are not saved. The robots.txt generator, tester and meta tag generator run entirely in your browser.

Can I check pages on localhost or my intranet?

No. For security, our server only fetches public websites. For a private page, use the 'Paste HTML' option of the meta tag checker.

Do the tools respect robots.txt?

The sitemap crawler follows your robots.txt rules and skips pages marked noindex, just like a search engine would.