Berq Markdown: Markdown Pages for AI & LLM Crawlers

描述

AI assistants and LLM crawlers (GPTBot, ClaudeBot, Googlebot-AI) struggle to read messy HTML, and content they cannot read cleanly is content they cannot cite. Berq Markdown exists to fix that and increase your AI visibility. It gives every page on your site a clean, structured Markdown version, so your content is more likely to be read accurately, cited correctly, and recommended by AI tools instead of getting skipped or misread. It does all of this without duplicating your content or creating any separate pages to manage.

Append .md to any public page or post URL (for example https://example.com/about.md) and you get a clean, structured Markdown file with YAML frontmatter, ready for AI tools to consume directly. The plugin handles content from all common sources: Gutenberg blocks, classic editor, shortcodes, and custom PHP templates, so your AI visibility gain covers your entire site, not just pages built one specific way.

How it works

When a visitor or crawler requests /about.md, the plugin intercepts the request, finds the matching WordPress post, converts the HTML content to Markdown, and serves it with the correct Content-Type: text/markdown header, the format AI tools expect. No new database entries or files are created, so you gain AI visibility without adding any clutter to your site.

Key Features

  • Automatic .md URLs: any public post or page URL plus .md returns clean Markdown, so AI crawlers get a direct, accurate read of your content instead of a messy HTML parse, increasing the odds your content gets cited.
  • YAML frontmatter: every .md file includes title, description, url, date_modified, author, and language, so AI tools can attribute and date your content correctly instead of guessing, which improves how confidently they cite you.
  • Full content compatibility: works with Gutenberg blocks, classic editor, shortcodes, and custom PHP page templates, so your AI visibility gain reaches every page on your site, not just a few.
  • HTML to Markdown conversion: pure PHP with no external dependencies. It handles headings, paragraphs, lists, links, images, tables, and code blocks, so AI tools get well formed Markdown instead of a broken or partial conversion.
  • <link> tag injection: injects a <link rel="alternate" type="text/markdown"> tag in every page’s <head>, so crawlers can discover the .md version of every page on their own, increasing the chance they read the clean version instead of the raw HTML.
  • Multilingual support: detects language via Polylang or WPML, adds language and alternates fields to the frontmatter, and emits hreflang link tags, so AI tools recommend the correct language version of your content to the right audience instead of the wrong one.
  • Post type selection: choose which post types (posts, pages, custom post types) are served as Markdown, so you decide exactly which content is eligible to be read and cited by AI tools.
  • URL exclusions with wildcards: exclude specific URL patterns using * as a wildcard (for example /cart/*, */checkout*), so private or transactional areas like checkout and account pages are kept out of AI crawling entirely.
  • Transient caching: optional cache with a configurable TTL that absorbs traffic spikes from AI crawlers, so your site stays fast for human visitors even under heavy bot traffic.
  • [btrmd313_md_link] shortcode: renders a link to the current page’s .md file and opens it in a new tab, giving readers an easy way to grab a clean copy of your content for their own AI tools.
  • Public content rendering: always renders content as a non logged in visitor, so member only sections are never exposed through the Markdown endpoint.
  • Admin settings page: every option is configurable under Settings > Berq Markdown, with a Settings link on the Plugins page, so you stay in control without touching code.

Other Products

Also check out our other WordPress products.

屏幕截图

安装

  1. Upload the berq-markdown folder to /wp-content/plugins/
  2. Activate the plugin through the Plugins menu in WordPress
  3. Go to Settings > Permalinks and click Save Changes to flush rewrite rules (required once after activation)
  4. Visit any page URL with .md appended, for example https://yoursite.com/about.md
  5. Optionally configure the plugin under Settings > Berq Markdown

常见问题

Does this create duplicate content?

No. The .md URLs are separate endpoints that return Content-Type: text/markdown, so search engines treat them differently from HTML pages. The plugin also sends X-Robots-Tag: noindex, follow by default, which keeps the Markdown versions out of the search index while still letting AI crawlers follow and read them, so you gain AI visibility without any SEO downside.

Will it work with my page builder?

Yes. The plugin renders content the same way WordPress does for a logged out visitor, using the full the_content filter chain, which covers Gutenberg blocks and most page builders. For pages using custom PHP templates with no post_content, it falls back to output buffering the template and extracting the body content, so even unusual page setups still get a working .md version.

How do I exclude certain pages from .md serving?

Go to Settings > Berq Markdown > URL Exclusions and enter one URL pattern per line. Use * as a wildcard. Examples: /cart/*, */checkout*, /my-account/. This keeps private or transactional pages out of AI crawlers entirely.

Does it work with Yoast SEO?

Yes. The plugin reads Yoast’s meta description and uses it for the frontmatter description field, so the description AI tools see matches the one you already wrote for SEO, keeping your AI visibility and search visibility consistent.

Does it support custom post types?

Yes. Go to Settings > Berq Markdown > Post Types and check any registered public post type you want included, so you can extend AI friendly content beyond just posts and pages.

Is caching enabled by default?

No, caching is disabled by default so you always see fresh content during development. Enable it under Settings > Berq Markdown > Cache when you are ready to scale and want to reduce server load from repeated crawler requests.

What shortcode is available?

[btrmd313_md_link] outputs a link to the current page’s .md version and opens it in a new tab.

Optional attributes:
* text: link label (default: “View as Markdown”)
* class: additional CSS class or classes

Example: [btrmd313_md_link text="Read as Markdown" class="my-button"]

Does it support multilingual sites?

Yes. The plugin detects language via Polylang or WPML. It adds a language field to the frontmatter and an alternates block listing all translated .md URLs. It also emits <link rel="alternate" hreflang="..."> tags in the HTML <head>, so AI tools and crawlers always land on the right language version of your content.

Does it work with caching plugins?

Yes. The plugin serves .md files before WordPress loads any theme template. Cache plugins that operate at the HTTP level, such as full page caching, may cache .md responses too. That’s fine and actually helps performance under heavy crawler traffic.

评价

此插件暂无评价。

贡献者及开发者

「Berq Markdown: Markdown Pages for AI & LLM Crawlers」是开源软件。 以下人员对此插件做出了贡献。

贡献者

更新日志

1.3.2

  • Set max 5 minutes cache age limit for .md endpoints.

1.2.0

  • Removed: Automatic internal link rewriting to .md equivalents. Links inside Markdown output now keep their original URLs.
  • Removed: Sitemap integration (Yoast provider and the standalone /md-sitemap.xml endpoint).
  • Change: .md responses now send a fixed Cache-Control: public, max-age=300 header, so browsers and CDNs cache them for a short, predictable window regardless of the server side cache TTL setting.

1.1.0

  • Fix: MD files now served with correct 200 HTTP status (WordPress was setting 404 before template_redirect fired)
  • Fix: Home page .md URL corrected to /index.md instead of broken site.local.md
  • Fix: url_to_postid() trailing slash sensitivity. The plugin now tries the URL both with and without a trailing slash
  • Fix: PHP template pages with empty post_content now rendered correctly via output buffer
  • Fix: HTML comments (including Gutenberg block delimiters) stripped from Markdown output
  • Fix: Heading whitespace normalized. Multi line headings now collapse to a single line
  • Fix: Inline tags inside headings stripped to plain text
  • Fix: Description frontmatter field no longer wrapped in unnecessary quotes
  • Fix: Cache disabled by default, opt in instead of opt out
  • Fix: WordPress Plugin Check compliance. Text domain corrected, parse_url switched to wp_parse_url, strip_tags switched to wp_strip_all_tags, variable prefixing fixed in uninstall
  • Feature: Post type selection. Choose which post types are served as .md files
  • Feature: Multilingual support with language and alternates frontmatter fields and hreflang link tags (Polylang and WPML)
  • Feature: Internal links inside Markdown output rewritten to .md equivalents automatically
  • Feature: [btrmd313_md_link] shortcode renders a link to the current page’s .md file
  • Feature: Template pages rendered as non logged in visitor for accurate public content
  • Feature: Admin bar suppressed during template output buffering

1.0.0

  • Initial release
  • .md URL serving with YAML frontmatter
  • HTML to Markdown conversion (pure PHP, DOMDocument)
  • Gutenberg block, shortcode, and custom template support
  • URL exclusions with wildcard support
  • Post type selection
  • Transient cache with configurable TTL
  • <link rel="alternate"> injection in HTML head
  • [btrmd313_md_link] shortcode
  • Yoast SEO sitemap integration plus generic md-sitemap.xml fallback
  • Multilingual support (Polylang, WPML) with language frontmatter and hreflang tags
  • Internal link rewriting to .md equivalents
  • Non logged in rendering for accurate public content
  • Admin settings page under Settings > Berq Markdown