Allison 1071e56173 feat(marketing): exempt public blueprints from noindex + fix / route collision ...

- add_no_crawl_headers now skips marketing.*, legal.*, billing.success,
  static, and robots_txt endpoints via _is_public_indexable_endpoint
  helper; all other routes keep the X-Robots-Tag noindex header
- recordings.index drops @login_required and instead redirects
  anonymous users to marketing.landing, resolving the URL-map
  collision between recordings_bp and marketing_bp at "/"
- robots.txt rewritten: public marketing pages and /legal/* allowed,
  /api/, /admin, /account, /share/, /app/, /checkout, /login, /signup,
  /webhooks/ disallowed; Googlebot, Bingbot, ClaudeBot, GPTBot,
  PerplexityBot, Applebot explicitly allowed
- New tests/test_no_crawl_headers.py (14 tests) covers exemption
  helper + integration on /, /robots.txt, /static, /admin, /login
- New tests/test_marketing_root_redirect.py (4 tests) verifies
  anonymous users at / never get a /login redirect

Tests verified via AST + logic walkthrough; pytest blocked on Windows
by pre-existing fcntl import in src/init_db.py (B-1.2 limitation).

2026-04-27 16:28:55 -04:00

4.6 KiB

Raw Blame History

View Raw