Blog
SavedSearch
Visit Hostlelo

Short Planned Maintenance: HTTP 503 and Retry-After

Configure HTTP 503 maintenance with Retry-After, no-store caching, Apache/Nginx rules, Express middleware, a Cloudflare edge fallback and recovery checks.

The short answer

For short planned maintenance, return HTTP 503 with a realistic Retry-After estimate and Cache-Control: no-store on affected URLs. Keep robots.txt available with its normal rules, use an unaffected proxy or edge if the origin stops, and verify GET, HEAD, deep routes and recovery. These steps reduce avoidable search risk; they do not guarantee rankings or AI citations.

Purple server boxes beside a coral wrench, a clock and a return arrow, illustrating short planned website maintenance.
Reading options

On this page

For short planned website maintenance, return HTTP 503 Service Unavailable on each affected URL, include a realistic Retry-After value, and prevent the maintenance response from being stored. Keep robots.txt reachable with its normal rules. Restore normal responses as soon as the work is complete.

This guide covers Apache/LiteSpeed, Nginx, Express and an origin-independent Cloudflare Worker. It also explains how to check deep URLs, cache behavior and recovery. The configurations are adaptation examples, not changes to your running server.

What a 503 response actually says

RFC 9110, sections 15.6.4 and 10.2.3 define 503 for temporary inability to handle a request, including overload and scheduled maintenance. Retry-After is an optional retry estimate. It can contain integer seconds or an HTTP date.

The header does not schedule server recovery or make a browser reload. Client retry behavior depends on the client; search crawlers do not promise to return at the exact time specified.

The complete response and both Retry-After formats

For a maintenance response, send a status, retry estimate, content type and cache policy together:

HTTP/1.1 503 Service Unavailable
Retry-After: 3600
Cache-Control: no-store, no-cache, must-revalidate, max-age=0
Content-Type: text/html; charset=UTF-8

The HTML notice follows a blank line. HTTP/2 and HTTP/3 use a different wire format; the status and header meanings remain the same.

  • Delay-seconds: Retry-After: 3600 asks the client to wait one hour after receiving the response. Use a non-negative integer, not 1h or a decimal.
  • HTTP-date: Retry-After: Sat, 24 Oct 2026 03:00:00 GMT gives a fixed end time. Replace this illustrative date before use.

A fixed delay is a rolling estimate. If maintenance ends at 03:00 UTC, a visitor arriving at 02:55 still receives a full hour with a constant 3600. Use the fixed HTTP date or calculate the remaining seconds for a known deadline. GMT is required for the generated HTTP date; local time belongs in the visitor-facing notice. The example end time is 07:00 in Dubai and 08:30 in India.

If work overruns, update both the estimate and the notice. Avoid past dates or Retry-After: 0 while the service is still unavailable. Never reopen an unhealthy service merely because the estimate has expired.

No-store and no-cache do different jobs

Under RFC 9111, no-store prevents compliant caches from storing this response. no-cache permits storage but requires successful validation before reuse. max-age=0 makes a stored response immediately stale; must-revalidate restricts stale reuse.

For this use case, Cache-Control: no-store is the key directive. The combined policy above is conservative; all four directives are not separate HTTP requirements. A 503 is not normally heuristically cacheable, but explicit CDN rules can change caching behavior. Response headers do not purge previously cached pages.

Before you start

  • You know which web server runs your site: Apache, LiteSpeed or Nginx. Your host can tell you, and our guide on what web hosting is explains the moving parts.
  • You can edit .htaccess or the Nginx config, through cPanel File Manager, SFTP or SSH.
  • You know the public IP address you will work from, so you can let yourself in.
  • You have a static maintenance.html page. Google recommends static HTML, inline CSS and few outside resources for a 503 page, plus the expected return time and how to contact you.
  • You have curl on your computer for testing.
  • You will back up the file before editing. The steps show how.

Made-up example: the owner of example.com plans a database upgrade on Saturday 24 October 2026 from 02:00 to 03:00 UTC. That invented one-hour window means Retry-After is 3600 seconds, and the owner works from the example IP 203.0.113.25.

Why a 200 maintenance page is a ranking risk

A maintenance notice served with 200 OK tells Google that the response is normal page content. Google may process that notice or classify an error-looking page as a soft 404. Google's status-code guidance says it ignores 5xx response content and slows crawling; persistent server errors can eventually remove URLs from the index.

There is no guaranteed 48-hour safe window

Google's pause guidance describes a 503 page as an option for an urgent one-to-two-day closure. Its crawl troubleshooting guide warns against returning 503 or 429 for more than about two days.

Treat those statements as reasons to minimize downtime, not a promise that every URL stays indexed until hour 48 or disappears at hour 49. Plan in minutes or hours, monitor the window and restore promptly. Correct maintenance signaling reduces avoidable search risk; it does not guarantee stable rankings.

Choose the response for the actual situation

  • Short planned outage: return 503 with a realistic Retry-After estimate for the affected resource.
  • Checkout alone is unavailable: keep useful information on 200 and disable the affected operation.
  • One client exceeds a request limit: use 429 Too Many Requests; a retry estimate may be useful.
  • Resource permanently deleted: return 404 or 410.
  • Resource moved permanently: use 301 or 308 to its actual replacement.

Do not use 403, sitewide noindex, blanket Disallow: / or Search Console removals to represent a short closure. For a longer closure, retain useful, accessible content wherever possible. Google recommends limiting functionality; if the whole business must be offline for longer, an informative indexable homepage can remain on 200. Do not turn every deep URL into a duplicate 200 holding page.

The common mistake: sending everyone to the home page

A tempting shortcut is one rule that redirects every URL to the home page or to a maintenance page that returns 200. It feels tidy, but Google's status code page says content from a redirecting URL is ignored and the target's content is processed instead.

Every product page and post now points at one 200 page saying "we're closed", so Google processes that single page instead of your real content. A 503 keeps each URL saying "I still exist, try again shortly".

Set up a 503 maintenance mode on Apache or LiteSpeed

Step 1: Make a backup copy first

A typo in .htaccess can take the whole site down, so copy it first. In cPanel, open Files then File Manager, click Settings and tick Show Hidden Files (dotfiles).

Download a private copy of .htaccess over SFTP, or use File Manager's Copy action with a destination your host has confirmed is outside every document root, alias and symlinked public path. Do not create .htaccess.bak alongside the public file.

For SSH, run this illustrative Bash block from the document root after replacing the private-parent placeholder with an existing writable directory confirmed by your host. These backup commands have not been executed on your server:

(
  set -eu
  umask 077
  private_parent='/absolute/non-public-backups'
  test -d "$private_parent" && test -w "$private_parent"
  backup_dir="$(mktemp -d "$private_parent/maintenance.XXXXXXXX")"
  cp -- .htaccess "$backup_dir/htaccess.backup"
  chmod 600 "$backup_dir/htaccess.backup"
  ls -ld "$backup_dir" "$backup_dir/htaccess.backup"
)

Record the generated absolute backup path and keep a protected off-server copy. The new backup directory should allow only its owner, and the file should have mode 600. A dot-prefixed name is not an access control. Only the static maintenance.html page belongs in the document root.

Step 2: Add the 503 rules

This block sends a 503 to everyone except 203.0.113.25, serves your maintenance page as the body, leaves robots.txt reachable, and only runs while a file named maintenance.enable exists. Put it before your application/front-controller rewrite rules in .htaccess. Apache needs mod_rewrite, mod_headers and suitable AllowOverride permissions; ask your host to confirm them. Each directive comes from the Apache docs, but this exact combination is not shown in them, so test with curl before relying on it (see the check section below).

# Maintenance mode: active only while maintenance.enable exists
ErrorDocument 503 /maintenance.html

RewriteEngine On
RewriteCond %{DOCUMENT_ROOT}/maintenance.enable -f
RewriteCond %{REMOTE_ADDR} !^203\.0\.113\.25$
RewriteCond %{REQUEST_URI} !^/maintenance\.html$
RewriteCond %{REQUEST_URI} !^/robots\.txt$
RewriteRule ^ - [R=503,L]

# These headers apply only to 503 responses
Header always set Retry-After "3600" "expr=%{REQUEST_STATUS} == 503"
Header always set Cache-Control "no-store, no-cache, must-revalidate, max-age=0" "expr=%{REQUEST_STATUS} == 503"

Why each piece is there, from the Apache 2.4 docs:

  • ErrorDocument uses a local path. Apache warns that a full remote URL makes the client receive a redirect status instead of the original error code.
  • The -f test checks that maintenance.enable exists, so you switch maintenance on and off with one file.
  • The - substitution leaves the path untouched.
  • [R=503] works because for codes outside 300 to 399, mod_rewrite drops the substitution and handles the response through the normal error path, including ErrorDocument.
  • Header always set is needed because headers in the always table are added even on errors and persist across internal redirects such as ErrorDocument handlers.

Heads up: behind a CDN or proxy, REMOTE_ADDR may hold the proxy's address. Configure mod_remoteip with trusted proxy ranges before using a visitor-IP exception. Never trust arbitrary client-supplied forwarded-IP headers. Keep admin bypass responses out of shared caches so an administrator's 200 response cannot reach public visitors.

Step 3: Switch maintenance on and off

Ensure a real static robots.txt and maintenance.html exist first. The robots exception must return your normal plain-text rules, not the application shell or the maintenance HTML with a 200. Then create an empty file named maintenance.enable in the document root. In File Manager, click + File in the toolbar; over SSH, run touch maintenance.enable.

When the work is done, delete maintenance.enable. The rules can stay in place for next time.

A note on LiteSpeed and cPanel

On a LiteSpeed/cPanel host, ask support to confirm that rewrite, error-document and conditional-header directives are supported in that configuration. Verify the public status and headers before relying on the Apache example.

Set up a 503 maintenance mode on Nginx

Step 1: Back up the server config

Find the file holding your site's server block and copy it into a protected configuration-backup directory outside all served paths. Use an explicit source and private destination confirmed by your administrator. Keep the original path and permissions in your rollback record. Do not put the backup in the website root.

Step 2: Add the maintenance rules

This schematic configuration uses an IP bypass and a trigger file. The geo block belongs in the http context; integrate the server directives into your actual virtual host. Preserve existing application routing, certificates and security headers.

The gate runs at server level, before location selection, so a separate PHP, API or regex location cannot accidentally escape it. The internal maintenance URI and robots file are explicitly exempt to avoid an error-page loop and preserve crawler rules.

# Inside http {}
geo $maintenance_public {
    default          1;
    203.0.113.25     0;  # Replace with your actual admin IP
}

server {
    listen 80;
    server_name example.com;
    root /var/www/example.com;

    set $maintenance_gate 0;
    if (-f /var/www/example.com/maintenance.enable) {
        set $maintenance_gate $maintenance_public;
    }
    if ($uri = /robots.txt) {
        set $maintenance_gate 0;
    }
    if ($uri = /maintenance.html) {
        set $maintenance_gate 0;
    }
    if ($maintenance_gate) {
        return 503;
    }

    error_page 503 /maintenance.html;

    location = /maintenance.html {
        internal;
        add_header Retry-After "3600" always;
        add_header Cache-Control "no-store, no-cache, must-revalidate, max-age=0" always;
        # Include your applicable security-header directives here too.
    }

    location = /robots.txt {
        try_files $uri =404;
    }

    location / {
        # Keep your existing normal application configuration here.
    }
    # Keep your other normal PHP/API/proxy locations.
}

Nginx documents that a local error_page URI is an internal redirect. Keeping the original error status avoids sending visitors to a public maintenance URL. internal prevents direct access to the file, and `always` attaches headers to the 503.

The listen 80 line alone does not cover HTTPS. Apply the policy to the actual TLS virtual host too, retaining its certificate configuration. Existing HTTP-to-HTTPS or hostname canonicalization redirects may remain; their final affected URL should return 503. Do not replace deep URLs with a redirect to a maintenance page.

Header inheritance matters: with Nginx's default rules, defining add_header in this location stops inheritance of parent add_header directives. Include applicable security headers here. Also check that no upstream or CDN adds a conflicting cache policy. Configure trusted real-client-IP handling before using the IP bypass.

Step 3: Validate once, then toggle the file

sudo nginx -t
# Run the reload only if the test succeeds.
sudo nginx -s reload

After the configuration is loaded, create /var/www/example.com/maintenance.enable to enable maintenance and remove that exact file to disable it. Those file changes do not require a reload. The flag must be visible to every serving instance. If you cache file-existence checks with open_file_cache, verify its effect on toggle propagation.

Check that the static robots file actually exists and returns 200 before starting. These Apache/Nginx combinations need staging tests against your own location, proxy and module configuration.

Express: return 503 before application routes

In Express, register your existing static /robots.txt handler first. Place the following middleware before body parsers, public routes, express.static and a frontend catch-all. Otherwise an earlier handler can already have returned 200.

// app is your existing Express application.
// Register the existing static robots.txt route before this middleware.
app.use((req, res, next) => {
  if (process.env.MAINTENANCE_MODE !== 'true') return next();

  res.set({
    'Retry-After': '1800',
    'Cache-Control': 'no-store, no-cache, must-revalidate, max-age=0'
  });

  return res.status(503).json({
    error: 'SERVICE_UNAVAILABLE',
    message: 'Planned maintenance. Please try again in about 30 minutes.'
  });
});
// Register normal application handlers after the gate.

This is an API-style JSON response; a website can send static HTML with the same status and headers. Express middleware order determines which handler runs first. Environment-variable changes normally require a process restart or redeploy; use a runtime feature flag for an instant operational switch.

If Node or PHP itself will stop, application middleware cannot answer. Put the maintenance response at an unaffected reverse proxy, load balancer or CDN edge before stopping the origin.

Cloudflare Worker: keep maintenance and robots independent of the origin

Use a Worker Route covering the existing proxied hostname, such as example.com/*. In this arrangement, fetch(request) forwards normal traffic to the origin when maintenance is off.

Copy the site's existing robots.txt unchanged into public/robots.txt in the Worker project. Configure the assets binding below in the existing Wrangler configuration. Do not replace established private-path or crawler rules with a new allow-all file.

[assets]
directory = "./public"
binding = "ASSETS"
run_worker_first = true

The assets binding serves that file without calling the origin. run_worker_first ensures assets cannot skip the maintenance gate.

export default {
  async fetch(request, env) {
    if (env.MAINTENANCE_MODE !== 'true') return fetch(request);

    const url = new URL(request.url);
    if (url.pathname === '/robots.txt' &&
        (request.method === 'GET' || request.method === 'HEAD')) {
      return env.ASSETS.fetch(request);
    }

    const html = '<!doctype html><html lang="en"><meta charset="utf-8">' +
      '<meta name="viewport" content="width=device-width,initial-scale=1">' +
      '<title>Planned maintenance</title><main><h1>Back shortly</h1>' +
      '<p>Scheduled upgrades are in progress. Please try again in about ' +
      '30 minutes.</p></main></html>';

    return new Response(request.method === 'HEAD' ? null : html, {
      status: 503,
      headers: {
        'Content-Type': 'text/html; charset=UTF-8',
        'Retry-After': '1800',
        'Cache-Control': 'no-store, no-cache, must-revalidate, max-age=0'
      }
    });
  }
};

Set the Worker variable MAINTENANCE_MODE to the string "true" for maintenance; switch it off after origin recovery. Response construction supports the status and headers above. Before stopping the origin, verify the route, asset binding and real robots response on the public hostname. For the strongest rehearsal, confirm both the 503 page and robots remain reachable in staging with its origin unavailable.

Keep the notice self-contained with inline CSS if needed. Include an honest expected return time, support contact and an independent status-page link in your production notice. If the notice needs same-domain images or styles, exempt only those exact resources or inline them; do not bypass all static/application paths.

Check CDN and browser caches as a separate layer

Cloudflare's cache-by-status configuration can override origin cache instructions. Review Cache Rules, status-code TTLs and custom Worker cache logic. Check Age and CF-Cache-Status where present; absence of either is not proof that every cache layer is bypassed.

An existing cached 200 can hide an origin-only maintenance gate. Put the gate before cache lookup or bypass the relevant cache when a full closure is required. If stale public content remains useful, keeping it available can be preferable, provided transactional operations are actually disabled. After recovery, clear any cached maintenance variants that were stored despite the intended policy, including service-worker caches where applicable.

Variants

  • Fixed end time. For the invented window, use Retry-After: Sat, 24 Oct 2026 03:00:00 GMT instead of 3600. Update it if work slips; a fixed delay also needs to remain a realistic estimate.
  • Several admin IPs on Nginx. geo accepts more lines, including CIDR ranges such as 192.0.2.0/24 and IPv6 prefixes such as 2001:db8::/32.
  • Apache server config. ErrorDocument, RewriteRule and Header are also allowed in server config and virtual host context. After editing the main config, run apachectl configtest, which reports Syntax Ok or the error.

How to check it worked

Test from a network outside the admin allowlist, such as mobile data. Use an actual GET while discarding the body, then compare HEAD:

# GET: print headers and discard the response body.
curl -sS --max-time 20 -D - -o /dev/null 'https://example.com/'
# HEAD should agree on status and maintenance headers.
curl -sS --max-time 20 -I 'https://example.com/'
# robots.txt must contain the real rules, not an HTML notice.
curl -sS --max-time 20 -D - 'https://example.com/robots.txt'

For an affected content URL, expect 503, one valid Retry-After value and Cache-Control containing no-store. Inspect the actual GET body separately to confirm the notice renders and API clients receive the intended format.

The following is an acceptance checklist, not a report of tests against your server:

  • Homepage and a real deep content URL, GET and HEAD: expect 503 plus retry/cache headers during the window; normal status and original content after recovery.
  • Affected API route: expect 503 in the intended response format during the window; normal API behavior afterwards.
  • `/robots.txt`, GET and HEAD: expect 200 and the actual plain-text rules throughout.
  • Admin exception, if configured: expect authorized access without shared caching during the window; normal access afterwards.
  • Intentionally exempt maintenance assets: expect 200 and correct rendering throughout.
  • HTTP/HTTPS and alternate public hosts: verify the intended canonical chain ends at the affected 503 during maintenance, then resumes normal content afterwards.

Compare the origin and CDN where your access policy permits it; repeat on multiple serving instances. A 200 on the homepage alone cannot prove that a PHP, API or regex location is gated.

For an API rehearsal, use a staging endpoint that is safe to call. Do not test maintenance by replaying a real payment or order request. A retry estimate does not make POST operations safe to repeat: use the API's documented idempotency and bounded retry behavior.

Recovery: remove the gate and verify the service

  1. Confirm the application, database and dependencies are healthy before opening traffic.
  2. Remove the exact trigger file or disable the application/edge flag on every instance.
  3. Repeat the same URL checks. Formerly affected pages must serve their original content, and maintenance-only Retry-After headers should be gone. Existing redirects, legitimate 404s and normal per-route cache policies should resume.
  4. Check CDN, browser and service-worker behavior. Purge mistakenly cached maintenance responses as needed; verify from a fresh session outside the admin bypass.
  5. Check access logs and monitors for continuing 503s, 5xx spikes or failed transactions. Record the actual start/end times so planned responses can be distinguished from an overrun without hiding a real outage.
  6. Review Search Console Crawl Stats and inspect important restored URLs. Request recrawling for a few key URLs if needed, or use a current sitemap for a broader set. This requests processing; it does not guarantee immediate indexing.

If work overruns, update the retry estimate and status notice, then reassess whether useful parts of the site can reopen. Avoid a blanket 503 lasting days.

Troubleshooting

Problem 1: the test shows 200 with the maintenance text. Symptom: the first line reads HTTP/1.1 200 OK but the body says "down for maintenance". Cause: a plugin is drawing the page, the rule is not matching, or Nginx has error_page 503 =200. Fix: turn off the plugin's maintenance mode, remove any =200, and keep the block at the top of .htaccess.

Problem 2: the test shows a 302 redirect. Symptom: a 302 with a Location header pointing at your maintenance page. Cause: ErrorDocument or error_page uses a full URL, which both Apache and Nginx document as producing a redirect. Fix: use a local path starting with /.

Problem 3: the 503 is there but the retry header is missing. Symptom: correct status, no Retry-After. Cause: Header set without always on Apache, or add_header without always on Nginx. Fix: add always and test again.

Problem 4: every page shows 500 Internal Server Error. Symptom: the whole site breaks right after editing. Cause: a typo, a module that is not enabled, or a status code httpd does not recognise, which the mod_rewrite docs say produces a 500. Fix: restore .htaccess from the recorded private backup path, then re-add the block carefully and check your error log. Use the generated path saved in Step 1, not a guessed backup filename.

Problem 5: you are locked out too. Symptom: your own office also gets the 503. Cause: your public IP changed, or a proxy hides your real IP. Fix: check your current IP, update the rule, and ask your host about mod_remoteip if a proxy is involved.

Problem 6: pages drop out of Search Console days later. Symptom: URLs stop appearing in Google Search. Possible cause: a prolonged outage or crawler access problem; confirm with Crawl Stats, URL Inspection and server logs rather than assuming a single cause. Fix: restore the intended responses and crawler access, then request recrawling of important URLs where needed. Recovery timing is not guaranteed.

Pro tips and security notes

  • Pick your quietest hour. Check your traffic stats and keep the window short.
  • Keep the maintenance page static. If the database is what you are upgrading, a PHP page will fail too.
  • Never block all crawling for a closure. Google warns that a robots.txt disallowing everything may remove your content and URLs from Search.
  • Keep the allowlist tight. Use single IPs and remove them afterwards, since anyone listed sees your half-finished site.
  • Do not leak details. No software versions, file paths or error messages on the maintenance page.

Choosing a host? Our small business hosting checklist covers what to ask about server access.

The one message worth forwarding

Hi, planned maintenance for example.com is [start] to [end] UTC.
Please return 503 on affected URLs with a realistic Retry-After value
and Cache-Control: no-store. Serve a static notice on the requested URL.
Keep robots.txt available with its normal rules, and make any admin
bypass private and uncacheable. If the origin will stop, serve these
responses from an unaffected proxy or edge. Test GET/HEAD, deep pages,
APIs, both hostnames and HTTPS, then repeat the checks after recovery.
Please confirm who owns the toggle, rollback and overrun update.

Does this also protect visibility in AI answers?

Correct HTTP behavior supports reliable crawler access, but 503 and Retry-After cannot guarantee rankings or AI citations. Google's AI Search guidance says normal SEO fundamentals apply, pages must be indexed and snippet-eligible, and there is no special AI schema or machine-readable file required for AI Overviews or AI Mode. Other assistants use their own retrieval systems.

For an educational article like this one, restore a normal 200 after maintenance and keep the useful instructions accessible, internally linked and supported by sources. The practical examples and verification checklist are editorial guidance, not evidence of measured ranking improvements.

Reader questions

What does HTTP 503 mean during planned maintenance?

The requested service is temporarily unavailable. Return the maintenance notice as the body of a 503 on the affected URL, with a realistic Retry-After estimate.

Is Retry-After required, and what formats are valid?

RFC 9110 makes it optional for 503, but it is useful for planned maintenance. Use non-negative integer seconds or an HTTP date in GMT. Neither format guarantees when a client or crawler will retry.

Will browsers automatically reload after Retry-After?

Do not rely on that. The header supplies a retry estimate; browser navigation, API retry and crawler behavior depend on the client. It does not turn maintenance off.

Is 48 hours of 503 maintenance guaranteed to be SEO-safe?

No. Google describes 503 as an option for very short closures and warns about outages lasting days. There is no guaranteed 48-hour grace period or promise of unchanged indexing or rankings.

Which Cache-Control header should the maintenance page use?

Use no-store to prevent compliant caches from storing the response. The examples also use no-cache, must-revalidate and max-age=0. Check CDN overrides and existing cached content separately.

Should robots.txt return 503 too?

Keep it reachable with its existing plain-text rules, normally on 200. Google initially pauses crawling after robots.txt server errors and later uses cached rules or other fallback behavior. Do not replace the rules with blanket Disallow: /.

Why is Retry-After missing from my error response?

Use Header always set in Apache or add_header with always in Nginx. Then inspect the final public response because a proxy, CDN or another handler can change the headers.

Can Express maintenance work while the origin is powered down?

No. An application cannot answer while its runtime is stopped. Put the response at an unaffected proxy, load balancer or CDN edge, and keep robots.txt available independently too.

What should I check after maintenance ends?

Remove the gate everywhere, verify original content and normal API behavior, confirm maintenance-only retry headers are gone, check cache variants and logs, and monitor crawler recovery in Search Console.

Does this guarantee Google AI Overviews or assistant citations?

No. Google says ordinary SEO fundamentals apply to its AI Search features and does not guarantee indexing or inclusion. Other assistants have their own retrieval behavior; correct maintenance signaling alone does not earn citations.

Sources & further reading

  1. RFC 9110: HTTP Semantics — Retry-After and 503
  2. RFC 9111: HTTP Caching — no-store, no-cache and revalidation
  3. Google: How HTTP status codes affect crawlers
  4. Google: Temporarily pause or disable a website
  5. Google: Troubleshoot crawling errors and emergency overcrawling
  6. Google: robots.txt server-error and caching behavior
  7. Apache 2.4: RewriteRule flags
  8. Apache 2.4: mod_headers
  9. Apache 2.4: ErrorDocument
  10. Apache 2.4: trusted client IP handling with mod_remoteip
  11. Nginx: rewrite, if and return
  12. Nginx: error_page and internal locations
  13. Nginx: response headers and inheritance
  14. Express: middleware order
  15. Cloudflare Workers: Response API
  16. Cloudflare Workers: routes in front of an origin
  17. Cloudflare Workers: static assets binding and run_worker_first
  18. Cloudflare: cache TTL by status code
  19. Google: AI features and your website

What changed

Added complete HTTP headers, Retry-After timing and no-store/no-cache distinctions; updated Apache/Nginx caching and file toggles; added Express and origin-independent Cloudflare examples, a GET/HEAD acceptance matrix, CDN cache checks and recovery steps. Clarified search-risk limits and AI visibility claims. Server/edge examples require staging validation; no production maintenance settings were changed.

Originally published . About our editorial updates.

Your next project deserves a better foundation.

Explore hosting built for your next chapter.

Explore hosting

Find your next answer.

Search published guides and articles.

Ctrl / ⌘ K to open · Esc to close