This document describes the pre-rendering functionality/endpoint, its associated HTTP header options, and possible error responses.

See the following links for related information:

Send your secret API token (you'll get it after creating an account) as a header with all your requests to avoid rate limiting.

• curl --header "X-Prerender-Token: secret-token"

You can think of this endpoint almost like a proxy, and in some ways a scraper. It returns the serialized HTML of the requested URL (the "origin") - useful for converting JavaScript apps to their JavaScript + HTML representation instead of just JavaScript.

All of our integration libraries (Node, Nginx, Apache etc.) proxy your requests through this endpoint. It can also be used as a conventional API - but if used as a conventional API, be mindful that the default options are designed to favor it as a proxy endpoint (caches, doesn't follow redirects, includes some server-side rendering compatibility add-ons)

curl https://service.headless-render-api.com/https://example.com/ > out.html

// 1. build/compile your JavaScript single-page app
// 2. cd into your build/dist directory

npx prerendercloud-server

// npm install prerendercloud
const app = require('express')();
const prerendercloud = require('prerendercloud');

app.use(prerendercloud);

See more low-level examples, or see high-level pre-rendering docs.

• Disables/ignores the default 300 second (5 minutes) cache on the headless-render-api.com servers

• curl --header "Prerender-Disable-Cache: true"

• Change the default cache duration from 300s (5 minutes) to anything up to 2592000s (1 month)
  (ignored if cache is disabled)

• curl --header "Prerender-Cache-Duration: 2592000"

• Updates/refreshes the cache even if it already exists. It will default to 300s, so use with Prerender-Cache-Duration
  (ignored if cache is disabled)

• curl --header "Prerender-Recache: true"

• Removes a JavaScript monkeypatch from the prerendered page that preloads your intial AJAX requests

  Read more and see  Ajax-Preload source code

• curl --header "Prerender-Disable-Ajax-Preload: true"

• Removes a JavaScript monkeypatch from the prerendered page that adds X-Prerendered to the header of all XHR and fetch requests signaling to the middleware to go straight to origin

  Read more and see  Ajax-Bypass source code

• curl --header "Prerender-Disable-Ajax-Bypass: true"

• Removes a JavaScript monkeypatch from the prerendered page that is intended to prevent duplicate meta/title/script/style tags

  Some libs/frameworks detect existing meta/title/style and don't need this, but in our experience this is still a worthwhile default.

• curl --header "Prerender-Disable-Head-Dedupe: true"

• Waits longer than normal for a page to finish rendering. Similar to Puppeteer's { waitUntil: 'networkidle' }. Useful for pages that depend on AJAX/XHR that fire late or IPFS hosted pages

• curl --header "Prerender-Wait-Extra-Long: true"

• By default, headless-render-api.com will wait for all ws activity to finish, but it doesn't make sense to "wait" for them to finish if they never stop. An example: real time prices on a stock price website.

• curl --header "prerender-dont-wait-for-web-sockets: true"

• By default, headless-render-api.com sends cookies back to the server. Use this to block them.

• curl --header "prerender-block-cookies: true"

• A CSV of headers to forward. (must start with prerendercloud-) By default, the GET request headless-render-api.com makes to your origin server is clean/pure (without any remnants of the original user request). This header allows you to forward certain headers to your origin.

  Usage of this header disables the headless-browser-api server-side cache

  see more examples here

• curl --header "origin-header-whitelist: prerendercloud-whatever,prerendercloud-b" \
    --header "prerendercloud-whatever: some-value \
    --header "prerendercloud-b: some-other-value"

• This option tells headless-render-api.com to only pre-render the <title> and <meta> tags in the <meta> section. The returned HTML payload will otherwise be unmodified.

• curl --header "Prerender-Meta-Only: true"

• Removes all script tags except for application/ld+json. If you're pre-rendering, this means no client-side JavaScript and thus your app will no longer be isomorphic. Useful for reducing byte payload for the crawling/scraping use case.

• curl --header "prerender-remove-script-tags: true"

• Removes trailing slash from URLs submitted to the API. e.g. example.com/docs/ -> example.com/docs. Originally intended for the pre-rendering use case, the primary purpose is to achieve a higher server cache hit rate. The SEO best practice is to use to 301s to redirect trailing slashes to non trailing slashes before service.headless-render-api.com is hit and/or using link rel canonical tag.

• curl --header "prerender-remove-trailing-slash: true"

• By default, if your origin server returns 301/302, headless-render-api.com will just return that outright - which is appropriate for the common use case of proxying traffic through headless-render-api.com. If using the API in a crawling/scraping/batching you may want to follow redirects.

• curl --header "Prerender-Follow-Redirects: true"

• Overrides device screen width (default: 1366)

• curl --header "prerender-device-width: 800"

• Overrides device screen height (default: 768)

• curl --header "prerender-device-height: 600"

• Whether to emulate mobile device (default: false).

  This includes viewport meta tag, overlay scrollbars, text autosizing and more. In other words, whether the meta viewport tag is taken into account.

  an example of "viewport meta tag":

• curl --header "prerender-device-is-mobile: true"

• Changes the response from text/html to application/json and returns an object 2 fields: {body, screenshot}

  The values are base64 encoded:

  • body is the normal pre-rendered response
  • screenshot is a base64 encoded PNG

  This is useful for savings screenshots or injecting them as open graph images. Often used with Prerender-With-Metadata

• curl --header "Prerender-With-Screenshot: true"

• Changes the response from text/html to application/json and returns an object 3 fields: {body, meta, links}

  The values are base64 encoded:

  • body is the normal pre-rendered response
  • meta is an object that includes  { title, metaDescription, h1, ogImage, ogTitle, twitterCard }
  • links is an array of URLs/paths on the page

  This is useful for capturing SEO-relevant metadata during the pre-render.

• curl --header "Prerender-With-Metadata: true"

• The API endpoint returns whatever status the origin returns, which for single-page apps (index.html) is usually 200, unless you change it with these tags. This is irrelevant if you're just using service.headless-render-api.com as a batch crawling/scraping tool.
• 404 status code informs a bot that the page should be removed from their index
• • <meta name="prerender-status-code" content="404">
• 301 status code informs a bot that their is newer content elsewhere
• • <meta name="prerender-status-code" content="301">
  • <meta name="prerender-header" content="Location: http://www.example.com"> (note: no other headers, currently, can be modified like this)

• Invalid Request
• There will be an error message in the HTML, fix your request and retry

• Example causes:

  • malformed URL
  • a localhost URL/IP
  • or a page responds with content-type: application/octet-stream

• Rate limited

• Requests made without API tokens (or expired/missing billing information) will get this - see

  pricing

• General Error

• Example causes

  :

  • 10s (timeout) while waiting for a page to finish rendering (waits until all in-flight requests finish, load event, domContent event etc.)
  • or HTTPS Page is making HTTP (non secure) requests
  • ...or a random bug on our end
• The error will show up in the headless-render-api.com web UI (after you sign in)

• Bad Gateway (your origin returned 5xx)
• It means we received a 5xx when trying to visit your page
• Or it means we received a 403 (forbidden). Typically this means your page is behind a login wall or a firewall (like Cloudflare) is blocking the headless-render-api user-agent (make an exception to allow user-agents matching /prerendercloud/)
• Probably not retryable. It depends on the site you're requesting. If it's your site, make sure it's up and running correctly

• Over capacity (Rare)
• Retry the request with some backoff
• We'll see the error and our autoscaler will increase capacity within 5 minutes, but you should email us anyway: support@headless-render-api.com

• Gateway Timeout (Rare)
• Retry the request with some backoff
• This is unexpected and should not happen. We'll see it, but you should email us anyway: support@headless-render-api.com