Self-hosting n8n at scale

Self-hosted n8n is the best value in automation right now, and it is also a service you now operate. That second part gets discovered around week six, usually at the moment the editor takes forty seconds to load and nobody knows why.

This is the runbook I wish I had. Everything here comes from instances running real AI workloads, not from a docker compose tutorial.

Queue mode, earlier than you think

The default single-process setup runs the editor, webhooks, and executions in one place. That is fine for a handful of quick workflows and wrong for AI work, because a single 90-second model call blocks the loop that also accepts your webhooks.

Queue mode splits this into a main process, Redis as the broker, and N worker processes. Webhooks stay responsive, long runs go to workers, and you scale by adding workers rather than by buying a bigger box.

• Main process: editor, API, webhook ingestion. Small.
• Workers: execute workflows. Size for concurrency, since AI runs are mostly idle waiting on providers. Concurrency per worker in the range of five to ten is a sane starting point.
• A dedicated webhook process once ingestion volume is high enough that you want it isolated from the editor.

Postgres is the whole ballgame

Every performance complaint I have investigated on self-hosted n8n came back to the database. n8n saves execution data, and execution data for AI workflows is large, because it includes every payload at every node.

Left alone, the executions table grows without limit. At tens of gigabytes the editor's execution list crawls, backups take hours, and disk alarms start firing.

1. Turn on execution pruning with a retention window you can actually justify. Fourteen days of full data covers nearly all debugging needs.
2. Do not save success data for high-volume workflows. Keep error executions in full. Successes you can sample.
3. Vacuum and reindex after the first big prune. Deleting rows does not return disk on its own, which surprises people once.
4. Use managed Postgres. Automated backups, point-in-time recovery, and a maintenance window you did not have to build. This is not the place to save thirty dollars a month.
5. Monitor connections. Every worker holds a pool. Workers times pool size must stay under your connection limit, or you get failures that look like random workflow errors.

Sizing, concretely

A shape that has handled a lot of real load for clients:

• Main: 2 vCPU, 2 GB RAM.
• Two workers: 2 vCPU, 4 GB RAM each.
• Managed Postgres with room to grow and automated backups.
• Managed Redis, small.
• Object storage for anything binary. Never let files live inside execution data if you can pass a reference instead.

Memory is the constraint that bites first. A workflow that loads a large JSON array and maps over it can spike a worker's heap, and Node will die rather than degrade. Batch your loops, stream where you can, and set explicit heap limits so a single bad run cannot take a worker down with it.

Reliability work that pays for itself

• Retries with backoff on every external call. Provider rate limits and timeouts are normal operating conditions, not incidents.
• An error workflow. One shared handler that captures failures, posts context to the channel your team reads, and writes to a dead-letter table you can replay from.
• Idempotency keys on writes. Retries will happen. Make them safe.
• Timeouts everywhere. A model call with no timeout can hold a worker slot for a very long time.
• Cost caps. A budget check node before expensive branches, and a daily spend alert. AI failures are expensive when they loop.

Nothing had changed. The executions table had simply crossed the point where every insert triggered an autovacuum fight.

Upgrades, credentials, and version control

Three operational habits separate the instances I enjoy maintaining from the ones I dread.

1. Pin your image tag. Never track latest. Upgrade deliberately, on a staging instance first, with a database backup taken minutes before.
2. Export workflows to Git. Either through native source control or a scheduled job that dumps JSON to a repository. The n8n database is not a version control system, and one accidental overwrite will teach you that.
3. Set the encryption key explicitly and back it up separately. Losing it means every stored credential becomes unreadable. This is the single most recoverable disaster people fail to prepare for.

Security basics

• Editor behind SSO or at minimum behind a reverse proxy with auth.
• Public webhook endpoints validated with a signature or a shared secret, never trusted by shape alone.
• Separate credentials per workflow scope. A workflow that reads invoices does not need write access to your CRM.
• Outbound egress rules if you handle regulated data, so a compromised node cannot exfiltrate quietly.

Where agents are involved, the same capability-first thinking from prompt injection is not a prompt problem applies directly to n8n credentials.

Is it worth it

Below roughly ten thousand executions a month, n8n Cloud is probably the better trade once you price your own attention. Above that, or the moment data residency enters the conversation, self-hosting wins decisively. The broader platform comparison is in n8n vs Make vs Zapier, and the workflow-level failure modes are in n8n AI agents in production.

If you want this set up correctly the first time, or an existing instance stabilized, that is n8n automation engineering. For a fast read on what is fragile right now, take the production readiness audit.

Frequently asked