Surface
Dropping an LLM node into n8n feels like progress. Sometimes it is: classify a ticket, summarize a long email, extract three fields into JSON. Sometimes it is chatbot theater with API keys: an open-ended prompt, tool access bolted on, and no evaluation bar. The model becomes a charismatic intern with production credentials.
Use AI inside n8n as narrow nodes with structured outputs. Do not treat the canvas as a place to reinvent unbounded agents. For product-level agent design, start with agentic AI: workflow, not chatbot theater and RAG vs fine-tuning vs agents. This essay stays on what belongs in an automation runtime.
What LLM nodes are good at in workflows
- Classify into a small, closed label set (priority, intent, team)
- Summarize a document into a fixed schema for a human queue
- Extract fields when formats vary but the target schema is stable
- Draft a suggestion that a human must approve before send
These tasks have evaluable outputs. You can keep a sample set and notice regressions when prompts or models change. That is the same spirit as an AI MVP eval set, scaled down to a workflow step.
Propose, validate, then commit
Input
Ticket / email / form payload
LLM node
One job · JSON schema · eval pocket
Validate
Parse fail → dead letter
Commit
Deterministic write or human approve
Insight: the model proposes; the workflow commits. Unbounded “figure it out” loops do not belong on production credentials.
Metrics for an AI step (before you enable write)
| Gate | Starter bar | What happens if you skip it |
|---|---|---|
| Eval pocket size | 20–50 real examples with acceptable outputs | Prompt drift ships unnoticed |
| Structured parse rate | ≥ 95% valid JSON / schema on eval | Junk fields hit CRM |
| Label accuracy (classify) | Agree a floor with the domain owner (often ≥ 85–90% on eval) | Misroutes look "automated" |
| Human checkpoint rate | 100% on customer-send / money paths | Confident wrong messages |
| Cost ceiling | $/successful outcome written down | Token bill surprises at volume |
| p95 latency | Fits the workflow SLA (often < 3–8s for interactive queues) | Queues back up; people bypass the bot |
The Stanford AI Index has documented how quickly model quality gaps compress year over year. That is why the durable investment is the eval pocket and schema, not a forever bet on one vendor's model card.
What they are bad at without product engineering
- Open-ended "figure out what to do next" loops across many tools
- Authority over money, access, or customer commitments without a checkpoint
- Answers that must be grounded in changing private data with no retrieval design
- Anything where a confident wrong answer is worse than a slow human
If you need multi-step tool use with permissions and audit logs, design that as a product workflow. n8n can call it. n8n should not improvise it.
Design rules for AI steps in n8n
Narrow the task
One node, one job, one output schema. "Classify support intent into five labels" beats "handle the customer."
Force structure
Ask for JSON (or another strict schema) and validate before downstream writes. If parsing fails, dead-letter; do not invent fields.
Separate read from write
Let the model propose. Let deterministic nodes or humans commit. Especially for email, CRM updates, and financial systems.
Keep an eval pocket
Twenty to fifty real examples with acceptable outputs is enough to catch prompt drift. Run them when you change model or temperature. Skipping this is how "it worked last month" becomes an incident.
Bound cost and latency
Workflow volume multiplies token cost. Cap max tokens, cache where safe, and alert on spend per workflow. A classify step that is fine at 20 runs/day can surprise you at 20 runs/minute.
Permissions checklist
| Capability | Default stance |
|---|---|
| Read ticket / email body | Usually fine for classify/summarize |
| Write CRM fields | Only after validation or human approval |
| Send customer email | Human checkpoint unless low-risk templates |
| Call payment or identity APIs | Product-owned service, not freeform LLM tools |
| Browse the public web unbound | Avoid in production glue workflows |
How this fits the series
Start from when to automate, pick a runtime, harden failure modes, then add AI nodes only where a narrow task clears an eval bar. Ownership of prompts and keys still needs a handoff plan in who owns the n8n instance.
If you are scoping AI features beyond automation glue, AI and ML solutions is the product conversation; keep n8n for the edges that stay edges.
