AWS Lambda Error – Out of Memory (OOM)

Problem

Your Lambda function stops abruptly during execution. CloudWatch logs appear truncated, no explicit error is thrown, and the function behaves inconsistently under load. In the Lambda metrics panel, “Max Memory Used” spikes sharply — often hitting the exact configured limit. This means Lambda exhausted its available memory and forcibly terminated the runtime.

Clarifying the Issue

When your function exceeds its memory allocation, Lambda does not raise a catchable exception. Instead, it:

Immediately kills the container
Stops generating logs mid-stream
Returns incomplete or missing output
Appears “mysterious” unless you know the signs

Common causes:

Large JSON payloads inflating in memory
Reading entire S3 objects into RAM
Big CSV/image/video processing
Heavy dependency footprints
Buffer growth and leaks
Repeated parsing/stringifying cycles

Lambda OOM failures feel ghostly because the runtime terminates without ceremony — but the root causes are mechanical.

Why It Matters

Out-of-memory errors are not minor glitches; they undermine the entire service boundary:

API requests fail
Batch jobs collapse mid-run
Retry storms trigger downstream stress
Data integrity becomes unpredictable
Customer-facing SLAs erode

Eliminating OOM conditions is fundamental to operational maturity.

Key Terms

OOM Kill — Lambda force-terminates execution after memory exhaustion.
Cold Start Footprint — Memory used before your handler runs.
JSON Inflation — JSON grows 2–4× when parsed into objects.
Buffer Overgrowth — Code loads full files or payloads into memory.
/tmp — Lambda’s ephemeral disk (512 MB → 10 GB) for offloading RAM pressure.

Steps at a Glance

Review CloudWatch’s “Max Memory Used” metric.
Add memory checkpoints to identify growth points.
Increase memory allocation to test CPU/memory scaling.
Remove full-buffer patterns and switch to streaming.
Use /tmp for large intermediates.
Reduce dependency bloat.
Add payload-size guards.
Load-test your updated function to confirm stability.

Detailed Steps

Step 1: Check CloudWatch Metrics

Go to: Lambda → Metrics → “Max Memory Used”

Interpret the numbers:

Above 80%: At risk
At 100%: Function already crashed
Sudden spikes: JSON.parse or buffer creation
Gradual incline: Memory leak

This chart tells you more than most logs.

Step 2: Add Memory Checkpoints in Code

These lightweight logs help you pinpoint where memory rises.

⚠️ Caution:
Stringifying extremely large events can consume extra memory. Use carefully in tight scenarios.

Node.js example:

// Warning: may create a large string if the event is huge
console.log("payload bytes:", Buffer.byteLength(JSON.stringify(event)));

const parsed = JSON.parse(bigJson);
console.log("after JSON.parse");

console.log("heapUsed:", process.memoryUsage().heapUsed);

Python example:

import json, resource

# Note: On AWS Lambda, ru_maxrss is reported in kilobytes.
print("memory usage (KB):", resource.getrusage(resource.RUSAGE_SELF).ru_maxrss)

Even three logs can reveal the exact hot spot.

Step 3: Increase Memory Allocation (Diagnostic Only)

Memory size controls both RAM and CPU allocation. Doubling memory often doubles throughput.

aws lambda update-function-configuration \
  --function-name MyFunction \
  --memory-size 1024

If the problem disappears after increasing memory, you’ve confirmed the core issue: workload exceeded resource allocation.

Step 4: Remove “Load Everything Into RAM” Anti-Patterns

These patterns cause sudden memory spikes:

s3.getObject(...).Body (full file in memory)
fs.readFileSync() on large files
json.loads() on massive strings
Base64 decoding entire binaries

Use streaming instead.

Node.js streaming:

const stream = s3.getObject({ Bucket, Key }).createReadStream();
stream.on("data", chunk => {
  // process chunk
});

Python streaming:

obj = s3.get_object(Bucket=b, Key=k)
for chunk in obj["Body"].iter_chunks():
    process(chunk)

Streaming is the single easiest way to prevent OOM.

Step 5: Use `/tmp` Instead of RAM

Move large intermediates from memory to disk.

Node.js:

const fs = require("fs");
fs.writeFileSync("/tmp/output.bin", buffer);

Python:

with open("/tmp/output.bin", "wb") as f:
    f.write(content)

Disk is cheap inside Lambda; RAM is precious.

Step 6: Reduce Dependency Bloat

Big libraries eat memory on import — before your handler runs.

Examples:

pandas
NumPy
Pillow
Chromium/Puppeteer
Heavy ML frameworks

Your cold start footprint might be 250 MB without you realizing it.

Best practices:

Remove unused imports
Replace heavy libs with lighter equivalents
Lazy-load modules inside functions
Use slim Lambda Layers

Step 7: Guard Against Oversized Payloads

JSON expands when parsed. Protect yourself early.

Node.js defensive check:

if (Buffer.byteLength(event.body || "") > 2_000_000) {
  throw new Error("Payload too large");
}

Failing fast prevents downstream OOM termination.

Step 8: Load-Test the Function

To catch memory leaks or spikes, run multiple invocations:

for i in {1..20}; do
  aws lambda invoke \
    --function-name MyFunction \
    --payload '{}' \
    out.json
done

Watch “Max Memory Used” over time to confirm:

no rising trend (leak)
no intermittent spike (large object)

Conclusion

Out-of-memory errors feel unpredictable, but once you understand how memory is consumed — and how quickly JSON, buffers, and dependencies expand inside Lambda’s runtime — the problem becomes systematic. By right-sizing memory, shifting heavy workloads to /tmp, embracing streaming, and eliminating full-buffer patterns, you transform Lambda from a brittle service into a resilient, production-ready component.

This is disciplined operational engineering: clean diagnostics, careful resource management, and intentional architecture choices.

Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.

Search This Blog

Tech-Reader.blog