The Secret Life of AWS - The Orchestrator (AWS Step Functions)

Why complex business logic requires a conductor, not a megaphone.

#AWS #StepFunctions #Microservices #Architecture

---

🎧 Audio Edition: Prefer to listen? Check out the expanded AI podcast version of this deep dive on YouTube.

📺 Video Edition: Prefer to watch? Check out the 7-minute visual explainer on YouTube.

---

Part 49 of The Secret Life of AWS

Timothy was mapping out the new Order Fulfillment workflow on the whiteboard. He was leaning heavily into the event-driven patterns he had just learned.

"When an order is placed," Timothy explained to Margaret, "EventBridge routes it to the Payment service to charge the credit card. The Payment service publishes a 'Card Charged' event. EventBridge catches that and routes it to the Warehouse service to reserve the inventory. Then, the Warehouse service publishes an 'Inventory Reserved' event, which triggers the Shipping service to print a label."

He stepped back, admiring the highly decoupled, asynchronous chain.

Margaret looked at the board. "What happens if the label printer is out of ink and the Shipping API times out?"

"The Shipping service fails," Timothy replied.

"Yes," Margaret said. "But the credit card has already been charged, and the inventory has already been reserved. How do you undo those actions?"

Timothy frowned. "I guess the Shipping service would have to publish a 'Shipping Failed' event. Then I would have to update the Payment service to listen for that event and issue a refund. And update the Warehouse service to listen for it and release the inventory."

The Limits of Choreography

"You are building a distributed mess," Margaret said. "In a purely event-driven architecture—called Choreography—every microservice acts independently. That is great for simple reactions, but terrible for complex business workflows. You are forcing every microservice to memorize the error-handling logic for the entire company."

"When a process requires strict order, state tracking, and complex rollbacks," Margaret explained, "we do not use Choreography. We use Orchestration. We need a conductor."

The Orchestrator

Margaret opened the AWS Console and navigated to AWS Step Functions.

"Step Functions is a visual workflow orchestrator," she said. "Instead of microservices blindly reacting to each other's events, we build a State Machine that explicitly coordinates the entire process from start to finish."

She opened the Workflow Studio and began dragging and dropping logical steps onto a visual canvas.

"The Step Function acts as the central brain," Margaret explained. "It tells the Payment Lambda function to run and waits for the exact response. If it succeeds, the Step Function moves to the next state and tells the Warehouse API to reserve the item. It passes the state payload from one step directly to the next."

"So the microservices no longer talk to each other?" Timothy asked.

"Exactly," Margaret said. "The Payment service does not know the Warehouse service exists. It only knows how to charge a card and return a result to the Orchestrator."

"It also gives us complete observability," she added. "Every execution leaves a visual audit trail. If an order gets stuck, you can open the console, see exactly which step it failed on, and inspect the JSON data payload at that exact moment in time."

Workflow Resilience

Timothy looked at the visual workflow. "But what about the label printer timing out?"

"Step Functions handles transient failures natively," Margaret said. She clicked on the 'Print Shipping Label' state and configured a Retry Policy.

"If the Shipping API throws a timeout error, we don't need to write custom loops in our application code. The Step Function will automatically wait three seconds and try again, up to three times, using exponential backoff. Just like our SQS queues provided buffer resilience, Step Functions provides workflow resilience. The network handles it."

Human Approval

"It can even wait for people," she noted. "Using a Callback Pattern, the state machine can pause the execution and wait for a human manager to click 'Approve' in an email before moving to the next step. Because we are using Standard Workflows, the execution can stay paused and retain its state for up to a full year. If we were building high-volume, short-duration data processing, we would use Express Workflows, but for order fulfillment, Standard is perfect. Standard workflows cost slightly more per state transition, but for business-critical operations, that reliability is worth it."

The Saga Pattern: Automated Rollbacks

"But what if the printer is completely broken, and all the retries fail?" Timothy pressed. "We still have a charged card and reserved inventory."

"This is where Orchestration shines," Margaret smiled. "We implement the Saga Pattern."

She dragged a 'Catch' failure path off the Shipping state. She routed it to two new states: 'Release Inventory' and 'Refund Credit Card'.

"The Saga Pattern is a failure management design for distributed transactions," she explained. "Think of the Saga Pattern as a transaction with an undo button for each step. If a downstream step fails, the Orchestrator automatically triggers Compensation Logic—specific steps designed to undo the previous successful work."

"If Shipping fails," Margaret pointed to the canvas, "the Step Function immediately catches the error. It executes the Warehouse release API, and then it executes the Payment refund API. The workflow ends in a 'Failed' state, but the business data is clean. And the best part? The microservices didn't have to orchestrate any of it. The State Machine managed the entire rollback."

Timothy erased the tangled web of EventBridge rules and compensation events from his whiteboard. He drew a single, authoritative Orchestrator directing the three microservices.

"So, what is the rule of thumb?" Timothy asked.

"Use EventBridge when you have simple reactions and don't need coordinated rollbacks," Margaret summarized. "Use Step Functions when the workflow has multiple steps, requires strict state tracking, or needs distributed transaction safety."

The architecture had memory. The business process was safe.

---

Key Concepts

Choreography vs. Orchestration: In Choreography (like EventBridge), independent services react to events without central control, which is highly decoupled but difficult to track for complex workflows. Orchestration uses a central controller to dictate the execution order, track state, and handle errors across multiple services.

Use EventBridge for simple, independent reactions, and Step Functions for complex, multi-step workflows requiring coordinated rollbacks.

AWS Step Functions is a serverless orchestration service that lets you integrate AWS services into visually defined workflows known as State Machines. These workflows manage the execution sequence, track the state of each step, and provide a visual execution history for complete observability.

Workflows can be configured as Standard Workflows for durable, long-running processes (up to a year), or Express Workflows for high-volume, short-duration executions.

The service provides built-in Retry Policies to automatically handle transient network or service failures using exponential backoff.

It also supports Human-in-the-Loop patterns, allowing a workflow to pause and wait for external manual approval.

For catastrophic failures in distributed systems, Step Functions enables the Saga Pattern. This pattern manages distributed transactions by executing Compensation Logic—a series of designated rollback steps—to undo previously successful operations, ensuring data consistency across your microservices when a workflow ultimately fails.

---

Aaron Rose is a software engineer and technology writer at tech-reader.blog. For explainer videos and podcasts, check out Tech-Reader YouTube channel.