The Secret Life of AWS: The Saga Pattern (AWS Step Functions)

How to manage distributed transactions and automated rollbacks across decoupled microservices

#AWS #StepFunctions #SagaPattern #Serverless

---

Margaret is a senior software engineer. Timothy is her junior colleague. They work in a grand Victorian library in London — the kind of place where code quality is the unspoken objective, and craftsmanship is the only thing that matters.

Episode 62

Timothy was staring at his customer support queue with a look of sheer defeat. He turned his monitor toward Margaret as she walked into the studio.

"The decoupled architecture is incredibly fast," Timothy said, "but we have a massive logic flaw. A customer ordered our flagship mechanical keyboard. The EventBridge router worked perfectly. The Payment Service charged their credit card. But milliseconds later, the Inventory Service processed the event and realized the keyboard was out of stock."

"So the order failed," Margaret observed.

"Yes, but the Payment Service had already succeeded," Timothy groaned. "The customer paid for an item we cannot ship. In our old, tightly coupled monolith, I would have just wrapped the entire checkout process in a single SQL database transaction. If the inventory check failed, the database would instantly rollback the credit card charge. How do I rollback a transaction when the services are completely independent and do not talk to each other?"

"You have discovered the hardest problem in distributed systems," Margaret said, pulling up a chair. "You cannot use traditional ACID database transactions across decoupled microservices. Instead, we must use a distributed transaction model called the Saga Pattern, orchestrated by AWS Step Functions."

The Saga Pattern

Margaret opened the AWS Console and navigated to AWS Step Functions.

"A Saga is a sequence of local transactions," she explained. "Each microservice does its job and then triggers the next step. But if any step fails, the Saga executes a series of Compensating Actions—essentially running the workflow in reverse to undo the work that already succeeded."

"So if Inventory fails, the compensating action is a command to refund the payment," Timothy reasoned. "But why can't EventBridge just handle that?"

"Because EventBridge is entirely stateless," Margaret clarified. "It routes events, but it does not remember where the workflow currently is or what steps have already succeeded. While EventBridge is great for choreography—where services react to events independently—a Saga requires strict orchestration. We need a central conductor to manage the state, handle the errors, and explicitly command the rollbacks. That conductor is a Step Functions State Machine."

Building the State Machine

Margaret opened the visual editor and began dragging and dropping states onto the canvas.

"We define the exact workflow using the Amazon States Language (ASL)," Margaret explained. "For long-running business processes, we use Standard Workflows, but for a high-volume, short-duration task like our checkout, we use Express Workflows to save on costs. First, the state machine commands the Payment Lambda to charge the card. If it succeeds, it commands the Inventory Lambda to reserve the item. If it succeeds, it commands the Shipping Lambda."

"And the error handling?" Timothy asked.

Margaret clicked on the Inventory state and added a Catch block. "We tell Step Functions: if the Inventory Lambda throws an OutOfStockException, immediately halt the forward progress. Route the workflow to a new state called RefundPayment, which triggers a specific Lambda function to reverse the credit card charge."

"And remember our lesson from yesterday," Margaret cautioned. "Each compensating action must also be idempotent. If the refund fails and Step Functions uses its built-in retries, we cannot accidentally refund the customer twice."

The Visual Audit Trail

Margaret saved the state machine and executed a test order for an out-of-stock item.

Timothy watched the visual execution graph in the console. The ChargeCard state lit up green. Then, the workflow moved to the ReserveInventory state, which turned red. Instantly, the workflow routed down the error path, and the RefundPayment state lit up green.

"The execution failed, but the system is in a perfectly consistent state," Timothy marveled. "The customer was automatically refunded, and I can actually see the exact path the execution took."

"That is the true power of Step Functions," Margaret smiled. "It maintains the state of the transaction. It handles retries with exponential backoff if a service is temporarily down, and it provides a perfect, visual audit trail of every distributed transaction. Many mature systems use a hybrid approach—choreography for simple reactions, and orchestration for critical sagas. But that is a lesson for another day."

Timothy updated his architecture diagram. His application was no longer just decoupled; it was orchestrating complex workflows with absolute, transactional safety.

---

Key Concepts Introduced:

Distributed Transactions: In a monolithic architecture, a single database transaction can ensure that all steps of a process (like payment and inventory) succeed or fail together (ACID properties). In a microservices architecture, services have independent databases, making traditional transactions impossible.

The Saga Pattern: A design pattern used to manage data consistency across distributed microservices. A Saga consists of a sequence of local transactions. If one of the transactions fails, the Saga executes Compensating Actions to undo the impact of the preceding transactions, returning the system to a consistent state (e.g., refunding a payment if inventory cannot be reserved). These compensating actions must also be idempotent.

Choreography vs. Orchestration: In Choreography (using stateless routers like EventBridge), services operate independently by reacting to events without a central controller. In Orchestration (using stateful managers like Step Functions), a central controller explicitly commands services to execute operations, making it ideal for complex workflows that require strict ordering and error handling.

AWS Step Functions: A visual workflow service that acts as a central orchestrator. It manages state, handles errors and retries natively, and provides a visual execution history. It offers two workflow types: Standard Workflows (ideal for long-running, auditable processes lasting up to a year) and Express Workflows (ideal for high-volume, short-duration event processing).

---

Aaron Rose is a software engineer and technology writer at tech-reader.blog. 

Catch up on the latest explainer videos, podcasts, and industry discussions below.