The Secret Life of Python: An Introduction to Concurrency

Synchronous vs. concurrent execution and the problem of race conditions

#Python #Concurrency #Threading #SoftwareEngineering

---

Margaret is a senior software engineer. Timothy is her junior colleague. They work in a grand Victorian library in London — the kind of place where code quality is the unspoken objective, and craftsmanship is the only thing that matters.

Episode 31

Timothy was feeling proud of his MatchLogger. It worked perfectly for a single game. But the Chess Club was growing. Today, two Grandmasters were playing two different matches at the exact same time, and Timothy wanted his app to record both simultaneously.

"I'll just start two loggers," Timothy told Margaret. "I'll have the program start Match A, then start Match B, and they can just... share the screen?"

But when he ran his script, Match B didn't start until Match A was completely finished. The Grandmasters were tapping their watches.

"Margaret," Timothy called out, "my code is waiting in line! It's like a grocery store with ten shoppers but only one cashier. How do I make Python do two things at once?"

---

The Illusion of Simultaneity

Margaret walked over to the whiteboard and drew a single, straight line.

"Everything you’ve written so far, Timothy, has been Synchronous," she said. "One task starts, finishes, and then the next begins. It’s safe, but it’s slow when you’re waiting on 'The Physical World'—like a file to save or a clock to tick."

She drew a second line parallel to the first.

"To handle the club's growth, you need Concurrency. It’s the art of managing multiple tasks at once. But be careful: when you invite more than one 'worker' into your code, you invite The Chaos of Many."

---

The Race to the File

"Imagine," Margaret continued, "if both matches try to write to the club's 'Daily Summary' file at the exact same millisecond. If Match A is halfway through writing 'Winner: Timothy' and Match B interrupts to write 'Winner: Margaret,' the file might end up saying 'Winner: TimMargaretothy.'"

Timothy’s eyes widened. "That’s a mess."

"That is a Race Condition," Margaret said. "It’s what happens when the logic of your program depends on which 'worker' reaches the finish line first. In a single-threaded world, you were the only one with a pen. Now, everyone has a pen, and they’re all fighting for the same piece of paper."

---

Why Use Concurrency

"Why wouldn't I always use concurrency if it's faster?" Timothy asked.

"Because concurrency adds complexity," Margaret cautioned. "For simple scripts, the single cashier is fine. But when speed matters, you need to choose the right tool for the job."

---

Three Approaches

• Threading: Waiting on the outside world (I/O) like files, networks, or databases.
• Multiprocessing: Heavy calculations and number crunching (CPU) that need raw power.
• Asyncio: Juggling thousands of simultaneous network connections in one loop.

"We’re going to start with Threading," Margaret decided. "We’re going to give your Chess Club two cashiers. But first, we have to teach them how to stay out of each other's way."

---

Margaret’s Cheat Sheet: The Concurrency Mindset

• Synchronous: The "One-at-a-Time" world. Easy to debug, but slow if you're waiting on external resources.
• Concurrent: The "Many-at-Once" world. Fast and efficient, but introduces "The Chaos of Many."
• The Race Condition: A bug where two tasks change the same data simultaneously. The result is corrupted or unpredictable (like "TimMargaretothy").
• The Trade-off: Only use concurrency when the performance gain is worth the extra complexity of managing multiple workers.

---

Aaron Rose is a software engineer and technology writer at tech-reader.blog. 

Catch up on the latest explainer videos, podcasts, and industry discussions below.