Tech-Reader.blog

  The Secret Life of AWS: The Fanout (Amazon SNS & Pub/Sub) How to broadcast one event to multiple microservices #AWS #SNS #PubSub #Microservices 🎧 Audio Edition: Prefer to listen? Check out the expanded AI podcast version of this deep dive on  YouTube . 📺 Video Edition : Prefer to watch? Check out the 7-minute visual explainer on YouTube . Part 47 of The Secret Life of AWS Timothy’s Checkout service was finally stable. By routing messages through Amazon SQS, the Checkout process was completely insulated from Inventory database outages. But success brings new requirements. "The product team wants to expand," Timothy told Margaret, mapping out the new flow on the whiteboard. "When a customer clicks 'Buy', we still need to update Inventory. But now, we also need to notify the Fulfillment service to print a shipping label, and we need to trigger the Notification service to email a receipt." Timothy drew three arrows pointing out of the Checkout service. ...

  The Secret Life of Azure: The Voice That Read Everything (RAG) Building private, grounded AI assistants with Azure OpenAI and RAG #Azure #OpenAI #AI #DevOps #GenerativeAI 🎧 Audio Edition: Prefer to listen? Check out the expanded AI podcast version of this deep dive on   YouTube . 📺  Video Edition :  Prefer to watch? Check out the 7-minute visual explainer on  YouTube . Data & AI The library was glowing with the light of the "Smart Index," but Timothy was still watching patrons struggle to synthesize information from twenty different scrolls at once. "Margaret," Timothy said, "the AI Search is brilliant—it finds the right books instantly. But the patrons are overwhelmed. They don't want a list of ten books about 'how to build a bridge'; they just want to know how to build the bridge. They want to talk to someone who has already read every book in the basement and can summarize the answer for them. But I can't be everywhere at once." ...

  The Secret Life of Go: The Mutex Protecting Shared Memory and The RWMutex. #Golang #Concurrency #SystemDesign #CodingTips 🎧 Audio Edition: Prefer to listen? Check out the expanded AI podcast version of this deep dive on  YouTube . 📺  Video Edition :  Prefer to watch? Check out the 7-minute visual explainer on  YouTube . Part 27: Protecting Shared Memory and The RWMutex Ethan stared at his terminal, utterly defeated. "It panics," he said. "Every time I run the load test, the whole server crashes." Eleanor peered over his shoulder at the error message glowing on the screen:  fatal error: concurrent map read and map write . "Ah," Eleanor nodded. "The classic map panic. Show me the cache implementation." Ethan brought up the code. var cache = make (map[string]string) func GetUser (id string) string { // If it's in the cache, return it if val, exists := cache[id]; exists { return val } // Simulate a database f...

  The Secret Life of AWS: The Buffer (Amazon SQS) Why tight coupling is a single point of failure. #AWS #SQS #Microservices #CloudArchitecture 🎧 Audio Edition: Prefer to listen? Check out the expanded AI podcast version of this deep dive on  YouTube . Part 46 of The Secret Life of AWS Timothy watched the graphs on his dashboard turn red. He had built the Checkout microservice and the Inventory microservice. Thanks to Margaret's guidance, they were securely connected across an AWS Transit Gateway. But today, they were both failing. "What happened?" Margaret asked, pulling up a chair. "The Inventory database locked up during a traffic spike," Timothy explained, frantically clicking through CloudWatch logs. "The Inventory service stopped responding. But the problem is... now the Checkout service is crashing too. Customers can't place orders at all." Margaret looked at the architecture diagram. "How does Checkout talk to Inventory?" "It...

  The Secret Life of Azure: The Basement with No Bottom (Blob Storage) Storing massive amounts of unstructured data with Azure Blob Storage. #Azure #BlobStorage #DataEngineering #SAS Data & AI The library lobby was overflowing. Timothy was standing next to a mountain of heavy crates containing thousands of high-resolution scrolls, fragile audio cylinders, and millions of archived maps. "Margaret," Timothy said, "we’ve given the library its notebooks—the  Azure SQL  ledger for our strict records and the  Cosmos DB  binder for our flexible data—but I can't fit these crates into a notebook. I’m running out of floor space, and these items aren't just heavy; there are millions of them. Where do I put the things that aren't 'text'?" Margaret walked over to a heavy oak door in the floor and pulled it open to reveal a staircase leading down into an expanse that seemed to have no end. "Timothy, you're looking for  Azure Blob Storage . In the ...

  The Secret Life of Go: Worker Pools How to Stop Crashing Your Server with 10,000 Goroutines #Golang #Concurrency # WorkerPools #SystemDesign Part 26: Controlling Concurrency and The Dispatcher Ethan was staring at a wall of red text on his monitor. "Out of memory," he muttered. "How? Go is supposed to be incredibly efficient." Eleanor walked by with her coffee. "What did you crash this time?" "The image processing service," Ethan sighed. "We had a backlog of ten thousand images to resize. Since goroutines are lightweight, I just launched one for every single image so they would all process at the same time." He showed her the code: func ProcessAll (images []string) { var wg sync.WaitGroup for _, img := range images { wg. Add ( 1 ) go func (imageName string) { defer wg. Done () resizeImage (imageName) // Opens the file, decodes, resizes, saves }(img) } wg. Wait () } ...

  LimitExceededException  When Starting Too Many Rekognition Video Jobs How to track active video jobs and design around Rekognition concurrency limits #AWS #AmazonRekognition #CloudArchitecture #DevOps Category: Service Quotas & Throttling Problem Your system submits multiple video analysis jobs using  Amazon Rekognition . You call: StartLabelDetection StartFaceDetection StartContentModeration Other async video APIs The first few requests succeed. Then new requests fail with: LimitExceededException: The number of concurrent jobs exceeds the limit. IAM permissions are correct. S3 access works. SNS role is configured. Yet new video jobs refuse to start. Clarifying the Issue Rekognition video APIs are  asynchronous . When you call a  Start*  operation, AWS: Accepts the job Queues it for processing Returns a  JobId That job remains active until it completes. Each AWS account has a  maximum number of concurrent active video jobs per region . If yo...