The Secret Life of AWS: The Bottomless Basement
.jpeg)
Why folders are a lie, why buckets must be unique, and the hidden cost of moving data.
Part 4 of The Secret Life of AWS
Timothy entered the West Wing Archives carrying a single piece of paper. He walked over to a wall of filing cabinets, opened a drawer, placed the paper inside, and closed it.
Then he opened it again, took the paper out, wrote a single word on it, and put it back.
"What are you doing?" Margaret asked. She was standing by the chalkboard, holding a cup of tea.
"I am saving my work," Timothy said. "I edit the file, I save the file. Just like a hard drive."
Margaret set her tea down. "Stop. Come to the board."
She picked up a cloth and wiped away the previous lesson. "Today, we discuss S3. And the first rule of S3 is: It is not a hard drive."
Buckets (The Container)
Margaret drew a large, U-shaped container on the board.
"In a traditional computer," she said, "you have a file system. You have a root drive (C:), folders, and subfolders. S3 (Simple Storage Service) is different. It is Object Storage."
She wrote BUCKET on the container.
"Everything begins with a Bucket. You cannot just leave files lying around; they must be inside a Bucket."
"Fine," Timothy said. "I will name my bucket 'files'."
Margaret shook her head. "You cannot. Bucket names must be Globally Unique. Not just unique to you—unique to the entire world."
"Why?"
"Because a bucket is not a folder on your computer," Margaret explained. "It is a web address. When you create a bucket, AWS creates a URL like https://files.s3.amazonaws.com. Since there can only be one of those URLs on the internet, there can only be one bucket named 'files'."
Timothy frowned. "I suppose I will call it timothy-london-archives-2026."
"Much better," Margaret said.
Objects (Not Files)
Margaret drew a piece of paper inside the bucket.
"In S3, we do not call them 'files'. We call them Objects."
She listed the components of an Object on the board:
- Key (The name)
- Value (The data itself)
- Metadata (Data about the data)
"This is the important distinction," Margaret said. "On a hard drive, if you want to change one word in a document, the computer finds that specific spot on the disk and rewrites just that word. This is called Block Storage."
She tapped the board. "S3 is Object Storage. Objects are immutable. You cannot open an object, edit a line, and close it."
"So how do I update my document?" Timothy asked.
"You don't," Margaret said. "You upload the entire new version of the document and overwrite the old one completely. It is atomic. It is either the old version or the new version. There is no 'half-saved' state."
The "Folder" Lie (Prefixes)
Timothy pointed to the filing cabinet. "But what about organization? I need folders. I want to store my image in photos/2026/january/image.jpg."
Margaret smiled a thin, knowing smile. "Timothy, I am going to tell you a secret. Folders do not exist."
Timothy blinked. "But I see them in the console. There are little folder icons."
"It is a lie," Margaret said calmly. "It is a hallucination created for the comfort of humans."
She wrote the string on the board:photos/2026/january/image.jpg
"To S3, that is not a file inside three nested folders. That is simply a Key—a very long filename that happens to contain slashes. S3 does not know what a 'folder' is. It just sees a single flat list of millions of objects."
"But... why does that matter?"
"Because if you try to 'rename' a folder with 1,000 files in it," Margaret warned, "you will discover that you cannot. Because the folder isn't there. You would have to copy all 1,000 objects to new names and delete the old ones. Do not be fooled by the console, Timothy. It is flat."
Storage Classes (Hot vs. Cold)
"I understand," Timothy said. "Unique buckets. Immutable objects. No real folders. Is that all?"
"Not quite," Margaret said. "How often will you look at this paper?"
"Maybe once a year."
Margaret drew a thermometer on the board.
"S3 charges you based on how accessible the data is. We have Storage Classes."
- S3 Standard: "The papers on your desk." Instant access, most expensive.
- S3 Infrequent Access (IA): "The filing cabinet." Instant access, but you pay a retrieval fee if you open the drawer.
- S3 Glacier: "The Deep Archive."
"Glacier is like freezing the document in a block of ice in the basement," Margaret said. "It is incredibly cheap—pennies to store terabytes. But if you want the document back, you cannot have it instantly."
"Why not?"
"Because we have to thaw the ice," she said. "It takes anywhere from minutes to 12 hours to retrieve data from Glacier. Never put your website's homepage in Glacier, Timothy. Your users will not wait 12 hours for the page to load."
The Hidden Cost (Data Transfer)
"So storage is cheap," Timothy said, "as long as I freeze it."
"Storage is cheap," Margaret corrected. "Movement is not."
She drew an arrow pointing out of the bucket.
"We charge you to keep the object on the shelf (Storage Costs). But we also charge you for Data Transfer Out. Every time someone downloads your file, you pay for the bandwidth."
"So if I put a massive video file in S3..."
"And one person watches it," Margaret finished, "it costs pennies. If a million people watch it, you will receive a bill that will make you weep. Plan accordingly."
The Lesson
Timothy looked at his single piece of paper. He realized it wasn't just a file. It was an immutable Object, identified by a globally unique Key, sitting in a flat Bucket.
"It is much more complex than a hard drive," Timothy admitted.
"It is," Margaret agreed. "But if you treat it right, S3 offers Eleven Nines of durability."
"Eleven nines?"
"99.999999999%," Margaret wrote on the board. "That means if you store 10,000 objects in S3, you can expect to lose one of them... once every 10 million years."
Timothy put the paper in his pocket. "I think I'll upload this. To the Standard class. I might need to read it tomorrow."
Aaron Rose is a software engineer and technology writer at tech-reader.blog and the author of Think Like a Genius.
Comments
Post a Comment