Problem: "No Space Left on Device" on Amazon Lightsail – Instance Out of Storage

Problem: "No Space Left on Device" on Amazon Lightsail – Instance Out of Storage

Bash

$ touch testfile.txt
touch: cannot create file ‘testfile.txt’: No space left on device

Issue:

Your Amazon Lightsail instance has run out of storage, preventing it from creating files, writing logs, or running applications. This can happen due to:

• Disk is completely full – Instance storage has reached 100% capacity.
• Docker/Log Files Overflow – Large Docker images or excessive log files are consuming space.
• Hidden Files or Orphaned Data – Deleted files are still holding disk space.
• Filesystem Inodes Exhausted – Even with free space, inode exhaustion can prevent file creation. 
• Misconfigured Storage Expansion – If you've added a disk, it may not be mounted properly.

Fix: Identify Storage Usage and Free Up Space

Bash

# Step 1: Check disk usage
$ df -h

# Expected Output (if disk is full)
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      20G   20G     0 100% /

# Step 2: Identify large files consuming space
$ du -ah / | sort -rh | head -10

# Expected Output (Largest Files/Directories)
4.8G    /var/lib/docker
2.3G    /var/log
1.5G    /home/ubuntu/large_backup.tar
800M    /var/cache

# Delete unnecessary large files
$ rm /home/ubuntu/large_backup.tar

# If Docker is using too much space, remove unused containers/images
$ docker system prune -a

# Step 3: Check for large log files
$ sudo du -sh /var/log/*

# Expected Output (Abbreviated)
500M    /var/log/syslog
320M    /var/log/auth.log
20M     /var/log/kern.log

# Clear large log files safely
$ sudo truncate -s 0 /var/log/syslog
$ sudo truncate -s 0 /var/log/auth.log

# Rotate logs to prevent rapid growth
$ sudo logrotate -f /etc/logrotate.conf

# Step 4: Check for orphaned/deleted but still-in-use files
$ lsof | grep deleted

# Expected Output
apache2   1234    www-data  DEL       REG  8,1    1.2G    /var/log/apache2/access.log
docker     5678    root      DEL       REG  8,1    2.5G    /var/lib/docker/tmp/somefile

# Restart processes to release deleted files
$ sudo systemctl restart apache2
$ sudo systemctl restart docker

# Step 5: Check inode usage (if disk is not full but new files can't be created)
$ df -i

# Expected Output (if inodes are full)
Filesystem       Inodes   IUsed    IFree IUse% Mounted on
/dev/xvda1      1280000  1280000       0  100% /

# Delete small unnecessary files to free inodes
$ find /var/log -type f -delete

# Step 6: Expand storage if needed
$ lsblk

# Expected Output (Listing Available Disks)
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda    202:0    0   20G  0 disk /
xvdf    202:80   0   50G  0 disk   # additional storage disk

# Mount additional storage disk if available
$ sudo mkdir -p /mnt/data
$ sudo mount /dev/xvdf /mnt/data

# Verify the disk is mounted
$ df -h

# Make the mount permanent (add to fstab)
$ echo "/dev/xvdf /mnt/data ext4 defaults,nofail 0 2" | sudo tee -a /etc/fstab

# If increasing the root volume, take a snapshot first
$ aws lightsail create-instance-snapshot \
    --instance-name WebServer \
    --instance-snapshot-name WebServer-Snapshot

# Create a new instance from the snapshot with a larger disk size
$ aws lightsail create-instances \
    --instance-names "WebServer-Larger" \
    --availability-zone "us-east-1a" \
    --blueprint-id "ubuntu_22_04" \
    --bundle-id "medium_3_0" \
    --source-instance-id "WebServer-Snapshot"

# Resize the filesystem after increasing the volume
$ sudo resize2fs /dev/xvda1# FINAL STEP 1: Verify disk usage is now healthy
$ df -h

# Expected Output (After Fix)
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      20G   12G   8G   60% /

# FINAL STEP 2 : Ensure file creation now works
$ touch testfile.txt
$ ls -l testfile.txt

# Expected Output (if successful)
-rw-r--r-- 1 ubuntu ubuntu 0 Feb 10 12:00 testfile.txt

Need AWS Expertise?

If you're looking for guidance on Amazon Lightsail or any cloud challenges, feel free to reach out! We'd love to help you tackle your Lightsail projects. 🚀

Email us at: info@pacificw.com

Image: Gemini