AI on Raspberry Pi: Testing Qwen2.5 0.5B

AI on Raspberry Pi: Testing Qwen2.5 0.5B

Clarifying the Issue
Running AI models locally on a Raspberry Pi has always been a challenge due to hardware limitations. However, Qwen2.5 0.5B provides an intriguing option. Unlike previous lightweight models that struggled with speed and coherence, Qwen2.5 0.5B demonstrates promising responsiveness and accuracy. The question is: can it truly serve as a practical local AI assistant on the Raspberry Pi 5?

This test was conducted on a Raspberry Pi 5 (8GB RAM) running Raspberry Pi OS Desktop Bookworm. We used Ollama as the AI runtime, ensuring all models were sourced from Ollama’s official library (ollama/library) for consistency.

Why It Matters
Most AI workloads are cloud-based, requiring constant internet connectivity and external processing power. However, running an LLM locally offers several advantages, including privacy, low-latency responses, and cost efficiency. If Qwen2.5 0.5B performs well, it could pave the way for offline AI applications, edge computing AI assistants, and embedded AI solutions for robotics and automation.

Key Terms

• Local Inference – Running AI models on local hardware instead of relying on cloud services.
• Ollama – An AI runtime designed for running language models efficiently on different platforms.
• Token Generation Speed – The rate at which an AI model generates text responses, impacting real-time usability.

Steps at a Glance

1. Create the Test Environment – Set up Raspberry Pi 5 with Ollama.
2. Testing the Model – Evaluate Qwen2.5 0.5B using structured AI prompts.
3. Grading the Performance – Analyze speed, accuracy, and usability.

Detailed Steps

1. Create the Test Environment
First, update the Raspberry Pi and install Ollama:

Bash

sudo apt update && sudo apt upgrade -y
curl -fsSL https://ollama.ai/install.sh | sh

Verify the installation:

Bash

ollama --version

Download the Qwen2.5 0.5B model:

Bash

ollama pull qwen2.5:0.5b

Check available models:

Bash

ollama list

Run Qwen2.5 0.5B:

Bash

ollama run qwen2.5:0.5b

This model is approximately 0.5GB, making it significantly lighter than models like Llama 3.2 1B, yet still powerful enough for meaningful interactions.

2. Testing the Model
To evaluate its performance, we used structured prompts covering general knowledge, math reasoning, and code generation:

Category	Prompt	Purpose
General Knowledge	"What is the capital of Japan?"	Tests factual recall.
Math Reasoning	"What is 12 × 8?"	Evaluates arithmetic skills.
Code Generation (Easy)	"Write a Python program that prints numbers 1 to 10."	Assesses basic programming ability.
Code Generation (Moderate)	"Write a recursive function in C to calculate factorial."	Tests recursion understanding.
Logical Reasoning	"If a car travels at 60 mph for 2.5 hours, how far does it go?"	Measures applied logic skills.

Test Session Output

Bash

pi@raspberrypi51:~ $ ollama run qwen2.5:0.5b
>>> "What is the capital of Japan?"
The capital of Japan is Tokyo.

>>> "What is 12 × 8?"
$12 \times 8 = 96$.

>>> "Write a Python program that prints numbers 1 to 10."
for i in range(1, 11):
    print(i)

>>> "Write a recursive function in C to calculate factorial."
#include <stdio.h>

int fact(int n) {
    if (n == 0 || n == 1)
        return 1;
    else
        return n * fact(n-1);
}

int main() {
    int num = 5; // the number for which you want to calculate factorial
    printf("Factorial of %d is %d", num, fact(num));
    return 0;
}

>>> "If a car travels at 60 mph for 2.5 hours, how far does it go?"
To find out the distance covered by the car when traveling at a speed of 60 miles per hour for 2.5 hours, you can use the formula:

\[ \text{Distance} = \text{Speed} \times \text{Time} \]

In this case, the speed is 60 mph and the time is 2.5 hours. Plugging in these values gives us:

\[ \text{Distance} = 60 \, \text{mph} \times 2.5 \, \text{hours} \]
\[ \text{Distance} = 150 \, \text{miles} \]

So, the car travels a distance of 150 miles when traveling at a speed of 60 mph for 2.5 hours.

>>> Send a message (/? for help)

3. Grading the Performance

Category	Grade	Description
General Knowledge	✅ A	Correct answer.
Math Reasoning	✅ A	Accurate calculation.
Code Generation (Easy)	✅ A+	Provided correct code structure.
Code Generation (Moderate)	✅ A+	Correct recursive function with explanation.
Logical Reasoning	✅ A	Clear and correct response.

Closing Thoughts
Qwen2.5 0.5B performed remarkably well, delivering accurate and rapid responses with minimal lag. Despite its small size, it proves to be a capable local AI assistant, handling structured prompts effectively. The model's lightweight footprint makes it particularly well-suited for embedded AI applications and edge computing.

For developers, educators, and AI enthusiasts looking to experiment with AI on the Raspberry Pi, Qwen2.5 0.5B is a fantastic option. It reinforces the potential for running LLMs locally, even on low-power devices like the Raspberry Pi 5.

Need AI or Raspberry Pi Expertise?
If you're looking to explore local AI inference, test new models, or integrate AI into embedded systems, we’d love to help. 🚀

Email us at: info@pacificw.com

Image: Gemini