From Factory Floor to Linux: Building an MQTT Sensor Data Pipeline with Raspberry Pi Pico

-
A factory motor produces a temperature reading. A Raspberry Pi Pico W reads that sensor, adds a timestamp and passes the observation across the network.


From Factory Floor to Linux: Building an MQTT Sensor Data Pipeline with Raspberry Pi Pico

A factory motor produces a temperature reading.

A Raspberry Pi Pico W reads that sensor, adds a timestamp and passes the observation across the network. An MQTT broker handles the message transport. A Linux workstation receives the observations, cleans the data with Python and Pandas, and turns the results into CSV and JSON.

The individual pieces are small.

What's interesting is watching the data change as it travels through the system.

Our pipeline looks like this: 

factory motor
   ↓
temperature sensor
   ↓
Pico W
   ↓
sensor ID + temperature + timestamp
   ↓
Python dictionary
   ↓
JSON + MQTT publish
   ↓
MQTT broker
   ↓
Linux subscriber
   ↓
Python + Pandas
   ↓
cleaned data
   ├── CSV
   └── JSON

MQTT uses a publish-and-subscribe architecture. A device publishes messages to a topic, an MQTT broker receives those messages, and subscribers receive the messages for topics they follow.

That makes it a natural fit for our small factory-floor telemetry project.

What We're Measuring

We'll keep the factory side deliberately simple.

Imagine a temperature sensor monitoring an electric motor.

For our example, the Pico W reads that sensor once every 60 seconds. That's a rule we've chosen for this demonstration, not a universal industrial sampling interval.

Each observation has a sensor identifier and temperature: 

motor_01,74.6

motor_01 tells us which machine the measurement belongs to.

74.6 is the temperature reading.

But we're still missing an important piece of information.

When did the measurement occur?

That's another job we'll give the Pico W.

The Hardware

Our conceptual arrangement is straightforward.

We have a temperature sensor monitoring a factory motor.

A Raspberry Pi Pico W reads the sensor once per minute. The Pico associates that observation with the motor's identifier and adds a timestamp.

The Pico connects over Wi-Fi to the local network and publishes the resulting observation through MQTT.

Our Linux workstation receives and processes the measurements.

Each layer has a particular job.

The sensor measures temperature.

The Pico collects and packages the observation.

MQTT transports it.

Linux performs the heavier data processing.

Follow the Data

Let's start with a temperature measurement: 

motor_01,74.6

The Pico reads the sensor and adds a timestamp.

Conceptually, we now have: 

2026-11-03T18:42:10,motor_01,74.6

But we don't have to treat this information as a CSV line inside the Pico.

MicroPython can represent the observation as a Python dictionary: 

record = {
    "timestamp": timestamp,
    "sensor_id": "motor_01",
    "temperature": 74.6
}

That distinction is useful.

The information is still simply a time, sensor ID and temperature reading. Its representation changes depending on what the next part of the pipeline needs.

For MQTT, we'll serialize that dictionary as JSON.

Conceptually: 

Python dictionary
       ↓
JSON serialization
       ↓
JSON string
       ↓
bytes
       ↓
MQTT publish

That's an important boundary in our system.

The Python dictionary exists inside the Pico's MicroPython program.

It isn't the dictionary itself that travels across the network.

The record is serialized as JSON and published as an MQTT payload. MQTT transports payload data, while JSON gives that payload a structure our receiving program can understand. A common Python MQTT pattern is to serialize an object with json.dumps() before publishing and parse the received payload with json.loads().

Code on the Pico W

The Pico's job remains small.

Every 60 seconds:

  1. Read the temperature sensor.
  2. Associate the reading with motor_01.
  3. Add a timestamp.
  4. Build a Python dictionary.
  5. Serialize it as JSON.
  6. Publish it through MQTT.
  7. Wait for the next reading.

A simplified MicroPython program might look conceptually like this: 

import time
import ujson
from umqtt.simple import MQTTClient

BROKER = "192.168.1.100"
TOPIC = b"factory/motor_01/temperature"

client = MQTTClient(
    "pico-motor-01",
    BROKER
)

client.connect()

while True:

    temperature = read_temperature()
    timestamp = get_timestamp()

    record = {
        "timestamp": timestamp,
        "sensor_id": "motor_01",
        "temperature": temperature
    }

    payload = ujson.dumps(record)

    client.publish(
        TOPIC,
        payload.encode()
    )

    time.sleep(60)

The sensor-specific read_temperature() and timestamp function are intentionally conceptual here. Their implementation depends on the particular sensor and clock arrangement being used.

The data transformation is our focus: 

temperature sensor
       ↓
MicroPython
       ↓
Python dictionary
       ↓
ujson.dumps()
       ↓
JSON string
       ↓
.encode()
       ↓
bytes
       ↓
MQTT publish

MicroPython MQTT clients can publish messages to MQTT topics from network-connected microcontrollers, including RP2/Pico W environments.

Our original measurement might therefore travel through MQTT looking like this: 

{
  "timestamp": "2026-11-03T18:42:10",
  "sensor_id": "motor_01",
  "temperature": 74.6
}

The Pico publishes it under: 

factory/motor_01/temperature

Then it waits for the next 60-second cycle.

The MQTT Broker

Between the Pico W and our Linux data-processing program sits the MQTT broker.

Conceptually: 

Pico W
   ↓
publish
   ↓
MQTT broker
   ↓
deliver
   ↓
Linux subscriber

The broker is responsible for MQTT messaging between publishers and subscribers. It isn't responsible for cleaning our temperatures, creating Pandas DataFrames, or producing Excel files.

That separation is useful.

The Pico needs to know about the broker and the topic.

The Linux application subscribes to that topic.

Publisher and subscriber don't need to perform each other's jobs. MQTT's broker-based publish/subscribe model provides that separation.

The Linux Workstation

Now the data reaches our Linux machine.

Suppose the Linux subscriber receives MQTT payload bytes representing this JSON: 

{
  "timestamp": "2026-11-03T18:42:10",
  "sensor_id": "motor_01",
  "temperature": 74.6
}

The program can decode and parse the incoming message: 

import json

record = json.loads(
    message.payload.decode()
)

records.append(record)

Now watch the representation change again: 

MQTT payload bytes
       ↓
.decode()
       ↓
JSON text
       ↓
json.loads()
       ↓
Python dictionary
       ↓
records collection

The same information that became JSON on the Pico has now become structured Python data again on Linux.

After several MQTT messages arrive: 

MQTT message #1 → one record
MQTT message #2 → one record
MQTT message #3 → one record
MQTT message #4 → one record
                    ↓
             collection of records
                    ↓
                  Pandas

Then we're back in familiar territory: 

import pandas as pd

df = pd.DataFrame(records)

MQTT has done its job.

Python and Pandas can take over.

Cleaning the Factory Data

Real data doesn't always arrive perfectly.

Our collected readings might contain something like this: 

2026-11-03T18:42:10  motor_01    74.6
bad_timestamp        motor_01    75.1
2026-11-03T18:44:10   MOTOR_01   75.3
2026-11-03T18:44:10   MOTOR_01   75.3
2026-11-03T18:45:10  motor_01    missing
2026-11-03T18:46:10  motor_01    75.7

We have an invalid timestamp, inconsistent sensor identification, a duplicate observation and a missing temperature.

That's dirty factory data.

Our Linux Python program can clean it: 

import pandas as pd

# INPUT
df = pd.DataFrame(records)

# PROCESSING
df["timestamp"] = pd.to_datetime(
    df["timestamp"],
    errors="coerce"
)

df["sensor_id"] = (
    df["sensor_id"]
    .str.strip()
    .str.lower()
)

df["temperature"] = pd.to_numeric(
    df["temperature"],
    errors="coerce"
)

# Remove incomplete or invalid records
df = df.dropna(
    subset=["timestamp", "sensor_id", "temperature"]
)

# Remove duplicate observations
df = df.drop_duplicates()

# Put the readings in time order
df = df.sort_values("timestamp")

# Reset row numbers
df = df.reset_index(drop=True)

# OUTPUT
df.to_csv(
    "motor_temperature_clean.csv",
    index=False
)

df.to_json(
    "motor_temperature_clean.json",
    orient="records",
    date_format="iso",
    indent=2
)

print(df)

print("\nCreated motor_temperature_clean.csv")
print("Created motor_temperature_clean.json")

After cleaning, our useful records look more like this: 

2026-11-03T18:42:10  motor_01    74.6
2026-11-03T18:44:10  motor_01    75.3
2026-11-03T18:46:10  motor_01    75.7

The invalid timestamp is gone.

The missing temperature is gone.

The duplicate is gone.

And our motor identifier is represented consistently.

One Dataset, Two Audiences

Now our cleaned factory data can move in two directions.

motor_temperature_clean.csv is for people.

A manager or analyst can open the file in Excel, inspect the readings, calculate summaries, and create a chart showing motor temperature over time. That analysis could then become part of ordinary reporting work in Excel and feed PowerPoint presentations or Word reports.

motor_temperature_clean.json is for software.

Developers can use the structured records in another Python program, application, API, monitoring system, automation process, or another stage of a larger data pipeline.

Same motor.

Same measurements.

Same cleaning process.

Two durable representations for two different consumers.

Watch the State of the Data

This remains the heart of the project.

Our observation starts on the factory floor as a temperature measurement associated with a motor: 

motor_01,74.6

The Pico adds time and represents the observation as a Python dictionary: 

{
    timestamp,
    sensor_id,
    temperature
}

That dictionary becomes JSON: 

Python dictionary
       ↓
JSON

And that JSON becomes an MQTT payload: 

JSON
 ↓
bytes
 ↓
MQTT

On Linux, the process reverses: 

MQTT payload bytes
       ↓
JSON
       ↓
Python dictionary
       ↓
collection of records
       ↓
Pandas DataFrame

Pandas cleans the observations.

Then the pipeline branches: 

cleaned data
   ├── CSV → Excel → charts and human reporting
   │
   └── JSON → developers, programs, APIs and pipelines

Follow the entire path and our little temperature reading has traveled quite a distance: 

factory motor
   ↓
temperature sensor
   ↓
Pico W reads every 60 seconds
   ↓
sensor ID + temperature + timestamp
   ↓
Python dictionary
   ↓
JSON
   ↓
MQTT publish
   ↓
MQTT broker
   ↓
Linux subscriber
   ↓
JSON → Python records
   ↓
Pandas
   ↓
clean data
   ├── CSV for people
   └── JSON for software

That's a small industrial data pipeline.

The sensor measures.

The Pico collects, timestamps, packages and publishes.

MQTT transports.

Linux receives.

Pandas cleans.

CSV puts the resulting information into a familiar form for people working in spreadsheets and reports.

JSON keeps the same cleaned observations available for developers, applications, APIs, automation, and whatever part of the software pipeline needs them next.

The individual programs are small.

The interesting part is what happens to the data as it moves through the system.


Copyright © 2026 Daily Tech-Reader.blog
All Rights Reserved

Popular posts from this blog

Insight: The Great Minimal OS Showdown—DietPi vs Raspberry Pi OS Lite

-
Image
Insight: The Great Minimal OS Showdown—DietPi vs Raspberry Pi OS Lite When you're building a headless server, IoT device, or lightweight project box, the last thing you want is bloatware eating your precious resources. Enter the world of minimal operating systems—where every megabyte matters and efficiency reigns supreme. Two contenders dominate this space: DietPi and Raspberry Pi OS Lite. Both promise lean, mean computing machines that boot straight to the command line. But scratch beneath the surface, and you'll find they take fundamentally different approaches to the "less is more" philosophy. The Minimalist's Dilemma Picture this: You've got a Raspberry Pi 3B+ sitting on your desk, destined to become a home media server. Do you go with the familiar comfort of Raspberry Pi OS Lite, or venture into DietPi's optimized territory? The choice isn't just about personal preference—it's about understanding what "minimal" means to each operatin...
Read more

Troubleshooting Raspberry Pi Connect Issues: A Comprehensive Guide

-
Image
  Troubleshooting Raspberry Pi Connect Issues: A Comprehensive Guide Raspberry Pi Connect simplifies communication between devices without typically requiring network or firewall adjustments. However, restrictive networks can hinder its functionality. To resolve such issues, the rpi-connect doctor command offers diagnostic tools, enabling you to test network connectivity and troubleshoot problems effectively. This guide will walk you through diagnosing and resolving connectivity issues, viewing service status, enabling enhanced logging, and accessing service logs. Diagnosing Network Issues To identify network-related issues, use the rpi-connect doctor command: $ rpi-connect doctor If your network supports Connect communication, the output should display successful test cases with checkmarks: ✓ Communication with Raspberry Pi Connect API ✓ Authentication with Raspberry Pi Connect API ✓ Peer-to-peer connection candidate via STUN ✓ Peer-to-peer con...
Read more