Solve: Defining Arrays of Structs in AWS Glue with CDK's aws_glue_alpha...A Cleaner Approach

Solve: Defining Arrays of Structs in AWS Glue with CDK's aws_glue_alpha...A Cleaner Approach

Introduction

AWS Glue is a powerful tool for defining data pipelines and transformations, and with the AWS CDK's experimental aws_glue_alpha module, developers can now define Glue tables directly in Python or TypeScript code. However, not all features are ergonomic yet—especially when working with nested types like arrays of structs. If you've ever tried to define a column as an array of structs with a well-formed schema, you may have run into a frustrating limitation.

Clarifying the Pain Point

Imagine you're working with semi-structured data that includes an array of JSON objects, such as:

json

[
  {
    "color": "blue",
    "hex": "#0000FF"
  },
  {
    "color": "green",
    "hex": "#00FF00"
  }
]

You'd naturally want to express this in CDK like so: 

python

glue.Schema.array(
    glue.Schema.struct(columns=[
        glue.Schema.column("color", glue.Schema.STRING),
        glue.Schema.column("hex", glue.Schema.STRING)
    ])
)

Unfortunately, the current construct doesn't allow this. Instead, you're forced to write something more opaque: 

python

glue.Schema.array(
    input_string="struct",
    is_primitive=False
)

This leaves your struct definition disconnected from your actual Glue table schema and makes the code harder to understand, refactor, or validate.


A Practical Workaround

To keep your code clean and intentions clear, you can still define the struct schema for reference elsewhere in your pipeline logic or documentation, like so: 

python

# Define the inner struct schema separately
my_struct_schema = glue.Schema.struct(columns=[
    glue.Schema.column("color", glue.Schema.STRING),
    glue.Schema.column("hex", glue.Schema.STRING)
])

# Use a fallback array definition for Glue table
my_array_schema = glue.Schema.array(
    input_string="struct",
    is_primitive=False
)

You can then use this in a full Glue Table definition: 

python

glue.Table(self, "ExampleTable",
    database=my_database,
    columns=[
        glue.Column(name="id", type=glue.Schema.STRING),
        glue.Column(name="metadata", type=my_array_schema)
    ],
    data_format=glue.DataFormat.JSON,
    bucket=my_bucket,
    s3_prefix="example-data/"
)

Try It Yourself

To see this pattern in action, we’ve created a minimal, runnable AWS CDK app that defines a Glue database and table using the aws_glue_alpha module. It includes the array-of-struct workaround, a self-contained app.py, and clear setup instructions. You can explore it or clone it from the following Gist:

👉 View the full working CDK example on GitHub Gist

This isn’t a framework or boilerplate—just a focused example to help you get going quickly.


Why This Matters

This gap may seem small, but it creates real challenges for teams working with schema-driven pipelines. When struct definitions are embedded as raw strings, you lose the benefits of composability, type validation, and centralized schema management. Worse, it introduces potential drift if the same schema is redefined inconsistently across jobs or layers of your stack.

Even if CDK doesn’t yet allow nesting struct() inside array() directly, you can still build a clean and maintainable workaround by isolating your struct definition and clearly documenting your intent. This sets you up for easier upgrades when the construct matures.

Conclusion

If you're using aws_glue_alpha today and trying to define arrays of structs, you’re not alone. This workaround offers a clear path forward until full nesting support arrives. Define your structs cleanly, and keep your Glue schema definitions expressive and future-proof.

Questions? Thoughts? Reach out in AWS Developers Slack #appdev or leave a comment below — we’d love to hear how you’re approaching this.

Need AWS Expertise?

We'd love to help you with your AWS projects.  Feel free to reach out to us at info@pacificw.com.

Written by Aaron Rose, software engineer and technology writer at Tech-Reader.blog.