datasette-files-s3 0.1a1

2026-03-25 · Source: Simon Willison's Weblog · Field: Technology & Digital — Software Development & Engineering, Cloud Computing & IT Infrastructure, Data Science & Analytics · Depth: Intermediate, quick

Summary

datasette-files-s3 0.1a1, released on March 25, 2026, introduces an S3 backend for the datasette-files plugin, enabling file storage and retrieval via Amazon S3 buckets. This alpha release specifically implements a new mechanism to periodically fetch S3 configuration from a specified URL. This feature allows the use of time-limited IAM credentials, which can be restricted to a particular prefix within an S3 bucket, enhancing security and access control for file operations within Datasette.

Key takeaway

For DevOps Engineers managing Datasette deployments, this release means you can now integrate S3 for file storage with improved security. You should consider implementing the new URL-based credential fetching to utilize time-limited IAM credentials, restricting access to specific S3 bucket prefixes. This approach significantly reduces the risk associated with long-lived or overly permissive S3 access keys.

Key insights

datasette-files-s3 0.1a1 adds S3 storage with dynamic, time-limited IAM credential fetching.

Principles

• Dynamic credential management enhances security
• Prefix-restricted access improves data isolation

Method

S3 configuration is fetched periodically from a URL, enabling the use of time-limited IAM credentials restricted to a bucket prefix.

In practice

• Store Datasette files in S3
• Use temporary, scoped S3 credentials

Topics

• datasette-files-s3
• S3 Backend
• IAM Credentials
• Cloud Storage
• Datasette

Code references

• datasette/datasette-files-s3
• datasette/datasette-files

Best for: Software Engineer, Data Engineer, DevOps Engineer

Related on AIssential

• datasette-files 0.1a2 — Datasette-files 0.1a2 enhances Datasette with direct file uploads, CSV/TSV import, and image thumbnail generation.
• AWS Introduces S3 Files, Bringing File System Access to S3 Buckets — S3 Files provides high-performance, file system access to S3 buckets, translating file operations to object storage.
• DuckLake 1.0: Data Lake Format with SQL Catalog Metadata — DuckLake 1.0 centralizes data lake metadata in a SQL database, improving performance and simplifying operations.
• Use Amazon Quick Suite custom action connectors to upload text files to Google Drive using OpenAPI specification — Amazon Quick Suite custom connectors enable secure, natural language-driven file management across cloud storage systems.
• Ingesting Data into Databricks | Data Engineering in Databricks — Databricks supports both direct file uploads and external cloud storage connections for data ingestion into Delta tables.
• Labs sharing their models via DropBox 😅 — Early model sharing relied on ad-hoc methods like Google Drive and raw AWS S3 buckets.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Simon Willison's Weblog.