**Exploring DuckDB’s New Quack Protocol: Concurrent Reads and Writes Across Remote Databases**
The DuckDB team recently introduced **Quack**, a new communication protocol designed to enable DuckDB databases on different servers to communicate over HTTP in a client/server arrangement. At a high level, Quack allows a DuckDB instance on one server to query or write data to a DuckDB instance on a remote server. Importantly, this capability is distinct from distributed query processing—Quack does not distribute a single query across multiple nodes, but it does enable coordination between separate DuckDB servers.
In this article, we explore how Quack can be used to perform **concurrent reads and writes** across multiple remote DuckDB databases. To demonstrate this in practice, we built a test environment using AWS EC2 instances and developed a GitHub repository called **cluster-duck**, which coordinates SQL execution across three remote DuckDB databases.
—
### The Test Setup
We deployed three AWS EC2 servers using a CloudFormation stack. Each server runs:
– Amazon Linux 2023 ARM64
– Python 3.12 and DuckDB 1.5.5
– The official DuckDB Quack extension
– A dedicated DuckDB database file
– A lightweight Quack server listening on port 9494
One server acts as the **coordinator node**, running Python code that orchestrates SQL execution across the three workers. The coordinator uses threading to fire SQL statements in parallel, synchronizes execution with a barrier, and collects results from each remote server.
> **Note:** This is not a distributed DuckDB cluster. Each server holds its own independent database file.
—
### How Coordination Works
The coordinator:
1. Accepts SQL statements via command line arguments, one per worker.
2. Wraps each SQL statement in a `QueryFragment` with a label (e.g., `query-1`, `query-2`).
3. Launches one thread per fragment, each connecting to its assigned worker via the Quack protocol.
4. Uses a barrier to ensure all threads start at roughly the same time.
5. Collects result sets, timing information, and returns a unified output.
Each worker opens its local DuckDB connection, loads Quack, attaches the coordinator’s endpoint, and executes the remote query via `remote.query()`. Results are returned to the coordinator and displayed together.
—
### Creating Test Data
Each server hosts a different table with 10 million synthetic records:
– **Worker 1:** `sales`
– **Worker 2:** `customers`
– **Worker 3:** `products`
Data is generated locally during EC2 bootstrap using Python scripts—no external data transfers are required.
—
### Usage Examples
#### 1. Running Simple SQL Statements
You can execute aggregate queries on each server simultaneously:
“`bash
cluster-duck-sql
–query “worker-1=SELECT sale_status, COUNT(*) FROM sales GROUP BY sale_status”
–query “worker-2=SELECT country, COUNT(*) FROM customers GROUP BY country”
–query “worker-3=SELECT category, COUNT(*) FROM products GROUP BY category”
“`
Output includes per-worker results and timing metrics, including a **start spread** metric showing how tightly the queries were launched.
#### 2. Running Complex SQL
The coordinator can also run complex, multi-step queries such as window functions, CTEs, and quantile calculations. Each worker executes its query independently, and results are gathered and displayed on the coordinator.
#### 3. Concurrent Writes and Reads
Using the `–allow-write` flag, you can insert new records into a remote table while simultaneously running read queries. The read queries see consistent snapshots—either before or after the insert— but never partial or corrupted data.
“`bash
cluster-duck-sql
–allow-write
–query “worker-1=INSERT INTO sales VALUES (30000001,1,1,1,’online’,’card’,’completed’,100.00,100.00,0.0,DATE ‘2026-08-09’)”
–query “worker-1=SELECT COUNT(*) AS visible_rows FROM sales WHERE sale_id BETWEEN 30000001 AND 30000020”
“`
You can observe how the count increases as inserts complete, demonstrating real-time, consistent concurrent access.
#### 4. Running DDL Across Servers
Quack also supports data definition language (DDL) commands. For example, you can create aggregated summary tables on each worker and query them immediately.
—
### Cost and Cleanup
The entire test environment consists of:
– Three t4g.nano EC2 instances
– Three 8 GB gp3 volumes
– Supporting AWS services (SSM, Parameter Store, Lambda)
Estimated cost for a 4-hour run is around **$0.12**, excluding potential free-tier benefits.
When you’re done, clean up with:
“`bash
aws cloudformation delete-stack
–region us-east-2
–stack-name cluster-duck-test-v2
“`
—
### Frequently Asked Questions (FAQ)
**Q: What is the difference between Quack and attaching a remote database in DuckDB?**
A: Attaching a database allows you to run SQL that spans local and remote tables in a single query. Quack, on the other hand, enables you to push SQL to a remote DuckDB server and retrieve results, but each query is executed independently. It is designed for coordination, not distributed query planning.
**Q: Does Quack support distributed transactions?**
A: No. Quack does not provide distributed transaction support. Each SQL statement is executed as an independent autocommit transaction. If one statement fails, others are not rolled back.
**Q: Is Quack production-ready?**
A: The DuckDB team describes Quack as experimental and a work in progress. It should not be used in production systems.
**Q: Can I use Quack over an untrusted network?**
A: Quack supports authentication via a token stored in SSM Parameter Store and can disable SSL for simplicity. For production use, you should enforce secure connections and manage tokens carefully.
—
### Conclusion
The **cluster-duck** project demonstrates how DuckDB’s Quack protocol can be used to execute parallel SQL statements across multiple remote databases. While not a distributed query engine, Quack provides a practical way to coordinate reads and writes in scenarios where each server maintains its own dataset.
In our tests, Quack handled concurrent queries and writes reliably, offering consistent snapshots and straightforward coordination via lightweight Python orchestration. For developers exploring federated analytics, edge computing, or multi-site data collection, Quack offers an intriguing building block—even in its experimental stage.
As always, we welcome feedback, extensions, and real-world use cases. You can explore the full code, CloudFormation templates, and examples in the [cluster-duck GitHub repository](https://github.com/your-repo/cluster-duck).
