DuckDB is a free and open-source SQL database that supports a wide range of data types and operations. It is designed to be fast, efficient, and easy to use, making it a popular choice for data analysis and manipulation.

In Latitude, DuckDB sources also has additional features like materialization, which allows you to store the results of your queries in a file so they can be used in other queries.

​
Install

Run the following command to install the connector from the root of your project:

npm install --save @latitude-data/duckdb-connector

​
How to configure sources

The sources files must be inside the queries/ folder.

To know how the source hierarchy works, please, visit How to configure sources.

​
Configuration

To configure the connection with your DuckDB database follow these steps:

1

Create the source configuration file

Create a new .yaml file in the queries folder. There can only be one source configuration file per directory, so if you already have another source file in your root queries folder, you will need to create a new subfolder for this source.

2

Add the duckdb type in the source configuration file

queries/duckdb/source.yaml 
type: duckdb
3

(optional) Add more details to connect to a remote duckdb database

queries/duckdb/source.yaml
type: duckdb
details:
  url: <your-duckdb-url>
4

Done!

​
Attributes

  • URL → (Optional) This refers to the location of the DuckDB database file. DuckDB is an embedded database, so instead of connecting over a network, you specify the path to the database file on your local system or a shared filesystem. This path tells your application exactly where to find the DuckDB database you want to work with. In its absence, duckDB starts a new database in memory that will not get persisted once the connection is closed.

​
Testing the connection

To test the connection you can:

  1. Create a query .sql in the queries directory that points to a table of your new connection in the FROM clause. See the section SQL Syntax Basics to learn more.
  2. Use the command line to run latitude run query_file_name where query_file_name is the name of your .sql file. This will display the results in your terminal. See the section Running queries to learn more about how to query your data.

​
Additional features

​
Materialization

Materialization allows you to store the results from any query to your Latitude server. Then, your DuckDB queries can access the materialized data instead of the original database. Read more about Materialized queries for more information.

DatabricksMicrosoft SQL