Substreams sink sql. DatabaseChanges mode improvements Fixed Clickhouse table fetching to ...

Substreams sink sql. DatabaseChanges mode improvements Fixed Clickhouse table fetching to ignore materialized columns when inserting data. Feb 16, 2026 · Delta update support in Substreams Sink SQL is a meaningful improvement for onchain data pipelines. If a column is materialized from another one, the column is skipped for data insertion coming from the Substreams. Requires adding annotations to your Protobuf messages (e. Sinks The Substreams:SQL Sink allows you to consume the data extracted from the blockchain through a SQL database. g. substreams-sink-clickhouse is a tool to pipe in data from the blockchain into a ClickHouse database The core function of the SQL sink is to translate your Substreams output (Protobuf data) into SQL tables. Sinks are integrations that allow you to send the extracted data to different destinations, such as a SQL database or a file. Choose one of the following methods depending on your needs: Using Relational Mappings "from-proto" Enables foreign key relationships in your SQL schema. Sinks are integrations that allow you to send the extracted data to different destinations, such as a SQL database, or a file. In those cases, it may be preferable to dump every files to CSV and then use COPYFROM to transfer data super quick to Postgres. Extract substreams-sink-sql binary into a folder and ensure this folder is referenced globally via your PATH environment variable. It supports two different Substreams output formats, each with distinct advantages: The Substreams:SQL Sink allows you to consume the data extracted from the blockchain through a SQL database. Sink your Substreams Choose a sink that meets your project’s needs. Quick Start Install and configure substreams-sink-sql in a project, define a manifest with a DSN for your database, run the setup to initialize the schema, and execute the sink to start streaming data into your SQL store. com Sep 19, 2025 · The substreams-sink-sql contains a fast injection mechanism for cases where big data needs to be dump into the database. The Substreams:SQL sink helps you quickly and easily sync Substreams modules to a PostgreSQL or Clickhouse database. , primary and foreign keys). . The Substreams SQL sink can be configured in multiple ways. Added handle_block_duration to postgres sink stats periodic logger to show how long from start to finish handling a block takes (this make more sense once live Oct 25, 2024 · Substreams Tutorial — Sink events from an Ethereum smart contract to an SQL database Ed from StreamingFast gives a quick tutorial on how to use Substreams to easily sink events from an Ethereum smart contract to an SQL database. While the sink configuration in the substreams. Note Or install from source directly go install github. Overview Once you find a package that fits your needs, you can choose how you want to consume the data. Pushing aggregation logic to Postgres keeps Substreams lighter and faster, while enabling native, incremental SQL aggregations (counts, sums, volumes). Install substreams-sink-sql by using the pre-built binary release available in the releases page. How-To Guides Consuming Substreams Once you find a package that fits your needs, you can choose how you want to consume the data. yaml file is optional, it can be useful for setting default parameters and advanced configurations. dtr qcuy xja ppvjo iujym