The Graph Indexer Office Hours #141

TL;DR: Steph from StreamingFast addresses questions on Substreams' compatibility with various chains, the exploration of Go language for Substreams, potential challenges with ClickHouse integration, handling Substreams cache during Graph Node restarts, and the versatility of Substreams in handling arbitrary trace contexts, providing insights into the current status and future considerations.

Opening remarks

Hello and welcome back to Indexer Office Hours!

Today is January 23, and this is episode 141.

Don’t miss this week’s GRTiQ podcast with Uri Kolodny, Co-Founder at StarkWare, the pioneering team behind Starknet, a Layer 2 solution designed to tackle Ethereum’s scalability challenges.

Repo watch

The latest updates to important repositories

Execution Layer Clients

• Erigon v2.0 New release v2.57.0 :
  • Schedule Dencun for Chiado
  • mdbx: hard dirtyPages limit
  • stagedsync: fix bor heimdall mining flow
  • Sepolia/Holesky/Chiado users: you must upgrade to this release or later before Dencun (30 Jan. for Sepolia, 31 Jan. for Chiado, 7 Feb. for Holesky).
• sfeth/fireeth: New release v2.2.2 :
  • After the Cancun hard fork on Goerli, this release changes the Protobuf definition for the new BlockHeader, Transaction, and TransactionReceipt fields that are related to blob transaction.
    These fields are optional in the Protobuf definition, which will render them in your language of choice using the appropriate “null” mechanism. For example, on Golang, those fields are generated as BlobGasUsed *uint64 and ExcessBlobGas *uint64 , which will make it clear that those fields are not populated at all.
  • The affected fields are:
    • BlockHeader.blob_gas_used, now optional uint64.
    • BlockHeader.excess_blob_gas, now optional uint64.
    • TransactionTrace.blob_gas, now optional uint64.
    • TransactionTrace.blob_gas_fee_cap, now optional BigInt.
    • TransactionReceipt.blob_gas_used, now optional uint64.
    • TransactionReceipt.blob_gas_price, now optional BigInt.
  • This is technically a breaking change for those that could have consumed those fields already but the impact should be minimal.
  • Operators will need to reprocess a small Goerli range. Update to new version to produce the newer version and then reprocess from block 10377700 up to when you upgraded to v2.2.2.
  • The block 10377700 was chosen since it was the block at the time of the first release supporting Cancun, where we introduced those new fields. If you know when you deploy either v2.2.0 or v2.2.1, you should reprocess from that point.
  • An alternative to reprocessing is updating your blocks by having a StreamingFast API Token and using .
• Nethermind: New releases
  • v1.25.3:
    • v1.25.3 version is a mandatory upgrade for all node operators on the following chains:
      1. Sepolia Dencun hard fork – Jan. 30, 2024 at 22:51:12 UTC
      2. Chiado Dencun hard fork – Jan. 31, 2024 at 18:15:40 UTC
      3. Holesky Dencun hard fork Feb. 07, 2024 at 11:34:24 UTC
  • v1.25.2:
    • This hotfix addresses the consensus issue in Nethermind that was introduced in v1.23.0. This release is mandatory for all Nethermind node runners.
    • No resync required. Consensus client restart is required; no need to resync your consensus client.
    • Version 1.22 and below wasn’t affected.
  • v1.25.1:
    • Consensus issue detected. This version is recognized to have a consensus issue. Please avoid using it in favor of 1.25.2.
• Avalanche: New release v1.10.18 :
  • Added avalanche_{chainID}_blks_build_accept_latency metric
  • Added avalanche_{chainID}_blks_issued{source} metric with sources:
    • pull_gossip
    • push_gossip
    • put_gossip , which is deprecated
    • built
    • unknown
  • Added avalanche_{chainID}_issuer_stake_sum metric
  • Added avalanche_{chainID}_issuer_stake_count metric
  • Added:
    • -consensus-frontier-poll-frequency
  • Removed:
    • -consensus-accepted-frontier-gossip-frequency
  • Deprecated:
    • -consensus-accepted-frontier-gossip-validator-size
    • -consensus-accepted-frontier-gossip-non-validator-size
    • -consensus-accepted-frontier-gossip-peer-size
      • Updated the default value to 1 to align with the change in default gossip frequency
    • -consensus-on-accept-gossip-validator-size
    • -consensus-on-accept-gossip-non-validator-size
    • -consensus-on-accept-gossip-peer-size
  • NOTE: This version is backwards compatible with v1.10.0. It is optional but encouraged.

Consensus Layer Clients

Information on the different clients

• Prysm: New release v4.2.1-rc.0 :
  • Fixes from Goerli Dencun fork
• Nimbus: New release v24.1.1 :
  • Nimbus v24.1.1 is a hotfix addressing a problem introduced in the v24.1.0 release. Nimbus was crashing immediately after being connected to an execution layer node that is not fully synced. All users of v24.1.0 are advised to upgrade at their earliest convenience.

Graph Stack

• Indexer Service & Agent: New release v0.21.2 :
  • indexer-agent: Fix bug in stake usage summary
  • indexer-agent: Support subgraphs syncing sepolia and arbitrum-sepolia
  • indexer-agent: Improve robustness of DAI contract contract calls
  • indexer-agent: add tests for allocation decision consolidation
  • indexer-agent: Improve batch action preparation
  • indexer-service: fix allocation monitor query
  • indexer-service: Various robustness improvements
  • indexer-service: Improve validation of operator wallet
  • indexer-service: Add graph-node version endpoint from status api
  • indexer-common: Reduce stringency of action validation when adding to queue
  • NOTE! This version is the same as v0.21.1 but fixes a bug that was affecting indexer-service startup
• Subgraph-radio New release 1.0.1-alpha.5 :
  • Adds new metrics and fixes memory leak

Graph Orchestration Tooling

Join us every other Wednesday at 5 pm UTC for Launchpad Office Hours and get the latest updates on running Launchpad.

The next one is on January 31. Bring all your questions!

Blockchains Operator Upgrade Calendar

The Blockchains Operator Upgrade Calendar is your one-stop solution for tracking hard fork updates and scheduled maintenance for various protocols within The Graph ecosystem.

Simplify your upgrade process and never miss a deadline again.

Add to your calendar!

Protocol watch

The latest updates on important changes to the protocol

Forum Governance

• Add support for opBNB
• Add support for zora network
• Add Support for BEVM Canary
• Add support for Scroll Mainnet
• Graph Horizon Explained: a Proposal for the Evolution of the Protocol

Forum Research

• Why do we need curation?

Contracts Repository

• test: add _curationTaxPercentage 0 case #931 (merged)
• fix: ensure L2 aliased addresses are the correct length #929 (merged)
• fix: ensure globbing is enabled in prepack #930 (merged)
• fix: correctly pass ts file list to tsc in prepack #928 (merged)
• fix: prepack instead of prepublishOnly for yarn 4 #927 (merged)
• chore: bump contracts to v6.1.0 #926 (merged)
• ci: retry yarn build to work around hardhat race condition #925 (merged)
• Update README.md #924 (merged)
• fix: typos #923 (merged)

Open discussion

• StreamingFast: Substreams Support for Avalanche C-Chain
  • Read the article, Substreams Support for Avalanche C-Chain

🚨 Substreams Support for Avalanche C-Chain! 🚨

StreamingFast is kicking off 2024 by adding support for yet another chain: @avax's C-Chain!
Developed for @graphprotocol, Substreams allow developers to access indexed data at parallelized speeds, unavailable elsewhere. ⏫

Need… pic.twitter.com/K6mHiYj72n

— StreamingFast ⏫ (@streamingfastio) January 18, 2024

• Steph answers a question about the current implementation of Substreams, supporting most EVM chains with logs but not parity Trace. Discussion about potential issues with different implementations and the need for some kind of handshake or identification mechanism.
• Steph talks about the ongoing work with extended tracers in Git, making it possible to have a thin implementation of Firehose instrumentation. Anticipates easier maintenance once the pull request is merged upstream.
• Mention of an issue related to ClickHouse not being run into Cloud and a question about Substream SQL. Steph encourages opening GitHub issues for more detailed investigation.
• Consideration of using TypeScript first and postponing work on alternative languages.
• Discussion on exploring TypeScript and potential side effects like Go.
• Question about the status of allowing subgraphs to index data emitted by Substreams.
• Discussion on the potential performance benefits of using Firehose with Graph Node and RPC for getting logs.
• Steph provides nuanced responses and technical details on RPC, Firehose, and Substreams.
• Further details on the versatility of Substreams and potential indexing features.
• Highlighting the versatility of Substreams, allowing for the integration of various chains without specific implementations.

Questions and Answers

• Derek asks if Substreams can handle arbitrary trace contexts or if it’s expected to work only with specific subgraphs.

Answer:

Steph clarifies that Substreams are designed to handle arbitrary trace contexts and be versatile.

• Mention of an issue related to ClickHouse not being run into Cloud and a question about Substreams SQL.

Answer:

Steph encourages opening GitHub issues for more detailed investigation.

• A participant raises a non-related question about Substreams cache handling during Graph Node restarts.

Answer:

Steph suggests opening a GitHub issue with a command to reproduce the issue for better analysis.