TL;DR: Delve into the evolution of The Graph protocol, exploring the debate between "brownfield" and "greenfield" approaches, strategies for fair competition, and concerns about diversity and redundancy among indexers.
Opening remarks
Hello everyone, and welcome to the latest edition of Indexer Office Hours! Today is January 30, and we’re here for Episode 142.
Consider this space your town square—a place where ecosystem participants gather to discuss crucial and relevant topics related to indexing.
Whether you’re an experienced indexer or just starting out, we encourage you to join the conversation and share your insights.
Don’t miss the latest GRTiQ podcast, part one of an intriguing two-part series with Rodrigo Coelho, Chief Spirit Officer at Edge & Node.
Repo watch
The latest updates to important repositories
Execution Layer Clients
- Erigon v2.0 New releases
- v2.57.3 :
- This release schedules the Napoli hard fork on the Mumbai testnet at block 45648608 (Feb. 7, 2024, around 8 AM UTC). Mumbai operators must upgrade beforehand.
- v2.57.2 :
- Fixes RpcDaemon doesn’t see recently retired blocks.
- v2.57.1 :
- Fixes a disk usage increment of almost 100 GB between v2.55.1 → v2.57.0 and other issues in tx pool.
- v2.57.3 :
- Geth: New release v1.13.11 :
- Changes how transaction indexing operates. As of 1.13.11, the behavior of eth_syncing is slightly changed, so that it now reports true until transaction indexing is finished.
- This release fixes a few bugs and enables the Cancun upgrade for the Sepolia and Holesky networks; Sepolia will upgrade on Jan. 31, and Holesky on Feb. 7, and naturally this is a required upgrade if you intend to follow either chain.
- sfeth/fireeth: New releases:
- v2.3.1:
- Firehose blocks that were produced using the RPC Poller will have to be extracted again to fix the Transaction Status and the potential missing receipts (e.g., arb-one pre-nitro, Avalanche, Optimism…)
- v2.3.0 :
- Reduces logging and logging “payload.”
- Fixes tools compare-blocks that would fail on new format.
- Fixes substreams to correctly delete .partial files when serving a request that is not on a boundary.
- v2.3.1:
- Nethermind: New release v1.25.3-exp.2:
- This version brings many performance improvements, the most important being a major state database optimization called Half-path. This new optimization is not a full path-based storage, but the way it arranges data helps with the performance of block processing and state database size.
- Avalanche: New release v1.10.19 :
- Added admin.dbGet call to the admin API.
- Added bloom filter metrics:
- bloom_filter_count
- bloom_filter_entries
- bloom_filter_hashes
- bloom_filter_max_count
- bloom_filter_reset_count to the following namespaces:
- avalanche_X_vm_mempool
- avalanche_P_vm_mempool
- avalanche_C_vm_sdk_atomic_mempool
- avalanche_C_vm_sdk_eth_mempool
- Fixed race condition during validator set creation.
- Fixed C-chain mempool bloom filter recalculation.
- NOTE: This version is backwards compatible with v1.10.0. It is optional but encouraged.
- Arbitrum-nitro: New release v2.2.3-beta.1
- This release improves L3 support and fixes other miscellaneous issues.
Consensus Layer Clients
Information on the different clients
- Prysm: New releases v4.2.1-rc.2 , v4.2.1-rc.3:
- Fixes from Goerli Dencun fork.
- Holesky and Sepolia Dencun fork epochs.
- Teku: New release 24.1.1 :
- NOTE: This is a required update for Sepolia, Holesky, and Chiado. Optional for other networks.
- Attention: Teku will require around 50 GB (35 GB for Chiado) of extra storage for blobs, but theoretically blob storage requirements can go up to 103 GB (200 GB for Chiado). This extra storage space WILL NOT grow above this limit over time.
- Nimbus: New release v24.1.2 :
- Nimbus v24.1.2 is a low-urgency point release bringing full support for the upcoming Cancun-Deneb hard fork on the networks Sepolia, Chiado (Gnosis Chain testnet), and Holesky.
- Lighthouse: New release v4.6.0
- NOTE: This medium-priority release provides valuable features, performance improvements, and bug fixes for mainnet users. It also enables the Deneb upgrade on the Sepolia, Holesky, and Chiado testnets.
Graph Stack
- Graph Node: New release v0.34.0 :
- Substreams as Source of Triggers for Subgraphs: This update significantly enhances subgraph functionality by enabling Substreams to act as a source of triggers for running subgraph mappings. Developers can now directly run subgraph mappings on the data output from Substreams, facilitating a more integrated and efficient workflow.
- indexerHints in Manifest for Automated Pruning: This update introduces the ability for subgraph authors to specify indexerHints with a field prune in their manifest, indicating the desired extent of historical block data retention. This feature enables graph-node to automatically prune subgraphs when the stored history exceeds the specified limit, significantly improving query performance. This automated process eliminates the need for manual action by indexers for each subgraph. Indexers can also override user-set historyBlocks with the environment variable GRAPH_HISTORY_BLOCKS_OVERRIDE .
- Initial Starknet Support: Introducing initial Starknet support for graph-node, expanding indexing capabilities to the Starknet ecosystem. The current integration is in its early stages, with notable areas for development, including the implementation of trigger filters and data source template support. Future updates will also bring Substreams support.
- Autogenerated Int8 IDs in graph-node: Introduced support for using Int8 as the ID type for entities, with the added capability to auto-generate these IDs, enhancing flexibility and functionality in entity management.
- Addressed a bug in the deduplication logic for Cosmos events, ensuring all distinct events are properly indexed and handled, especially when similar but not identical events occur within the same block.
- Fixed compatibility issues with ElasticSearch 8.X, ensuring proper log functionality.
- Resolved an issue when rewinding data sources across multiple blocks. In rare cases, when a subgraph had been rewound by multiple blocks, data sources “from the future” could have been left behind. This release adds a database migration that fixes that. With very unlucky timing, this migration might miss some subgraphs, which will later lead to an error assertion failed: self.hosts.last().and_then(|h| h.creation_block_number()) <= data_source.creation_block(). Should that happen, the migration script should be rerun against the affected shard.
- Graphman Deploy Command: A new graphman deploy command has been introduced, simplifying the process of deploying subgraphs to graph-node.
- NOTE! This version is the same as v0.21.1 but fixes a bug that was affecting indexer-service startup.
- Subgraph-radio New release 1.0.1-alpha.6 :
- Fixed pubsub topic generation issue.
Graph Orchestration Tooling
Join us every other Wednesday at 5 pm UTC for Launchpad Office Hours and get the latest updates on running Launchpad.
The next one is on February 14. Bring all your questions!
Blockchains Operator Upgrade Calendar
The Blockchains Operator Upgrade Calendar is your one-stop solution for tracking hard fork updates and scheduled maintenance for various protocols within The Graph ecosystem.
Simplify your upgrade process and never miss a deadline again.
Protocol watch
The latest updates on important changes to the protocol
Forum Governance
- Graph Horizon Explained: a Proposal for the Evolution of the Protocol
- Add Sport for SKALE Network
- Add Support for Fraxtal (Frax Finance Chain)
- Add support for Scroll Mainnet
- Add Support For Linea Mainnet
Forum Research
Contracts Repository
- test: add _curationTaxPercentage 0 case #931 (merged)
- chore: deploy GGPs 31, 34 and 35 to Arbitrum One #933 (draft)
- feat: add ops script for query dispute testing and sdk modifications … #934 (open)
- fix: ensure L2 aliased addresses are the correct length #929 (merged)
Project watch
A detailed update from core developers on Graph Explorer, Indexer components, Graph Node, and more
Open discussion
Graph Horizon Explained: a Proposal for the Evolution of the Protocol
In simple terms, Graph Horizon is about iterating upon the existing protocol to address things we’ve learned since the protocol launched and implement improvements we’ve heard suggested in conversations with many users and contributors, to ensure The Graph fulfills its mission and remains core and reliable infrastructure for web3.
Pablo Carranza Vélez
- A proposal was introduced for the evolution of The Graph protocol and discussion around the need for a “brownfield vs. greenfield” approach and various points raised in the proposal.
- Pablo explained the design decisions regarding allocations and their potential impact on network discovery.
- They discussed the potential benefits and drawbacks of increased consumer choice in data services, as well as strategies to ensure fair competition among data service providers.
- They clarified the importance of spreading out indexing agreements with as many indexers as possible to maintain diversity and quality of service, addressing concerns about gateways centralizing indexing choices.
- Marc-Andre expressed his preference for a brownfield approach and discussed the potential benefits of modularizing everything in Horizon, opening up possibilities for various applications beyond data services.
- Concerns were raised about the potential negative impact of removing indexing rewards and the diversity and redundancy of indexers, shifting the responsibility for diversity from the protocol to gateway operators.
- There was discussion questioning the sustainability of moving the responsibility for diversity from the protocol to indexers, citing real-world examples where diversity is not typically prioritized in supplier selection.
No Comments