TL;DR: In this recap of IOH #161, you’ll find a rousing edition of Project Watch, covering possible Graph Node developments and the arrival of Sunrise, plus an interactive discussion on the effectiveness of chain integration communications.
Opening remarks
Hello everyone, and welcome to the latest edition of Indexer Office Hours! Today is June 11, and we’re here for episode 161.
GRTiQ 172
Don’t miss the GRTiQ Podcast with Dan French, a real estate expert who, as CEO at ResProp, is exploring how blockchain and cryptocurrencies can revolutionize the future of real estate investment, purchase, and management.
Repo watch
The latest updates to important repositories
Execution Layer Clients
- Geth New releases:
- v1.14.5 :
- v1.14.4 :
- Geth v1.14.4 is a usual maintenance release, but it does ship a 5-7% block import speed improvement. Furthermore, v1.14.4 also finally includes an Ether supply live tracer, that you can enable via –vmtrace supply. Also please note that the default value for miner tip enforcement was dropped from 1 gwei to 0.001 gwei (block producers can change this via –miner.gasprice).
- Nethermind: New release v1.27.0-rc.0 :
- This release brings 159 performance improvements and critical bug fixes to the Nethermind client, significantly enhancing its efficiency and reliability. It is a continuation of big block processing improvements:
- v1.25.4 to v1.26.0: Improved block processing by about 70% to 80% (from 68 to 107 MGas/s)
- v1.26.0 to v1.27.0: Improved block processing by about 150% (from 107 to 254 MGas/s)
- One of the key advancements in this release is the implementation of an intra-block cache. This feature optimizes the processing of transactions within blocks. By leveraging caching mechanisms, the system can avoid recalculating the state for transactions within the same block. This not only reduces redundant computations but also accelerates block execution, leading to a notable boost in overall performance.
- This release brings 159 performance improvements and critical bug fixes to the Nethermind client, significantly enhancing its efficiency and reliability. It is a continuation of big block processing improvements:
- Avalanche: New release v1.11.7 :
- This version is backward compatible with v1.11.0. It is optional but encouraged.
- Changed the undocumented pebble option for -db-type to be pebbledb and documented the option.
- Added peer’s trackedSubnets that are not locally tracked to the response from info.peers.
- Arbitrum-nitro New release v3.0.0 :
- Entrypoint changes: New default flags for the entrypoint. Ensure to replicate these if overriding the entrypoint.
- ArbOS 30 finalized: Required by June 17 for Arbitrum Sepolia; pending on-chain vote for Arbitrum One and Arbitrum Nova.
- Configuration updates:
- New -validation.wasm.allowed-wasm-module-roots path.
- -log-level now accepts string arguments (crit, error, warn, debug, info, trace).
- User-facing Improvements:
- Finalize ArbOS 30.
- Add pebble config options and HTTP body limit configuration.
- Adjust EIP-4844 batch size.
- Enhance blocks re-execution compatibility.
- Internal updates:
- Arbitrum Stylus support.
- Disable LLVM in JIT build.
- Separate wasm asm database.
- Various bug fixes and performance improvements.
- Heimdall: New release v1.0.7 :
- This release contains minor improvements and fixes over the previous release.
Consensus Layer Clients
Information on the different clients
- Lighthouse: New release v5.2.0
- This medium-priority release contains a major improvement to how states are managed in memory (tree-states) and an optimized epoch and block processing implementation. The beacon node now uses considerably less memory than the previous version thanks to more efficient state caching. It also includes other performance improvements, bug fixes, and groundwork preparation for the upcoming Electra fork and PeerDAS upgrade.
- Epoch and block processing have been optimized, resulting in improved performance and security. The optimized epoch processing implementation is accompanied by a proof of correctness detailed in this post.
- Introduce in-memory tree-states, providing more efficient state caching, improved overall performance and reduced memory usage.
- Add server-sent events (SSE) for proposer slashing, attester slashing, and BLS to execution change.
- Encode execution engine client version in block graffiti to probe execution layer client diversity, more details below.
- Add new timing metrics for block availability to better reflect the actual delays caused by blob processing and other factors.
- Update the proposer re-orgs feature to align with the latest spec.
- Introduce new containers, presets, and configurations in preparation for the Electra fork.
- Refactor and simplifications in sync and block lookup logic in preparation for PeerDAS.
- Add a new builder-header-timeout flag, which allows for configuring timeouts for the get builder header API call.
- Fix attestations not getting added to the aggregation pool. This improves the quality of aggregate attestations that Lighthouse produces.
Graph Stack
- Graph Node: New release v0.35.1-rc.0 :
- Enhancements and optimizations:
- Made the Ethereum Firehose codec more robust to avoid crashes.
- Optimized the subgraph pointer update process.
- Added deployment ID to firehose connection headers.
- Improved error handling for “too many bind params” in FindDerivedEntityQuery.
- Stopped using the stat API in IPFS.
- New features:
- Added zkSync fingerprint to the TOO_MANY_LOGS_FINGERPRINTS.
- Added support for Ethereum getCode in mappings.
- Added support for fetching block receipts.
- Bug fixes:
- Fixed check for deterministic error.
- Fixed static filters restart behavior when using thresholds.
- Fixed incorrect comparison of transaction hash in Ethereum chain.
- Fixed block receipts call failure for some providers.
- Configuration and infrastructure:
- Added zkSync exceptional case for transaction-less events.
- Updated the use of the Timestamp type for aggregation attributes.
- Improved the LayoutCache by periodically removing entries.
- Enhanced graphman rewind to use pause and resume logic.
- Improved Firehose log messages to include ‘provider’.
- Technical debt and refactoring:
- Refactored manual index creation and added DDL tests.
- Centralized and clarified the use of futures.
- Refactored graph, server to reinstitute Graph-Attestable header.
- Refactored code to support select by specific attributes by default.
- Tests and maintenance:
- Updated integration tests for ethereum.hasCode.
- Added a test for number_gte block constraints.
- Various dependency updates and version bumps (e.g., tokio, proc-macro2, num-integer, etc.).
- Enhancements and optimizations:
- Subgraph-radio: New release 1.0.6 :
- Update Graphcast SDK dependency.
Graph Orchestration Tooling
Join us every other Wednesday at 5 PM UTC for Launchpad Office Hours and get the latest updates on running Launchpad.
The next one is on June 19. Bring all your questions!
Polygon namespace now supported by Launchpad
This means that the Polygon mainnet and Amoy testnet are officially supported by the Launchpad stack. The namespace will deploy everything you need for self-hosted Polygon, including Erigon, Heimdall, proxyd, etc.
You can also seamlessly inherit updates from the stable or canary release channels.
Protocol watch
The latest updates on important changes to the protocol
Forum Governance
Contracts Repository
- ❄️ Graph Horizon and Subgraph Service ❄️ #944 (open)
- Graph Horizon: tests and internal review fixes #983 (open)
- fix: ensure rewards manager looks at subgraph service and staking for… #982 (merged)
- chore: horizon allocation fix and tests #981 (merged)
Project watch
Graph Node and subgraphs: high-level update
Recently, some of the core dev teams were thinking about the future of The Graph, and Graph Node was one of the main topics that was discussed. Subgraph composition came up as an important topic. A general theme is how we can enhance the network effects of subgraphs and make them more useful for developers. There are some limitations to reusing and tweaking the data contained in a subgraph.
Some ways we could overcome these limitations so that developers can modify and combine data:
- Query time composition where you’re “mushing” data together in Graph Client.
- Indexing time composition, enabling reuse of datasets across multiple subgraphs to optimize infrastructure setups.
Sunrise is here
The Graph’s hosted service endpoints are unavailable as of June 12, so indexers must update some of their configurations and practices.
Arbitrum Network Subgraph
Most indexers are probably using the hosted service endpoint. It’s time to generate an API key and update your Arbitrum network subgraph endpoint for indexer components.
The API key will give you 100,000 free queries per month.
- The network documentation will be updated.
- Reach out to felix@edgeandnode.com if you have questions or need help.
To clarify…
The hosted service endpoints will not exactly be unavailable as of June 12. They will be rate-limited. There also may be an exception for the Arbitrum network subgraph to give everyone more time to move from the hosted service.
The goal was to avoid people relying on the hosted service after the deadline.
Attestation signer cache limitations
Ford, a Software Engineer at Edge & Node, mentioned an issue that came up with a hard limit on a cache the indexer service uses. It only keeps 5,000 allocations in memory, so indexers should update this to avoid allocations getting blocked if they’re nearing 5,000.
- Hear more about this at 26:44 of the recording
- service: Increase size of attestation signer cache #906
Open discussion
Chain integration process
For the latest updates on chains integrating with The Graph Network, refer to the Chain Integrations Tracking Doc.
The following chains have indexer docs available:
- Scroll
- Linea
- Base
- BSC
Chain Integration Process Overview
Chain integration communications
To indexers:
- What do you think about the current communications for the chain integration process?
- Are they useful?
- Do you feel like you have enough time to respond and support these chains?
General feedback from indexers:
- Need more communications, more often, more updates—and more transparent.
- The process feels hidden, there’s nothing really public going on in terms of how the chains are progressing and what we should expect—need to work on transparency quite a lot.
- It’d be good to go over upcoming chains in IOH as another venue.
- Any changes to that doc [Chain Integrations Tracking Doc] should be part of the usual agenda in IOH.
- Network robustness over marketing splash. Having better transparency so indexers can actually prepare and serve the new networks as they come in.
Responses:
Note: Some comments have been lightly edited.
From Abel:
- Sometimes there are IOH episodes dedicated to chains that are about to be integrated or have already been integrated, like Astar and ZKsync.
From Pedro:
- I can also ping all indexers here whenever I update that Forum thread/tracking doc.
- I understand there’s not a lot of visibility. We can start providing more detailed weekly updates here on what’s being tested, what’s being discussed with the Council/TAB, and a list of upcoming chains (basically whenever that document is updated).
From Nick:
- We always try to market the announcements without disclosing too much in advance. It’s big news to announce a new chain! So it’s a delicate balance, but I see that we need to do better with this aspect of the comms.
- We can be more present here in IOH providing these updates.
- This is coming from a directive of a co-marketing strategy where we need to work with these chain partners and make sure we activate their dev communities. I do fundamentally agree that a more robust network is more important than a big marketing splash, but there is still some credence to making sure when new partnerships happen that there is marketing buzz and comms coordinated with the partners for their communities.
At the end of the episode, Abel confirmed that Scroll will be enabled for indexing rewards on June 25. He’s working on getting the Scroll team to attend IOH that day.
Indexing rewards and Substreams-powered subgraphs
How do indexers feel about rewards being enabled for Substreams-powered subgraphs on chains like Base, BSC, Linear, etc.?
Do indexers feel like operating the Firehose and Substreams stack is accessible for these chains?
Listen to this spirited discussion at 48:35 of the recording.
No Comments