The Graph Indexer Office Hours #146

Events By Wisdom Nwokocha Mar 01, 2024 No Comments

TL;DR: Exciting discussion on Pinax's new RPC service for indexers and an approach for saving the blobs (EIP-4844)!

Opening remarks

Hello everyone, and welcome to the latest edition of Indexer Office Hours! Today is February 27, and we’re here for Episode 146.

Listen to the latest GRTiQ podcast with Ayoola John, the CEO and co-founder of Astronaut, an innovative AI assistant designed specifically for community managers.

Repo watch

The latest updates to important repositories

Execution Layer Clients

Erigon v2.0 New release v2.58.1 :
- This patch release fixes syncing mainnet from scratch and the Index out of range issue.
Geth New releases:
v1.13.14 :
- Geth v1.13.14 is a small maintenance release with a handful of polishes to the blob pool.
- This release is NOT critical for the Cancun fork, but recommended to make Geth lighter in anticipation of unknown blob load.
v1.13.13 :
- This is a minor release with fixes for several issues related to the upcoming Cancun mainnet fork. As such, it is recommended for all mainnet users.
sfeth/fireeth: New releases:
- v2.3.4 :
  - Fix JSON decoding in CLI tools (firehose-client, print merged-blocks, etc.).
- v2.3.3 :
  - Known issue: the block decoding to JSON is broken in the CLI tools (firehose-client, print merged-blocks, etc.). Use version v2.3.1 for those tools.
  - Fix block caller panic in the previous version.
- v2.3.2 :
  - Added missing metering events for sf.firehose.v2.Fetch/Block responses.
  - Substreams:
    - Fixed bug in scheduler ramp-up function of sometimes waiting before raising the number of workers.
    - Fixed load-balancing from tier1 to tier2 when using dns:/// (round-robin policy was not set correctly).
    - Added trace_id in gRPC authentication calls.
Avalanche: New release v1.11.1 :
- Fixes suspended transaction re-push gossip in the P2P SDK.
Celo: New release v1.8.1 :
- Includes a collection of bug fixes and enhancements for the blockchain client. This update is recommended for all users.
Arbitrum-nitro New release v2.3.0 :
- This release adds support for ArbOS 20, which includes EIP-4844 batch posting support and new EVM opcodes such as MCOPY and TSTORE. The Arbitrum DAO forum proposal documents the full extent of the upgrade. This release will be a required upgrade for any Arbitrum networks that choose to adopt ArbOS 20.
- To run a layer 2 Arbitrum Nitro node (this does not apply to layer 3), you will now need to supply a beacon chain RPC connection. This allows the node to query the contents of EIP-4844 batches. If you’re running a layer 1 node locally, your consensus client should expose this RPC. Prysm exposes this RPC on port 3500 by default, and Lighthouse exposes this RPC on port 5052 by default.

Consensus Layer Clients

Information on the different clients

Prysm: New release v5.0.0 :
- Deneb is scheduled for mainnet epoch 269568 on March 13, 2024, at 01:55:35 PM UTC. All operators MUST update their Prysm software to v5.0.0 or later before the upgrade in order to continue following the blockchain.
- This release brings improvements to the backfill functionality of the beacon node to support backfilling blobs. If running a beacon node with checkpoint sync, we encourage you to test the backfilling functionality and share your feedback. Run with backfill enabled using the flag –enable-experimental-backfill.
- Known issue: –backfill-batch-size with a value of 1 or less breaks backfill.

Nimbus: New release v24.2.2 :
- Nimbus v24.2.2 is a hotfix release addressing a consensus violation issue affecting Deneb-transitioned networks such as Holesky. Please upgrade as soon as possible if your node is affected.

Lighthouse: New releases v5.0.0 :
- This release is mandatory for all mainnet and Gnosis users. It enables the Deneb/Cancun (“Dencun”) upgrade on:
  - Mainnet (#5233), which will occur on March 13, 2024, at 01:55:35 PM UTC
  - Gnosis (#5242), which will occur on March 11, 2024, at 06:30:20 PM UTC
- Any node that is not updated to a Lighthouse v5.x.x release by this time will stop following the chain and will need to be resynced. For stakers, this would result in missed rewards and penalties.

Graph Orchestration Tooling

Join us every other Wednesday at 5 PM UTC for Launchpad Office Hours and get the latest updates on running Launchpad.

The next one is on March 13. Bring all your questions!

Blockchains Operator Upgrade Calendar

The Blockchains Operator Upgrade Calendar is your one-stop solution for tracking hard fork updates and scheduled maintenance for various protocols within The Graph ecosystem.

Simplify your upgrade process and never miss a deadline again.

Add to your calendar!

Protocol watch

The latest updates on important changes to the protocol

Forum Governance

Graph Horizon Explained: a Proposal for the Evolution of the Protocol

Contracts Repository

Network Subgraphs

We’re now tracking five subgraphs (instead of three). All have been deployed to the decentralized network. Read the full announcement on Discord.

Network Subgraphs
- Core network subgraph:
  - Staging release (v1.0.0)
  - Decentralized network release 1.0.0
- Analytics subgraph:
  - Staging release (v1.0.0)
  - Decentralized network release 1.0.0
- Activity subgraph:
  - Decentralized network release 1.0.0
- Epoch Block Oracle subgraph:
  - Decentralized network release 0.0.2
- Billing subgraph:
  - Decentralized network release 0.0.1

Project watch

Agenda:

Indexer components
Graph Node
Miscellaneous

For more info, check out the presentation slides.

Open discussion

Pinax’s RPC service for indexers

Pinax recently announced it has RPCs available for indexers to use:

Introducing RPC Endpoints from Pinax

Matthew shared the exciting news from Pinax: they’ve made RPC services available for public use. Since Pinax runs RPCs for their own indexer, they thought other people might want to use them too.
Matthew further explained that the RPCs specifically target indexers. Features like mempool and WebSockets, which developers often seek in an RPC, are not enabled. To access the service, you’ll need an API key, which you can obtain from the Pinax app. The best part? It’s currently free! Pinax will see how the usage goes and if people use it a lot, they might have to charge for it.
Matthew said the list of chains that is supported is smallish at the moment, but they will be adding more. They have a lot more RPCs running than are listed on the website, so if you want a chain that’s not listed, please reach out. Pinax is looking for feedback on the service, so please let them know about your experience.
Marc-André [Ellipfra] shared in the chat: Fantastic option for failover or sporadic use! Thanks for making this available. But it’s best for a data integrity pov that most indexers use their own.
- Matthew agreed it would not be good for people to rely on a single point for RPCs, as that would not be helpful for the network.
Mickey [Edge & Node] shared in the chat: We can bring Muhammad to a future IOH since he has hands-on experience with this [service] on our side.

Matthew leads a discussion on EIP-4844

EIP-4844 is a way to store blobs without bloating the main execution layer chain, so blobs are going to be handled by the consensus layer. They’re only going to be stored for 18 days, and as indexers, we need to be able to replay old subgraphs, so we need old data.
You can change your consensus layer client so that it stores blobs forever, but Pinax wanted to do something more than that.
Pinax has built a “beta” Firehose Beacon using StreamingFast’s Firehose RPC poller stack.
- Source code: https://github.com/pinax-network/firehose-beacon
- Protobuf: https://buf.build/pinax/firehose-beacon
And two Substreams:
- https://github.com/pinax-network/substreams/tree/develop/eth.blobs
- https://github.com/pinax-network/subtivity-substreams/blob/main/substreams.beacon.yaml
- Note: development is ongoing, contact Matthew for the latest info and access.
Also need:
- https://github.com/pinax-network/subtivity-substreams

For more info from this discussion, including workflows, see the presentation slides.

Pinax wants to know how you want to use blobs! What kind of blob APIs do you want?

Questions

Q1: What are blobs going to be used for?

Answer: Rollups are the obvious answer because that’s what they’re designed for. Someone’s probably going to do inscriptions with them. Who knows what other uses are coming?

Q2: If indexers don’t sync the blobs at hf [hard fork], can they get full blob history, or do the consensus clients need to add something in if, say, you started six months from now?

Answer: If you’re not saving the blobs at the time of the hard fork, there’s no way to get them from syncing to the peer-to-peer network.

You need to get them from somebody who has them, so either, you know, Pinax’s Firehose, or some other RPC provider, or get a backup from somebody, you know, a snapshot, but yeah, you cannot sync them from the network, so somebody will have to send them to you.

The idea behind the blob-compatible RPC endpoint is to use a proxy server within your environment. This proxy can handle regular requests by directing them to your normal Beacon node. However, for blob requests that you don’t have, the proxy forwards them. This approach allows your normal consensus layer queries to proceed as usual while the proxy handles blob-specific requests. The API was designed with this purpose in mind.

Can share merged blocks as well. Sharing blobs via File Hosting Service (once it’s ready) will work too.

API keys, Blobs, firehose, indexers, RPC endpoints, subgraphs, substreams, The Graph

The Graph Indexer Office Hours #146

Opening remarks