Menu

The Graph Indexer Office Hours #187

Events By Dec 13, 2024 No Comments 
TL;DR: Rem from Edge & Node discussed the proposal for evolving The Graph protocol economics. The proposal aims to better meet participant needs, with this discussion focusing on consumers and providers (indexers), particularly aligning incentives with value delivery and transitioning to a demand-driven network. A key change involves having gateways play a larger role in matching indexers to consumer demands rather than having indexers try to predict consumer demand.

Opening remarks

Hello everyone, and welcome to episode 187 of Indexer Office Hours!

GRTiQ 198

Catch the GRTiQ Podcast with Sean Robb, Technical Product Lead at Flow. Based out of St. Louis, Sean’s journey into web3 started with his background in computer engineering, where he worked on flight control systems before moving into roles at CenturyLink and MasterCard.

At Dapper Labs, he helped shape the Dapper Wallet during the NBA Top Shot craze, driving the evolution of consumer-facing applications in web3.

⬇️ Skip straight to the open discussion ⬇️

Repo watch

The latest updates to important repositories

Execution Layer Clients

  • Celo: New release v1.8.7
    • Date: 2024-12-06 02:45:00 UTC
    • Celo v1.8.7 sets the L2MigrationBlock for Alfajores testnet (already migrated but to keep a clean client). Can be skipped for mainnet.
    • Urgency indicator: Green
    • Urgency reason: Low-priority update for convenience.

Ana | GraphOps: An issue with Heimdall came up last week. The newest fork, Polygon Amoy, has been broken for many of us, so we had to resync from a snapshot and use an external bor RPC while everything was resyncing. Here’s the bug: Amoy testnet heimdall 1.0.10/1.1.0-beta broke before jorvik-hardfork height, so do check it out if you’ve had this issue. Use the latest version, resync from a snapshot, and then you should be good.

Protocol watch

The latest updates on important changes to the protocol

Forum Governance

Forum Research

Core dev updates:

Contracts Repository

Project watch

Graph Node Update

Mickey | The Graph | E&N posted: Marko is working on some bug fixes in graph-node by the way. Should hopefully get those pushed out soon-ish.

Open discussion [6:53]

Today, we have a conversation with Rem from Edge & Node about GIP-0070: Evolving The Graph Protocol Economics (2024). He’ll walk us through parts of his draft proposal for the next phase of The Graph protocol, which weaves together lessons from the original protocol and insights gained during the Graph Horizon evolution.

The Forum post: Evolving The Graph Protocol Economics (2024)

Note: Some of the content below is from the post and GIP, and some is from the discussion. Comments have been lightly edited.

Why are we looking at protocol evolution?

Rem | Edge & Node: I’m curious to hear from the group in the chat why you think we’re looking at protocol evolution?

Derek | Data Nexus: More services, more money.

Payne | Stake🦑Squid: Number go up?

AbelsAbstracts.eth | GraphOps: Many aspects of the protocol could work better – curation as an example.

  • Rem: Yes, curation is definitely one of the things that has been a hot discussion topic.

Marc-André | Ellipfra: Good things evolve.

  • Rem: I think that’s a really valid point actually, and it’s a topic that’ll come up again because I think the protocol that we’ve got has served us pretty well, but there are things that now need to change, otherwise it won’t keep serving us as well as it has in the past.

AbelsAbstracts.eth | GraphOps: Very crowded market and we need to compete.

Pierre | Chain-Insights.io: Efficiencies, features, improvements, fixing issues.

Ricky: Aligning incentives with valuable work being done, and creating self-sustaining models.

  • Rem: Incentive alignment is one of the key themes as you’ll see.

AbelsAbstracts.eth | GraphOps: Data indexing is super competitive now – we need a SaaS service to compete.

Derek | Data Nexus: In the initial stages of the protocol, our incentive models didn’t have the benefit of hindsight. Evolving can better suit participants and intended behaviors.

Rem: I’m getting a sense of the themes, and they are basically the right themes, but I’ll frame it slightly differently as I saw it. About a year ago when we were discussing Horizon and The Graph protocol, Horizon had identified some key issues with the original protocol, some things that needed to be improved. As we started discussing how that would be implemented, it also brought up some things to the surface, like what’s going to happen to curation? What’s going to happen to delegation? Because there’s an implication that these things were going to change, but it wasn’t yet clear how they were going to change.

I don’t know how many people have had a chance to read the forum post yet. It’s a reasonably long post, and it’s at a high level, so it covers lots of points and doesn’t actually, in my opinion, go into all the details of them, and we’re expecting more of the details to come later.

Participant Needs

Rem: I want to start talking through some of the participants and what their roles are in the network. To explore the protocol evolution for participants, their needs, and how to evolve to meet those needs.

  • Consumer – uses The Graph Network to satisfy data needs, querying data through a gateway
  • Provider – a more generic term for indexer, but there are some data services we run that don’t really fit neatly into the indexer framing
  • Gateway – manages relationships between many data consumers and many indexers
  • Contributor – subgraph developers are a primary example
  • Delegator – allocates GRT tokens to indexers, supporting their operations in exchange for a share of the rewards earned from query fees and indexing rewards
  • Curator – signals on specific subgraphs by staking GRT tokens, which indicates the quality and usefulness of a subgraph to indexers
  • Sponsor – person or party that is paying for the subgraph to be indexed: it might be a gateway, a consumer, another indexer

From the chat:

Mickey | The Graph | E&N: What is the difference between consumer and sponsor?

Derek | Data Nexus: Consumers are paying based on the queries they send. Sponsors are funding a payment stream to gain indexer support regardless of traffic.

Mickey | The Graph | E&N: hmm okay. So you see those other roles as paying for indexing?

Derek | Data Nexus: Imagine a DAO wants to pay for a few indexers to support a subgraph so that third-party consumers can then query it, but they cover the query fee traffic. I’d be willing to bet that 90+% will be paid by the person who intends to query the subgraph.

Rem elaborated: A consumer is someone who is using The Graph Network to satisfy the data needs, so broadly speaking, they’re typically querying data, doing so through a gateway. A sponsor is someone who’s paying for indexing. I consider a sponsor more of a role than a persona, so multiple participants might be sponsors at different times.

Consumers

Rem: I’ll start briefly looking at consumers. At any stage, give me your thoughts if you think I’m missing anything or if you disagree with something. This is very much intended to be an exploration to make sure that we as a community find the best answer here, and I’m sure there’ll be some aspects of change.

Key considerations for satisfying consumer needs and optimizing their value:

  1. Conveniently matching individual consumer needs: Providing the best available match for each consumer’s specific requirements, alongside the right information and support to help them maximize value. (The right information, at the right time, in the right way.)
  2. Consumer payments are a tangible metric of value delivered: Payment for services is the most direct and measurable confirmation of value delivered to consumers.
  3. Enabling consumer contribution and role progression: The consumer experience is enhanced when consumers can also contribute value back to the network and potentially take on additional roles, increasing the value of the network for all participants. This would be laying some of the foundations we need to make the entire user community more collaborative so that we get more reuse and network effects.
  4. Facilitating reuse to boost efficiency and value: Reuse of existing captured value (rather than duplication) improves network efficiency and consumer value. It also encourages collaborative improvement and maintenance, further enhancing consumer value. This can also improve accessibility, expand the consumer base, and thereby further increase network value for all participants.
  5. Expanding the value proposition to serve additional consumers: We should not limit our focus to our current consumer base or our current solutions. The protocol and ecosystem should evolve to better serve both existing and new consumers.

Providers (Indexers) [21:31]

For both indexers and The Graph Network to thrive, two critical indexer profiles must be facilitated, incentivized, and balanced:

An indexer goes through two phases in their lifecycle:

  1. New indexers joining the network: The health of The Graph Network relies on new indexers being able to join, prove themselves, and transition into established indexers. We’re constantly trying to make the technical barrier to entry as low as possible, but this is just a starting point.
  2. Established indexers operating in the network: The network depends on experienced indexers providing high-quality service to maintain the network’s efficiency and performance. From the perspective of the established indexers, they’re coming here so that through their expertise, they can earn some reward.

I’m going to focus on what I think matters from the point of view of the established indexers, and at the end, it goes back to how you join the network and become an established indexer.

Established Indexers and Consumer Demand

As the network matures, indexer income will increasingly depend on external consumers paying for service (typically to gateways rather than directly to indexers). Such income depends on:

  1. The value created and delivered to consumers: The more value we create for consumers, the more they will be willing to pay for services, thereby increasing indexer income.
  2. How well individual consumers are matched to indexers: Consumers need to be matched with indexers that can provide the best value for their needs. The gateways and the rest of the network match consumers to the most suitable indexer. For example, if you are in Southeast Asia, then an indexer on the other side of the world is not going to have the best latency.
  3. The uniqueness of consumer value created by The Graph: This uniqueness enhances the competitive position of both the network and individual indexers in comparison to other data sources.

Efficiency and Profitability of Indexers

The margins and profitability of indexers will also increasingly depend on how efficiently they provide services. This efficiency is determined by:

  1. Efficient allocation of indexing resources: Indexing must be efficiently distributed to satisfy demand, minimizing unnecessary costs for indexers.
  2. Accurate matching of indexers to consumer needs: The better indexers are matched to the needs of consumers, the more value they can deliver.
  3. Efficient use of GRT to provide services: Efficient management of resources, including GRT, will play a key role in maintaining profitability.

Additional Dependencies for Maximizing Income and Efficiency

Several additional factors will influence the success of indexers:

  1. The overall quality of service provided by the network: High-quality service across the network will help maintain a positive reputation and attract more users.
  2. The reliability and trustworthiness of services and data: Consumers need to trust that the data and services they receive are accurate and reliable.
  3. The ease of permissionless entry for new indexers: Entry into the network must remain permissionless, aligning with our decentralized principles of trustless operation. Additionally, it should be as easy as possible, both technically and operationally, minimizing barriers to entry. This is critical for maintaining network diversity, scaling to meet growing demand, and fostering innovation.
  4. Opportunities for specialization: Indexers should have the chance to specialize and optimize their services for particular niches, improving efficiency and value.
  5. The effectiveness of The Graph in attracting and retaining users: The more users The Graph can attract and retain, the greater the demand for indexer services.

Aligning Incentives with Value Delivery

Rem: Some of these are a bit different from how you might have thought of things with the protocol so far.

Under the original protocol, indexers were expected to determine what indexing to perform based on curation signals. However, with the transition to Sunrise and increasing demand, this approach no longer serves indexers well for several reasons:

  1. Indexers are not well-positioned to predict consumer demand. The network needs to do more to make sure that indexers are matched to the consumers who actually see value in that indexer’s service. It’s not the role of an indexer to figure out what queries are going to happen on which subgraphs.
  2. Curation signals are not reliable predictors of query demand for any particular indexer.
  3. Curation signals do not facilitate the efficient allocation of indexing resources to match consumer demand.

Transition to a Demand-Driven Network

We don’t really have a demand-driven network. We’ve done a really good job collectively of getting such a good network in place, and indexers should take a huge amount of credit for that. I think you have been really good partners and supporters of the whole ecosystem and it’s fantastic that we’ve got this far. But we need to transition to the network being reliant on the subsidy of issuance to actually the network being driven by external income because that’s what ultimately will bring us the most success.

  1. Maintain permissionless entry: The easier it is to join the network and compete, the better for the overall health of the network. A mature understanding of this means acknowledging that we cannot guarantee indexer profitability, but we can design the network to seek an equilibrium that attracts and sustains quality indexers.
  2. Build reputation and relationships: Indexers should have opportunities to establish reputations, build service relationships, and fairly compete with other providers.
  3. Match indexers to useful work: Indexers should be matched to the work that maximizes the value of their services, allowing them to focus on excelling in service provision.
  4. Recognition and rewards for service excellence: Indexers providing excellent service should be recognized and rewarded accordingly, further incentivizing high-quality service provision.

Future Considerations

  • The original indexing rewards system served the network well during its early stages.
  • However, as the network matures and becomes more demand-driven, indexers will no longer be well served by the current mechanisms.
  • We must align incentives to focus on delivering real value to consumers and ensure that indexers are rewarded based on their contribution to that value.

Questions [42:32]

Pierre | Chain-Insights.io: Curators are not working, according to Yaniv in the Graph Builder Office Hours. But in the proposal it seems the opposite.

Rem: The way curation is working is being changed as part of this proposal. Curation as it was, was flawed. I’ve seen suggestions to get rid of curation, and actually, at one stage, from a terminology point of view, I did. My earlier version of the proposal didn’t have the curator role, but we realized this new role actually shares a lot of the characteristics of curation, and it’s working as it should, and it’s probably better to still call it curator, but just upgrade the mechanism.

Pierre | Chain-Insights.io: Curation, in my opinion, should be in the protocol to determine the subgraph to index and how many indexers are required to fulfill the demands.

Rem: Exactly, and curation at the moment doesn’t really specify very effectively how much and another flaw is that it doesn’t necessarily find the right indexers to match the consumer’s demand.

Gateways

Gateways play a crucial role in The Graph Network by conveniently matching the supply of data to consumer needs. Their success depends on how effectively and conveniently they meet the needs of their customers.

The success of gateways depends on:

  1. The health of The Graph Network: Gateways rely on the availability of high-quality indexers to provide the services needed to fulfill consumer demand.
  2. Effectively incentivizing indexers: Gateways must be able to incentivize indexers to index and serve the data required to meet their consumer needs.

Rem: I’m going to skip over the rest of these and give you a slightly different view.

To review the rest of the participants (Contributors, Delegators, Curators, and Sponsors), refer to the post, Evolving The Graph Protocol Economics (2024) > Contributors.

Design Principles [45:37]

The overall approach focuses on creating a coherent protocol upgrade that, based on the network’s operating principles, optimizes participation for consumer value creation. The diagram below summarizes the scope of the proposal. Rem covered the first two columns, Goal and Protocol Aspect.

A diagram with four columns: Goal, Protocol Aspect, Principle, and Proposed Mechanism. Most significant for this session are the 3 goals: maintain network balance, enhance incentive alignment, and sustain continuous improvement.
Source: GIP-0070: Evolving The Graph Protocol Economics (2024)

Maintain Network Balance

The Graph protocol holistically balances global incentives across all network activities. It is crucial to maintain this established global incentive balance during protocol upgrades.

Issuance distribution maintains indexing and network health

Rem: There are some things in the original protocol that we want to maintain, and this is one of them. That there is a good, sound way of distributing issuance.

Integrated role incentives to holistically balance network

Rem: The network is actually designed as a set of integrated mechanisms, not just one or two mechanisms working or a set of mechanisms working in isolation. Curation works in combination with indexing rewards, etc., all the pieces fit together, and so as we upgrade, the whole protocol has to maintain that balance.

Enhance Incentive Alignment

Where incentive alignment gaps exist, we aim to enhance the overall incentive structure of The Graph protocol. Our goal is to ensure local decisions are more aligned with creating consumer value while extending incentive coverage across all network contributions that generate consumer value.

Network rewards are not linked to consumer value delivered

While indexing rewards incentivize global indexing supply, and curation signals interest in subgraphs and incentivizes indexing, rewards are not linked to the quality of service delivered to users at query time. Curation neither provides predictable indexing nor effectively matches consumer needs.

Limited incentives for network contribution to consumer value

Currently, query fee cuts incentivize and reward curation; however:

  1. There is no equivalent mechanism for data services or indexing payments.
    1. No corresponding cut exists to incentivize indexing payments or reward indexing signals.
    2. The data services framework neither requires nor facilitates such a cut.
  2. The mechanism is inflexible and tightly coupled to subgraph indexing.
    1. It could potentially apply to other data modules but less easily to other capability-building use cases.
    2. The fixed cut proportion is typically suboptimal, creating either inadequate incentives or excessive overhead.
  3. It provides limited coverage of the value network required to create or enhance value.
    1. For example, it does not incentivize creators and maintainers of subgraphs.
    2. It also fails to incentivize user support or collaboration.

Sustain Continuous Improvement

The Graph Network has successfully established itself, with The New Era of The Graph demonstrating a continued trajectory of decentralized value creation. In upgrading The Graph protocol’s incentive structure, we should anticipate and build into the protocol long-term provisions for sustaining healthy ongoing innovation.

To Wrap Up

Rem [52:53]: There are really three areas of focus:

  1. Maintain the network balance: There are some things about the original protocol that work and we have to carry on and make sure that’s still the case in the upgraded protocol.
  2. Enhance incentive alignment: We know the incentives are always as well aligned as they should be and that’s a key theme that’s informing a lot of the decisions.
  3. Sustain continuous improvement: We have to design the network to keep growing and innovating over time, and the way we think about this protocol evolution is not just what we need over the next year or two but making sure the protocol is set up for the long term.
A diagram showing the 3 goals with their corresponding protocol aspect: maintain network balance with issuance distribution and integrated role incentives, enhance incentive alignment with rewards linked to consumer value and network contribution to consumer value, and sustain continuous improvement with balance issuance allocation and establish long-term model for network improvement.
Protocol evolution goals

Questions [47:41]

Derek | Data Nexus: Is there a goal to abstract the decision away from the indexer in terms of which subgraphs they index and serve queries on?

Rem: Essentially, yes, because gateways are the aggregators of demand, and other sponsors can play and curators can play in the game as well, but I’ll describe it mainly from the gateway point of view. Gateways aggregate demand from many users and gateways are in a good position to match the right indexer for their particular consumer, and that will be a combination of factors including geography but also other criteria like different price sensitivity or different data services that are required.

Derek: That decision process that the gateway is using to determine which indexer is going to actually serve the queries, is that something that will be periodically reevaluated, or is that something that is set up once and then that person just kind of sits there? What does that look like right now?

Rem: We don’t know all of the details yet, and that’s both for how we start and how it evolves over time. I do think we’ll need to find a mechanism quite carefully and get input in there, but we’re also going to need to see it running and see where it needs adjusting. It will be dynamic in nature. There are two different selection sources, one of them is the protocol itself from issuance and it has different criteria from a gateway acting on behalf of users, so the protocol will be looking very much at the health of the network and enabling any indexer to come onto the network, it won’t be so latency-sensitive, for example. It’s more interested in making sure that new indexers can join the network, whereas the gateway will be looking at the track record of what’s been served and how that best satisfies the known demands from that consumer.

Derek | Data Nexus: Ahh, I see. So the ‘dips’ payments will have slightly different criteria and agreement compared to the gateways query routing algo.

[54:00]

Mickey | The Graph | E&N: Rem, what is the churn rate of prospective or new indexers who gave up? Have we interviewed them to understand what caused them to leave (e.g., startup capital vs attracting delegation vs infra costs, etc.)?

Rem: The honest answer to that is I don’t know, it’s a very interesting question. I’d love to know the answer. It’s possible other people do, but I don’t.

Mickey | The Graph | E&N: Ok, we should look into that. Rem, how will we address the issue of indexers earning indexing rewards but not serving any queries (in the spirit of adding value to consumers)?

Rem: Basically, the quality of service of an indexer is taken into account. Essentially, quality of service becomes a factor and if you’re not serving any queries, then the network will not be requesting you to do a lot of work. You’ll always have an opportunity, but you won’t get a lot of work.

Pierre | Chain-Insights.io: Are there going to be more IOH on each topic to talk in-depth?

Rem: I’ll be doing as many Indexer Office Hours as you want me to because there are a series of other improvement proposals. This broad one is actually not really posing all that much but it’s an agreement on intent saying this is the shape of the target, but there will be subsequent GIPs to deal with the individual parts.

AbelsAbstracts.eth | GraphOps: Yes sir. We’re planning this now.

Derek | Data Nexus: Is this proposal something that will go to TAB & The Council, or will this be a stakeholder vote? Or both?

Rem: We’ll go to The Council. I don’t know the exact details of that process. I would be very surprised if The Council did anything on this without understanding community sentiment. I expect that they would look at sentiment before taking a vote on this.

Mickey | The Graph | E&N: What is that threshold for calculating an indexer’s Quality of Service (QoS) score? If an indexer collects a ton of indexing rewards but serves queries on one tiny subgraph, will the network view them as meeting the QoS bar? Or do they need to serve above some level of queries per cycle?

Rem: We’ve already played around with a few different algorithms and a few different ways of doing this, but we do not have a final answer. That’s something we want to do openly with the community is figure out the scoring. It will not be as trivial as you get a high quality service on one subgraph, right, your reputation will accumulate based on how much you’ve proven yourself over time. The importance of self-stake delegation won’t go away, but effectively, quality of service will help determine almost like a multiplier. When an indexer first comes on the network, they haven’t proven themselves, so they don’t get a very high multiplier, but as they start improving themselves, the multiplier increases, so they get more work.

, , ,
Author

We're a web3 service provider specializing in blockchain indexing operations. Our mission is to enable creators to achieve their true potential with web3 technology. We want to help developers reliably access blockchain data in a consistent format so you can create amazing experiences for your applications.

No Comments

Leave a comment

Your email address will not be published. Required fields are marked *