Reddit AMA Deep Dive: Unpacking The Graph’s Vision for a New Web3 Era

Events By Dec 18, 2023 No Comments

Hey Graph’ers! The recent AMA (Ask Me Anything) on The Graph’s transformative New Era brought together core development team members and the community, sparking an insightful dialogue on the future of decentralized data and web3 development.

Here’s a recap of the key discussions, tailored to our tech-savvy audience of developers and indexers.

Event overview

Dates: November 28-30, 2023

Format: Continuous Reddit Q&A over three days

Participants: Core dev team members, community members, moderators

Focus: The Graph’s New Era roadmap and its impact on web3

Core team insights

  • Adam Fuller (Edge & Node): Discussed product management aspects and future developments.
  • Alex Bourget (StreamingFast): Shared insights on technological advancements and upcoming features.
  • Chris Wessels (GraphOps): Highlighted the importance of File Data Service for indexers.
  • Daniel Keyes (Pinax): Spoke about protocol evolution and supporting more blockchains.
  • Eva Beylin (The Graph Foundation): Emphasized community engagement and protocol development.
  • Sam Green (Semiotic Labs): Provided updates on deploying SQL and LLMs in The Graph Network.
  • Uri Goldshtein (The Guild): Discussed contributions to protocol efficiency and developer tooling.
  • Vincent Wen (Messari): Focused on expanding subgraph coverage and SQL integration.
  • Yaniv Tal (GEO): Shared visions for the interconnected graph of data and knowledge graphs.

Detailed Q&A highlights

Q: As a Graph Advocate, how can I stay updated with the roadmap and practically implement new features, while also assisting other Advocates and communities? (Asked by jagtapyash2512)

Q: How soon can we expect the implementation of ClickHouse SQL in The Graph, considering it relies on Substreams and Firehose? Also, is an early release of “event substream” feasible for OP stack chains using RPC, before the full Firehose deployment? (Asked by Drewsapple)

  • A: Sam Green from Semiotic Labs outlined the current status of integrating Substreams with ClickHouse and developing the SQL API. He mentioned active work on database tooling (DBT) experiments and plans to have SQL queries on the network by Q1 2024.
  • A: Daniel Keyes, CEO of Pinax, provided further insights into SQL data services. He explained that Pinax is exploring ways to deploy these services efficiently and in a modular fashion, in collaboration with StreamingFast and Semiotic Labs. Regarding the Firehose development, he mentioned that Pinax is adding RPC nodes for various EVM chains, with a current list of supported blockchains. Additionally, the StreamingFast team is creating a “light” version of Firehose that requires less extensive instrumentation.
  • A: Alex Bourget from StreamingFast addressed the issue of developers spending time on data aggregation during transformation steps. He suggested focusing on Substreams to make this process simpler and announced that StreamingFast is working on:
    • A WASI-compatible target to allow the use of multiple programming languages, not just Rust.
    • Code generation tools to speed up development, such as automatic building of dynamic data source patterns, which you can read about in the Substreams CLI change log.
    • The development of DSLs in Rust to enable more efficient coding.
    He also discussed the importance of the SQL sink, which you can learn more about at the SQL sink repository on GitHub. This sink has features like a high throughput injector and reorg navigation for Postgres and ClickHouse, and it’s being developed into a deployable unit for The Graph Network, demonstrated in the Substreams SQL tutorial. Bourget mentioned the recent completion of a significant milestone with the new RPC poller for Firehose Ethereum, detailed in this GitHub issue, and the backfilling of the Arbitrum network as a case study of this new approach. StreamingFast has also started syncing Bitcoin Core with this method and plans to address OP stack chains next. The team aims to speed up chain coverage and simplify the extraction process, as described in the Firehose extractor GitHub issue.

Q4: What goals or accomplishments do you hope to be completed in a year, and what are some stretch goals to aspire to in future years? (Asked by PaulieB79)

  • A: Chris Wessels from GraphOps expressed enthusiasm for The Graph’s shift towards a diverse data services marketplace. Their key focus is the File Data Service (FDS) for monetizing and sharing file data, especially beneficial for indexers. In a year, the aim is active FDS usage for increased efficiency and revenue. Additionally, they’re developing tooling to enhance FDS user experience. GraphOps is also updating Launchpad (Launchpad intro), their Kubernetes toolkit, to include Firehose and Substreams support. Plus, they’re eager about Graphcast’s contributions to data determinism and network intelligence (Graphcast intro), reflecting broader core developer ecosystem advancements.
  • A: Sam Green from Semiotic Labs outlined key objectives for the year. They plan to deploy Scalar TAP, enhancing The Graph’s micropayment system once hosted service subgraphs are upgraded to The Graph Network. Another goal is to introduce SQL for analytics queries as a new data service in The Graph Network, as detailed in this Reddit thread. Deploying open-source large language models to the network as a new service is also on the agenda. The team is working on implementing a verifiable Firehose. As a stretch goal, Semiotic aims to deploy verifiable SQL queries, which would allow The Graph’s data to be used for critical applications, such as tax accounting. Read more on the Timeline Aggregation Protocol.
  • A: Vincent Wen from Messari highlighted their ongoing efforts to expand subgraph coverage in both scope and detail, including support for unique protocols like FriendTech. They’re also enthusiastic about integrating SQL capabilities into The Graph ecosystem, aiming to enhance analytics use cases by building upon StreamingFast’s and Semiotic’s work and exploring composable SQL-based subgraphs.
  • A: Daniel Keyes, CEO of Pinax, envisions The Graph supporting a wider range of blockchains and data sources by next year. He anticipates The Graph will achieve significant network effects and service levels, surpassing centralized providers. This growth is expected to be fueled by a collaborative ecosystem of developers, service providers, and community members. For more insights into Pinax’s contributions to this vision, he suggested visiting their blog.

Q5: How will The Graph address cross-chain data consistency in the evolving ecosystem, and what are the specific plans for integrating an open-source large language model, possibly in collaboration with OpenAI or for AI-based on-chain data queries? (Asked by Proxima-95803)

  • A: Sam Green discussed the integration of open-source Large Language Models (LLMs) into The Graph, aiming to enable natural language queries. He highlighted the early stages of this service and the goal of fine-tuning LLMs for effective GraphQL generation.
  • A: Alex Bourget from StreamingFast discussed their work on the Substreams:SQL initiative, focusing on merging multiple data streams into a single database, which could enable joining data from various chains. He mentioned an ongoing effort to develop deployable units and referenced the dbt layer in their tutorial. Bourget also noted plans to research verification layers for these technologies, including SQL verification, runtime verification for Firehose and Substreams, and economic security through cross-validation.

Closing thoughts

The AMA session illuminated The Graph’s commitment to shaping a future where decentralized data plays a pivotal role. For developers and indexers, these discussions offer a glimpse into the exciting possibilities and challenges ahead in the web3 space.

Continue exploring these discussions and more at The Graph’s official subreddit. Visit the various links provided throughout the blog for a more comprehensive understanding.

Author

I am a dedicated member of the Graph Advocates DAO and proud to be a part of the Graphtronauts community. As a passionate crypto investor and enthusiast, I have delved into the world of decentralized technologies, with a strong focus on Graph Protocol. My journey includes writing insightful blogs for Graphtronauts and contributing to the development of Subgraph documentation for various projects within the Graph Protocol ecosystem. Most recently, I have taken on the role of a Pinax technical writer, further expanding my commitment to advancing the adoption and understanding of blockchain and graph-based technologies.

No Comments

Leave a comment

Your email address will not be published. Required fields are marked *