Hey Graph’ers! The recent AMA (Ask Me Anything) on The Graph’s transformative New Era brought together core development team members and the community, sparking an insightful dialogue on the future of decentralized data and web3 development.
Here’s a recap of the key discussions, tailored to our tech-savvy audience of developers and indexers.
Event overview
Dates: November 28-30, 2023
Format: Continuous Reddit Q&A over three days
Participants: Core dev team members, community members, moderators
Focus: The Graph’s New Era roadmap and its impact on web3
Core team insights
- Adam Fuller (Edge & Node): Discussed product management aspects and future developments.
- Alex Bourget (StreamingFast): Shared insights on technological advancements and upcoming features.
- Chris Wessels (GraphOps): Highlighted the importance of File Data Service for indexers.
- Daniel Keyes (Pinax): Spoke about protocol evolution and supporting more blockchains.
- Eva Beylin (The Graph Foundation): Emphasized community engagement and protocol development.
- Sam Green (Semiotic Labs): Provided updates on deploying SQL and LLMs in The Graph Network.
- Uri Goldshtein (The Guild): Discussed contributions to protocol efficiency and developer tooling.
- Vincent Wen (Messari): Focused on expanding subgraph coverage and SQL integration.
- Yaniv Tal (GEO): Shared visions for the interconnected graph of data and knowledge graphs.
Detailed Q&A highlights
Q: As a Graph Advocate, how can I stay updated with the roadmap and practically implement new features, while also assisting other Advocates and communities? (Asked by jagtapyash2512)
- A: Paolo Diomede from The Graph Foundation recommended focusing on specific roadmap areas, leveraging resources like the weekly Graph Builders Office Hours, Indexer Office Hours, and the Graph Core Dev calls. He emphasized the importance of staying updated through the monthly newsletter (subscribe at thegraph.com), participating in the GIP process, and checking the calendar.
- A: Chris Ewing from Pinax chimed in to suggest Discord as a great resource.
- A: Alex Bourget recommended keeping an eye on the change log for Substreams and the StreamingFast Medium blog.
Q: How soon can we expect the implementation of ClickHouse SQL in The Graph, considering it relies on Substreams and Firehose? Also, is an early release of “event substream” feasible for OP stack chains using RPC, before the full Firehose deployment? (Asked by Drewsapple)
- A: Sam Green from Semiotic Labs outlined the current status of integrating Substreams with ClickHouse and developing the SQL API. He mentioned active work on database tooling (DBT) experiments and plans to have SQL queries on the network by Q1 2024.
- A: Daniel Keyes, CEO of Pinax, provided further insights into SQL data services. He explained that Pinax is exploring ways to deploy these services efficiently and in a modular fashion, in collaboration with StreamingFast and Semiotic Labs. Regarding the Firehose development, he mentioned that Pinax is adding RPC nodes for various EVM chains, with a current list of supported blockchains. Additionally, the StreamingFast team is creating a “light” version of Firehose that requires less extensive instrumentation.
- A: Alex Bourget from StreamingFast addressed the issue of developers spending time on data aggregation during transformation steps. He suggested focusing on Substreams to make this process simpler and announced that StreamingFast is working on:
- A WASI-compatible target to allow the use of multiple programming languages, not just Rust.
- Code generation tools to speed up development, such as automatic building of dynamic data source patterns, which you can read about in the Substreams CLI change log.
- The development of DSLs in Rust to enable more efficient coding.
Q4: What goals or accomplishments do you hope to be completed in a year, and what are some stretch goals to aspire to in future years? (Asked by PaulieB79)
- A: Chris Wessels from GraphOps expressed enthusiasm for The Graph’s shift towards a diverse data services marketplace. Their key focus is the File Data Service (FDS) for monetizing and sharing file data, especially beneficial for indexers. In a year, the aim is active FDS usage for increased efficiency and revenue. Additionally, they’re developing tooling to enhance FDS user experience. GraphOps is also updating Launchpad (Launchpad intro), their Kubernetes toolkit, to include Firehose and Substreams support. Plus, they’re eager about Graphcast’s contributions to data determinism and network intelligence (Graphcast intro), reflecting broader core developer ecosystem advancements.
- A: Sam Green from Semiotic Labs outlined key objectives for the year. They plan to deploy Scalar TAP, enhancing The Graph’s micropayment system once hosted service subgraphs are upgraded to The Graph Network. Another goal is to introduce SQL for analytics queries as a new data service in The Graph Network, as detailed in this Reddit thread. Deploying open-source large language models to the network as a new service is also on the agenda. The team is working on implementing a verifiable Firehose. As a stretch goal, Semiotic aims to deploy verifiable SQL queries, which would allow The Graph’s data to be used for critical applications, such as tax accounting. Read more on the Timeline Aggregation Protocol.
- A: Vincent Wen from Messari highlighted their ongoing efforts to expand subgraph coverage in both scope and detail, including support for unique protocols like FriendTech. They’re also enthusiastic about integrating SQL capabilities into The Graph ecosystem, aiming to enhance analytics use cases by building upon StreamingFast’s and Semiotic’s work and exploring composable SQL-based subgraphs.
- A: Daniel Keyes, CEO of Pinax, envisions The Graph supporting a wider range of blockchains and data sources by next year. He anticipates The Graph will achieve significant network effects and service levels, surpassing centralized providers. This growth is expected to be fueled by a collaborative ecosystem of developers, service providers, and community members. For more insights into Pinax’s contributions to this vision, he suggested visiting their blog.
Q5: How will The Graph address cross-chain data consistency in the evolving ecosystem, and what are the specific plans for integrating an open-source large language model, possibly in collaboration with OpenAI or for AI-based on-chain data queries? (Asked by Proxima-95803)
- A: Sam Green discussed the integration of open-source Large Language Models (LLMs) into The Graph, aiming to enable natural language queries. He highlighted the early stages of this service and the goal of fine-tuning LLMs for effective GraphQL generation.
- A: Alex Bourget from StreamingFast discussed their work on the Substreams:SQL initiative, focusing on merging multiple data streams into a single database, which could enable joining data from various chains. He mentioned an ongoing effort to develop deployable units and referenced the dbt layer in their tutorial. Bourget also noted plans to research verification layers for these technologies, including SQL verification, runtime verification for Firehose and Substreams, and economic security through cross-validation.
Closing thoughts
The AMA session illuminated The Graph’s commitment to shaping a future where decentralized data plays a pivotal role. For developers and indexers, these discussions offer a glimpse into the exciting possibilities and challenges ahead in the web3 space.
Continue exploring these discussions and more at The Graph’s official subreddit. Visit the various links provided throughout the blog for a more comprehensive understanding.
No Comments