Enterprise Software for Data Analytics
Historically, one of the key ways of creating leverage in an organization was through enterprise software - transforming manual, error-prone processes into more systematic, robust processes. At scale, process improvements in an organization provide immense business benefit in the form of increasing revenue and decreasing cost.
In the past decade, data analytics infrastructure and applications have transformed how organizations operate. Cheap, flexible cloud-based data warehouses enabled businesses of all sizes to analyze all of their business and product telemetry. Best practices around cloud-native ETL (rebranded as “ELT”) developed, allowing data ingestion and transformation pipelines to be more robust, scalable, and composable. And finally, easy-to-use business intelligence tools allowed data analysts of all skillsets to create data dashboards, primarily designed to summarize high-dimensional information in a compact, intuitive way. As of today, the data dashboard remains king as the primary interface of choice for last-mile analytics delivery, the interface for data insights.
LLMs disrupt data interfaces
In the past five years, the data analytics industry has seen innovation in the interface layer for last-mile analytics delivery. New form factors for data insights were developed and marketed at scale including notebooks, data apps, spreadsheets, and canvases. More than five years into this experiment, it has become more clear that none of these new form factors were fundamentally different enough and better enough to take hold as the new primary interface for data insights.
Enter Large Language Models. The introduction of ChatGPT to the world at the end of 2022 caused a (perhaps unexpected, even to its creators) sensation, leading to mass adoption and change of user behavior that we have not seen since the mobile and cloud waves of the late 2000s and early 2010s. Unlike the false chatbot wave of the mid 2010s, this time, the technology works at scale, and users are sufficiently primed to take advantage of this new technology.
Two years into the availability of chat-based AI models, it is becoming more clear that users of data analytics software are gaining a larger appetite for natural language-based systems for self-serving data questions. The proposed explanation is simple yet profound: when direct answers are possible and available, users do not want to go through an intermediary output such as ten blue links or generic data dashboards. Users want direct answers, now.
Thus, for the first time in decades, we believe that a fundamental platform shift will result for data analytics. We believe that the interface of choice for data insights will shift from data dashboards to data agents, which are AI agents that know how to reason about all of one’s enterprise data, agents that communicate to the end user using natural language. This does not mean the death of data dashboards or other form factors, but a shift in primacy, much like the desktop to mobile platform shift.
Introducing Oxy, an open-source framework for agentic analytics
To enable a future where data agents become the primary interface for data insights, we built Oxy, an open-source framework for agentic analytics. The core of Oxy is an open-source, declarative framework built in Rust that is purpose-built for agentic data analytics. Oxy is built with the following product principles: open-source, performant, code-native, declarative, composable, and secure.
We were founded by second-time venture-backed (Khosla) entrepreneurs with research backgrounds from Harvard and MIT who led data and ML initiatives at companies like Airbnb, Wayfair, and QuantCo.
With Oxy, users can build, test, and deploy data agents that can reason about enterprise data. Data agents enable English-to-Analytics experiences for the entire organization, wherein the lingua franca becomes English, not SQL. Oxy provides a rigorous and systematic way for builders to semantically model their data and test their data agents, ensuring the accuracy necessary to utilize data agents in production.
More complex workflows can be hardened into workflows, which are agents chained together with increased determinism. Teams can use Oxy to automate difficult-to-automate reporting workflows traditionally not possible with BI tools that require larger amount of information synthesis. Further, teams can also use Oxy to automate a family of ad-hoc analytics questions around business metrics and their relations to other business events. And finally, Oxy can be used to let businesses embed data agents into their own products, allowing their customers to self-serve data questions in plain English.
Without Oxy, one would have to duct-tape ChatGPT, LangChain, and other generic tools together, or use agents bolted onto old BI tools with limited customizability. Oxy enables a user to build powerful data agents easily and intuitively with a purpose-built, opinionated framework that natively enables greater accuracy and speed of answers.
The Human-to-AI interface
Philosophically, the reason why a framework like Oxy is so powerful is that the AI interface it creates closely mimics existing human behavior, the human-to-human interface. As an analogy, seasoned human data analysts are valuable in an organization because of the degrees of freedom a human-to-human interface provides, as opposed to working with a pure software form factor. The human-to-human interface is superior at being robust against slightly erroneous input and output information. The human-to-human interface is superior at covering a very wide range of question-workflow pairs, starting with the most simple question about a particular metric in a point of time but also expanding into a very complex workflow that entails the synthesis of thousands of findings from hundreds of metrics and their correlations that result in a major business decision. This interface gets to the point directly, rather than using a rudimentary artifact like the dashboard as a repeatable yet poorly-powered intermediary. It is not difficult to envision that a human-to-AI interface that functions accurately and quickly that is engineered to recapture this natural paradigm will swiftly rise in primacy.
In this decade ahead, we will be witnessing the rise of data agents.
