The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
Welcome to the GitLab Product Analytics group direction.
The Product Analytics group vision is in alignment and contributes to the Monitor Stage vision and simply stated is ". . to continue to extend DevOps across its most painful gap - measuring user value." Giving teams the tools they need to stay user-focused can have positive impact on their performance, job satisfaction and productivity as noted in the Accelerate State of DevOps 2023.
When conducting user research about the product analytics space, a common theme we heard is that developers find it hard to add instrumentation code to their app or to successfully gather together the reported data. This could be because they are unfamiliar with their app's architecture, the specific SDK of the instrumentation tool, or because they had to use another tool to configure reporting and monitor results. GitLab is the One DevOps platform and therefore has a unique opportunity to address these pain points that other solutions may have.
One area in which we have an opportunity is to make it easy for developers to add instrumentation code. This is the first step in any analytics workflow. Given that developers represent a key persona we collaborate with, we have a better sense of what they like, do not like, are good at, and what sorts of experiences they prefer. Additionally, they are already familiar with GitLab and many conventions of our product. This gives us the ability to create an approach to adding product instrumentation that is easy and familiar to our users. If developers are excited and easily able to add instrumentation to their apps, they will add more, which means their teams will get more valuable insights. Other functional counterparts, such as Product Managers, will see how easy it is for developers to use GitLab to instrument their apps, so they will also be excited to use GitLab, rather than another tool. We are collaborating with the Analytics Instrumentation Group who are providing interfaces to developers to enable the gathering of insights.
Another opportunity we have is to do the entire analytics reporting and analysis directly within GitLab, rather than requiring users to use another tool. We know from talking to users that accessing another tool is time-consuming and friction filled. Being able to go from reading a GitLab issue to looking at an MR to visualizing the latest Product Analytics data in a single place will make it easy for users to get access to the insights from their data quickly. Because it is so painless to view analytics data, these users will do so more and more often, which means that they get even more value out of GitLab.
In addition to the benefits mentioned previously, our ability to store the user's application code presents a significant longer-term opportunity. By leveraging this contextual information, we can offer more comprehensive analytics solutions than any other product. For instance, we could explore the possibility of suggesting or automatically adding analytics instrumentation code to different parts of the app. While this requires extensive research, one potential outcome could be direct suggestions within merge requests (MRs) along with code snippets to automatically instrument newly added sections of code.
Finally, GitLab's unique capabilities with respect to AI provide opportunities that will be difficult for other companies to replicate. GitLab Duo and other AI features will give us the ability to help address pain points we've identified as well as helping improve user workflows. Some ideas around this relate to offering automatic code for instrumentation, automated suggestions for reporting and charting of collected data, and providing a question-and-answer interface for engaging with the data. Since AI is so new, we are actively exploring in this area and tracking it in this epic.
There are many use cases for Product Analytics. One way you can think about these is to segment them by the type of digital product analyzes and the subsequent questions those who create it would seek to be answer. For example, a team that publishes a blog is primarily interested in page views of blog posts.
Our initial use case is focused on our ability to dogfood. As a result, we are focused on Web Apps. Our initial use case is:
The ideal customers for Product Analytics are customers already invested in the GitLab DevSecOps platform. These customers have no usage tracking today and cannot explore, visualize or share data that shows the value the software they develop provides or where they should invest in features next. We have found this may be, but is not limited to, an internal tools or platform team who offers self-serve capabilities to other development teams through webapps but do not know if that software investment is paying off today.
It is important to identify the use cases and personas that we want to pursue and it is equally important to identify those that we do not wish to pursue. Because analytics can mean many things to many different people, this is especially important.
We do not plan at this time to pursue use cases and personas around:
Note that while the above are use cases and personas we are not actively pursuing, this does not mean that those personas will not find some value in what we provide. It means that they are not our primary focus. We do not intend to block functionality around these, just that they are not our primary focus. For example, page views and traffic trends are also useful for marketing personas, but we intend to use them for guiding product-related use cases, not for addressing marketing-related use cases. Similarly, understanding error reporting for a given feature may be important to understand a feature, but we are not focused on making a product quality or bug tracking platform using those pieces of data.
Our Strategy to achieve this vision is to start by helping users store, query and visualize quantitative data to measure user value. We will collaborate with the Analytics Instrumentation team to give users the tools they need to instrument and collect data from deployed applications.
Given our focus on developers, the software delivery value stream, and DevOps - we will compose our new DevOps stage, Analyze, based on the set of categories we commonly see in User Engagement competitors.
Currently the Product Analytics group includes two categories.
In the future we envision more categories as we broaden our scope to cover additional personas and user cases. Those new categories include:
There are a number of existing (or considered) product categories in GitLab that could be considered part of the outer loop that the Analytics section will partner closely with to ensure we provide a cohesive experience. Those include:
We plan to collaborate and build with these categories where possible, rather than re-inventing new solutions for these related use cases.
We have a plan to iteratively develop Product Analytics features. We will start small and incrementally add new capabilities. Each iteration will let us learn more and solicit feedback to improve, before we start our next iteration. Our first iteration was primarily internally focused and we are now turning our attention to the first customer facing releases.
With our Experimental Release complete we are engaging with external users to show them Product Analytics to improve the Product Analytics offering. We now turn our focus to the work for the Beta release to enable customers to opt into Product Analytics themselves. This is the next iteration outlined as an epic below.
These roughly correlate to our current guidance in the handbook and documentation about phased releases. We will continue to provide timing guidance for each as we make progress.
This iteration will be available to any Ultimate Customer interested in signing up to try Product Analytics on GitLab.com. Our goal with this release will be to identify any potential issues, bugs, or edge cases with our features that would prevent us from being ready for production-grade usage and fully opening up for GA. This aligns with our FY24'Q3 OKR.
Once we are in Beta, we will be feature complete for what will be available in the GA release. The Beta is intended primarily to understand tweaks that may need to be made to existing features, ensure we are ready for production-grade loads, and fix any bugs. We are tracking this work in the Beta Epic.
Pricing and packaging should be relatively complete by this release, though may change before the GA release.
A key difference between the Experiment and Beta is that we will expect much greater load on GitLab during this period, since any Ultimate SaaS customer can sign up for it.
As we continue to use Product Analytics and gather feedback from users we have heard that while the default dashboards are helpful what users really want is to track custom events and query the data so we will work to make it easier to access that data in the Visualization Designer. We have also received feedback about the default dashboards themselves and have captured that into an epic to improve the Single Statistic visualization.
In our own usage of Product Analytics we know that seeing how many users transition from one step to the next like from the dashboard list to a specific dashboard, or through the Visualization Designer is important to see where there could be friction in the process. To address that we will take the next steps to enable the visualization of funnels and implement an MVC designer for them.
These features and improvements are being considered separately from work towards the Beta or General availability release. None of them are required to be complete before entering those phases of Product Analytics.
This iteration will be when the Product Analytics capabilities are available for all users to take advantage of, just like any other capability within GitLab.
Our minimal feature set that we will work on to get us to our first GA will likely include:
Once our first user-facing release is available, we will respond to feedback on where we'll next focus our efforts to build a "Lovable" product. We have several ideas, and have validated some of these with customers, for follow-on iterations after this release, which are listed below. Some of these may also be included in our initial GA release.
Since releasing Product Analytics internally and as an Experiment we have received excellent feedback on existing features and have better understood the problems our users have that influenced the roadmap of planned features. One key theme that stuck out was that users do not want to learn another query language to view their data. This reinforced that we should release the Custom Dashboard Designer, Visualization Designer which was recently completed.
Our latest iteration was focused on the Experiment Release of Product Analytics that onboards the first external customer(s). As these customers scale it allows us to learn about how the managed product analytics stack handles load and make improvements to the feature set based on a wider set of customer feedback.
Our first iteration was focused on an internal preview of Product Analytics that we can dogfood. This let us work through the technical questions of how to best develop Product Analytics, how to host and maintain relevant infrastructure, as well as how to use it like an end-user would. This culminated with us adding Product Analytics to the internal handbook, Pajamas, Metrics Dictionary, and VersionApp for dogfooding purposes.
Due to the heavy emphasis on SaaS and the high data volumes - most pricing in this market is consumption-based.
Our Performance Indicator is Weekly Active Users defined as users viewing a dashboard and is being tracked in Sisense (internal link).
We are conducting research on critical jobs to be done for the Analytics section.
The existing personas we serve are below, in priority order. We will likely need additional personas in the future.
Some nuance can be added to our personas and how we approach them. Nearly all analytics questions, workflows, funnels, or any metrics gathering will require technical work to add instrumentation, test, and deploy it. This is the reason we are focusing on Sasha as our primary persona before Parker. We are addressing Sasha in the context that they are supporting Analytics efforts for their team. This means they are interested in how to do tasks related to adding instrumentation code, deploying it, and debugging it in support of analytics-related questions and projects. This is a more focused version of the Sasha overall persona.
As part of considering these personas, consider what personas we are not including in this initial list. Specifically, we are not targeting executive personas or Directors with the initial offering. Sasha and Parker are individual contributors and have unique needs different than Directors or executives. They are focused mainly on specific applications and the analytics related to them, whereas executives and Directors will be concerned about multiple, or a "fleet", of applications. We intend to go after these personas eventually and will not intentionally create capabilities that exclude them, but they are not our primary focus at this point.
The market is divided between big tech entrants building on top of complete Marketing Automation platforms marketed towards enterprise marketing orgs and stand-alone tools user engagement tools that are marketed towards Product (and occasionally Development) teams.