The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.
| Stage | Maturity | Content Last Reviewed |
|---|---|---|
| Govern | Viable | 2022-03-04 |
Legal, regulatory, and business requirements call for reliable, secure, and performant audit logging capabilities. Users interact with shared resources like projects, comments, and groups inside GitLab. Administrators need the ability to see and review activities to understand what happened in GitLab and who performed those actions. Audit events are individual records of what happened, when it happened, and who did it for various actions inside GitLab.
A comprehensive set of audit events for the most critical and sensitive types of actions inside GitLab allows for a detailed record of what happened, when it happened, and who did it. This empowers administrators to quickly understand what changes happened. It also empowers compliance managers, like Cameron (Compliance Manager), to confirm that compliance policies have not been violated or to maintain records for if exceptions have occurred.
The Audit Events category is focused on capturing the most important user-driven events within GitLab, making that data easily available (via APIs, webhooks, and exportable reports), and providing a reasonable duration of storage for the data that follows compliance requirements (7 years).
As users use more of GitLab's capabilities, there will be more potentially sensitive changes to monitor for. We will focus on ensuring that key areas of GitLab generate audit events to support administrators and compliance managers. GitLab moves quickly, releasing new features every month, so a challenge we will need to focus on is to add and capture new actions associated with these new capabilities. This aspect is key, because we can export audit events for users to post-process themselves, but if we never logged the event in the first place, there is not a good way forward.
It's important that we’re presenting information in a way that makes answering questions simple. Users goal is to uncover quickly insights from a set of logs, not to comb through all the logs. Separation of duties questions like “who opened and merged this MR?” aren’t readily available in a logging system that logs individual events. GitLab should promote traceability; when a change is logged, we should enable an administrator to see the full context around the change.
Organizations, especially enterprises, have complex services and tooling built to integrate with their existing data analytics processes. This means that GitLab needs to ensure data parity between the audit events table within the UI and the audit events API. It also means incorporating different types of data ingestion for organizations, such as webhooks for various components within GitLab that are critical for an organization's custom tooling and analytics.
Excellence in this category means that we’re solving for all of these needs: transparency, traceability, configuration, and monitoring.
Audit events are a necessary aspect of compliance, but more important is leveraging audit events to inform proactive decisions and actions. Every customer differs in their policies and procedures, but also in their security and compliance culture and mindset. It will be critical to provide customers with smart, proactive alerts about anomalous activity in a way that's easy to setup and take action on. Flexibility is key because some customers will want to take programmatic action (e.g. automatically block a user's account) and some will want to take a "lighter touch" approach (e.g. notify an administrator, but take no further action).
Audit Events are a core component of any compliance program. An organization needs to know what activity is occurring on their systems in order to proactively manage those systems. Traditionally, audit logs have been technical, verbose, and difficult to manage. The Audit Events category wants to ensure that anyone can easily find the data they're looking for, in as little time as possible, and at the granularity that they need.
The three key themes for this category are: granularity, simplicity, and time savings.
An audit events system is incomplete if it cannot answer any questions about who did what, when, and on what system. GitLab has the breadth and depth of insight to be able to shed light on the complexities of the devops workflows your organization employs. Not only should GitLab provide logs for the most important user-driven events, but those events should be traceable to other entities or events in the system. We want to provide you with an experience that answers your auditing questions, but also delights you while you're answering them.
The capture of all necessary audit events leads to a more reliable auditing process for your team and organization. This will enhance your ability to identify and deal with any incident or anomaly.
We want to make it simple to work with GitLab audit events. When users look at the logs to find out what happened in their instance, we will make it easy to find out what happened and to identify any anomalies or policy violations. Our goal is to enable this with straightforward workflows that do not require coding or using complicated interfaces.
By making the Audit Events experience super simple, we hope to alleviate the time commitment and headache our users typically experience when searching for data in this context.
Audits and compliance efforts can be a major time commitment. Time spent building custom tooling and reporting to searching through the data in these, or other systems, for specific answers is time that could be used on other activities. The person doing the analysis frequently does not specialize in compliance. This introduces friction because these people may not understand what events are important, how to get relevant artifacts, or exactly what the person requesting compliance artifacts needs.
By making GitLab audit events simple and friendly, we will enable everyone to contribute to compliance efforts and get the visibility needed to do so.
Our top priority is to evolve our storage database architecture so we may improve performance and scalability. This will be achieved through migration to Clickhouse. Improvements in this area provides a number of advantages:
We'll then be working to expand support for filtering and adding capabilities across group and instance levels, while also extending support into 3rd-party storage systems. Third-party audit event storage solutions will make it a lot easier for users to get started with streaming audit events and use the tools they already have, rather than require them to build logic to ingest and process HTTP events.
| Destination | Feature | Group | Instance |
|---|---|---|---|
| HTTP Destination | |||
| Sending audit events | ✅ | ✅ | |
| Filter by event type | ✅ | ✅ | |
| Filter by subgroup/project | 🔄 (https://gitlab.com/groups/gitlab-org/-/epics/11384) | 🔄 Epic | |
| GCP Logging | |||
| Sending audit events | ✅ | ✅ | |
| Filter by event type | 📆 Epic | 📆 Epic | |
| Filter by subgroup/project | 📆 Epic | 📆 Epic | |
| AWS S3 | |||
| Sending audit events | 🔄 Epic | 📆 Epic | |
| Filter by event type | 📆 Epic | 📆 Epic | |
| Filter by subgroup/project | 📆 Epic | 📆 Epic | |
| Azure Blob storage | |||
| Sending audit events | 📆 Epic | 📆 Epic | |
| Filter by event type | 📆 Epic | 📆 Epic | |
| Filter by subgroup/project | 📆 Epic | 📆 Epic |
Contributors outside of the Compliance group can add support for new audit event collection into GitLab by following our guide to contributing new audit events. An example of this is how the DAST group was able to contribute support for auditing their events.
Additionally we work to add in support for new audit events on a regular basis to expand our comprehensive audit log.
As we add new audit events in the future, we will thoughtfully consider which to add first and which to add later. To help decide which events to add before others, we consider these priorities, in stack rank order, when deciding which events to focus on next:
Please note that the list above is not exhaustive, but shows a stack rank of some of the most important types of events to record and our perspective as we work on prioritizing them. The individual events we are working on are captured in this epic and we would love your feedback.
Note that when an action is taken very often, the audit events will likely generate more data than we can store. In those cases, we will deliver audit events for those actions as streaming-only audit events. This provides a good balance between recording the actions that users care about while not overwheliming GitLab servers with needing to store large amounts of data.
As part of the Audit Event category, the Compliance group is responsible for delivering a standardized and robust auditing system as part of the GitLab product. Our group will occasionally add events ourselves using the priority list above, but it is not scalable for one group to add them for all features in GitLab since all groups ship so much so often.
Individual groups are responsible for adding audit events for their own features as they develop them. We maintain a self-service development guide in the documentation for how people can add a new audit event. If there are questions about the guide or about how to add an audit event for a specific feature, the Compliance group is happy to help.
Audit Events is a very deep domain and can mean different things to different people. Above we highlight areas we will focus on but it is also important to highlight areas that we intentionally will not focus on.
At this time, we do not intend to build a full SIEM experience around Audit Events. A SIEM can do many different tasks and can consume many different data streams simultaneously, correlate them, and handle high data volumes. We envision audit events as being able to be consumed by an external SIEM, but we will not build a SIEM into GitLab itself.
We do not intend Audit Events to be used in such a way that they could be used for product analytics or measuring feature adoption. There are purpose-built tools, such as GitLab Value Stream Management that can be used to understand how end-users use a product and measure adoption over time.
We do not intend to log every single user-driven action inside GitLab. Because GitLab is such a large application, it is not feasible to process and store the amount of data that would be generated if every single action was recorded. Additionally, not every single event inside of GitLab will be as sensitive as others or relevant during audits. Instead, our plan is to identify the most critical and sensitive actions and focus on those.
Currently, GitLab’s maturity in Audit Events is viable. Here’s why: GitLab currently offers an audit log system, but it does not capture all of the required user-driven events. While we’re iterating and capturing more events over time, our users demand a comprehensive view of activity in application logs. However, the events being captured are useful at answering valuable questions on GitLab instances, and are used to solve real problems.
Advancing this category to a complete state means having a robust, complete set of logs that captures the vast majority of user activity. These logs are structured and easily parsed; while GitLab may not actively guard against threats, an administrator can easily track down the history behind nearly any change. Additionally, these audit logs should be usable; an administrator should be able to ingest them into their Security Information and Event Management (SIEM) tool of choice or search for events easily in the GitLab UI and via our API.
Lovable consists of compliance adding active monitoring for Audit Events to be truly trusted by users. The application is able to demonstrate security and compliance on an ongoing basis, in a single view, and actively monitors for (and defends against) suspicious behavior. An administrator or auditor can check a dashboard to be brought up to speed - or simply get some peace of mind.
We'll know we're on the right track with Audit Events based on the following metrics:
GitLab has several competitors in this area with mature product offerings. To compete with these experiences, we need to first understand the value they offer to users. This will enable us to find opportunities to differentiate GitLab as we build out our own capabilities.
GitHub provides a comprehensive and versatile audit log experience. Their search capability empowers users to find any data they need and categorizes audit events into different types. The delineation between operation, repository, user, action performed, and various other categories, provides convenience and easy searchability for their users. They provide a detailed breakdown of each event category, reducing the friction of learning about the various events and lowers the barrier of instrumenting custom tooling around these logs.
Another convenience feature they provide is the ability to configure the retention period of audit logs. Public repositories can be configured to retain logs for 1 - 90 days. Private, internal, and enterprise repositories can be configured to retain logs for 1 - 400 days.
This audit log experience is currently in public preview but shows a methodical approach to audit events. The primary segmentation is between Categories (event type) and Areas (areas of the application). This allows both Microsoft and users to navigate the various event types more easily and ingest that data into SIEM tools for longer retention periods. This audit log experience only provides 90-day retention and instructs the user to leverage the API to backup the data elsewhere for longer storage.
Filtering and exporting audit events is supported, which makes it easier for users to parse and analyze the data outside of Azure. They also provide a useful ID and CorrelationID relationship so an event can be linked to other events. This is important for traceability and makes it much easier to analyze the series of events that lead to an incident or undesirable action.
Azure DevOps Audit also supports audit event streaming, which is a request we receive regularly from our customers to work with their existing tooling to ingest data into SIEM or other systems. They seem to have built-in some native integrations with tools like Splunk to provide this streaming experience.
Another offering is the ability to export a list of users and their access levels, further emphasizing the fact that organizations need to pull specific reports that can be automated and standardized.
This audit log experience from Atlassian provides flexibility to define the level of detail users wish to capture in their audit logs. They segment the logs by "coverage area" and offer an additional setting for "coverage". They provide 26 categories and their coverage options break down into: Off, Base, Advanced, and Full.
Users can create a custom-tailored logging solution, in-app, that meets their needs which may require Full coverage of the Repositories category but Base coverage of the Users and groups category. This allows organizations to focus on the areas that are most important to them, which reduces the volume of logged activity and therefore makes it easier to search through the logs.
Bitbucket allows users to retain data up to 99 years, which is well beyond even the more conservative logging policies and compliance requirements.
Each competitor seems to follow a similar theme of breaking out their event types into categories or coverage areas. This creates a hierarchical structure for building out audit logs which has benefits for both the service provider and the customer organization. They also provide a sensible amount of granularity in the audit log configuration, such as defining the level of detail to log (Bitbucket) or delineating between specific event types like GitHub. This categorization makes it easier to work with the audit event system through consistency and human-readable syntax.
All of these competitors seem to be focused on making audit events complete in their coverage as well as the usability: searching, filtering, and integrating event streaming.
When it comes to data retention, there's significant variance, but each company provides the flexibility to customize the retention period to some maximum value and offers parity with an API to ensure an organization can capture this data outside of the respective audit log systems for longer retention periods and other analysis or workflows.
To summarize:
The primary feedback we've received from analysts about this category has centered around programmatic alerting. Providing smart audit logs that can notify appropriately personnel of anomalous activity or specific, defined events can really elevate the value of Audit Events for our customers.
Thanks for visiting this direction page for Audit Events in GitLab. If you'd like to provide feedback or contribute to our vision, please open a merge request for this page or comment in the corresponding epic.