Category Direction - Secret Detection

The following page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features or functionality remain at the sole discretion of GitLab Inc.

Secret Detection

Secret Detection


Stage	Secure
Maturity	Viable
Content Last Reviewed	`2024-01-26`

Introduction and how you can help

This direction page describes GitLab's plans for the Secret Detection category, which protects you against leaking credentials, tokens, or other secrets on GitLab.

This page is maintained by the Product Manager for Static Analysis, Connor Gilbert.

Everyone can contribute to where GitLab Secret Detection goes next, and we'd love to hear from you. The best ways to participate in the conversation are to:

Comment or contribute to existing Secret Detection issues in the public gitlab-org/gitlab issue tracker.
If you don't see an issue that matches, file a new issue.
- Post a comment that says @gitlab-bot label ~"group::static analysis" ~"Category:Secret Detection" so your issue lands in our triage workflow.
If you're a GitLab customer, discuss your needs with your account team.

Overview

GitLab Secret Detection helps you avoid a particularly dangerous type of mistake: leaking credentials or other secrets in your code repositories.

We want GitLab to be a safe place to develop software, so we're working to make Secret Detection a standard part of the software development lifecycle (SDLC). No one should have to think about secrets to be protected from leaking them.

The problem

Even experienced developers and teams can slip up and cause serious risk by committing secrets into their code repositories.

The potential damage is significant:

Secrets often provide access to sensitive data, production systems, or cloud resources that can be abused.
If a repository is public, automated tools or malicious users can detect and abuse leaked secrets—there are even public sites dedicated to listing these leaks.
Even if a repository is private within a team or organization, leaked secrets can no longer be trusted to uniquely identify the authorized user(s) in a non-repudiable way.

GitLab's solution

GitLab Secret Detection helps you prevent the unintentional leak of sensitive information like authentication tokens and private keys.

Secret Detection checks your Git repositories to detect secrets or credentials, then it reports potential findings. Today, Secret Detection jobs run in your CI/CD pipelines.

We want everyone to be secure, so:

Parts of Secret Detection are available in every GitLab tier.
Secret Detection is on by default in Auto DevOps.

In GitLab Ultimate, after you enable Secret Detection:

You can see and address new Secret Detection findings in merge requests.
Results appear in the Vulnerability Report.
Automatic responses help mitigate leaks as soon as they occur.

Secret Detection doesn't target a specific language, so you can easily enable it in any project. Our approach takes advantage of patterns for well-identifiable credentials like service account keys and API tokens, but also searches for more generic secret types like passwords in certain contexts.

To learn more, check the Secret Detection documentation.

Outside of the Secret Detection category, GitLab also offers other features that relate to secret values:

Push rules block files with certain names from being pushed to your repositories.
The Secrets Management category focuses on enabling GitLab to use and manage secret values.

Strategy and Themes

We're specifically focusing on solving the following user problems:

Coverage: Protecting against credential leaks wherever they occur—in a Git repository, an issue, a job log, or anywhere else.
Time to Remediate: Reducing the window of time between when a secret is leaked and when it no longer poses a threat.
Prioritization: Knowing which leaks are the most urgent and serious, so teams can respond to those issues more quickly.

Those problems support two overall goals that are related but distinct:

Helping organizations achieve their security and compliance requirements in a unified DevSecOps platform.
- This goal is primarily focused on organizations using GitLab Ultimate, in accordance with our tiering philosophy and rationale for paid-only features.
Making GitLab a safe place for people to develop software.
- This overall goal applies across tiers, though different features will be available in each tier.

1 year plan

Our strategic one-year focus is a balance between:

Investing in the next generation of Secret Detection, which includes:
- Pre-receive scanning, which blocks commits from being pushed if they contain secrets. This functionality is in development, with an early experimental release available in GitLab Dedicated.
- Pipelineless post-receive scanning, which replaces the existing scanning system that runs in CI/CD pipelines after content is pushed.
- Client-side protections, which help users avoid leaking secrets in issues, merge requests, and comments.
Addressing problems with how Secret Detection works in its current architecture.

We generally are focusing more on the next-generation approach, but will still make high-impact improvements to the current architecture.

Specifically, we plan to focus on:

Bringing pre-receive scanning to broader availability—including specifically to GitLab.com as an opt-in Experiment or Beta release.
- This contributes to Coverage goals in the next-generation system.
Tracking leaks better as they move through a codebase, to avoid repeatedly surfacing the same leaks.
- This contributes to Prioritization goals in the current system.
Speeding triage by allowing more exceptions and ruleset customization. This builds on our existing customization features, including the shared configurations introduced in 16.1.
- This contributes to Prioritization goals in the current system.
Scanning codebases without pipeline jobs, replacing the current pipeline-based approach. This will likely involve rethinking the end-to-end workflow for detected secrets. This includes providing better organization-wide visibility of possible leaks and better context as leaks are detected.
- This contributes to Time to Remediate and Coverage goals in the next-generation system.

What is next for us

In the next 3 months, we are planning to:

Building, tuning, and performance-testing pre-receive secret scanning so that it can be made available on GitLab.com as an Experiment and then a Beta. (See the technical design issue for this milestone.)
- We plan to target GitLab.com next because it allows real-time monitoring and faster iteration cycles than self-managed release cycles. We will make pre-receive scanning available in all environments as soon as we can.
A better tracking mechanism for findings as they move through a codebase. We expect this to make a significant improvement against the overall problem of repeated/duplicated secret findings.

We are also looking forward by:

Designing user experience (UX) workflows for pipeline-less post-receive scanning.
Refining the system architecture for pipelineless post-receive scanning, which should share significant architectural elements with the new pre-receive secret detection feature.

What we are currently working on

We are currently working on:

Completing the MVC of pre-receive secret scanning, including known optimizations and improvements.
Identifying the path to enabling pre-receive scanning on GitLab.com.
Finalizing Job to be Done (JTBD) UX researh for Secret Detection.

What we recently completed

Our recent work includes:

An initial MVC release of pre-receive secret scanning (16.7). This is currently available as an Experiment in GitLab Dedicated. We are currently working to expand availability.

Check older release posts for our previous work in this area.

What is Not Planned Right Now

Repositories outside of GitLab: We don't plan to offer protections for projects hosted outside of GitLab.
Credentials not common in software: We plan to focus on the types of credentials that are most common in DevSecOps workflows and in software development. This means we won't actively pursue rules that are:
- For services unlikely to be used in a software project.
- Closer to Data Loss Prevention, for example searching for personally identifiable information (PII) or credit card numbers.

Best in Class Landscape

ℹ️ Best In Class (BIC) is an indicator of forecasted near-term market performance based on a combination of factors, including analyst views, market news, and feedback from the sales and product teams. It is critical that we understand where GitLab appears in the BIC landscape.

Key Capabilities

Secret Detection products should:

Help developers avoid making leaks in the first place.
Enable quick responses to any leaks that are made.
Deliver trustworthy, accurate findings, with appropriate priority, so that the right findings are remediated quickly.
Automatically respond to leaks, without human intervention, when possible.
Provide security users with overall visibility into exposures anywhere within their organization.

Roadmap

The plan described above reinforces our competitive standing.

In addition to those main themes, we will likely pursue additional detection techniques including:

Checks for whether detected credentials are still active.
Machine Learning or other solutions for identifying generic or lower-confidence secrets. These secrets, which include passwords and tokens that don't have an identifiable structure, are more difficult to detect while minimizing false-positive results.

Target Audience

As with many security categories, Secret Detection is a bridge between different communities:

Security teams, who are responsible for protecting their organization from credential leaks.
- These teams often drive the implementation of tools like Secret Detection.
- They're also often responsible for responding to alerts and incidents related to compromised credentials.
- User personas include Alex, the Security Operations Engineer and Sam, the Security Analyst.
Development teams, who are responsible for building software.
- These teams may accidentally leak credentials in the course of their work.
- They don't want to be the source of a security breach, but they also don't want to be blocked by unreliable tooling or onerous processes.
- User personas include Sasha, the Software Developer, and Sasha's teammates and managers.

Pricing and Packaging

Secret Detection products vary in how they're provided:

Some are integrated features within the platforms they protect.
Others are standalone products.
Others are modules within other products, like SAST scanners.

GitLab packages and prices Secret Detection primarily as part of Ultimate. Basic protection features are available in all tiers. We intend to expand the level of protection available in all tiers, while still delivering unique organization-level value in Ultimate.

Analyst Landscape

Analysts usually include Secret Detection as a secondary feature of Application Security Testing (AST) coverage. See Category Direction - Static Application Security Testing (SAST) for up-to-date analyst coverage.