1. You are here:
2. GitLab Direction
3. Product Stage Vision - Create
4. Group Direction - Code Creation
5. Category Direction - Code Suggestions

On this page

• Code Suggestions
  • Introduction and how you can help
  • Overview
    • Vision
    • Strategy
      • Short-term Strategy
      • Long-term Strategy
        • Privacy
        • Security
        • Enterprise Control
      • Target Audience
    • 1-year plan
      • Roadmap
      • Possible projects
    • What we recently completed
    • What is Not Planned Right Now
      • Maturity Plan
    • User success metrics
  • Competitive Landscape

Code Suggestions

Stage	Create
Group	Code Creation
Maturity	minimal
Content Last Reviewed	2023-10-17

Introduction and how you can help

Thanks for visiting this category direction page on Code suggestions in GitLab. This page belongs to the Code Creation group of the Create stage and is maintained by Kevin Chu (kchu at gitlab dot com).

Overview

Code suggestion systems use Generative AI to suggest relevant code snippets and autocomplete code while software developers write code. These systems aim to boost programmer productivity and reduce time spent on rote coding tasks. Some key points about AI code suggestion systems:

• Code suggestion solutions often uses large language models, trained on huge data sets, such as open source code repositories or other scraped content, to repeatedly predict the next token or code snippet, provided a prompt.
• Code suggestions should empower developers to focus on higher logic and architecture rather than routine coding.
• Limitations and risks of using AI code suggestions include:
  • Bias perpetuation - GenAI may inadvertently amplify biases found in the training data.
  • Quality control - Risk generating incorrect or faulty code without proper evaluation.
  • Legal - Copyright, attribution, and licensing for all AI-generated content need to be thought through.
• Code suggestion solution today is integrated into IDEs, but code suggestion is not only limited to writing code in the IDE.

Vision

Be the best-in-class enterprise code suggestion solution for end-to-end DevSecOps by helping increase engineering productivity without sacrificing security, privacy, and enterprise control.

Strategy

There are more details on how we developed the strategy in this confidential (internal to GitLab) epic.

Short-term Strategy

Our short-term (3 months) goal is to launch a generally available code suggestion product as soon as feasibly possible.

As every sector aims to gain a competitive edge with AI, GitLab must supplement our product suite with AI-powered workflows. Code suggestions are among the most obvious (even if it is not the biggest SDLC efficiency driver) and mature AI categories. By empowering customers to adopt code suggestions, we can then jointly prioritize and augment the remaining DevSecOps workflow with AI.

If we don't move quickly, we risk alienating customers who need to adopt AI immediately. The current AI hype cycle makes these needs so strong that they pose significant churn risk. By delivering code suggestions, we can mitigate this risk while collaborating with customers to determine other high-impact AI integrations.

For a GA product, GitLab Duo Code suggestions will have the following attributes:

• Best in class LLM for the right task
• Minimized code/repo exposure - customers won't have to ship their code to another vendor
• GitLab doesn't scan/store code prompts or suggestions
• Viable code suggestion quality

Between Beta and GA, we are focused on the following:

• Best-in-class models
• Improve acceptance rates
• Operational readiness

Long-term Strategy

Longer-term, we plan to position code suggestions, along with the rest of our AI offerings, to compete on the axis of privacy, security, and enterprise control. In addition, we plan to continue to improve the quality of suggestions, because after all, if code suggestions are too limited or of a subpar quality, none of the other strategic themes would matter. Code suggestions should at least:

1. Increase productivity: Empower Developers to write code faster and more efficiently, with code suggestions for code snippets and improvements in real time.
2. Reduce errors: Detect and suggest corrections for errors as they occur, reducing the time and effort spent on debugging.
3. Improve code quality: Suggest refactoring and best practices, leading to more maintainable and performant code.
4. Enable more efficient collaboration: Developers should be able to share and reuse code snippets with their team members, leading to more efficient collaboration.
5. Empower access to a broader range of knowledge: Suggest code snippets and improvements across a wide range of programming languages and platforms, providing developers access to a broader range of knowledge.

Privacy

Code is enterprise IP. How we enable privacy without compromise even as new AI-powered features become available is a top concern for organizations. Data collection and management, access control, authorization, secure communication, compliance, governance, responsible AI, and other ethical consideration will be strategic vectors we focus on.

Security

One of the potential risk to AI can be introducing vulnerability thorough suggested code, or perpetuation of security problems. Instead of treating code suggestion as more potential risk, we plan to leverage the rest of the DevSecOps capabilities to make suggestions that improves our customer's security posture.

Enterprise Control

Managing feature access at scale is an important problem to solve for enterprises. Furthermore, providing visibility into how code suggestions is used, how code suggestion impacts overall productivity will help GitLab customers make data-informed decisions.

Target Audience

People who code:

1. Sasha (Software Developer)
2. Priyanka (Platform Engineer)
3. Simone (Software Engineer in Test)

In the future, we may expand to include security personas to help write more secure code and review code for security vulnerabilities and fix them early in the software development lifecycle (SDLC), before you commit.

1-year plan

Roadmap

The main body of work we are currently focused on is improving the quality of code suggestions. This effort is captured in the code suggestion evolution and its sub-epics and issues. The table below highlights some of the most important needle movers as we get ready for GA.

Theme	Metric	Tactics/Epics	Timing
Best in class model	Acceptance Rate	Introduce Anthropic	FY24'Q3
Improve code suggestion quality	Acceptance Rate	Improve formatting, Reduce code suggestion requests, Increase when we use code-generation	FY24'Q4 FY25'Q1
Operational Readiness	Ready to support Code Suggestions GA	Identifying projects in issue	FY24'Q4
Enterprise Control	?	Granular control to enable/disable Code Suggestions in Project/Groups/Sub-Groups/Users	FY25'Q1 FY25'Q2
Performance	Acceptance Rate	Streaming	FY24'Q4 FY25'Q1
Smarter context	Acceptance Rate	Repository X-Ray, Increase context window	FY25'Q1 FY25'Q2
Chat	Acceptance Rate, Number of users	Code Tasks	FY25'Q1 FY25'Q2

Possible projects

We plan to invest according to our long-term strategy. Here are some loosely prioritized projects that we may work on over the next 6-12 months.

1. Improve code suggestion performance - code suggestion quality: The speed by which code suggestion is returned has a direct impact on whether or not code suggestion is helpful to the developer.
2. Improved code suggestion setting - code suggestion quality: Empower developers to fine tune how they want to use code suggestions.
3. Multiple choice - code suggestion quality: Provide developers with multiple suggestions so they can choose the most appropriate.
4. More automation - code suggestion quality: Things like automate code commenting, test case generation to enable developers to be more efficient.
5. Vulnerability/Refactor improvement on open/committed file - security: Bring security left as the developer is writing code.
6. Increase context from the repository - code suggestion quality: The code that's in the repository can include libraries, coding style, and other resources that improve code suggestion quality.
7. Code suggestion dashboard - enterprise control: View how code suggestion is used.
8. Code suggestion content guardrail/filter - privacy: organizations do not wish to have problematic code suggested.
9. Generate suggestions for APIs and frameworks - code suggestion quality: allowing developers to fill in low-value common boilerplate code.

What we recently completed

1. Generate code suggestions from plain text allowing suggestions to take direction and have a greater context of developer intention.

What is Not Planned Right Now

• Fully original GitLab-trained model: We can iterate faster now by partnering with best-in-class LLM vendors.
• Self-hosted models: We are prioritizing using SaaS Gen AI capabilities for the time being. We will reconsider the priority of model hosting flexibility post GA.
• Generating code outside of the IDE: While we may eventually allow developers to generate code outside of the IDE, we are starting with code generation only in the IDE.

Maturity Plan

This category is currently at minimal maturity. At GA we may move the category to viable maturity.

User success metrics

• Acceptance Rate - This metric is currently the primary metric for quality. We assume that if the suggestion is relevant, timely, and helps the developer, the more likely the developers are to accept the suggestion.
• Retention Rate - This metric indicates whether or not the suggested code made it into the repository. This shows the real impact that code suggestion has on the code base.
• Developer cycle time - Code Suggestions may make developers more efficient and we believe help reduce the number of bugs, security vulnerabilities, and stylistic issues improving developer cycle time.
• Time to merge - With more efficient coding, merge requests will merge faster with less review time and back and forth between reviewers and authors.

Competitive Landscape

Please see the content in our internal handbook