Engineering Practices in Scrum: Branching, CI, Feature Flags and Technical Debt

Version note: This post is based on the Scrum Guide 2020 (November 2020, Ken Schwaber and Jeff Sutherland). The code in this post was built and run with .NET 10 and Microsoft.FeatureManagement.AspNetCore 4.8.0; the HTTP (Hypertext Transfer Protocol) output comes from real requests sent with curl to a local Kestrel instance.

Guess First

It is the last day of Sprint 14. Ayşe and Zeynep have been building the "self-service returns" feature on the feature/self-service-returns branch for three weeks; the feature spans two Sprints and needs at least another week. Meanwhile more than 60 commits have gone into main; when they try to bring the branch up to date with main, seven files conflict. The time zone fix finished in the same Sprint is waiting for release, because the team says "let's ship everything together after the big merge".

Is there a usable Increment at the end of this Sprint? And is it possible to merge a half-finished feature into main, or would that violate the Definition of Done? Write down your guess; the answer lies where the three practices in this post intersect.

Scrum Does Not Impose Engineering Practices

The Guide is deliberately silent: it says various processes, techniques and methods can be employed within the Scrum framework, and that Scrum wraps around existing practices or renders them unnecessary. Branch strategy, testing approach and deployment method are not in the Guide. But the Guide sets two conditions, and those conditions effectively narrow the choice of practices: a usable Increment that meets the Definition of Done every Sprint, and quality that does not decrease during the Sprint.

If every Increment must be verified to work together with all prior Increments, integration has to happen continuously, not once at the end of the Sprint. In his "Dark Scrum" article, Ron Jeffries ties the Developers' way out of badly applied Scrum precisely to these practices: acceptance testing, incremental design, refactoring, programmer testing and continuous integration. Extreme Programming (XP), where these practices come from, is covered in the Agile Mindset series post "Extreme Programming (XP): The Engineering Side of Agile"; here we only look at how they fit Scrum's rhythm.

Short-Lived Branches and Trunk-Based Development

Bookstore's three-week branch runs against Scrum's rhythm. The longer a branch lives, the further it drifts from main, the bigger the conflict risk, and worst of all, whether the integration really works only shows up at merge time. That moment is usually the last day of the Sprint.

Trunk-based development reverses this: everyone integrates often, in small changes, into a single main line (the trunk, main at Bookstore); branches, if any, live for a few hours or a day or two at most. Bookstore's new working agreement has three points: a pull request stays open for at most one working day, if a pull request changes more than 400 lines the team discusses splitting it, and main is always deployable. Branch strategies themselves are covered in the Git & GitHub series post "Teamwork with Git: Branching Strategies, Rulesets and Conventional Commits", and the review flow in the same series' post "Pull Requests on GitHub: GitHub Flow, Reviews and Merge Methods".

The Scrum counterpart shows up in how the Sprint Backlog is broken down. The Guide says that in Sprint Planning items are often decomposed into work items of one day or less; a one-day work item means a one-day branch. Planning and branch lifetime feed each other.

CI: Not Leaving Integration to the End of the Sprint

Continuous Integration (CI) means every Developer integrates their work into the main line at least once a day, and every integration is verified by an automated build and tests. The Definition of Done pipeline we saw earlier in the series is built on this: build, tests, vulnerability and OpenAPI checks on every pull request. The definition of CI and how it differs from Continuous Delivery are in the CI/CD Fundamentals series post "CI, Continuous Delivery and Continuous Deployment: The Differences and Why They Matter".

From Scrum's point of view, CI's payoff is transparency. If main is integrated and tested every day, there is no "integration might break" unknown in the Daily Scrum; the real state of the product is visible at any moment. A three-week branch, on the other hand, carries an invisible risk behind the "In Progress" card in the Sprint Backlog.

Merging Unfinished Work into Main with a Feature Flag

So how does a three-week feature fit into one-day branches? The answer is a feature flag: unfinished code is merged into main and deployed, but kept off by a configuration key. Deploy and release are decoupled. The Bookstore team wrapped the self-service returns feature with Microsoft.FeatureManagement like this:

using Microsoft.FeatureManagement;
using Microsoft.FeatureManagement.AspNetCore;

WebApplicationBuilder builder = WebApplication.CreateBuilder(args);
builder.Services.AddFeatureManagement();

WebApplication app = builder.Build();

// unfinished Sprint 14 work: merged to main, hidden behind the flag
app.MapPost("/api/orders/{orderId:int}/returns", (int orderId, ReturnRequest request) =>
        Results.Created($"/api/returns/{orderId}-1", new { orderId, request.Reason, Status = "Requested" }))
    .WithFeatureGate(Features.SelfServiceReturns);

// existing endpoint: shows the return button only when the flag is on
app.MapGet("/api/orders/{orderId:int}", async (int orderId, IFeatureManager features) =>
    new
    {
        orderId,
        Status = "Delivered",
        CanRequestReturn = await features.IsEnabledAsync(Features.SelfServiceReturns)
    });

app.Run();

public record ReturnRequest(string Reason);

public static class Features
{
    public const string SelfServiceReturns = "SelfServiceReturns";
}

The flag's value lives in appsettings.json, using the Microsoft Feature Management schema:

{
  "Logging": { "LogLevel": { "Default": "Warning" } },
  "feature_management": {
    "feature_flags": [
      { "id": "SelfServiceReturns", "enabled": false }
    ]
  }
}

In production the flag is off. The real responses to two requests while the app is running (the Date and Server headers removed):

$ curl -i -X POST http://127.0.0.1:5391/api/orders/1042/returns -H 'Content-Type: application/json' -d '{"reason":"Damaged cover"}'
HTTP/1.1 404 Not Found
Content-Length: 0

$ curl http://127.0.0.1:5391/api/orders/1042
{"orderId":1042,"status":"Delivered","canRequestReturn":false}

With the flag off, WithFeatureGate returns 404 as if the endpoint did not exist; the order details also tell the Angular side not to show the return button. On staging the flag is turned on with an environment variable; same build, different configuration. After restarting the app with feature_management__feature_flags__0__enabled=true, the same two requests:

$ curl -i -X POST http://127.0.0.1:5391/api/orders/1042/returns -H 'Content-Type: application/json' -d '{"reason":"Damaged cover"}'
HTTP/1.1 201 Created
Content-Type: application/json; charset=utf-8
Location: /api/returns/1042-1
Transfer-Encoding: chunked

{"orderId":1042,"reason":"Damaged cover","status":"Requested"}

$ curl http://127.0.0.1:5391/api/orders/1042
{"orderId":1042,"status":"Delivered","canRequestReturn":true}

The team now merges into main every day; the feature is tested on staging with the flag on, and Selin and the support team try it on staging at the Review. The time zone fix went to production on day three of the Sprint without waiting for a big merge. Designing flag types and lifecycles is covered in the Solution Patterns series post "Designing Feature Flags in Code: Toggle Types, Microsoft.FeatureManagement and Flag Debt"; the strategy of decoupling deploy from release, and OpenFeature, in the Deployment Strategies series post "Feature Flags: Decoupling Deploy from Release with Microsoft.FeatureManagement and OpenFeature".

A Flag Does Not Pierce the Definition of Done

There is a subtle distinction here. The code behind the flag meets the quality lines of the Definition of Done when it enters main: it was reviewed, its tests are green, the build has no warnings. But the "self-service returns" item is not Done; users cannot use it and the criteria the Product Owner accepted are not yet complete. At the end of the Sprint, by the rule we saw earlier in the series, this item returns to the Product Backlog and Selin orders it again.

The payoff of the flag is not artificially enlarging the Increment; it is making sure unfinished work does not hold the release and integration of finished work hostage. Sprint 14's Increment consisted of the time zone fix and two small items; the unfinished feature was waiting in main, integrated and tested.

A flag also has a cost: every flag adds a condition to the code and a dimension to the test matrix. That is why a line entered Bookstore's Definition of Done: the flag of a released feature is deleted from the code within two Sprints at the latest. The deletion work is written into the Product Backlog as an item on the day the flag is turned on.

Making Technical Debt Visible

The technical debt metaphor comes from Ward Cunningham's 1992 OOPSLA (Object-Oriented Programming, Systems, Languages and Applications) experience report: shipping code quickly with your first understanding is taking on debt; if you do not adjust the code as your understanding improves, you pay interest. From Scrum's point of view the real question is not what the debt is but whether it is visible. The three ways Bookstore tried, and their trade-offs:

Bookstore combined the first two. New debt is stopped by the Definition of Done; old debt sits in the backlog as items, ordered with a "why it is here" reason. If a debt item directly slows down a feature (like the 900-line method in post 03), it is folded into that feature's plan. The capacity share was dropped because it quietly eroded the Product Owner's ordering accountability. The technique of paying debt in small steps is in the Clean Code series and the upcoming Refactoring series.

Spike: Buying Knowledge

Spike is a term from XP: a timeboxed research item whose purpose is to produce knowledge, not production code. The first item in Bookstore's backlog in post 03 was exactly that: "Shipping return label API (Application Programming Interface): produce a label with the test account", two days. It produced three things: the knowledge that the API returns the label only as a PDF (Portable Document Format), more clearly written versions of the next items, and a prototype written to be thrown away.

The word spike is not in the Guide; but the "what" topic of Sprint Planning and ordering the Product Backlog by risk and learning give it a natural place. Two rules worked at Bookstore: the spike's duration is written up front, and when it runs out the team stops with whatever it has learned; the spike's code does not go to production, and the right item is rewritten with what was learned.

In Production: What to Watch

Bookstore looks at four numbers for the health of its practices: the age of open branches, the time from opening a pull request to merging it, the number of open feature flags and the age of the oldest one, and the total time main stays red. The DORA (DevOps Research and Assessment) metrics that measure delivery performance are in the CI/CD Fundamentals series post "Measuring and Speeding Up a Pipeline: DORA Metrics, Caching and Parallelism".

Feature flags become a load of their own at scale. When Bookstore's open flag count reached 14, nobody knew which combinations had been tested; the two-Sprint rule in the Definition of Done was added after that experience.

Common Pitfalls

A long-lived feature branch for every Product Backlog item. If the item takes two Sprints, so does the branch; integration risk piles up at the end of the Sprint and the Increment's usability depends on a merge.

Flags that are never deleted. As long as the flag of a released feature stays in the code, dead conditions and untested combinations pile up; the flag turns into a bigger debt than the problem it solved.

The "technical debt Sprint". A Sprint every few Sprints in which only debt is cleaned up is a cousin of the hardening sprint: it pushes quality outside the Sprints and effectively suspends the Guide's "quality does not decrease" rule in the other Sprints.

The spike with no end date. An item that starts as "we're researching the shipping API" and lasts a whole Sprint is not learning but hidden development.

A Small Experiment

★☆☆ List your team's open branches with their ages. For each one older than a working day, write why it is still open.

★★☆ Run the example from this post on your machine, then turn the flag on with appsettings.Development.json instead of an environment variable. Use curl to see the same build behave differently in two environments.

★★★ Find every feature flag in your code and fill in the columns "date turned on, date released, is there a removal item". Write a Product Backlog item for each released but not deleted flag and take them to the next Sprint Planning.

Without these practices, Scrum's events turn into an empty shell. In the next post we look at the best-known forms of that shell: Dark Scrum, Zombie Scrum, ScrumBut, Sprint 0, hardening sprints and mini-waterfall Sprints; for each, the symptom, the root cause and the first step.