tags : General Programming
Workflow
Git branching strategy against Gitflow
How do we unusually use dev
branch?
Usually don’t think clearly what’s the usecase of the dev
branch, its a relic from gitflow branching strategy but we end up using it for deployment pipeline as a way to deploy to development env(they don’t have to be coupled!), pre-prod testing(eg. always goes to dev before main kinds), sandbox testing(just want to try out this feature and see how it works etc.)
Do we really need develop
branch?
- When we use
dev
, what we essentially end up havingdevelop
andmain
as two forks of the codebase. And these two forks go out of sync due to whatever reason.- Eg. You tested the change on
develop
, you made a PR tomain
, you made some micro adjustments to the PR before it made its way tomain
. Boom! out of sync, We pushed something to develop and never cherry-picked it tomain
. etc. etc.
- Eg. You tested the change on
- The
develop
branch not only becomes a long lived feature-branch, it becomes a long-lived feature-branch that everyone keeps changing without any guard-rails. - It also has a more fundamental flaw in a way that “if we need
develop
andmain
in sync”, why are we merging something todevelop
which is also not ready formain
.
What would make this simpler?
One line answer: better git branching strategy(trunkbased) + IssueOps + invariants/development practices + branch protection rules
Better git branching strategy
- Instead of fighting
develop
we get rid ofdevelop
. - There’s just one primary branch, and it’s
main
.main
gets deployed toproduction-environment
,main
gets deployed todevelopment-environment
- Everything else are short lived branches which come as PRs and get deleted as PRs get reviewed and merged/closed.
- If the PR change needs its changed to be on
development-environment
, we can simply deploy the changes of the PR to thedevelopment-environment
directly from the PR. This allows review a PR while its live on thedevelopment-environment
.
- There are certain invariants and practices which allow such a flow to happen at teams of different sizes. (Eg. you might wonder, well I have 2 PRs that I need to test in
development-environment
how do I do that? these question are addressed by the certain invariants and practices)
-
What’s under the hood? (More on the branching strategy)
- See this reference on branching strategy
- What we’re currently following is a stripped down version of the gitflow model. (The author themselves suggests using better alternatives, see linked article)
- What I am suggesting here is the trunkbaseddevelopment(TBD). There’s also GithubFlow model which is a subset of TBD.
IssueOps
IssueOps is just a buzzword for being able to trigger operations from github issue/PR comments. In this case, we wish to deploy changes of a PR into development-environment
without merging our changes into main
because we want to test it out. We just comment something like /deploy to develop
and it’ll provision the development-environment
with the changes of our PR.
-
post-merge vs pre-merge deployments
Usually, we do deployments after merge. But if it’s for testing, deployments before merge make more sense if the environment allows it. IssueOps lets us do pre-merge deployments
post-merge
: merge tomain
> deployproduction-environment
> something broke > open a PR > roll back > pass checks/approvals > merge againpre-merge
: deploy on PRdevelopment-environment
> something broke > fix in PR > deploy on PR … > merge tomain
-
What about multiple PRs
- In certain cases, teams try to provision an entire ephemeral preview environment. Which solves if
Emiway
andKRSNA
both try to see their changes on top ofmain
. The deployment infra will spin two different environments and give both of them separate ephemeral links. - In most cases, ephemeral environments are not possible due to the nature of our environments. We resort to bathroom etiquette and let people pee in peace. We use locks.
- Once we
/deploy
if we’re not done testing our changes, we can/lock
any other deploys todevelopment-environment
. People can come/unlock
it and then deploy their changes once they feel they need to test their changes. If you just/deploy
and don’t/lock
then their/deploy
will simply override your changes if they were not merged tomain
yet. (This tooling is very nicely provided with branch-deploy)
- In certain cases, teams try to provision an entire ephemeral preview environment. Which solves if
-
Decoupling branch to environment
- We don’t want to map git branches to deploy environments. We’ve seen the problems it has caused us.
- Additionally it just allows more ways for
production-environment
to drift fromdevelopment-environment
- We just have one branch
main
, that gets deployed to all of our environments. - With IssueOps, we can test the changes of the PR on top of
main
and see those changes in the environment that we want.(in our case,development-environment
)
Invariants/development practices (for TBD)
There are additional practices/tooling that makes TBD more realistic. These are written in details in the TBD website.
- PR: TDB doesn’t mandate using
PRs
, you could push direct to trunk(main
). But that’s only okay for very small teams. We need to be using ONLY PR based workflow with shortlived-branches. - Ready to release:
main
should always be in a ready to go toproduction
state. Your PR changes must take that into account. - Feature flags: A way to work against
main
for features that are bigger than one PR. - Commit sizes: small incremental commits, tested and rolled out. Ideally should be as small as possible while still being “whole”.
- Others
- Features writing practices such as branching by abstraction, few more abstract practices we need to keep in mind when writing our code.
- Since we’ve only
main
, now we won’t be making PRs againstmain
ordevelop
, we won’t be making PRs againstdevelopment-environment
orproduction-environment
. This is a little mindshift in how we do development.
-
More on Feature flags
- For features that take longer to implement we can use feature toggles.
- This way, we don’t need to keep track of what features is in
main
, if it’s still getting tested, we would have a feature flag active ondevelopment-environment
and it’ll be live there whileproduction-environment
will not have it even if the feature is there inmain
(guard-rail by the feature flag) - If we’re writing unit tests, we need to make sure we write tests for feature regardless of they’re getting enabled in any environment.
- Links: Feature toggle, Feature Toggles (aka Feature Flags), Testing code that uses feature flags
Rollout strategy for IssueOps
Maybe a mix of issueOps and a bit post merge deploy magic
- Use issueOps to deploy to
development-environment
- Use merge to
main
as a trigger for deploy toproduction-environment
- Or we can go full overboard and use
issueOps
completely for bothdevelopment-environment
andproduction-environment
as-well as keepingdeploy-proudction-on-merge-to-main
Using branch-deploy
- We could something that allows deploying
main
todevelopment-environment
. We could try different tooling or even write our own but Github already has a very nice tool as an action: branch-deploy - It uses combination of github environments, github actions, branch protection rules, and github issue commands to allow us with a nice flow to deploy
main
intoenvironments
we want to deploy our change to. - Gotchas about
branch-deploy
- Required reviewers is
Enterprise
feature for private repos, so if you haveproduction
env specified forbranch-deploy
anyone will be able to deploy toproduction-environment
if their PR is approved.- Required-reviewers feature needs the PR to be approved by certain
- We keep deploy to
production
onmerge
and usebranch-deploy
only to deploy branch changes todevelopment
env.
- Required reviewers is
Other notes
dev/prod deploy vs branch
- Ideally we should not have production deploy/ci scripts in dev branch at all if branch per env is what we’re doing
- In env<>branch mapping ideally, we’d have a
poc
andproduction
branch which is not the case. So we’re operating in a mixed mode which is not ideal.
What about schema change?
- For databases, we expect the data to drift massively in
development-environment
andproduction-environment
but not the schema. - Database schema changes are tricky because unlike code, they may not be same in different environments as these changes are persistent unlike code. Changes can go from outside, changes can be deployed from a PR which did not land on
main
but because we deployed todevelopment-environment
it altered the schema etc. - Make sure the schema change can be rolled back. (Use
UP/DOWN
convention that the respective migration library provides) - You want to be extra careful with these changes. If you’re making a schema change that is not going in
main
, maybe rollback the schema change using deploy commands. - If ever schema migration conflicts occur between environment, resolving them is a manual process dealing with the DB. with newer migrations fixing the broken migration etc.
- Hopefully this will only ever be a rarity.
Do we want to cut releases?
- I think we only do it for
martini
at the moment - But in most of our setups we don’t really need to do the releasing ceremony. Whatever is in
main
is expected to be live in production and life is good.
Github repo stuff
Actions to be taken for each repo
- Enable “Automatically delete head branches” after PR merge
- This will delete short lived feature branches after PR is merged
- If existing repo, scrub any unused branches.
- Add
.gitmessage
Branch protection to be added
Need it? | Description |
---|---|
YES | Require a pull request before merging, w min 1 required approval |
YES | Dismiss stale pull request approvals when new commits are pushed |
YES | Require status checks to pass before merging |
Optional | Require branches to be up to date before merging* |
Preferred | Require linear history (No merge commits!, only ff commits) |
On “Require branches to be up to date before merging”, we don’t really need it at the moment so optional
- Eg. When some other merged PR added a new test case which this change might fail with
- Can be automated using a merge queue but we don’t have a strong usecase for this
git message
Add .gitmessage
so that the following automatically is populated when we try to write a commit message
See Better Commit Messages with a .gitmessage Template
# <type>(optional scope): <description>
# <type>(<opt scope>): <message in present tense>
# [opt body]
# [opt footer(s)]
#
# types: fix,feat,chore,docs, style, refactor, perf, test etc.
# scopes: api,lang etc.
#
# Breaking change:
# 1. The text "BREAKING CHANGE" anywhere in the commit message
# 2. "!" after type is breaking change. Eg. fix!: some description
# Types:
# - `feat`: new feature for the user, not a new feature for build script
# - `fix`: bug fix for the user, not a fix to a build script
# - `docs`: changes to the documentation
# - `style`: formatting, missing semi colons, etc; no production code change
# - `refactor`: refactoring production code, eg. renaming a variable
# - `test`: adding missing tests, refactoring tests; no production code change
# - `chore`: updating grunt tasks etc; no production code change
#
# See https://www.conventionalcommits.org/