Bitbucket

Challenge

Atlassian's internal engineering organization faced a significant infrastructure challenge: they were running their own Bitbucket Data Center instance hosting over 18,000 repositories used by 8,000 engineers across Sydney, Bangalore, San Francisco, Austin, and Gdansk. The on-premise instance required a dedicated team of 6 SREs to maintain, costing approximately $2.4 million annually in infrastructure and personnel. The instance experienced 3-4 partial outages per quarter during peak usage, each affecting developer productivity for 2-4 hours. Smart Mirrors in remote offices helped but added operational complexity. The decision to migrate to Bitbucket Cloud was part of Atlassian's broader 'Cloud First' strategy, but the scale — 18,000 repositories with 12 years of git history totaling 8TB of data — made this one of the largest Bitbucket Cloud migrations ever attempted.

Solution

Atlassian's internal developer experience team built a phased migration framework called 'Project Nimbus' that migrated repositories in waves of 500-1,000 per week over a 5-month period. The migration tooling used Bitbucket's REST API for repository creation and configuration, combined with custom git mirror operations that verified data integrity by comparing commit SHA chains between source and destination. Critical to the migration was preserving all pipeline configurations — the team built an automated pipeline-config-translator that adapted Data Center-specific pipeline features (like self-hosted runner references) to Cloud equivalents. Branch permissions, CODEOWNERS files, and webhook configurations were migrated using a declarative configuration format that allowed teams to review and approve their repository settings before cutover. The team implemented a dual-write period where both the Data Center instance and Cloud received pushes for 2 weeks per batch, using a custom git hook that replicated pushes to the Cloud workspace. This allowed teams to validate that their pipelines, integrations, and workflows functioned correctly on Cloud before the Data Center instance was decommissioned for their repositories. A migration dashboard built on Bitbucket Cloud's API showed real-time progress, flagging repositories with migration issues (LFS objects exceeding Cloud limits, incompatible pipeline configurations, or webhook endpoints unreachable from Cloud IP ranges).

Outcome

SRE team for Bitbucket infrastructure reduced from 6 to 1 (maintaining integrations only). Infrastructure cost reduced from $2.4M to $680K annually (72% reduction). Developer-reported git operation latency improved by 40% globally due to Bitbucket Cloud's CDN-backed architecture. Outage frequency dropped from 3-4 partial outages per quarter to zero self-inflicted outages.

Scale

18,000 repositories, 8,000 engineers, 8TB of git data, 5 global offices, 5-month migration window

Key Learnings

• Dual-write migration periods are essential for large migrations — they provide a safety net and allow teams to validate integrations without a hard cutover deadline
• Pipeline configuration translation between Data Center and Cloud cannot be fully automated — approximately 15% of pipelines required manual adjustments for self-hosted runner dependencies
• Repository-level migration batching by team (not alphabetically) reduces coordination overhead because each team handles their own validation window

Challenge

BMW's Connected Drive and ADAS (Advanced Driver Assistance Systems) division had development teams spread across Munich (Germany), Spartanburg (USA), Shanghai (China), and Pretoria (South Africa). Each location had independently evolved their CI/CD practices, resulting in 4 different pipeline architectures: Munich used Jenkins, Spartanburg used CircleCI, Shanghai had a custom build system, and Pretoria used Azure DevOps. This fragmentation made it impossible to enforce consistent quality gates, security scanning, or deployment procedures across the division. When a safety-critical bug was found in the lane-keep-assist module, the root cause analysis revealed that the Shanghai team's custom build system didn't run the same static analysis checks as Munich's Jenkins pipeline, and the bug would have been caught by those checks. The regulatory team flagged this as an ISO 26262 compliance gap that needed to be resolved before the next model year's software could be certified.

Solution

BMW's platform engineering team selected Bitbucket Cloud as the unified source control and CI/CD platform for the entire Connected Drive division, migrating all 4 locations to a single Bitbucket workspace with standardized pipeline templates over a 9-month period. The migration was executed location-by-location, with Munich (the largest team at 85 engineers) going first to establish the patterns. The team built a library of 12 custom Bitbucket Pipes encapsulating BMW-specific operations: automotive static analysis (MISRA C/C++ compliance checking), functional safety test execution, hardware-in-the-loop (HIL) test orchestration via self-hosted runners connected to physical test benches, and regulated artifact signing using BMW's internal PKI infrastructure. Pipeline templates were distributed as a 'golden pipeline' repository that teams imported via git submodules, ensuring that updates to quality gates propagated automatically. The golden pipeline enforced a mandatory stage gate model: code compilation, MISRA compliance scan, unit tests, integration tests with HIL simulation, security scan, and deployment — with each gate producing auditable evidence stored in BMW's compliance artifact repository. Self-hosted Bitbucket Pipeline runners were deployed in each location's data center to handle HIL test execution, which required physical connectivity to automotive ECU test benches that couldn't be replicated in the cloud. The runners were managed using Ansible with BMW's standard hardened Linux image, ensuring consistent runner environments across all locations.

Outcome

Pipeline configuration divergence eliminated — all 87 repositories use the same golden pipeline template. MISRA compliance scan coverage increased from 62% to 100% of repositories. Security vulnerability detection improved by 340% due to consistent scanning across all locations. ISO 26262 audit passed with zero findings on CI/CD process consistency. Average time from commit to deployable artifact reduced from 4.2 hours to 52 minutes.

Scale

200 engineers, 4 countries, 87 Bitbucket repositories, 12 custom pipes, 450 daily pipeline executions

Key Learnings

• Custom Bitbucket Pipes are the right abstraction for domain-specific CI operations like automotive static analysis — they encapsulate complexity while maintaining a simple YAML interface
• Self-hosted runners are essential for industries with physical test infrastructure (HIL benches) that cannot be cloud-hosted, but they must be treated as managed infrastructure with automated provisioning
• Golden pipeline templates distributed via git submodules provide the right balance between standardization and team-specific customization

Challenge

Samsung's Mobile Communications division, responsible for Galaxy smartphone firmware, had one of the largest Bitbucket Data Center deployments in the world, serving over 3,000 engineers across Suwon (South Korea), Noida (India), Ho Chi Minh City (Vietnam), and Warsaw (Poland). The Bitbucket instance hosted 2,400 repositories containing firmware source code, device drivers, kernel modules, and OTA update packages. The primary challenge was performance at scale: during the pre-launch development sprint for each new Galaxy device, the instance experienced peak loads of 15,000 git operations per minute, causing clone and fetch operations to take 5-10 minutes for large firmware repositories. The monolithic firmware repository for the Galaxy S series alone was 45GB, and cloning it consumed significant network bandwidth across Samsung's international WAN links. Smart Mirror synchronization for the Noida and Ho Chi Minh City offices lagged by up to 30 minutes during peak periods, causing engineers to work with stale code.

Solution

Samsung's SCM platform team implemented a multi-layered performance optimization strategy for their Bitbucket Data Center deployment. The first layer was repository architecture reform: the monolithic 45GB firmware repository was decomposed into 35 component repositories (kernel, drivers, HAL, framework, apps) using git submodules with a manifest repository that pinned component versions for each device model. This reduced the largest single repository from 45GB to 8GB and allowed engineers to clone only the components they were actively developing. The second layer was Bitbucket Data Center cluster scaling: the cluster was expanded from 4 nodes to 12 nodes behind an NGINX load balancer with sticky sessions based on repository hash, ensuring that git operations for the same repository consistently hit the same node's warm filesystem cache. The third layer was Smart Mirror optimization: mirrors in Noida and Ho Chi Minh City were upgraded from a single mirror server to 3-node mirror clusters, each with NVMe storage and 10Gbps network connections. The team implemented a priority-based synchronization system where repositories tagged as 'active-development' were synchronized every 30 seconds, while archived repositories synced hourly. The fourth layer was the introduction of partial clone support: by enabling Bitbucket's partial clone (git clone --filter=blob:none) configuration, engineers could clone the repository structure without downloading all historical blobs, reducing initial clone times from 10 minutes to 45 seconds. Blob content was fetched on-demand during git checkout and git diff operations.

Outcome

Galaxy S series firmware repository clone time reduced from 10 minutes to 45 seconds using partial clone. Smart Mirror synchronization lag reduced from 30 minutes to under 60 seconds for active repositories. Bitbucket Data Center uptime improved from 99.2% to 99.95%. Peak git operation throughput increased from 15,000 to 42,000 operations per minute without latency degradation.

Scale

3,000 engineers, 2,400 repositories, 12-node Bitbucket Data Center cluster, 3 Smart Mirror sites, 15,000 git ops/minute peak load

Key Learnings

• Monolithic firmware repositories must be decomposed before performance optimization — no amount of infrastructure scaling compensates for a 45GB repository that every engineer clones daily
• Partial clone (blob:none filter) is the single highest-impact optimization for large repositories, but it requires Bitbucket Data Center 7.6+ and git 2.22+ on all client machines
• Smart Mirror synchronization priority tiers are essential at scale — not all repositories need sub-minute sync, and over-synchronizing low-activity repos wastes network bandwidth

Challenge

Commonwealth Bank of Australia (CBA), the country's largest bank by market capitalization, processes over $2.3 billion in daily digital transactions through services built and deployed via Bitbucket. Their digital banking platform team of 280 engineers manages 156 Bitbucket repositories containing payment processing, customer authentication, and account management code. Following a PCI-DSS v4.0 assessment, the QSA (Qualified Security Assessor) identified three critical compliance gaps in CBA's software development lifecycle: (1) insufficient code review enforcement for payment-related code changes, with 18% of production deployments bypassing the two-reviewer requirement; (2) lack of segregation of duties between developers who write code and operators who deploy it; and (3) inadequate audit trails for tracing production deployments back to specific code reviews and approvals. The QSA gave CBA 90 days to remediate these findings or face restrictions on their ability to process card-present and card-not-present transactions.

Solution

CBA's DevSecOps team implemented a comprehensive compliance framework using Bitbucket's branch permissions, merge checks, deployment environments, and audit logging capabilities. For code review enforcement, they configured branch permissions on all 156 repositories to block direct pushes to main, release/*, and hotfix/* branches. Merge checks were configured to require a minimum of 2 approvals for standard code changes and 3 approvals for changes to files in the payments/, crypto/, and auth/ directories (enforced via a custom merge check plugin on their Bitbucket Data Center instance). CODEOWNERS files were deployed to every repository, mapping payment-critical directories to the security-review team and requiring their explicit approval before merge. For segregation of duties, the team configured Bitbucket deployment environments (staging, pre-production, production) with distinct permission groups: developers could trigger staging deployments but only members of the deployment-operations group could approve production deployments. The deployment approval was implemented as a manual trigger step in Bitbucket Pipelines with the deployment: production designation, and the production environment was configured to only accept approvals from the ops team. For audit trail completeness, the team deployed a Bitbucket webhook consumer that captured all PR merge events, deployment approvals, and branch permission changes, streaming them to an immutable log store in AWS CloudTrail. A custom compliance dashboard aggregated this data, showing for each production deployment: the PR that introduced the change, all reviewers and their approval timestamps, the pipeline execution ID, and the operator who approved the production deployment — creating a complete chain of custody from code change to production.

Outcome

PCI-DSS compliance gaps fully remediated within 67 days (23 days ahead of deadline). Code review bypass rate dropped from 18% to 0%. Production deployment audit trail completeness reached 100%. Segregation of duties between development and deployment achieved across all repositories. QSA follow-up assessment passed with zero findings.

Scale

280 engineers, 156 repositories, $2.3B daily transaction volume, PCI-DSS v4.0 compliance, 90-day remediation deadline

Key Learnings

• PCI-DSS compliance for CI/CD is achievable with Bitbucket's native features (branch permissions, merge checks, deployment environments) without requiring expensive third-party compliance tools
• CODEOWNERS files are the most underutilized Bitbucket feature for regulated environments — they provide path-based review requirements that map directly to PCI-DSS segregation of duties requirements
• Immutable audit logging must be implemented outside Bitbucket (in CloudTrail or similar) because Bitbucket's native audit logs can be modified by workspace administrators

Challenge

Lufthansa's IT subsidiary, Lufthansa Industry Solutions (LHIND), develops and maintains safety-critical flight operations software including crew scheduling, flight planning, weight and balance calculations, and maintenance tracking systems used by Lufthansa Group airlines (Lufthansa, Swiss, Austrian, Brussels Airlines, Eurowings). These systems are subject to EASA (European Union Aviation Safety Agency) certification requirements that mandate rigorous change management, full traceability from requirements to deployed code, and evidence that every change was reviewed by qualified personnel. The existing development process used SVN repositories with manual build procedures documented in 200-page runbooks, making it impossible to achieve the deployment velocity needed to respond to operational changes (e.g., rapid schedule modifications during weather disruptions or airspace closures). A single software update to the crew scheduling system took 6 weeks from code freeze to production deployment due to the manual certification evidence gathering process.

Solution

LHIND migrated their flight operations software development to Bitbucket Data Center with a pipeline architecture specifically designed to generate EASA-compliant certification evidence automatically. The migration from SVN to Bitbucket was executed using SubGit to preserve complete revision history, with a custom mapping that converted SVN properties (svn:log metadata used for traceability) to git notes attached to corresponding commits. The pipeline architecture implemented a concept called 'Certification-as-Code' where each pipeline step produced structured evidence artifacts that were assembled into the EASA-required Software Accomplishment Summary (SAS) document. The pipeline enforced EASA DO-178C objectives automatically: static analysis at Level C assurance (detecting all undefined behavior, data coupling errors, and control flow anomalies), test coverage measurement confirming MC/DC (Modified Condition/Decision Coverage) at 100%, and formal verification of safety-critical mathematical functions using the SPARK/Ada verification toolchain. Bitbucket branch permissions were configured to model the EASA-required organizational roles: Developer, Reviewer (independent from author), Quality Assurance, and Configuration Manager — each with specific branch-level permissions ensuring that code could only progress through the pipeline with the correct sequence of approvals. A custom Bitbucket plugin tracked the complete traceability chain from JIRA requirements to Bitbucket commits to test results to deployment records, generating a traceability matrix that was previously compiled manually over 2-3 weeks.

Outcome

Software update cycle reduced from 6 weeks to 8 days (including mandatory regulatory hold periods). Certification evidence generation time reduced from 2-3 weeks manual effort to 4 hours automated. First DO-178C certified software pipeline in the Lufthansa Group. Annual cost savings of EUR 1.8 million from eliminated manual certification processes.

Scale

140 engineers, 68 repositories, 5 airline brands, EASA DO-178C Level C certification, 12 safety-critical systems

Key Learnings

• Aviation software certification is compatible with CI/CD — the key is generating certification evidence as pipeline artifacts rather than as manual documents
• Bitbucket's branch permission model maps naturally to EASA organizational roles (developer, reviewer, QA, configuration manager) when configured with role-based groups
• The traceability matrix (requirements → code → tests → deployment) is the single most time-consuming certification artifact to produce manually and benefits most from automation

Challenge

Tesla's Autopilot team develops the neural network models, planning algorithms, and control systems that power Full Self-Driving (FSD) capabilities across the Tesla vehicle fleet. The team's simulation testing infrastructure needed to validate every code change against a dataset of 4 million recorded driving scenarios collected from the Tesla fleet, requiring GPU-accelerated processing that couldn't run on standard CI/CD cloud runners. The existing workflow involved developers manually submitting simulation jobs to an internal HPC cluster, waiting 4-12 hours for results, and then manually correlating simulation outcomes with specific code changes in Bitbucket. This disconnection between code review in Bitbucket and simulation testing on the HPC cluster meant that simulation failures were discovered hours or days after the code was merged, requiring expensive bisection to identify which commit introduced the regression. During the critical FSD v12 development cycle, the team identified that 34% of simulation regressions were caused by changes that had already been merged to main before simulation results were available.

Solution

Tesla's Autopilot platform team built a deep integration between Bitbucket Pipelines and their GPU simulation cluster using self-hosted Bitbucket Pipeline runners deployed on NVIDIA DGX A100 nodes. The architecture consisted of 24 self-hosted runner instances, each running on a DGX node with 8 A100 GPUs and 1TB of RAM, registered with the Bitbucket workspace as labeled runners (label: gpu-simulation). The bitbucket-pipelines.yml for the Autopilot repository was restructured into a multi-phase pipeline: Phase 1 (CPU steps on cloud runners) handled code compilation, unit tests, static analysis, and neural network model validation checks in under 8 minutes. Phase 2 (GPU steps on self-hosted runners) triggered automatically upon Phase 1 success, executing the simulation test suite across the 4 million scenario dataset. The simulation workload was distributed across the 24 DGX nodes using a custom Bitbucket Pipe that implemented intelligent scenario sharding based on scenario complexity ratings — simple highway scenarios ran on fewer GPUs while complex urban intersection scenarios received more GPU resources. Simulation results were streamed back to Bitbucket as pipeline step logs and summarized as a PR comment using the Bitbucket API, providing developers with immediate feedback on which driving scenarios regressed and linking to replay visualizations. For PR pipelines, a representative subset of 50,000 scenarios (selected by a stratified sampling algorithm that ensured coverage of all scenario categories) was used to provide simulation feedback within 35 minutes, while the full 4 million scenario suite ran post-merge as a branch pipeline on main.

Outcome

Simulation regression detection shifted from post-merge (34% missed) to pre-merge (97% caught in PR). Time to simulation feedback reduced from 4-12 hours to 35 minutes for PR subset. Full simulation suite completion time reduced from 18 hours to 2.5 hours through GPU-aware sharding. Developer simulation workflow fully integrated into Bitbucket PR review process, eliminating the separate HPC submission step.

Scale

24 DGX A100 nodes (192 GPUs total), 4 million driving scenarios, 160 Autopilot engineers, 180 daily pipeline executions, 50,000 scenario PR subset

Key Learnings

• Self-hosted Bitbucket Pipeline runners on GPU infrastructure must be configured with Docker GPU passthrough (--gpus all) and NVIDIA Container Toolkit — this is not supported on Bitbucket Cloud runners, making self-hosted runners mandatory for GPU workloads
• Stratified scenario sampling for PR pipelines is more effective than random sampling — it ensures every driving scenario category (highway, urban, parking, weather conditions) is represented in the fast-feedback subset
• Streaming simulation results as PR comments with linked replay visualizations dramatically improved code review quality for perception algorithm changes