OpenTofu

Symptom

Every subsequent `tofu plan`/`apply` in the pipeline fails immediately with "Error acquiring the state lock", even though no apply is currently running against that workspace.

Error Message

Error: Error acquiring the state lock

Lock Info:
  ID:        7f3a2c1e-...
  Path:      workspaces/prod/terraform.tfstate
  Operation: OperationTypeApply
  Who:       ci-runner-42
  Created:   2026-06-30 02:14:11 UTC

Root Cause

When a `tofu apply` using the HTTP state backend is interrupted (CI runner killed, job timeout, network drop), the client's deferred unlock call never fires because the process exits before it runs. The HTTP backend has no server-side lease or TTL on the lock, so it stays held indefinitely until someone force-unlocks it. This exact failure mode is documented upstream in opentofu/opentofu#3624 ("http backend fails to unlock state after interrupted tofu apply due to context canceled error").

Diagnosis Steps

1. 1.Check CI pipeline history for any run still marked in-progress or forcibly killed against the same workspace
2. 2.Inspect the lock's Who/Created fields in the error output to identify which run created it and how long ago
3. 3.Confirm with the team in chat that no one is running tofu manually against the same state before forcing an unlock
4. 4.Review backend logs (HTTP endpoint / reverse proxy) for the original lock acquisition request to corroborate the timeline

Solution

Confirm via CI run history that no other apply is genuinely in-flight against the same workspace, then run `tofu force-unlock <LOCK_ID>`. Follow up by having the platform team add a server-side lock TTL/reaper on the HTTP backend endpoint so future interruptions self-heal without manual intervention.

Commands

tofu force-unlock <LOCK_ID>

tofu plan -lock-timeout=5m

curl -s "$TF_HTTP_LOCK_ADDRESS" | jq .

Symptom

tofu apply hangs or times out trying to acquire the state lock, even though no other tofu process is running against that workspace and the state file itself shows no in-progress operation.

Error Message

Error: Error acquiring the state lock

Error message: convert pg advisory lock: ERROR: could not obtain lock on relation ... (SQLSTATE 55P03)

Root Cause

The pg backend implements state locking with Postgres advisory locks keyed by a hash of the state address. When the same Postgres instance/database is shared with an unrelated application that also uses pg_advisory_lock for its own coordination, the two can collide on the same lock key. opentofu/opentofu#2218 ("pg backend new state causes lock conflicts") documents production deployments failing this way when unrelated components share the locking database.

Diagnosis Steps

1. 1.SELECT * FROM pg_locks WHERE locktype='advisory'; join to pg_stat_activity to see which session/query holds it
2. 2.Compare the advisory lock key against the one OpenTofu computes for the workspace to confirm it's a collision, not a genuine tofu lock
3. 3.Check whether any non-Terraform/OpenTofu process connected to the same database uses pg_advisory_lock for its own purposes

Solution

Identify the colliding session via pg_locks joined to pg_stat_activity, confirm it is not a legitimate in-flight tofu apply, and either wait for it to release naturally or terminate it. Migrate the OpenTofu state/lock table to a dedicated database or schema that no other application touches, so this collision cannot recur.

Commands

SELECT pid, locktype, mode, granted FROM pg_locks WHERE locktype = 'advisory';

SELECT pg_terminate_backend(<pid>);  -- only after confirming it is NOT a live tofu apply

tofu force-unlock <LOCK_ID>

Symptom

Right after switching a workspace from the terraform binary to tofu (same state file, no config changes intended), `tofu plan` errors out on one or more providers with a schema mismatch or provider-not-found error, even though `terraform plan` against the same state worked cleanly the day before.

Error Message

Error: Failed to install provider

Error while installing hashicorp/somevendor: provider registry registry.opentofu.org does not have a provider named registry.opentofu.org/hashicorp/somevendor

Root Cause

OpenTofu resolves shorthand provider source addresses (e.g. hashicorp/aws) against registry.opentofu.org by default, not registry.terraform.io. Mainstream providers are mirrored, but providers that have not yet been published/mirrored to the OpenTofu registry (or state files carrying a legacy/implicit provider source recorded by an older Terraform version) fail to resolve. This is a known migration pitfall documented in OpenTofu's own migration guides and reported in Spacelift's knowledge base ("provider schema errors after migrating from Terraform to OpenTofu").

Diagnosis Steps

1. 1.Run tofu init -upgrade in a scratch copy of the workspace to see exactly which provider fails to resolve
2. 2.Inspect the state file's recorded provider source strings for legacy/implicit addresses from an older Terraform version
3. 3.Cross-check the failing provider against registry.opentofu.org to see whether it has been mirrored yet

Solution

For each provider that fails to resolve, pin an explicit `source = "registry.terraform.io/<namespace>/<name>"` in the required_providers block instead of relying on the shorthand default, then re-run `tofu init -upgrade` and confirm `tofu plan` shows no unexpected diff before letting anyone apply.

Commands

tofu init -upgrade

tofu providers

tofu state show <resource_address> | grep provider