Repository navigation
sfs: provider panics with a nil pointer dereference when the API returns an undecodable response #1743
Description
Description
The SFS resources and data sources crash the provider process with a nil pointer dereference when an API call
returns a response that the SDK decodes to nil. Terraform reports it as The plugin encountered an error, and failed to respond to the plugin6.(*GRPCProvider).ReadResource call, so the operator gets a plugin crash instead of
a diagnostic.
Two routes lead to a nil response, neither of which requires anything unusual from the SFS API:
1. A 2xx answer with an empty body. The generated SDK decoder returns early for an empty body
(v1api/client.go):
func (c *APIClient) decode(v interface{}, b []byte, contentType string) (err error) {
if len(b) == 0 {
return nil
}localVarReturnValue therefore stays nil and Execute() returns (nil, nil).
2. A single retryable gateway error during a wait. WaiterHelper.Wait returns true for waitFinished on any
fetch error, handleError swallows a 502 or 504 and returns a nil error, and the loop then returns the nil response
because it is already done — the retry never happens. Reported separately as
stackitcloud/stackit-sdk-go#11084 with a runnable reproduction.
The call sites check only the returned error. Several of the checks that are meant to validate a response are
themselves the dereference — they test x.Field == nil without first testing x == nil:
// stackit/internal/services/sfs/resourcepool/resource.go, v0.113.0
if response.ResourcePool == nil || response.ResourcePool.Id == nil { // panics when response is niland in both create paths the wait handler result is read by tflog.SetField one line before its guard.
Steps to reproduce
A plain Terraform configuration cannot trigger this deterministically — it needs the API to answer a GET with an
empty body or a single 502, which is not something a config can provoke. The configuration involved is unremarkable:
resource "stackit_sfs_resource_pool" "this" {
project_id = var.project_id
name = "sfs-pool"
availability_zone = "eu01-m"
performance_class = "Standard"
size_gigabytes = 512
ip_acl = ["192.168.2.0/24"]
}The crash reproduces deterministically as a unit test against a mock server. Add this to
stackit/internal/services/sfs/sfs_test.go and run
go test ./stackit/internal/services/sfs/ -run TestSfsResourcePoolReadHandlesEmptyResponse:
func TestSfsResourcePoolReadHandlesEmptyResponse(t *testing.T) {
projectId := uuid.NewString()
resourcePoolId := uuid.NewString()
const region = "eu01"
s := testutil.NewMockServer(t)
defer s.Server.Close()
tfConfig := fmt.Sprintf(`
provider "stackit" {
default_region = "%s"
sfs_custom_endpoint = "%s"
service_account_token = "mock-server-needs-no-auth"
enable_beta_resources = true
}
resource "stackit_sfs_resource_pool" "resourcepool" {
project_id = "%s"
name = "sfs-instance"
availability_zone = "eu01-m"
performance_class = "Standard"
size_gigabytes = 512
ip_acl = ["192.168.2.0/24"]
}
`, region, s.Server.URL, projectId)
resource.UnitTest(t, resource.TestCase{
ProtoV6ProviderFactories: testutil.TestAccProtoV6ProviderFactories,
Steps: []resource.TestStep{
{
// Fail the create wait so the IDs are in state and the next step can refresh them.
PreConfig: func() {
s.Reset(
testutil.MockResponse{
Description: "create resource pool",
ToJsonBody: sfs.CreateResourcePoolResponse{
ResourcePool: &sfs.ResourcePool{Id: new(resourcePoolId)},
},
},
testutil.MockResponse{Description: "failing waiter", StatusCode: http.StatusInternalServerError},
)
},
Config: tfConfig,
ExpectError: regexp.MustCompile("Error creating resource pool"),
},
{
PreConfig: func() {
s.Reset(
testutil.MockResponse{Description: "refresh with an empty body", StatusCode: http.StatusOK},
testutil.MockResponse{Description: "delete", StatusCode: http.StatusAccepted},
testutil.MockResponse{Description: "delete waiter", StatusCode: http.StatusNotFound},
)
},
RefreshState: true,
ExpectError: regexp.MustCompile("Error reading resource pool"),
},
},
})
}Actual behavior
The provider process panics:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x0 pc=0x105620e48]
github.com/stackitcloud/terraform-provider-stackit/stackit/internal/services/sfs/resourcepool.(*resourcePoolResource).Read(...)
stackit/internal/services/sfs/resourcepool/resource.go:375
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ReadResource(...)
terraform-plugin-framework@v1.19.0/internal/fwserver/server_readresource.go:156
Line 375 is err = mapFields(ctx, region, response.ResourcePool, &model); on the released v0.113.0 it is line 372.
Affected sites, all of them reachable by one of the two routes above:
| file | method |
|---|---|
sfs/resourcepool/resource.go |
Create (wait result, post-create GET), Read, Update (PATCH response, wait result) |
sfs/resourcepool/datasource.go |
Read |
sfs/share/resource.go |
Create (POST response, wait result, post-create GET), Read, Update (PATCH response, wait result) |
sfs/share/datasource.go |
Read |
sfs/snapshots/datasource.go |
Read |
export-policy and snapshot-policy already guard their responses, project-lock uses nil-receiver-safe getters,
and every SFS Delete discards the response, so the delete waiter's legitimate nil return on a 404 is harmless.
Expected behavior
A response the SDK could not decode should produce a Terraform diagnostic, not a plugin crash. The package already
has the right shape for this in resourcepool/resource.go — one combined condition covering the response and the
fields actually used:
if resourcePool == nil || resourcePool.ResourcePool == nil || resourcePool.ResourcePool.Id == nil {
core.LogAndAddError(ctx, &resp.Diagnostics, "error creating resource pool", "Calling API: Incomplete response (id missing)")
return
}Environment
- OS: macOS 15 (darwin/arm64)
- Terraform version (see
terraform --version): reproduced throughterraform-plugin-testingv1.16.0; not version-specific - Version of the STACKIT Terraform provider:
v0.113.0, andmainat a7d114b
Additional information
A fix is open as #1741. It guards the call sites so the provider reports a diagnostic; the underlying
(nil, nil) return from the wait handler is stackitcloud/stackit-sdk-go#11084.
Activity
devpie commentedon Sep 7, 2026
This is not an SFS-only problem, and the sweep turned up one crash that is plainer than any of them.
I ran NilAway over
main(a7d114b), one service at a time. The raw output is3122 findings, but that number is misleading in two ways: the same site is reported once per service that pulls it in,
and 94% of the raw hits come from acceptance-test code indexing
config.Variablesmaps, all funnelling into a singlehelper in
internal/testutil. Deduplicated and split:The plainest finding first
stackit/internal/services/iaas/server/resource.go:840dereferences a server that may be nil, because the call thatproduced it never returns on error:
Any failing
GetServer— a 500, a timeout, a transient network error — logs a diagnostic and then panics onserver.Status, so the operator gets a plugin crash instead of the error message that was just recorded. This needs noassumption about response bodies.
The identical missing
returnsits at:577in the create path, but there it is saved by accident:mapFieldsrejects a nil response at
:949and the caller returns at:583-585. So:588, which NilAway also flags, is a falsepositive.
Filed separately as #1746 with a fix in #1747, since it is a different defect from the one this issue describes.
The bulk of it
Of the 130 production sites:
Execute()returning(nil, nil), dereferenced uncheckednil— 11 provider functions, 6 generated SDK models (GetActualInstance())iaas/serversites aboveThese buckets follow NilAway's message wording rather than the underlying mechanism; grouped by mechanism the SDK's
share is larger than the first row alone suggests.
The 97 are the class this issue is about. They span 34 of the 43 service directories under
stackit/internal/services/— 33 distinct STACKIT services, sinceiaasandiaasalphaare the stable and the alphaAPI of the same one:
iaas17,observability7,sfs6,albwaf4,scf4,dns3,loadbalancer3,logme3,and 25 more. A site counts here when NilAway traces the nil back to the generated client's unassigned
localVarReturnValue; the twoiaas/serversites are excluded because it flags them only as an unguardedExecute()result.
What that means for #1741: it covers the SFS subset only. Measured with the same tool, SFS production code goes
from 6 sites to 0 with that branch applied, and the remaining 91 are untouched — the commit changes six files, all
under
stackit/internal/services/sfs/, and no shared package or dependency.A caveat I want to state plainly: the 97 are candidates, not confirmed crashes. NilAway does produce false
positives —
iaas/server/resource.go:588above is one — and "the response can be nil" is formally always true here.The real question per site is whether that particular endpoint can answer 2xx with a body the generated decoder turns
into
nil.I reproduced two, both by pointing the provider at a mock server answering
200with an empty body: the SFS shareCreate(row 80,share/resource.go:246) and the SFS resource poolReaddescribed above. Both panic onmain. TheReadis not a row of its own here — NilAway reportsresourcepool/resource.go:375as a second dereference point ofthe same nil source as row 77. That leaves 96 rows I have not reproduced, and forcing an empty body from a mock does
not show that any real endpoint sends one.
On the other nil route: the wait handlers can also hand back
(nil, nil), reported asstackitcloud/stackit-sdk-go#11084. If retrying 5xx responses moves into the SDK's HTTP client instead of living in the
wait handlers, that route closes by itself — the handler would no longer swallow an error and then fall through to its
donebranch. It would not touch the 97 here, which come from the empty-body path indecode().Ways forward
x == nilguard covering the response and thefields actually used, matching the pattern already used elsewhere in the codebase.
decode(), which returns early for an emptybody and leaves the return value
nilon the success path. Whether that should change in the SDK rather than in 97call sites is your call — it would be a behaviour change for every consumer, so I did not assume it.
Before doing 1 at scale I would sample 10–15 sites and check how many are real, so the effort goes where it pays. I am
happy to do any of these.
All 97 sites (file:line, SDK call, dereferenced field)
stackit/internal/services/alb/applicationloadbalancer/resource.go:1180CreateLoadBalancerExecute()Namestackit/internal/services/alb/applicationloadbalancer/resource.go:1302UpdateLoadBalancerExecute()Namestackit/internal/services/albcertificates/certificate/resource.go:241CreateCertificateExecute()Idstackit/internal/services/albwaf/custom_rule_group/resource.go:413CreateCustomRuleGroupExecute()Namestackit/internal/services/albwaf/managed_rule_set/resource.go:300CreateManagedRuleSetExecute()Namestackit/internal/services/albwaf/waf_configuration/resource.go:237CreateWAFExecute()Namestackit/internal/services/albwaf/waf_configuration/resource.go:369UpdateWAFExecute()Namestackit/internal/services/cdn/distribution/datasource.go:417GetDistributionExecute()Distributionstackit/internal/services/cdn/distribution/resource.go:813CreateDistributionExecute()Distributionstackit/internal/services/dns/recordset/resource.go:238CreateRecordSetExecute()Rrsetstackit/internal/services/dns/zone/datasource.go:261ListZonesExecute()TotalItemsstackit/internal/services/dns/zone/resource.go:337CreateZoneExecute()Zonestackit/internal/services/edgecloud/plans/datasource.go:163ListPlansProjectExecute()ValidPlansstackit/internal/services/git/instance/resource.go:223CreateInstanceExecute()Idstackit/internal/services/iaas/affinitygroup/resource.go:221CreateAffinityGroupExecute()Idstackit/internal/services/iaas/image/resource.go:447CreateImageExecute()Idstackit/internal/services/iaas/imagev2/datasource.go:433ListImagesExecute()Itemsstackit/internal/services/iaas/machinetype/datasource.go:186ListMachineTypesExecute()Itemsstackit/internal/services/iaas/network/resource.go:491CreateNetworkExecute()Idstackit/internal/services/iaas/networkarea/resource.go:179CreateNetworkAreaExecute()Idstackit/internal/services/iaas/networkarea/resource.go:340ListNetworkAreaRegionsExecute()Regionsstackit/internal/services/iaas/networkarearegion/resource.go:702ListNetworkAreaRangesExecute()Itemsstackit/internal/services/iaas/networkarearoute/resource.go:366CreateNetworkAreaRouteExecute()Itemsstackit/internal/services/iaas/networkinterface/resource.go:310CreateNicExecute()Idstackit/internal/services/iaas/publicip/resource.go:216CreatePublicIPExecute()Idstackit/internal/services/iaas/securitygroup/resource.go:233CreateSecurityGroupExecute()Idstackit/internal/services/iaas/securitygrouprule/resource.go:493CreateSecurityGroupRuleExecute()Idstackit/internal/services/iaas/server/resource.go:552CreateServerExecute()Idstackit/internal/services/iaas/server/resource.go:775UpdateServerExecute()MachineTypestackit/internal/services/iaas/volume/resource.go:479CreateVolumeExecute()Idstackit/internal/services/iaas/volume/resource.go:611UpdateVolumeExecute()Sizestackit/internal/services/iaasalpha/vpcroutingtable/resource.go:259AddVPCRoutingTableExecute()Idstackit/internal/services/intake/runner/resource.go:247CreateIntakeRunnerExecute()Idstackit/internal/services/loadbalancer/loadbalancer/resource.go:785CreateLoadBalancerExecute()Namestackit/internal/services/loadbalancer/observability-credential/resource.go:217CreateCredentialsExecute()Credentialstackit/internal/services/loadbalancer/observability-credential/resource.go:275GetCredentialsExecute()Credentialstackit/internal/services/logme/credential/resource.go:228CreateCredentialsExecute()Idstackit/internal/services/logme/instance/resource.go:478CreateInstanceExecute()InstanceIdstackit/internal/services/logme/instance/resource.go:982ListOfferingsExecute()Offeringsstackit/internal/services/logs/accesstoken/resource.go:290CreateAccessTokenExecute()Idstackit/internal/services/mariadb/credential/resource.go:254CreateCredentialsExecute()Idstackit/internal/services/mariadb/instance/resource.go:794ListOfferingsExecute()Offeringsstackit/internal/services/modelserving/token/resource.go:307CreateTokenExecute()Tokenstackit/internal/services/mongodbflex/instance/resource.go:1085ListFlavorsExecute()Flavorsstackit/internal/services/objectstorage/credential/resource.go:305CreateAccessKeyExecute()KeyIdstackit/internal/services/observability/alertgroup/datasource.go:169GetAlertgroupExecute()Datastackit/internal/services/observability/alertgroup/resource.go:329CreateAlertgroupsExecute()Datastackit/internal/services/observability/instance/resource.go:1033CreateInstanceExecute()InstanceIdstackit/internal/services/observability/instance/resource.go:2564ListPlansExecute()Plansstackit/internal/services/observability/log-alertgroup/datasource.go:165GetLogsAlertgroupExecute()Datastackit/internal/services/observability/log-alertgroup/resource.go:276CreateLogsAlertgroupsExecute()Datastackit/internal/services/observability/scrapeconfig/datasource.go:219GetScrapeConfigExecute()Datastackit/internal/services/opensearch/credential/resource.go:254CreateCredentialsExecute()Idstackit/internal/services/opensearch/instance/resource.go:416CreateInstanceExecute()InstanceIdstackit/internal/services/opensearch/instance/resource.go:864ListOfferingsExecute()Offeringsstackit/internal/services/postgresflex/instance/datasource.go:270GetInstanceExecute()FlavorIdstackit/internal/services/rabbitmq/credential/resource.go:268CreateCredentialsExecute()Idstackit/internal/services/rabbitmq/instance/resource.go:412CreateInstanceExecute()InstanceIdstackit/internal/services/rabbitmq/instance/resource.go:875ListOfferingsExecute()Offeringsstackit/internal/services/redis/credential/resource.go:257CreateCredentialsExecute()Idstackit/internal/services/redis/instance/resource.go:483CreateInstanceExecute()InstanceIdstackit/internal/services/redis/instance/resource.go:953ListOfferingsExecute()Offeringsstackit/internal/services/resourcemanager/folder/resource.go:218CreateFolderExecute()ContainerIdstackit/internal/services/resourcemanager/project/resource.go:222CreateProjectExecute()ContainerIdstackit/internal/services/scf/organization/resource.go:262CreateOrganizationExecute()Guidstackit/internal/services/scf/organization/resource.go:280ApplyOrganizationQuotaExecute()QuotaIdstackit/internal/services/scf/organization/resource.go:424GetOrganizationExecute()QuotaIdstackit/internal/services/scf/organizationmanager/resource.go:241CreateOrgManagerExecute()Guidstackit/internal/services/secretsmanager/instance/resource.go:216CreateInstanceExecute()Idstackit/internal/services/secretsmanager/instance/resource.go:560ListACLsExecute()Aclsstackit/internal/services/secretsmanager/user/resource.go:206CreateUserExecute()Idstackit/internal/services/serverbackup/schedule/resource.go:280CreateBackupScheduleExecute()Idstackit/internal/services/serverbackup/schedule/resource.go:560ListBackupsExecute()Itemsstackit/internal/services/serverupdate/schedule/resource.go:257CreateUpdateScheduleExecute()Idstackit/internal/services/serviceaccount/account/datasource.go:135ListServiceAccountsExecute()Itemsstackit/internal/services/serviceaccount/accounts/datasource.go:178ListServiceAccountsExecute()Itemsstackit/internal/services/sfs/resourcepool/datasource.go:118GetResourcePoolExecute()ResourcePoolstackit/internal/services/sfs/resourcepool/resource.go:441UpdateResourcePoolExecute()ResourcePoolstackit/internal/services/sfs/share/datasource.go:112GetShareExecute()Sharestackit/internal/services/sfs/share/resource.go:246CreateShareExecute()Sharestackit/internal/services/sfs/share/resource.go:406UpdateShareExecute()Sharestackit/internal/services/sfs/snapshots/datasource.go:128ListResourcePoolSnapshotsExecute()ResourcePoolSnapshotsstackit/internal/services/ske/cluster/resource.go:1060ListProviderOptionsExecute()KubernetesVersionsstackit/internal/services/ske/kubeconfig/resource.go:496GetClusterExecute()Statusstackit/internal/services/ske/kubeconfig/resource.go:514GetClusterExecute()Statusstackit/internal/services/sqlserverflex/flavors/datasource.go:195ListFlavorsExecute()Paginationstackit/internal/services/sqlserverflex/instance/datasource.go:287GetInstanceExecute()FlavorIdstackit/internal/services/sqlserverflex/instance/resource.go:736CreateInstanceExecute()Idstackit/internal/services/telemetryrouter/accesstoken/resource.go:399UpdateAccessTokenExecute()Idstackit/internal/services/telemetryrouter/destination/resource.go:690UpdateDestinationExecute()Idstackit/internal/services/telemetryrouter/instance/resource.go:429UpdateTelemetryRouterExecute()Idstackit/internal/services/valkey/credential/resource.go:256CreateCredentialsExecute()Idstackit/internal/services/valkey/instance/resource.go:480CreateInstanceExecute()InstanceIdstackit/internal/services/valkey/instance/resource.go:944ListOfferingsExecute()Offeringsstackit/internal/services/vpn/connection/datasource.go:290GetGatewayConnectionExecute()-stackit/internal/services/vpn/connection/resource.go:1024CreateGatewayConnectionExecute()-stackit/internal/services/vpn/gateway/resource.go:332CreateGatewayExecute()IdSerseusWasTaken commentedon Sep 7, 2026
Thanks @devpie for reporting this issue.
I have created a internal tracking issue and we will be taking a look at this in the near future.
Kind regards,
Jonas from the STACKIT Developer Tools team
veriton-dev commentedon Sep 7, 2026
This class of crash is usually a missing null-guard at the resource Read boundary, not deep in the SDK.
Pattern that works for generated clients:
resp == nil/ empty 2xx body as a first-class outcome — never dereference fields.Errorwith the HTTP status + path), not a plugin panic.ReadResource/ReadDataSource, preferRemoveResourceFromState(or equivalent) when the API says “gone/empty” only if that is the real semantics; otherwise return a hard error so apply stops cleanly.Assert: provider process stays alive + diagnostic string contains status/path.
If you want a tight PR for the SFS Read path only, I can take a paid micro-pass ($1–$3 findings+diff, pay-after-delivery on first orders): acer-openclaw@agentmail.to · https://veriton-dev.github.io/veriton-micro-dev/
devpie commentedon Sep 8, 2026
I sampled the 97 to find out how many are real, before proposing work on the rest.
Method. 14 sites, stratified rather than the first 14 of the table: 14 distinct services and all four call
kinds (4 create, 4 get, 4 list, 2 update). Each site was traced from the
Execute()to the dereference, followinginto every function called in between, and each verdict was then checked by two independent reviewers whose brief was
to overturn it. Row 78, which #1741 already fixes, went in as a control.
Result: 13 of 14 are real, 0 are prevented by a downstream guard.
The fourteenth,
scf/organization/resource.go:424(row 67), is unresolved rather than clean. The objection raisedagainst it was that the
v1apigetters are nil-receiver-safe, which is true fororg.GetName()at :399 andorg.GetQuotaId()at :415 — but :424 isorg.QuotaId = applyOrgQuota.QuotaId, a direct field write that no getterprotects. The path that leaves
orgnil at that point is convoluted, and I have not settled it either way.Not one of the 14 turned out to be prevented by something downstream. That is structural rather than luck: the
"saved" shape needs the dereference to sit after a nil-rejecting
mapFields, and in this class it sits before —the field is read to build the state entry or the follow-up request.
iaas/server/resource.go:588, which I cited asa false positive in my previous comment, is the exception, not the pattern.
A correction to my previous comment
I wrote there that an empty body on a 2xx would be "a server fault, not normal protocol". That understates it. For at
least
alb,albcertificatesandalbwafthe success schema is declared but carries norequiredarray, so aperfectly spec-conformant 200 that simply omits the field crashes the provider too.
alb/applicationloadbalancer/resource.go:1180shows why:
That is two dereferences on one line.
createRespnil panics, andcreateRespnon-nil withNamenil panics aswell. The second needs no misbehaving server at all.
Why it matters beyond the panic
In the create paths the dereference happens before the state write —
SetAndLogStateFieldsis on the followingline at best, and in the case above the crashing expression is an argument to it. So the resource the API has just
created never reaches Terraform state. The user gets a plugin crash, no diagnostic, and an orphaned resource that the
next apply collides with.
That is the same failure shape as the SFS pool in #1737 that started this, and it is reachable in ~90 places.
Suggestion
On this evidence I would not sample further; the class is real. Two things follow:
iaas/server/resource.go:840(iaas: stackit_server panics when GetServer fails during update #1746, fix in fix(iaas): return after a failing GetServer in the server resource #1747) is unrelated to the 97 but the same crash shape, and it is aone-line fix.
diagnostic roughly ninety times. At that volume a small shared helper in
internal/coreis probably worth morethan ninety inline copies — but that is a judgement about your codebase, not mine, so I would rather ask than
assume. Say which you prefer and I will start.
Happy to hand over the per-site analyses for the 14 if that is useful for review.