You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request updates the Helm chart configuration for release preparation. The modifications are compliant with automated quality standards; however, the pull request lacks a description. Providing context for configuration changes is recommended to assist with long-term maintainability.
About this PR
The PR description is empty. Please provide context regarding the reasons for the configuration updates to improve documentation and traceability.
Test suggestions
Verify that the Helm chart values file is valid YAML after manual indentation changes.
Validate that the installation and documentation strings render correctly in templates.
Confirm the availability and integrity of the worker image.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify that the Helm chart values file is valid YAML after manual indentation changes.
2. Validate that the installation and documentation strings render correctly in templates.
3. Confirm the availability and integrity of the worker image.
The deploy_to_doks_release job (job 30732) timed out because the
CircleCI no_output_timeout (15m) matched the helm --timeout (900s).
Helm produces no output while waiting for pods to roll out, so CircleCI
killed the process just as helm was about to hit its own timeout.
Fix: raise no_output_timeout to 30m and helm --timeout to 1500s (25m)
so helm has room to complete or fail cleanly before CircleCI intervenes.
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…ded (#836)
* Clean up stale Kubernetes jobs before helm upgrade in release deployment
Previous failed deployments leave hook jobs (e.g. DB migrations) behind in
the codacy-release namespace. When helm tries to run the same post-upgrade
hook on the next attempt it finds an already-failed job and immediately
reports BackoffLimitExceeded, causing the deploy_to_doks_release CI job to
fail (see job 30795).
Add a clean_jobs Makefile target that deletes all jobs in the namespace
before the helm upgrade runs. The codacy-release namespace is a dedicated
Helm release namespace, so any lingering jobs are stale hook artifacts from
prior failed runs and are safe to remove.
AI-Generated: true
* Apply suggestion from @codacy-production[bot]
Co-authored-by: codacy-production[bot] <61871480+codacy-production[bot]@users.noreply.github.com>
* Reduce the delete jobs scope
---------
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
Co-authored-by: Ricardo Bernardino <ricardo.bernardino@codacy.com>
Co-authored-by: codacy-production[bot] <61871480+codacy-production[bot]@users.noreply.github.com>
…837)
The clean_jobs Makefile target ran kubectl delete jobs without specifying
the --namespace flag, so it searched the default namespace instead of the
release namespace (e.g. codacy-release). This meant stale hook jobs (DB
migrations etc.) from prior failed deployments were never cleaned up, and
helm's next attempt would find the existing BackoffLimitExceeded job and
immediately fail again.
Add --namespace ${NAMESPACE} to the kubectl delete command so it targets
the correct namespace for every environment (dev, sandbox, release).
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…838)
When a post-upgrade helm hook job fails with BackoffLimitExceeded, the
failed job lingers in the namespace. The --atomic flag causes helm to
roll back, leaving the hook job behind. On the next CI run, clean_jobs
removes it before the upgrade, but the hook can fail again transiently.
This change converts deploy_to_doks_from_chartmuseum from a
prerequisite-chain target to a recipe that cleans up and retries the
helm install once if the first attempt fails, covering transient hook
failures (e.g. delete-rabbitmq-queues running before RabbitMQ is ready).
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…ness wait (#839)
The post-upgrade hook job was spawning 100 pods (backoffLimit: 100, restartPolicy: Never)
each failing in ~9s because MinIO isn't ready immediately after a Helm upgrade, totalling
~15 minutes before BackoffLimitExceeded and causing helm --atomic to roll back.
Changes:
- Add a wait loop in the hook container that polls MinIO (aws s3 ls) every 10s for up
to 10 minutes before attempting put-bucket-lifecycle-configuration
- Reduce backoffLimit from 100 to 3 since retries are now handled within the container
- Add before-hook-creation to hook-delete-policy so any stale failed job is always
cleaned up before the next upgrade attempt creates a new one
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
… pre-push delete (#840)
Two fixes for the persistent deploy_to_doks_release failure (job 31035):
1. codacy/templates/fluentd/lifecycle-police-job.yaml: Add "Filter":{} to
the lifecycle configuration JSON. Newer MinIO versions strictly follow the
AWS S3 spec which requires a Filter element in each lifecycle rule, even
when empty. Without it, put-bucket-lifecycle-configuration returns a
MalformedXML error immediately, causing 3 fast pod failures and
BackoffLimitExceeded (the ~15 min total time is main-deployment rollout;
the hook itself fails in seconds).
2. .circleci/config.yml: Delete the existing chart version from chart museum
before pushing. Chart museum rejects re-uploads of the same version with
409, which failed the first pipeline run (job 31005). Adding a pre-push
DELETE via the chart museum REST API (with || true to tolerate missing
charts) ensures subsequent CI runs can always push the updated chart.
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
…IO (#841)
The previous fix (PR #839) waited for MinIO to respond to `aws s3 ls`
before calling put-bucket-lifecycle-configuration. However, MinIO can
be accessible while the specific bucket doesn't exist yet (bucket
creation is a separate hook). This caused rapid failures of the
lifecycle config command, hitting BackoffLimitExceeded (with
backoffLimit:3) long before Helm's 1500s timeout.
Changes:
- Wait for the specific bucket (`aws s3api head-bucket`) instead of
just MinIO (`aws s3 ls`), so the lifecycle config only runs once the
bucket actually exists
- Extend MAX_WAIT to 1200s (fits within Helm's 1500s timeout as a
single container, no Kubernetes backoff overhead)
- Set backoffLimit:0 so Kubernetes doesn't add exponential backoff
delays between retries — all retry logic is in-shell
- Add an in-shell retry loop (up to 5 attempts) for
put-bucket-lifecycle-configuration to handle transient API errors
after the bucket is confirmed ready
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
Co-authored-by: Ricardo Bernardino <ricardo.bernardino@codacy.com>
The stable/minio chart (v5.0.33) uses old-style Helm labels
(release: <name>, app: minio-make-bucket-job) and has
hook-delete-policy: hook-succeeded only. When the job fails,
it is not deleted by Helm on rollback.
clean_jobs was filtering by app.kubernetes.io/instance which
doesn't exist on old-style chart resources, so the stuck job
was never cleaned up. The second helm upgrade attempt would
then fail with "failed to deploy codacy-minio-make-bucket-job"
because the job already existed.
Add a second kubectl delete with the legacy release=<name>
label selector so clean_jobs catches both new- and old-style
Helm-labelled jobs.
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
PR #841 accidentally removed the `"Filter":{}` field that PR #840 had
added to the S3 lifecycle rule JSON. Newer MinIO versions strictly follow
the AWS S3 spec and require a Filter element in each lifecycle rule (even
when empty). Without it, put-bucket-lifecycle-configuration returns
MalformedXML immediately, causing the in-shell retry loop to exhaust its
5 attempts and exit with failure, which Kubernetes surfaces as
BackoffLimitExceeded on the post-upgrade hook.
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
The codacy/tag_version orb job was failing with:
"Invalid username or token. Password authentication is not supported for Git operations."
GitHub no longer accepts token-based HTTPS push for git operations.
Replace both codacy/tag_version orb calls in the release_pipeline workflow
with a custom tag_version_ssh job that uses SSH key authentication,
bypassing the HTTPS credential issue entirely.
AI-Generated: true
Co-authored-by: circleci-app[bot] <127350680+circleci-app[bot]@users.noreply.github.com>
--version is not a flag the helm-push plugin recognizes for printing
its own version (it errored with "flag needs an argument: --version"
- that flag expects a chart version to push). Use `helm plugin list`
instead, which reports installed plugins and their versions directly
from each plugin.yaml.
Still temporary, to be removed once captured.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Root cause confirmed via chartmuseum/helm-push#93: the bundled
helm-push 0.9.0 plugin loads an apiVersion v1 parent chart (ours)
using its own bundled Helm 2 library, which cannot read v2 subcharts
nested inside it (rustfs-1.0.3). It fails before anything reaches
ChartMuseum - matching the "error unpacking rustfs-1.0.3.tgz ...
apiVersion 'v2' is not valid" error from both prior CI runs. Neither
the system helm binary (confirmed v3.2.1) nor the plugin being merely
"old" (confirmed v0.9.0, already past the v0.8.0 Helm-3-client
threshold) was the actual issue - it's this specific, previously
unfixed bug.
Fixed upstream in helm-push v0.10.0 (Oct 2021), which also renamed the
plugin/command to cm-push (to avoid colliding with Helm 3.8+'s native
OCI `helm push`). Install v0.10.4 (the last release built against
Helm 3, matching this job's helm v3.2.1) in the helm_push job itself -
no codacy/ci-base image rebuild needed - and switch the push
invocation to `helm cm-push`. Shared by both helm_push_incubator and
helm_push_unstable via the helm_push step anchor.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
codacy/values.yaml sets rabbitmq-ha.memoryHighWatermark.enabled: true
(added in e99f895 "test highwatermark"), but never gave rabbitmq-ha
its own resources.limits.memory default, unlike every other subchart
in this file (codacy-api, engine, portal all set both limits and
requests). The rabbitmq-ha subchart's rabbitmq.toBytes helper renders
.Values.resources.limits.memory directly through a regex and errors
on a nil value:
template: .../rabbitmq-ha/templates/_helpers.tpl:127:48: executing
"rabbitmq.toBytes" at <.>: invalid value; expected string
This broke any install path that doesn't separately layer in
values-production.yaml (which happens to set it), including the
CircleCI microk8s_install job - its own independent test-values block
(microk8s_values_content in .circleci/config.yml) never set it
either. It only started showing up now because the RustFS/ChartMuseum
fixes unblocked the pipeline far enough to reach this install step for
the first time in a while.
Added the same 500m/1200Mi limits and 200m/200Mi requests that
values-production.yaml already overrides this with, so production
installs see no behavior change.
Verified: bare `helm template codacy codacy/` no longer fails here (it
now stops only at the separate, pre-existing, intentionally-blank
global.akka.sessionSecret); `-f values-production.yaml` still renders
total_memory_available_override_value = 1258291200 (1200Mi) unchanged;
a reconstructed copy of the CircleCI job's exact microk8s_values_content
renders successfully end to end; helm lint clean.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Error: UPGRADE FAILED: parse error at
(codacy/charts/rustfs/templates/_helpers.tpl:151): unclosed action
codacy/ci-do:2.8.0 - the image behind deploy_to_doks_release and every
other job using the default_doks_image anchor (validate/plan/create_
dev_cluster, setup_sandbox_db_values, setup_release_db_values, deploy_
to_doks_dev, deploy_to_doks_sandbox) - bakes in helm v3.2.1 (2020,
go1.13.10). That binary's Go template parser can't read the rustfs
chart's multi-line `{{- $mtls := dict ... }}` action in its
_helpers.tpl. Confirmed by pulling the actual image and running its
helm directly against the vendored rustfs-1.0.3.tgz: reproduces the
exact error. A current helm (v3.19.0 and v4.2.4 both tested) parses
the same chart with no error, so this is purely a client-version
limitation on a valid, modern chart - same root category as the
helm-push plugin issue already fixed, but this is a separate,
unrelated code path (a direct `helm upgrade --install`, no plugin
involved) that needed its own fix.
Added ensure_modern_helm, which replaces the binary with a current
v3.19.0 build (same release line already trusted elsewhere in this
pipeline, for the microk8s_install jobs) before anything is installed.
Made it a prerequisite of helm_install_infrastructure, helm_install_
from_local_dir and helm_install_from_chart_museum, so every one of the
.do/Makefile-driven deploy paths is covered, not just the release one.
Verified: curl/tar/mv are present in the real ci-do:2.8.0 image and
the download/extract steps succeed against it; a genuine v3.19.0
binary (tested natively, no emulation) renders the vendored rustfs
chart with exit 0; `make -n helm_install_from_chart_museum` shows the
new step correctly ordered before the helm upgrade call.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
No description provided.