Skip to content

chore(deps): update dependency siderolabs/talos to v1.14.0 - #3713

Draft
renovate[bot] wants to merge 9 commits into
mainfrom
renovate/siderolabs-talos-1.x
Draft

chore(deps): update dependency siderolabs/talos to v1.14.0#3713
renovate[bot] wants to merge 9 commits into
mainfrom
renovate/siderolabs-talos-1.x

Conversation

@renovate

@renovate renovate Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package Update Change
siderolabs/talos minor 1.13.91.14.0

Warning

Some dependencies could not be looked up. Check the warning logs for more information.


Release Notes

siderolabs/talos (siderolabs/talos)

v1.14.0

Compare Source

Welcome to the v1.14.0-beta.1 release of Talos!
This is a pre-release of Talos

Please try out the release binaries and report any issues at
https://github.com/siderolabs/talos/issues.

DNS over TLS (DoT) and DNS over HTTPS (DoH) Support

Talos now supports DNS over TLS (DoT) and DNS over HTTPS (DoH) for secure DNS resolution.
These features allow Talos to encrypt DNS queries and responses, enhancing privacy and security for DNS traffic.
The DNS protocol can be configured on a per-name server basis in the ResolverConfig document, allowing for flexible configuration of DNS resolution.

noexec on EPHEMERAL (/var)

Talos 1.14 clusters now default the EPHEMERAL volume (/var) to noexec in addition to the existing nosuid and nodev
mount options through generated machine configuration.

Existing machines are not affected on upgrades.

Note: Workloads that execute binaries placed under /var can break on new machines.
Longhorn v1 and vCluster are known to be affected.
For example, Longhorn v1's instance-manager executes engine binaries that the engine-image DaemonSet places under
/var/lib/longhorn/engine-binaries/, which now fails with permission denied.
Affected users can opt out via a VolumeConfig document:

apiVersion: v1alpha1
kind: VolumeConfig
name: EPHEMERAL
mount:
  secure: false

NOTE: Setting secure: false will also disable nosuid and nodev, which may have security implications. Use with caution.

Longhorn v2 (SPDK data engine) runs the data plane inside the instance manager process and is not affected.

Apply Configuration Modes

The '--mode=reboot' option has been removed from the talosctl apply-config command; by default, configuration is applied without a reboot.
Most configuration changes don't require a reboot; the documentation lists the changes that do.

Native BGP

Talos now supports running native BGP routing instances on the host via embedded GoBGP servers, configured with BGPInstanceConfig documents.
This removes the need to ship FRR as a system extension for the common fabric-facing use case.

List of changes:

  • Added repeatable, named BGPInstanceConfig documents to configure local ASN, router-id, optional Linux VRF, advertised interfaces, neighbors, and per-route preferred source (routeSource).
  • Peer hold-time and BFD behavior are configured inline on each concrete neighbor, which selects either an address or a link.
  • Numbered and unnumbered (IPv6 link-local, RFC 8950 extended next-hop) peering are supported, including IPv4 prefixes learned over an IPv6 link-local next-hop.
  • Neighbor-local ASN overrides and passive sessions are supported. ECMP (multipath) and BFD (fast failure detection) are supported for fabric peering.
  • BFD is currently supported only by the instance in the default routing domain; GoBGP's embedded BFD listener is not VRF-aware.
  • Each instance owns an isolated BGP RIB and, by default, installs learned routes into its default or VRF routing table through the existing route controllers. Set installRoutes: false to retain learned routes in the BGP RIB without installing them into the Linux routing table.
  • Instances can selectively import best neighbor-learned routes from other named instances with importRoutes prefix selectors. Imports are one-way, preserve path attributes, and do not recursively import locally originated or previously imported paths.
  • Peer state is observable via instance-qualified BGPPeerStatus resources (talosctl get bgppeerstatus).
  • RouteSpec/RouteStatus now carry a multipath next-hop list to support ECMP and cross-family (RFC 8950) next-hops.
Btrfs Support

Talos now supports mounting and provisioning btrfs filesystem for user volumes and existing volumes.

Support for btrfs is enabled by installing btrfs system extension.

CRI Base Runtime Specification Configuration

Talos now supports overriding the default OCI runtime specification for CRI containers with a
CRIBaseRuntimeSpecConfig document:

apiVersion: v1alpha1
kind: CRIBaseRuntimeSpecConfig
overrides:
  process:
    rlimits:
      - type: RLIMIT_NOFILE
        hard: 1024
        soft: 1024

The .machine.baseRuntimeSpecOverrides field is deprecated and remains supported during the deprecation
period. It is mutually exclusive with CRIBaseRuntimeSpecConfig; configurations containing both are rejected.

Applying, updating, or removing either source regenerates the base runtime specification and restarts CRI
automatically. A machine reboot is no longer required.

CRI Customization Configuration

Talos now supports customizing the CRI containerd configuration with named CRICustomizationConfig
documents. Each document contains a TOML fragment; fragments are merged in lexicographical order by name.
Applying, updating, or removing these documents updates the generated CRI configuration and restarts CRI
automatically.

The legacy /etc/cri/conf.d/20-customization.part machine-file configuration remains supported during the
deprecation period and is exposed under the reserved name customization. A CRICustomizationConfig document
cannot use that name.

NOTE: a machine reboot is no longer required to apply changes to CRI configuration.

Containerd NRI

Talos no longer disables NRI (Node Resource Interface) for the CRI containerd instance by default, so NRI is available
to use without any machine config patches.

To bring back the old behavior of NRI disabled by default, add the following machine configuration document:

apiVersion: v1alpha1
kind: CRICustomizationConfig
name: disable-nri
content: |
  [plugins]
    [plugins."io.containerd.nri.v1.nri"]
       disable = true
Default Installer Image

The default installer image has been updated to use the Image Factory.
The ghcr.io/siderolabs/installer image is no longer published with releases; use the Image Factory installer image instead.

DHCP

DHCPv4 search domains are now applied to the resolver configuration.

DHCPv4 configuration now supports ignoreRoutes option to ignore routes provided by DHCPv4 servers.

Cluster Discovery

Talos introduces support for configuring multiple discovery service endpoints.
Talos introduces new document for configuring the cluster discovery identity.

List of changes:

  • Deprecated .cluster.discovery in the v1alpha1 config; use the DiscoveryServiceConfig document for discovery service configuration. The v1alpha1 config and DiscoveryServiceConfig are mutually exclusive.
  • Deprecated .cluster.secret and cluster.id in the v1alpha1 config; use the DiscoveryIdentityConfig document for discovery identity configuration. The v1alpha1 config and DiscoveryIdentityConfig are mutually exclusive.
  • Changed cluster ID encoding in the generated secret bundle, from base64.URLEncoding to base64.StdEncoding. This aligns the encoding with the rest of Talos.
Encryption Discards

Volume encryption now supports an allowDiscards option (disabled by default) which passes TRIM/discard requests
through to the underlying device when the encrypted volume is opened.

This only enables passing discards through to the underlying device; Talos does not perform any fstrim/discard operation by itself.

etcd

Talos is now compatible with etcd v3.6.x only (the default etcd version was 3.6.x since Talos v1.11).
The default version is 3.7.0+ now.

etcd now serves its HTTP-only endpoints (/metrics, /health, the gRPC-gateway JSON API) on a dedicated
listener on port 2383, while the client port 2379 serves gRPC only. This keeps gRPC off Go's net/http
HTTP/2 server, avoiding watch-stream starvation under TLS (see etcd-io/etcd#15402, golang/go#58804,
etcd-io/etcd#21605).

Upgrade note: etcd metrics and the HTTP health endpoint are no longer reachable on 2379; scrape them on
port 2383 instead (same client mTLS as before). etcd gRPC clients and the Talos health check are unaffected.

Firewall might need to be adjusted to block the port 2383 if previously 2379 was blocked.

If --listen-metrics-urls was customized, the metrics should not move.

EtcFileConfig

Talos now supports managing user-owned files under /etc with the new EtcFileConfig multi-document
configuration kind. The document name is the path relative to /etc, and each document owns the complete
file contents and mode.

This can be used to configure files such as /etc/nfsmount.conf or /etc/multipath.conf. Talos-managed
paths, including resolv.conf, hosts, machine-id, CRI and Kubernetes configuration, trust bundles, and
identity files, are rejected to prevent overriding files owned by Talos.

Filesystem Trim

Talos can now periodically trim (the equivalent of the fstrim command) mounted filesystems which support trimming,
discarding unused blocks. This is useful for SSDs and thin-provisioned storage.

Trimming is opt-in via a new FilesystemTrimConfig document which sets the global trim interval:

apiVersion: v1alpha1
kind: FilesystemTrimConfig
interval: 168h0m0s # one week

The default machine configuration for Talos 1.14+ includes a FilesystemTrimConfig document with a default trim interval of one week,
so trimming is enabled by default for eligible filesystems. For cluster which were upgraded from older versions, the FilesystemTrimConfig document will be missing,
so trimming will be disabled by default until the document is added.

When the document is present, Talos builds a stable schedule (hashed by node ID and volume ID, so trims are spread out
across volumes and across nodes in a cluster) and trims eligible volumes (ready disk/partition volumes with a
trim-capable filesystem; for encrypted volumes only when allowDiscards is set).

The trim interval can be overridden or disabled per-volume via a trim block on the volume documents
(VolumeConfig, UserVolumeConfig, ExistingVolumeConfig, ExternalVolumeConfig):

trim:
  enabled: true
  interval: 24h0m0s
Flannel CNI

Talos now configures Flannel with the EnableNFTables option enabled, which uses nftables native backend instead of iptables-nft compatibility layer.

FlexVolume Host Path Removed

Talos no longer provisions the deprecated FlexVolume executable host path at
/usr/libexec/kubernetes. FlexVolume has been deprecated since Kubernetes 1.23.
Modern CSI plugin paths under /var/lib/kubelet are unaffected.

Host DNS Configuration

HostDNS configuration was moved from the v1alpha1 config .machine.features.hostDNS field to the new hostDNS in the ResolverConfig document.

HTTP Probe Support

Talos now supports HTTP network probes, allowing for monitoring of HTTP endpoints.
HTTP responses with status 200-399 are considered successful, while connection and transport errors are treated as failures.

Image Cache Configuration

Talos now supports a new ImageCacheConfig document for configuring the Image Cache feature, replacing the old machine.features.imageCache field in the v1alpha1 config.
Old configuration is still supported for backwards compatibility.

Kernel Multi-document Configuration

Talos introduces new multi-document configuration for kernel parameters (sysctl and sysfs settings), replacing the old v1alpha1 config fields.
The old configuration is still supported for backwards compatibility, but new deployments should use the new documents.

If both old and new configuration sources are used, the new multi-document configuration takes precedence over the old v1alpha1 config on conflicting fields.

List of changes:

  • Deprecated .machine.sysctls in the v1alpha1 config; use the SysctlConfig document for kernel sysctl configuration.
  • Deprecated .machine.sysfs in the v1alpha1 config; use the SysfsConfig document for sysfs configuration.
  • Deprecated .machine.kernel in the v1alpha1 config; use the KernelModuleConfig document for kernel module configuration.
Kernel Module Status

Talos now reports the status of both dynamically loaded, and built-in kernel modules.

The LoadedKernelModule resource has been deprecated and superseded by the new KernelModuleStatus resource.

In-tree Volume Plugins Deprecated

Because the kubelet now runs inside the sandbox namespace (see the workload isolation note), the in-tree
Kubernetes volume plugins that require the kubelet to reach host-level daemons no longer work. In particular
the in-tree iscsi volume plugin, which drives the kubelet's iscsiadm wrapper to talk to the host iscsid,
can no longer locate it across the sandbox PID namespace boundary.

Use CSI drivers instead — a CSI node plugin performs the attach/mount itself in its own privileged pod and is
unaffected by the sandbox. For iSCSI, kubernetes-csi/csi-driver-iscsi (or democratic-csi) consumes a
target the same way. All in-tree (non-CSI) volume plugins are deprecated for the kubelet and support for them
may be removed in a later release.

Kubernetes Multi-document Configuration

Talos introduces new multi-document Kubernetes configuration, which allows for more flexible and modular configuration of Kubernetes components.
Talos still supports the old v1alpha1 config for backwards compatibility, but new features and fields will only be available in the new multi-document format.
The kube-proxy is now using configuration to manage its settings instead of command line arguments (with new KubeProxyConfig document).

List of changes:

  • Deprecated .cluster.secretboxEncryptionSecret in the v1alpha1 config; use the KubeEtcdEncryptionConfig document for full etcd encryption configuration.
  • Deprecated .cluster.apiServer in the v1alpha1 config; use the KubeAPIServerConfig, KubeAdmissionControlConfig, KubeAuditPolicyConfig, KubeAuthenticationConfig and KubeAuthorizerConfig documents for kube-apiserver configuration.
  • Deprecated .cluster.ca, .cluster.acceptedCAs and .cluster.aggregatorCA in the v1alpha1 config; use the KubeAPIServerCAConfig, KubeAggregatorCAConfig documents.
  • Deprecated .cluster.controllerManager in the v1alpha1 config; use the KubeControllerManagerConfig document for kube-controller-manager configuration.
  • Deprecated .cluster.scheduler in the v1alpha1 config; use the KubeSchedulerConfig document for kube-scheduler configuration.
  • Deprecated .cluster.proxy in the v1alpha1 config; use the KubeProxyConfig document for kube-proxy configuration.
  • Deprecated .cluster.network in the v1alpha1 config; use the KubeNetworkConfig document for Kubernetes network configuration; Flannel can be configured using the KubeFlannelCNIConfig document.
  • Deprecated .cluster.coreDNS in the v1alpha1 config; use the KubeCoreDNSConfig document for CoreDNS configuration.
  • Deprecated .cluster.name and .cluster.controlPlane.endpoint in the v1alpha1 config; use the KubeClusterConfig` document for cluster name and control plane endpoint configuration.
  • Deprecated the following list of fields, all of them moved into `KubeNodeConfig:
    • .cluster.allowSchedulingOnControlPlanes
    • .machine.kubelet.skipNodeRegistration
    • .machine.kubelet.registerWithFQDN
    • .machine.kubelet.nodeIP
    • .machine.nodeLabels
    • .machine.nodeAnnotations
    • .machine.nodeTaints
  • The default NoSchedule taint for controlplane and label are now explicitly listed in KubeNodeConfig.
  • Deprecated the rest of .machine.kubelet fields in the v1alpha1 config; use the KubeNodeConfig and KubeCredentialProviderConfig documents for kubelet configuration.
  • Deprecated .machine.pods in the v1alpha1 config; use the KubeStaticPodConfig document for static pod configuration.
  • Deprecated .machine.files in the v1alpha1 config; use dedicated configuration documents such as EtcFileConfig and CRICustomizationConfig instead.
  • Deprecated .machine.baseRuntimeSpecOverrides in the v1alpha1 config; use the CRIBaseRuntimeSpecConfig document for base runtimespec overrides.
  • Deprecated .cluster.inlineManifests in the v1alpha1 config; use the KubeInlineManifestConfig document for inline manifests.
  • Deprecated .cluster.extraManifests and .cluster.extraManifestHeaders in the v1alpha1 config; use the KubeExternalManifestConfig document for external manifests.
  • Deprecated .machine.features.kubePrism; use the KubePrismConfig document for KubePrism configuration (or remove it to disable KubePrism).
  • Deprecated .machine.features.kubernetesTalosAPIAccess; use the KubeTalosAPIAccessConfig document instead.
  • Added nodeCIDRMaskSizeIPv4 (default 24) and nodeCIDRMaskSizeIPv6 (default 64) settings to the KubeNetworkConfig document to control the per-node pod CIDR mask size and validate the pod and service subnet sizes.
LVM Logical Volume Creation

Logical volumes can now be declared with a new LVMLogicalVolumeConfig multi-doc config kind. Each document
names a logical volume, its parent volumeGroup, a type (linear, raid0, raid1 or raid10) and a
maxSize (absolute, e.g. 50GiB, or a percentage of the volume group, e.g. 80%). RAID layouts accept
optional mirrors (raid1/raid10, default 1) and stripes (raid0/raid10, default: all available physical
volumes) fields. Once the volume group is assembled the logical volume is created via lvcreate.

Raising maxSize grows an existing logical volume via lvextend; percentage-sized volumes also grow when
their volume group is extended. Shrinking is never performed (it risks data loss) - a request to reduce the
size surfaces an LVMValidationError instead. Removal stays an explicit operation via the LVMService LV
remove RPC (talosctl wipe lv).

LVM Status

Talos now provides detailed LVM status information, allowing for better monitoring and management of LVM volumes.
New resources LVMPhysicalVolumeStatus, LVMVolumeGroupStatus, and LVMLogicalVolumeStatus expose PV, VG, and LV details.
DiscoveredVolume resources for logical volumes are listed by their kernel name (e.g. dm-0). To resolve the <vg>/<lv> for a given device, use the Disks or BlockSymlinks resources, which carry the udev-managed symlinks (e.g. /dev/disk/by-id/dm-name-<vg>-<lv>).

LVM Volume Group Creation

Talos can now create and grow LVM Volume Groups declaratively through a new LVMVolumeGroupConfig multi-doc
config kind. Each document names a Volume Group and a CEL volumeSelector over the disk inventory; matched
disks are initialised as Physical Volumes (pvcreate) and aggregated into the requested VG (vgcreate).
Newly matched disks added to an existing VG are attached via vgextend.

Reconciliation is strictly additive and safe-by-default.

LVM Wipe

Talos now provides the ability to securely wipe LVM metadata from logical volumes, volume groups, and physical volumes.
This feature allows for selective wiping of logical volumes, volume groups, and physical volumes.

With talosctl wipe lv/vg/pv <name>, users can wipe LVM metadata from a specific logical volume, volume group, or physical volume.

NTS for Time Synchronization

Talos now supports Network Time Security (NTS) for secure time synchronization.
This feature enhances the security of NTP by providing cryptographic authentication of time sources.

NTS is enabled by default (without any configuration sources) for the default time.cloudflare.com time server
NTS can be enabled for custom time servers via the new useNTS field in the TimeServerConfig document.

RAID Array Creation

Talos can now create and grow Linux MD (software RAID) arrays declaratively through a new RAIDArrayConfig
multi-doc config kind. Each document names an array, its level (raid1) and a CEL volumeSelector over
the disk inventory; matched disks are assembled into the requested array with mdadm and exposed at the stable
/dev/disk/by-id/md-name-<name> path. New matching disks added to an existing array are attached automatically.

Reconciliation is strictly additive and safe-by-default. Arrays are never destroyed by removing the config;
removal stays an explicit operation via talosctl wipe md <device>. The new MDArrayStatus resource reports the
assembled array, level, device path, and members.

Booting from a RAID Array

Talos can now be installed onto and boot from a Linux MD (software RAID) array. Define a RAIDArrayConfig for the
array and point the install disk selector (UnattendedInstallConfig) at the resulting /dev/disk/by-id/md-name-<name>
device.

Only raid1 arrays with metadata: "1.0" can be used for booting: the 1.0 format keeps its superblock at the end of
each member, so the partition table written to the array stays visible at the start of every disk, allowing the
firmware to boot from any member. metadata defaults to 1.0; other levels and metadata formats are not bootable.

Workload Isolation (sandboxd)

The container runtime plane — CRI containerd, the kubelet, and all pods — now runs inside a dedicated PID and
mount namespace anchored by a new sandboxd service, instead of sharing machined's namespaces.

sandboxd runs in its own least-privilege SELinux domain (sandboxd_t). if it dies the kernel tears down the
namespace and Talos recreates it — relaunching CRI, the kubelet, and pods — without rebooting the node.
Its logs are available via talosctl logs sandboxd.

Workload isolation is controlled by the workloadIsolation field of the new SecurityProfileConfig document.
talosctl gen config emits it with workloadIsolation: true for Talos 1.14+, so new clusters are isolated by
default
. Clusters upgraded from older versions do not have this document and therefore keep the previous
(non-isolated) behavior until it is added — upgrades change nothing on their own. To enable on an existing
cluster, add the document:

apiVersion: v1alpha1
kind: SecurityProfileConfig
workloadIsolation: true

NOTE: With workload isolation enabled, the deprecated in-tree Kubernetes iSCSI volume plugin does not work
(the kubelet cannot reach the host iscsid across the sandbox); use a CSI driver instead. See the in-tree
volume plugin deprecation note.

ICMP send_redirects Disabled by Default

Talos now sets net.ipv4.conf.all.send_redirects=0 and net.ipv4.conf.default.send_redirects=0 by default,
preventing the node from emitting ICMP redirect messages. This aligns with CIS Benchmark recommendations and
does not affect normal Kubernetes pod or service traffic. Nodes that deliberately act as L3 gateways relying
on ICMP redirects can override this via machine.sysctls.

Support Bundle Encryption

The talosctl support command now encrypts support bundles using the age encryption tool, enhancing the security of support data.
The default set of recipients includes the 'siderolabs' GitHub organization members, but it can be overridden with custom recipients.

Dedicated System Volumes

The ETCD, CRI, KUBELET and LOG system volumes (/var/lib/etcd, /var/lib/containerd, /var/lib/kubelet and /var/log) can now be placed on dedicated partitions via a VolumeConfig document with provisioning set (optionally encrypted). By default they remain directories under the EPHEMERAL volume.

apiVersion: v1alpha1
kind: VolumeConfig
name: ETCD
provisioning:
  minSize: 1GB
  maxSize: 2GB

The backing (directory vs. dedicated partition) is fixed at cluster creation: switching an already-provisioned node between the two is rejected.

A dedicated partition has its own mount, so the mount.secure option (nosuid/noexec/nodev, enabled by default) can be set per volume; directory-backed volumes inherit the EPHEMERAL mount options.

Note that with ETCD on a dedicated partition, etcd data no longer lives under EPHEMERAL. Resetting a control plane node with only the EPHEMERAL partition wiped will not clear etcd data; wipe the ETCD volume to reset etcd.

TLS 1.3 Minimum Version

Talos now runs etcd and kube-apiserver with a minimum TLS version of 1.3, improving security by leveraging the latest TLS features and cipher suites.
Custom settings for cipher suites have been removed, as they are ignored when TLS 1.3 is used, which simplifies configuration and ensures the use of modern, secure defaults.

Udev Rules Multi-document Configuration

Talos introduces new multi-document configuration UdevRulesConfig document for configuring custom udev rules.
The old v1alpha1 .machine.udev.rules field is still supported for backwards compatibility, but new deployments should use the new document.

If both old and new configuration sources are used, UdevRulesConfig takes precedence.

List of changes:

  • Deprecated .machine.udev.rules in the v1alpha1 config; use the UdevRulesConfig document for custom udev rules.
Unattended Install Configuration

Talos introduces a new UnattendedInstall multi-document config kind which replaces the deprecated .machine.install
section of the v1alpha1 config. The document carries the installer image and a provisioning section with a CEL
volumeSelector to match the install disk, plus a wipe option.

When the UnattendedInstall document is present, the install is driven by the new UnattendedInstallController
(exposing an UnattendedInstallStatus resource) instead of the legacy install sequence.

talosctl gen config and talosctl cluster create now generate the UnattendedInstall document by default.
The .machine.install field remains supported for backwards compatibility and is still used for older version contracts.

Component Updates

Linux: 6.18.41
Kubernetes: 1.37.0-beta.0
containerd: 2.3.3
etcd: 3.7.1
Flannel: 0.28.8
runc: 1.5.1
CoreDNS: 1.14.6

Talos is built with Go 1.26.5.

Virtual Ethernet Pairs

Talos now supports declarative virtual Ethernet (veth) pairs through the new VethConfig multi-document
configuration kind. Both endpoints are created in the host network namespace and support the common link settings,
addresses, routes, and multicast configuration.

For example, the following configuration creates a pair named veth-host and veth-router with an address on each
endpoint:

apiVersion: v1alpha1
kind: VethConfig
name: veth-host
addresses:
  - address: 10.3.0.1/30
peer:
  name: veth-router
  addresses:
    - address: 10.3.0.2/30
XFS Allocation Group Geometry

On non-rotational devices mkfs.xfs sizes the allocation group count to the number of CPUs, bounding the
allocation group size from below at 4 GiB only. On machines with many cores and a modest disk this produces
hundreds of tiny allocation groups, which squeezes the AG-local reflink/rmap metadata (leading to spurious
ENOSPC on reflink-heavy workloads while the filesystem still has plenty of free space) and inflates the
journal at the same time.

Talos now keeps XFS allocation groups at 64 GiB or above when it formats a volume. The bound can be changed
per volume, and setting it to zero restores the stock mkfs.xfs behavior:

apiVersion: v1alpha1
kind: VolumeConfig
name: EPHEMERAL
filesystem:
  xfs:
    minAllocationGroupSize: 128GiB

The same filesystem.xfs.minAllocationGroupSize setting is available for UserVolumeConfig.

Note: allocation group geometry is fixed when the filesystem is created, so this only affects volumes
formatted by Talos 1.14 or later. Existing volumes keep their current geometry until they are wiped and
re-created (e.g. talosctl reset --system-labels-to-wipe=EPHEMERAL).

XFS scrub

Talos now supports automatically running background online filesystem maintenance tasks. Currently,
only XFS using xfs_scrub tool is supported.

This behavior can be enabled globally using a FilesystemScrubConfig document, or on per-volume
basis using a field in corresponding VolumeConfig documents.

Contributors
  • Andrey Smirnov
  • Noel Georgi
  • Mateusz Urbanek
  • Maja Bojarska
  • Utku Ozdemir
  • Orzelius
  • Erwan Leboucher
  • Kevin Tijssen
  • Lukasz Raczylo
  • Mickaël Canévet
  • Oguz Kilcan
  • Dmitrii Sharshakov
  • Dmitriy Matrenichev
  • Edward Sammut Alessi
  • Rokoucha
  • buckaroo
  • immanuwell
  • kastakhov
  • Aleksei Sviridkin
  • Andras Elso
  • Ansgar Dahlen
  • Artem Chernyshev
  • Benoît Knecht
  • Calin
  • Christian Korneck
  • Dario Emerson
  • David Orman
  • Dharsan Baskar
  • Dmitry Sharshakov
  • Filip Boye-Kofi
  • Fritz Schaal
  • Immanuel Tikhonov
  • Jaakko Sirén
  • Jonny
  • Justin Garrison
  • Konstantin Nesterov
  • Mario Cole
  • Mark Glants
  • Maxime Bertin
  • Nico Berlee
  • Pranav Patil
  • YANG JOO WOONG
  • Zadkiel AHARONIAN
  • appkins
  • ctr49
  • imusmanmalik
Changes
487 commits

  • 1ab88f743 fix: load the raid1 module for configured MD arrays
  • 9e8568d26 fix: use inmem containerd for installs/upgrades
  • 8cf28da5f fix: preserve kmsg reconciliation after config changes
  • 311b6fde9 fix: make reset during boot phase more robust
  • 945d1cdab feat: bump kernel to 6.18.41
  • 4e77d40e8 fix: restore the systemd-timesync best-sample check in spike detection
  • 4444a187d test: inspect host mount state from the current thread
  • b7c0497b0 fix: race between vol.cfg. and vol.mgr. ctrl
  • ef9a091ec feat: allow attaching extra disks to controlplane machines
  • a418c0e1e test: fix CRI restart event assertion race
  • a9bfdbdd4 fix: ignore insecure-only imager assets
  • 54e3b20e8 fix: hold the darwin vmnet bridge open for the network lifetime
  • bc59389fa fix: drop the OOM config rule about overall system memory PSI
  • 69be56ea9 chore: add some initial set of libvirt SELinux policies
  • 6170ad8b9 fix: validate kubespan & discovery config correctly for multi-doc
  • 6e58c6d5a test: fix ded. vol. plumbing in integration tests
  • b644d1640 fix: allow directory-backed volumes in reset api
  • 28e7a8742 feat: bump etcd to v3.7.1
  • 2c657c224 feat: bump kernel to 6.18.40
  • b1bdc8c07 test: restart qemu process several times on startup failures
  • 7533057a7 feat: support route imports between BGP instances
  • a94783704 docs: remove duplicated docs in the markdown for CLI
  • 6f17c5033 fix: verify the public key signed images correctly
  • 570fe34f0 fix: generate backwards compatible etcd encryption config
  • f7790816e fix: use context without cancelation for etcd locks
  • 67e61ef30 feat: add the fs_scrub controller
  • 1c156458a fix: override DHCP search domains via explicit ResolverConfig domains
  • fd8dbd8a1 fix: skip pod check if desired number of pods is zero
  • 9aede5429 feat: add kubeimportlinter for versioned k8s imports
  • ae93d1462 fix: redact resource specs in the merge controllers
  • fc5743cd0 feat: add GrubUseUKICmdline install option
  • 6bba77724 feat: add named native BGP instances
  • c5ab22f1d feat: move Talos API access from Kubernetes config to multi-doc
  • b1abd9c03 fix: split the up/finished service events
  • 644ecbc66 feat: add discovered volumes status controller
  • 6be2b1384 feat: add veth pair support
  • 7514401de fix: drop the controlplane static pod change
  • 9a521f667 fix: preserve container tasks across CRI restarts
  • 9048d4157 fix: fix a nil-map assignment panic in configpatcher
  • 45eaf2037 release(v1.14.0-beta.0): prepare release
  • 7e58e0442 feat: add dedicated CRI configuration documents
  • 076c38136 fix: race with PCR extensions and volume unlock
  • 88884194c fix: teardown ephemeral mount request during reset
  • c793bcbf5 fix: configure bonds during initial link creation
  • 9b3bf6e51 fix(talosctl): prevent duplicate QEMU config server ports
  • fa6cd1ca8 fix(machined): preserve health when services reach running
  • 9d5554e69 fix(machined): wait for host namespace commands through reaper
  • fc08533bf chore: update dependencies
  • c08863cdd feat: provide different heuristics for xfs allocation groups
  • e955d9bd7 feat: update CoreDNS to 1.14.6
  • c3f757f9e feat: update Flannel to 0.28.8
  • fada0d960 fix: provide non-sensitive KubeletStatus resource
  • c68085286 fix: volume mount race (third attempt) around service restart
  • b185752e5 feat: refactor KubePrism config into multidoc
  • 499d4ebf9 test: update Calico in canal reset test
  • 5b6ed0068 test: add a test for kata-qemu runtime class
  • 1a075383a feat: allow "duplicate" kinds in the config patches
  • 06943be9e feat: update Kubernetes to 1.37.0-beta.0
  • 01f2a1423 fix: preserve trailing rate-limited trigger events
  • 46fab8057 test: stabilize AWS readiness and Talos 1.13 QEMU config
  • a26ac746d feat: move static pods and manifests into multi-doc
  • 67464cbef fix: update the vulnerability dates and description
  • 4920ee06f feat: update Linux to 6.18.39
  • 286fa8006 feat: include CA into kube-apiserver serving certificate
  • 6d65e223b feat: drop kubernetes flexvolume mounts
  • 4935e9452 feat: refactor kubelet's config into KubeletConfig
  • 241bd0ff1 feat: custom cfg for system volumes (cri, kubelet, etcd)
  • ea9557816 fix: talosctl build
  • c2b763608 feat: add UFSHC and some other modules
  • 2193b5781 feat: native BGP support via embedded GoBGP
  • 2e42c5900 fix: add ca-certificates to talosctl
  • 0f55e1f05 feat: refactor Kubernetes configs into KubeNodeConfig
  • 6efdc8f71 fix: zero MD superblock via block wipe on destroy
  • f78f5e5a1 fix: vrf sorting
  • 77385181a fix: oom podruntime protection
  • c1184d38e feat: update to runc 1.5.1
  • 4bff7eb90 feat: support reboot and sync for remote provisioner
  • c791fa8c0 feat: add host-namespace debug profile
  • e370e40b7 feat: implement KubeClusterConfig
  • 37c78bfc0 fix(ci): skip ephemeral noexec test on 1.13
  • 0ab6695e6 feat: update Kubernetes to 1.37.0-alpha.3
  • 443ca17e1 test: bump test dependencies
  • c4242088b fix: enable noexec for EPHEMERAL only for new machines
  • fc9f72648 feat: bump CoreDNS, Flannel
  • 352b1bdeb fix: use symlinks for init aliases
  • 883775a9e fix: move sandboxd into a separate cgroup
  • 099a2ceda fix: remote provisioner name
  • ff67aaf32 feat: bump go dependencies
  • 79c0c5414 feat: add iommufd as a kernel module
  • f34e93fe2 fix: do proper backoff for NTP Kiss-of-Death responses
  • a3e644d8d chore: bump tools and pkgs
  • efa88f2f6 fix: flaky tests
  • 17a134711 feat: add ignoreRoutes option to DHCPv4 config document
  • 2519bf231 fix: make audit restartable
  • 54b4bbc03 fix: provide correct handler for Ctrl-Alt-Delete sequence
  • 87e126ab7 feat: isolate cri, kubelet and pods in a sandbox namespace
  • 3fb8f4e9e fix: avoid image cache mount request churn
  • 9753fc27f fix: e2e test flakes
  • f756ff232 feat: kubenetworkconfig supports per-node pod cidr configuration
  • b42c42976 fix(ci): fix more flaky tests
  • 5d97eccdf feat: bring in ifb.ko module
  • 6769a1d5c fix: terminate log persistence a bit harder
  • 98cce792f fix(ci): extensions test
  • 057d554d2 test: assert dm transport for device-mapper disks
  • 9fd16a21e feat: bump etcd to 3.7.0
  • 3048eeb23 feat: support booting from MD RAID1 array
  • e1fc7a4a1 fix: do not block volume lifecycle teardown on failed user volumes
  • 147dea148 feat: add --no-reboot flag to upgrade cmd
  • 1b23b11fc chore: update pkgs and tools
  • bfa9fb4e8 fix: flaky tests
  • a1ede48cb test: fix testremovemember etcd integration flake
  • ea90e690d feat: add MD RAID gRPC service and reconcile controllers
  • 74486ef6d chore: update deps
  • f59c3ccad feat: implement service account configuration
  • baff2d3f9 test: fix some test flakiness
  • 5450ec303 fix: use a forked version of secure-io/siv-go
  • 33fac3f85 test: stabilize netapp trident csi fio runs
  • afdde2a8f chore(ci): add netapp trident csi integration tests
  • 21eca156f fix: print link status changes
  • 210f4e369 fix: shutdown/reboot via usermode helpers
  • d193f278d test: fix cilium test config patching
  • e06898069 fix: flaky tests
  • b7398ec00 feat: move kernel module config into multi-doc
  • 55bc643af fix: flaky serviceaccount suite test
  • dced7d570 fix: correctly treat guaranteed QoS pods in the OOM handler
  • f783f6636 feat: implement controlplane only config validation
  • d0291bb0b feat: extract Kubernetes CA config into a separate document
  • 97ed958a8 chore: use lefthook globs to skip noop jobs
  • a145c6356 chore: lefthook USERNAME env, post-commit hook
  • f836707ad fix: use UnattendedInstallConfig for extensions
  • 67293c809 chore: add lefthook.yml
  • 726ea8fc2 chore: switch v1alpha1 validation to use cluster config struct
  • d1d848022 feat: add mdadm tooling and udev rules
  • 020de3f51 chore: update go dependencies
  • ae84f56a0 chore: remove orphaned unattendedinstall.md
  • 416073748 feat: add UnattendedInstall config and controller
  • 4e5b4c6a7 feat: extract clusterid and clustersecret to discoveryidentityconfig
  • 0a641f268 refactor: simplify device status controller
  • 99da7f27f fix: data race in manifest sync
  • 54ac1cbd6 fix: provide cooldown period for the QoS trigger
  • 788562586 feat: udevd controller and udev rules config document
  • 6e34da25c feat: delegate drain ops to go-kubernetes/nodedrain
  • e9e027c63 fix: kubelet stuck restarting
  • 6f481b420 fix: decode extraArgs list values correctly
  • c8bdcc252 feat: update runc to 1.5.0
  • eae11ab0c feat: allow user managed etc files
  • 47d4bd87e feat: set user-agent for Kubernetes client
  • ba926c6ce chore: update golangcilint config
  • 45497bd5b feat: bring systemd 261.1
  • 8d9ecec93 refactor: improve stability for process_test.go
  • 31221e7ee refactor: talosctl running tasks are yellow
  • b268a6b08 feat: refactor CoreDNS config into multi-doc
  • 416d5fe4b fix: race in etcd member add
  • c244e4c46 fix: building integration test binary on darwin
  • b15a64b31 chore: bump rekor for GHSA-47q9-m4ww-924m
  • cd8b0fe39 release(v1.14.0-alpha.2): prepare release
  • 917820cb3 chore: sync pkgs/tools
  • b34be14e9 fix: cli.md codeblock generation
  • 25abcc6b5 docs: update kubespanconfig to match discoveryserviceconfig
  • 742589f50 feat: support multiple discovery service configs
  • fc3f27d79 chore: enrich the SBOM with Go module licenses
  • 47d5c3351 fix: handle image cache being disabled
  • 1a965aec3 test: disable LongHorn ublk test and add more cores
  • 6d03b3f61 fix: align documented image cache partition label
  • 6447d854f fix(talosctl): use aio threads on darwin
  • f856d1808 fix: image verification with referrers
  • 11a7fbe4c feat: extract kube-apiserver config into multi-doc configs
  • 337654d2b test: fix rook-ceph tests
  • e33a86825 feat: add AMD XGBE driver to initramfs
  • bd2d6242a fix: revert coredns to 1.14.2
  • 7c4e644f8 feat: update Linux to 6.18.36
  • 6e23a5c2f chore: refactor bare opentree_clone into a mount helper
  • dfbd30959 fix(talosctl): prevent appending type 11 smbios values on restart
  • 5926dd70d test: support running integration test against remote provisioner
  • f146c6a18 feat: refactor /etc mounts
  • ebe364117 chore: bump containerd to 2.3.2
  • bc30c61a1 chore: bump deps (go, k8s, docker)
  • 00d739d0a test: skip fstrim default schedule on

Important

✂ PR body was truncated to here.


Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@botantler-1
botantler-1 Bot enabled auto-merge September 10, 2026 08:25
@devantler
devantler marked this pull request as draft September 12, 2026 14:13
auto-merge was automatically disabled September 12, 2026 14:13

Pull request was converted to draft

The version and the digest are pinned on adjacent lines because they have to
move together. Renovate bumps the version from its datasource but cannot know
the new digest, so this bump left the pair inconsistent and the install-time
guard refused to install unverified bytes.

The guard reported the stale-pin fault specifically: the served bytes matched
this release's own published sha256sum.txt, so the bytes are right and only the
pin was behind. Re-running the guard with the digest below is what verifies it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Adapted: the bump was missing its digest

This PR could not finish on its own. setup-talosctl.sh pins the Talos version and the
talosctl SHA-256 on adjacent lines because they have to move together; Renovate bumps the
version from its datasource but cannot know the new digest, so the pair was left
inconsistent and the install-time guard refused to install unverified bytes.

The guard reported the stale-pin fault specifically — the bytes served for v1.14.0
matched that release's own published sha256sum.txt, so the download was right and only
the pin was behind. I set TALOSCTL_SHA256 to the value the guard named. Re-running the
guard is what verifies it: if the pin were wrong, the same check fails closed again rather
than installing anything.

The PR was converted to draft before this commit, which also disarmed the auto-merge that
was armed on it — an agent-authored commit restores the ordinary review gate, so it should
not merge on the pre-existing arming.

This PR gates #3534. That one bumps Kubernetes to v1.37.0 and fails with
Kubernetes v1.37.0 is not verified compatible with Talos v1.13.9 ... complete any Talos upgrade separately before raising Kubernetes. Landing this Talos upgrade first is what
unblocks it.

@renovate

renovate Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor Author

Edited/Blocked Notification

Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.

You can manually request rebase by checking the rebase/retry box above.

⚠️ Warning: custom changes will be lost.

The compatibility validator asks talos/pkg/machinery whether the pinned
Kubernetes version is supported by the pinned Talos version. machinery was still
at v1.13.9, which has no knowledge of the 1.14 release family, so it failed
closed — by design, rather than inheriting an old ceiling — and the validator's
own error names this as the remedy.

RED at machinery v1.13.9:
  Kubernetes v1.36.4 is not verified compatible with Talos v1.14.0:
  compatibility with version 1.14.0 is not supported
GREEN at v1.14.0:
  ksail.prod.yaml: pinned Kubernetes/Talos versions are compatible

The pinned Kubernetes version is unchanged; only the reviewed knowledge of which
Talos releases exist moves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Second lockstep pin: machinery

With the digest corrected the install succeeded and the job got one step further, to
validate-talos-kubernetes-compatibility, which then failed. That validator asks
talos/pkg/machinery whether the pinned Kubernetes version is supported by the pinned Talos
version, and go.mod still pinned machinery at v1.13.9 — which has no knowledge of the
1.14 release family. It fails closed rather than inheriting an old ceiling, which is the
documented intent, and its error names the remedy directly.

Reproduced locally against the same command CI runs:

RED — machinery v1.13.9:

ksail.prod.yaml: Kubernetes v1.36.4 is not verified compatible with Talos v1.14.0:
compatibility with version 1.14.0 is not supported
exit status 1

GREEN — machinery v1.14.0, nothing else changed:

ksail.prod.yaml: pinned Kubernetes/Talos versions are compatible
exit 0

The Kubernetes pin is untouched; what moved is only the reviewed knowledge of which Talos
releases exist. Full Go suite after the bump: 1633 tests across 16 packages, all passing.

Why this keeps happening

A Talos upgrade needs four coordinated edits, and Renovate can make only two of them:

edit who does it
TALOS_VERSION in setup-talosctl.sh Renovate
spec.cluster.talos.version in ksail.prod.yaml Renovate
TALOSCTL_SHA256 beside the version nobody — bump lands broken
talos/pkg/machinery in go.mod Renovate, but as a separate PR

The siderolabs/talos packageRule documents the ISO and lockstep-Kubernetes edits a
maintainer must complete, but not these two — so every Talos bump arrives red and needs this
same manual repair. Filed separately so it can be fixed once rather than rediscovered each
time.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Status: mechanically unblocked, deliberately not promoted

This PR is step 1 of #3626, which sequences the Talos upgrade ahead of the Kubernetes
one. What I have done is clear the two mechanical blockers that were keeping it red for
reasons unrelated to its actual merits — the stale TALOSCTL_SHA256 and the lagging
machinery pin. It can now be judged on the upgrade itself rather than on pin drift.

I am leaving it in draft, and it should not be merged on CI green alone. Merging rolls a
fleet-wide Talos OS upgrade, and this file documents the risk in its own comments:

Note the iso and version pins are already out of step on main (v1.13.9 against an ISO
the comment labels v1.12.4), so the "pin Talos to match the ISO" comment appears to
describe install media rather than the running upgrade target. That is worth confirming
rather than assuming in either direction.

Blocker: Hetzner ISO id selection + live-upgrade sequencing per #3626 | authority |
last-verified 2026-09-12: open — needs a maintainer decision on the ISO and on when to roll
the node upgrade | asked pr 2026-09-12

Once that is settled, the remaining path is the ordinary one: CI green, a current-head
review, promote, and enqueue.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Note: GitHub did not dispatch any workflow for 721c5843

The machinery commit was pushed at 2026-09-12T14:22:55Z and, eleven minutes later, none
of this PR's four pull_request workflows had started
— not CI, Dependency Review, Scan for
Workflow Vulnerabilities, or Enable Auto-Merge. Only the eight CodeQL check-runs are present at
that head.

This is not a path filter: ci.yaml triggers on pull_request: branches: [main] with no
paths: restriction, and the same workflows dispatched normally for the previous two pushes on
this branch (3b7d3436 and d5a25eee). The d5a25eee run was still in flight when I pushed
and was not cancelled despite cancel-in-progress: true, which points at the synchronize
event never reaching Actions rather than at anything in the workflow.

⚠️ Worth knowing before reading this PR's checks: a filtered "any failing checks?" query
returns empty here, which reads as green but actually means the jobs never ran. The required
CI - Required Checks is absent, not passing.

I have deliberately not forced a re-dispatch. The only lever that fires a default-type
pull_request event is close-and-reopen, and doing that to a Renovate PR risks Renovate
treating the update as rejected and declining to recreate it — a worse outcome than a missing
run, especially as this PR is parked on the iso decision anyway and is not a merge candidate
yet.

The next push to this branch (or the maintainer completing the iso edit) will dispatch
normally. The two fixes themselves are verified independently of CI: the digest by the guard's
own published-checksum comparison, and the machinery bump by a local RED/GREEN plus the full
Go suite at 1633 tests across 16 packages.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Refreshed this Talos 1.14 update onto current main with signed merge commit 877c9630797d662f688c3531135836902ccad155. The resulting diff is limited to the verified talosctl 1.14.0 pin, the production Talos 1.14.0 pin, and regenerated transitive checksums.

Validated locally:

  • go test ./scripts/validate-talos-kubernetes-compatibility
  • go run ./scripts/validate-talos-kubernetes-compatibility ksail.prod.yaml → pinned Kubernetes/Talos versions are compatible
  • shellcheck .github/scripts/setup-talosctl.sh scripts/tests/test-setup-talosctl.sh
  • bash scripts/tests/test-setup-talosctl.sh
  • the full Go package sweep completed except for two sandbox-only failures involving a loopback listener and /dev/stdout; both exact packages passed when rerun outside that sandbox

ksail workload validate was not counted as positive evidence: unrelated current-main Helm overlays reported render failures before the sweep was stopped. Fresh GitHub checks are running on the exact pushed head. The PR remains draft pending terminal CI and a substantive exact-head review.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Closing diagnosis at exact head 877c9630797d662f688c3531135836902ccad155:

  • CI run 34731189022 is RED in 🧪 Validate Talos Machine Config. Talos v1.14.0 rejects the existing legacy overlays because the generated 1.14 base already owns the same settings through multi-document config (machine.install, kubelet/node annotations, cluster network/Flannel, API server, controller manager, and proxy).

  • The deployed KSail v7.184.0 path has a second independent blocker. Its Talos 1.14 migration converts a shared legacy cluster.apiServer patch to KubeAPIServerConfig, but preserves cluster scope and emits that control-plane-only document into the worker bundle. A minimal current-head reproduction fails talosctl validate with:

    * KubeAPIServerConfig: kube-apiserver image cannot be empty
    * the following document kinds are only allowed on control plane machines: [KubeAPIServerConfig]
    

    Tracked as fix(talos): keep migrated API-server patches off workers ksail#7010 with the RED fixture and acceptance criteria.

  • The Platform patch migration also needs an explicit decision for machine.kubelet.extraMounts used by the Longhorn worker overlay; the Talos 1.14 KubeletConfig schema does not expose an equivalent field. I am retaining this as QUERY-UNKNOWN, not silently dropping the mount.

  • Live availability of the pinned Hetzner Talos ISO remains unverified because the current local hcloud credential cannot describe ISO 125127.

The PR therefore remains draft. I did not push a speculative patch migration or enable auto-merge; the next safe sequence is to fix/release KSail #7010, resolve the Longhorn mount mapping, migrate the Platform overlays with exact rendered-role tests, and then re-run the immutable ISO prerequisite plus the full PR gate.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Pushed the Talos 1.14 configuration migration at exact head 54b5f122bf2881ca2f91ba5fb73b362cbf9df73a, refreshed onto current main.

The previous head reproduced the CI failure locally with the same eight Talos 1.14 conflicts reported by run 34731189022. The repaired render now validates production and local control-plane and worker configs independently, while asserting the behavior that must survive the migration:

  • Flannel and kube-proxy remain disabled;
  • Dex and API audit policy remain control-plane-only;
  • the local control plane remains schedulable;
  • the GHCR revision annotation and Longhorn worker label use KubeNodeConfig;
  • the legacy kubelet representation is retained because Talos 1.14 KubeletConfig cannot express the required Longhorn extraMounts; its v1.36.4 image pin, seccomp default, and static-manifest lock remain explicit;
  • KSail is pinned to released v7.184.1, which folds the AppArmor kernel argument into the Image Factory schematic and keeps migrated API-server documents off workers;
  • fresh Talos 1.14 Longhorn workers receive the documented EPHEMERAL execution exception, while control planes retain secure mount defaults.

RED→GREEN evidence includes the exact eight-conflict machine-config validation and an ablation proving the Talos 1.14 worker lacked the Longhorn execution exception. GREEN checks: both rendered profiles and roles pass talosctl validate, compatibility Go tests and the production pin check pass, both Kustomize overlays build, all changed YAML parses, actionlint passes, and all new commits have verified signatures.

The PR remains draft. Live Hetzner ISO 125127 availability is still QUERY-UNKNOWN: the current local hcloud credential is rejected, so I did not promote or enable auto-merge on static evidence alone.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 13, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-13T20:17:58.355194Z 5e475ff Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e475ffbc5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread talos/control-planes/gc-terminated-pods-sooner.yaml
Comment thread ksail.prod.yaml
Comment thread ksail.prod.yaml
# Pin Talos to match the ISO so 'ksail cluster update' doesn't attempt
# an unwanted in-place upgrade to ksail's default version.
version: v1.13.9
version: v1.14.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Use a boot ISO that understands the Talos 1.14 config

On a fresh production create or the autoscaler snapshot rebuild triggered by this version change, the temporary node still boots ISO 125127, documented immediately below as Talos v1.12.4, before installing the target image. The generated configuration and the new patches now contain Talos 1.14-only document kinds such as KubeNodeConfig, KubeAuthenticationConfig, and UnattendedInstall; Talos' strict document decoder on the v1.12 boot environment cannot register those kinds, so bootstrap is rejected before it can install v1.14. Update the Hetzner ISO to a compatible release or upgrade the maintenance environment before applying the 1.14 configuration.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Generated by the Agentic Engineer

Still open. The two P2 findings are fixed in 5f5d186; this one needs the id of a Hetzner ISO that ships Talos 1.14, and I could not look that up from this run because the local Hetzner API credential was rejected (credentials/auth). Until a v1.14 ISO id replaces iso: 125127, this PR should not merge: a fresh create or snapshot rebuild would boot the v1.12.4 ISO against 1.14-only config documents.

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Codex findings at 5e475ffb — verified, not yet fixed

Codex was requested at this head because CodeRabbit skips this title. It reported one P1 and two P2 findings. I checked each against this head; all three hold, so this PR is not ready.

  • P1 — the boot ISO predates the new configuration. ksail.prod.yaml pins talos.version: v1.14.0 but still boots Hetzner ISO 125127, which its own comment documents as Talos v1.12.4. The migrated patches now use Talos 1.14 document kinds, so a fresh create or an autoscaler snapshot rebuild would hand them to a v1.12 maintenance environment. This needs a Hetzner ISO at a compatible Talos release (or a maintenance-environment upgrade first) before the config change ships.
  • P2 — the terminated-pod GC threshold is lost. The branch deletes talos/cluster/gc-terminated-pods-sooner.yaml and adds no equivalent controller-manager setting anywhere, so the Kubernetes default of 12,500 terminated pods returns. The value should move to the structured controller-manager document.
  • P2 — two recovery commands read the old install-image path. docs/rwx-storage.md:51 and docs/dr/runbook.md:132 still read .spec.machine.install.image, which the multi-document configuration no longer populates, so both would yield an empty image during node recovery.

The three review threads stay open until each is fixed. Because agent commits adapted this dependency PR, it keeps the normal review gate: after the fixes, the review loop restarts at CodeRabbit (then Codex while CodeRabbit keeps skipping this title). #3534 (Kubernetes 1.37) stays parked behind this upgrade.

Talos 1.14 moves kube-controller-manager args into a KubeControllerManagerConfig
document and the installer image into UnattendedInstallConfig. Restore the
100-pod terminated-pod GC threshold the migration dropped, and make the manual
recovery commands read the new document with a fallback for older configs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Still blocked on the Talos 1.14 boot ISO, rechecked today (2026-09-14). This is outside our control.

  • Hetzner's public changelog has no Talos 1.13 or 1.14 ISO announcement.
  • KSail's scheduled Hetzner ISO tracker last ran successfully at 2026-09-13T18:31Z. Its newest known ISO is still Talos v1.12.4 (125127), the one this branch boots.
  • The other route, a custom Talos 1.14 ISO uploaded to the Hetzner project, needs Hetzner API access. The local token is rejected (credentials/auth), so this run can't check it.

The two P2 findings are fixed at 5f5d1863. Only the P1 thread is open. This PR stays a draft until Hetzner publishes a compatible ISO or someone takes the custom-ISO route. platform#3534 waits on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: 🫴 Ready

Development

Successfully merging this pull request may close these issues.

1 participant