Failed to create snapshot due to an internal error (4.21.0.0-RC3)

[weizhouapache]

problem

more error logs

2025-08-15 05:58:06,216 ERROR [o.a.c.a.c.u.s.CreateSnapshotCmd] (API-Job-Executor-6:[ctx-ae78d929, job-179, ctx-e779649f]) (logid:2d361233) Failed to create snapshot due to an internal error creating snapshot for volume 6be22094-454f-466f-9f57-17522847e881 java.lang.RuntimeException: Unexpected exception
    at com.cloud.storage.VolumeApiServiceImpl.takeSnapshotInternal(VolumeApiServiceImpl.java:3910)
    at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:3821)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:109)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:52)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
    at jdk.proxy3/jdk.proxy3.$Proxy283.takeSnapshot(Unknown Source)
    at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:253)
    at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:173)
    at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:110)
    at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:689)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
    at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
    at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
    at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:637)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Can't find snapshot strategy to deal with snapshot:Snapshot {"id":10,"name":"aaa","state":"Allocated","uuid":"8ea4acea-f665-479f-840a-56f336ea4760","version":"2.2","volumeId":5}
    at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:1624)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
    at jdk.proxy3/jdk.proxy3.$Proxy260.takeSnapshot(Unknown Source)
    at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:2794)
    at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:3981)
    at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:5303)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:102)
    at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:5317)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
    at jdk.proxy3/jdk.proxy3.$Proxy283.handleVmWorkJob(Unknown Source)
    at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:99)
    ... 12 more

2025-08-15 05:58:06,217 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-6:[ctx-ae78d929, job-179]) (logid:2d361233) Complete async job-179, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":"530","errortext":"Failed to create snapshot due to an internal error creating snapshot for volume 6be22094-454f-466f-9f57-17522847e881"}

versions

4.21.0.0-RC3
KVM

The steps to reproduce the bug

1. deploy a vm
2. create a vm snapshot without memory
3. stop the vm
4. create a volume snapshot

What to do about it?

No response

[weizhouapache]

Got an error on another management server, which seems to be the root cause
2025-08-15 05:58:04,557 WARN [o.a.c.s.s.DefaultSnapshotStrategy] (Work-Job-Executor-6:[ctx-bcc76f85, job-179/job-180, ctx-5bbd5fed]) (logid:2d361233) VM [b50bdd89-1b6b-4ef3-bf85-c97d8cfa01a2] already has KVM File-Based storage VM snapshots. These VM snapshots and volume snapshots are not supported together for KVM. As restoring volume snapshots will erase the VM snapshots and cause data loss.

This is related to #10632

@JoaoJandre
would it be possible to make the error message more clear ?

[JoaoJandre]

Got an error on another management server, which seems to be the root cause 2025-08-15 05:58:04,557 WARN [o.a.c.s.s.DefaultSnapshotStrategy] (Work-Job-Executor-6:[ctx-bcc76f85, job-179/job-180, ctx-5bbd5fed]) (logid:2d361233) VM [b50bdd89-1b6b-4ef3-bf85-c97d8cfa01a2] already has KVM File-Based storage VM snapshots. These VM snapshots and volume snapshots are not supported together for KVM. As restoring volume snapshots will erase the VM snapshots and cause data loss.

This is related to #10632

@JoaoJandre would it be possible to make the error message more clear ?

@weizhouapache I can change the error message that is displayed to the end user. But this behavior is expected, as the log you posted explains.

If you want to use VM snapshot and volume snapshot at the same time, you should not use VM snapshot strategy introduced in #10632. You may disable it by adding it to the KvmFileBasedStorageVmSnapshotStrategy value to the vmSnapshot.strategies.exclude configuration.

[weizhouapache]

Got an error on another management server, which seems to be the root cause 2025-08-15 05:58:04,557 WARN [o.a.c.s.s.DefaultSnapshotStrategy] (Work-Job-Executor-6:[ctx-bcc76f85, job-179/job-180, ctx-5bbd5fed]) (logid:2d361233) VM [b50bdd89-1b6b-4ef3-bf85-c97d8cfa01a2] already has KVM File-Based storage VM snapshots. These VM snapshots and volume snapshots are not supported together for KVM. As restoring volume snapshots will erase the VM snapshots and cause data loss.
This is related to #10632
@JoaoJandre would it be possible to make the error message more clear ?

@weizhouapache I can change the error message that is displayed to the end user. But this behavior is expected, as the log you posted explains.

@JoaoJandre
Agree the behaviour is expected. I do not think it is a major issue we need to fix asap.

I can change the error message that is displayed to the end user.

that would be better I think. thanks in advance @JoaoJandre

If you want to use VM snapshot and volume snapshot at the same time, you should not use VM snapshot strategy introduced in #10632. You may disable it by adding it to the KvmFileBasedStorageVmSnapshotStrategy value to the vmSnapshot.strategies.exclude configuration.

[sureshanaparti]

If you want to use VM snapshot and volume snapshot at the same time, you should not use VM snapshot strategy introduced in #10632. You may disable it by adding it to the KvmFileBasedStorageVmSnapshotStrategy value to the vmSnapshot.strategies.exclude configuration.

@JoaoJandre does this need to be documented?

[JoaoJandre]

If you want to use VM snapshot and volume snapshot at the same time, you should not use VM snapshot strategy introduced in #10632. You may disable it by adding it to the KvmFileBasedStorageVmSnapshotStrategy value to the vmSnapshot.strategies.exclude configuration.

@JoaoJandre does this need to be documented?

@sureshanaparti it is documented in #9524. But I can raise a doc PR for it.

[sureshanaparti]

@JoaoJandre does this need to be documented?

@sureshanaparti it is documented in #9524. But I can raise a doc PR for it.

thanks @JoaoJandre , it's good to have this in docs.

[sureshanaparti]

Fixed in #11455