Repository navigation
[BUG] xrun on TGLH RVP NOCODEC ZEPHYR when testing multiple pause resume #6091
Description
Describe the bug
When multiple pause resume on TGLH RVP NOCODEC ZEPHYR, xrun happened
in inner test result 14447
To Reproduce
TPLG=/lib/firmware/intel/sof-tplg/sof-tgl-h-nocodec.tplg MODEL=TGLH_RVP_NOCODEC_ZEPHYR ~/sof-test/test-case/multiple-pause-resume.sh -r 50
Reproduction Rate
Observe only once in CI so far
Environment
Kernel Branch: topic/sof-dev
Kernel Commit: 655b48ffac48
SOF Branch: main
SOF Commit: 8881a406af67
Zephyr Commit: zephyr-v3.1.0-2058-g63a497280f22
Topology: sof-tgl-h-nocodec.tplg
Platform: TGLH_RVP_NOCODEC_ZEPHYR
Screenshots or console output
sof-logger
[ 49988287.805311] ( 45.104164) c0 pipe 1.3 ....../pipeline-stream.c:270 INFO pipe trigger cmd 6
[ 49988417.805305] ( 42.239582) c0 zll-schedule src/schedule/zephyr_ll.c:69 INFO task complete 0xbe081200 pipe-task
[ 49988131.294900] ( 57.291664) c0 dw-dma src/drivers/dw/dma.c:1128 ERROR dw_dma_get_data_size(): xrun detected
[ 49988191.034481] ( 59.739582) c0 dai 1.2 src/audio/dai-legacy.c:905 ERROR dai_report_xrun(): underrun due to no data available[ 49988242.701146] ( 51.666664) c0 dai 1.2 src/audio/dai-legacy.c:787 ERROR comp_underrun(): dev->comp.id = 2, source->avail = 192, copy_bytes = 0
[ 49988330.826142] ( 43.020832) c0 pipe 1.3 ....../pipeline-stream.c:143 ERROR pipeline_copy(): ret = -61, start->comp.id = 2, dir = 1
[ 49988375.565724] ( 44.739582) c0 pipe 1.3 ..../pipeline-schedule.c:208 ERROR pipeline_task(): xrun recovery failed! pipeline is stopped.
[ 49988437.701138] ( 19.895832) c0 zll-schedule src/schedule/zephyr_ll.c:71 INFO num_tasks 3 total_num_tasks 3
[ 49990044.836491] ( 1607.135376) c0 host 7.36 src/audio/host-legacy.c:475 INFO no bytes to copy, 0 free in buffer, 384 available in DMA
[ 49992040.721828] ( 1995.885376) c0 host 7.36 src/audio/host-legacy.c:475 INFO no bytes to copy, 0 free in buffer, 384 available in DMA
[ 49994040.930082] ( 2000.208252) c0 host 7.36 src/audio/host-legacy.c:475 INFO no bytes to copy, 0 free in buffer, 384 available in DMA
[ 49996040.982086] ( 2000.052002) c0 host 7.36 src/audio/host-legacy.c:475 INFO no bytes to copy, 0 free in buffer, 384 available in DMA
Activity
marc-hb commentedon Aug 5, 2022
@XiaoyunWu6666 is a brand new regression? Did it start with
just like this other new regression?
marc-hb commentedon Aug 5, 2022
TGL: enable CONFIG_INCOHERENT correctlywas reverted in #6094plbossart commentedon Aug 5, 2022
possible similar sighting in https://sof-ci.01.org/linuxpr/PR3796/build485/devicetest/?model=ADLP_RVP_SDW_ZEPHYR&testcase=multiple-pause-resume-5
xrun leading to IPC error.
It's a SoundWire v. nocodec configuration though, it could be a separate issue.
lgirdwood commentedon Aug 5, 2022
Pause is always a P2.
plbossart commentedon Aug 5, 2022
I disagree @lgirdwood, If we end-up with a fatal IPC error it's a P1. Xruns are tolerable since they are not fatal.
lgirdwood commentedon Aug 5, 2022
It's P2 as it's real world usage is negligible and I'd rather focus on issues that will hit the majority of users (and we have some). Sure, become a P1 is it causes IPC timeout or crash.
plbossart commentedon Aug 5, 2022
https://sof-ci.01.org/linuxpr/PR3796/build485/devicetest/?model=ADLP_RVP_SDW_ZEPHYR&testcase=multiple-pause-resume-5 is an IPC error and crash @lgirdwood
it's like a kernel oops, it doesn't matter how we got there, it shouldn't happen. Rather, it shall not happen.
lgirdwood commentedon Aug 5, 2022
This looks like the same issue/use case and it now qualifies as a P1 since it's a FW crash/fatal. i.e no longer a survivable IPC error reply as reported initially but now a fatal IPC timeout.
9 remaining items
lyakh commentedon Aug 18, 2022
I found that daily test result Nr. 14447 and I see the xrun messages as quoted in this report, but I don't see IPC timeouts there?
lyakh commentedon Aug 18, 2022
ah, I see that @lgirdwood changed the original report title and added an "IPC timeout" there. Interestingly, I don't see an IPC timeout in that daily result, but I found a similar xrun in daily result Nr. 14634 on ADLP RVP nocodec XTOS and there I do see an IPC timeout, so that one is more similar to #5850 although the latter one is on a Chrome device and it's the HDA DMA that's hitting an xrun there, unlike the former, where it's DW DMA.
[-][BUG] IPC timeout TGLH RVP NOCODEC ZEPHYR when testing multiple pause resume [/-][+][BUG] xrun TGLH RVP NOCODEC ZEPHYR when testing multiple pause resume [/+][-][BUG] xrun TGLH RVP NOCODEC ZEPHYR when testing multiple pause resume [/-][+][BUG] xrun on TGLH RVP NOCODEC ZEPHYR when testing multiple pause resume [/+]lyakh commentedon Aug 19, 2022
indeed, I'd say this one is more like #5850
lyakh commentedon Aug 25, 2022
Having studied 5 daily and 1 PR test failures with IPC timeouts I came to the conclusion, that we are looking at 2 failure types:
I tried to reproduce the problem with the multiple-pause-resume test on an UP-Xtreme TGL board with both topologies (nocodec was modified to only include SSP0) - no success so far.
plbossart commentedon Aug 26, 2022
@lyakh I find the table above confusing, if you have a timeout on DMIC, isn't that using the DW DMA? how can we have xruns on the HDA DMA here?
lyakh commentedon Aug 26, 2022
@plbossart I meant, that there are no xruns on DW DMA in those cases. HDA DMA xruns - no idea if they're related, e.g.
see https://sof-ci.01.org/linuxpr/PR3796/build485/devicetest/?model=ADLP_RVP_SDW_ZEPHYR&testcase=multiple-pause-resume-5
plbossart commentedon Aug 26, 2022
@lyakh The HDAudio host DMA would be used, there's really no notion of xrun at that level. There's a very large DDR buffer as destination.The xruns only make sense at the link level, no?
lyakh commentedon Sep 6, 2022
The most recent occurrence of this bug, known to me was on ADLP_RVP_NOCODEC in daily test result 14634 of August 14 2022. Let's wait for another week until a full month with no reproduction, then we can close this.
kfrydryx commentedon Sep 13, 2022
No longer reproducing