Repository navigation
ipc_send_queued_msg() has a hard dependency on the EDF scheduler #4356
Description
Activity
Thanks @kv2019i and @lyakh for trying to solve this issue offline, can you please share the "best bits" here?
cc: @lgirdwood , @andyross , @plbossart and @nashif
@marc-hb I think we should use the Zephyr ktimer API (like used in the ll-scheduler in zephyr_domain.c) to create a repeating task to flush out the IPC send queue. This could be registered from IPC code if needed (DMA traces or position updates requested).
Reacted by Liam Girdwood@marc-hb lets do as @kv2019i says, we either DMA copy the buffer to host on
- Buffer free space < some limit - we can then wake the DMA thread
- Periodic empty - lets say every second we wake the DMA thread.
We have the logic to determine if conditions 1 or 2 are true today, but do not have the thread today.
The thread should be similar to
dma_trace_copy_thread() { /* sleep until we are woken */ sleep(); /* something woke us - copy the available data */ dma_copy_data_to_host(); }
To really simplify the logic for 2 (and reusing the LL timer) I would attach a counter to the LL thread as the dma_trace period counter and wake the thread from here (instead of creating a separate timer).
@marc-hb I think we should use the Zephyr ktimer API (like used in the ll-scheduler in zephyr_domain.c) to create a repeating task to flush out the IPC send queue. This could be registered from IPC code if needed (DMA traces or position updates requested).
@kv2019i
k_timerdirectly? That runs in the interrupt context...And a plain k_thread which sleeps, flushes IPCs and sleeps again, would do as well.
Reacted by Liam Girdwood@marc-hb lets do as @kv2019i says, we either DMA copy the buffer to host on
Buffer free space < some limit - we can then wake the DMA thread
Periodic empty - lets say every second we wake the DMA thread.I think the
trace_work()DMA thread already does that and it even runs in Zephyr already. However it merely adds theposnupdate IPC to a (shared?) IPC queue and it's only that IPC queue that does not get flushed (yet) in Zephyr.
I found this IPC scheduling and dependency issue while troubleshooting DMA traces on Zephyr but I suspect it has consequences even without Zephyr. In fact I wonder whether it could maybe explain the very long standing #4333 "lag" in DMA traces too.
In Zephyr the problem is simple (once you know it!): no (legacy) EDF scheduler ->
task_main_primary_coredoes not run -> no IPC TX flush -> no DMA traces.I tried to schedule
task_main_primary_core()in Zephyr too but I hit the following circular dependency with the EDF scheduler where the EDF scheduler wants to initialize the main task in the middle of its own initialization andtask_main_init()needs the half-initialized EDF scheduler to schedule itself.Here's one super-ugly and probably unreliable workaround. Warning: many other code changes are required to get DMA traces with Zephyr.
Even in plain SOF, this does not "feel right" and as I wrote above I wonder whether it could explain the very regular DMA traces "lag" exposed in #4333
The circular dependency and all the code above is as old as the entire scheduler itself: Aug 2019 20acc16 in giant PR #1710
There was a small
ipc_send_queued_msg()refactor in Feb 2020 commit 9e0b2e5 but I don't think it changed the logic.