Linux QBMan QMan driver safe_copy_dqrr

[tim8]

The Linux SDK for QorIQ processors includes qman_high.c in the driver staging tree. This includes the function safe_copy_dqrr(), which the comments say is ensuring reads reach QBMan in order and //preventing a WRAP transaction to be seen by the QBMan//.

I presume this is referring to an AXI WRAP burst transaction, rather than anything to do with the DQRR index wrap.

Can anyone explain why this is problem that needs to be made safe?

Thanks in advance.

[tim8]

Thanks for you reply.

If I understand correctly, with DQRR stashing disabled the CPU read of a DQRR entry (after a cache invalidation and memory barrier) triggers a cache line fill but this can be "broken" by a AXI wrap burst transaction while the DQRR entry is being updated?

I'm using LS1046A RDB and can see no reference to this behaviour in NXP manuals. Is it documented somewhere? I'm looking for a reference to justify this safe copy function.

With DQRR stashing enabled, QMan writes the DQRR entry to cache. Is this subject to the same issue?

Thanks again for your help.

[Bio_TICFSL]

Hello,

Yes — the comment is most plausibly about a bus/cache-line WRAP transaction , not the DQRR producer/consumer index wrapping.

The reason it needs to be “safe” is that a DQRR entry is not ordinary RAM. It is a QBMan portal structure whose contents are produced by QMan, possibly stashed into the CPU cache, and consumed by software using the valid bit in the VERB byte . The DPAA manuals state that even though QMan updates a DQRR entry atomically, a GPP core’s reads may not observe all bytes of the entry changing at the same time; software must ensure the VERB byte containing the valid bit is loaded before the rest of the DQRR entry.

So safe_copy_dqrr() is trying to avoid this unsafe pattern:

CPU wants to copy DQRR entry
  -> compiler/CPU/cache subsystem turns that into a burst or cache-line fill
  -> transaction reaches QBMan as a WRAP-style read
  -> QBMan/interconnect may see/read beats in an order different from the logical field order
  -> software may observe a valid VERB byte associated with stale or not-yet-visible entry data

That matters because the valid bit is the synchronization point. With DQRR entry stashing enabled, the documentation specifically warns that DQRR.PI can be updated before the associated DQRR entry stash arrives in the processor cache, creating a race; therefore software should use the valid bit in the DQRR entries, not DQRR.PI , to detect production. The valid bit itself is an alternating-polarity bit that toggles each time the ring wraps from the last entry back to entry 0.

Regards