2400864_en-US

Yes

MCU-Link - Linkserver - Injection of unwanted cyclecounts at performance tests

Hello there,

First of all: everything written is from a claude agent, because everything I make is done inside an obsidian Repo and it is faster to let the agent write

I am doing cycle-accurate timing measurements on an FRDM-MCXN947 for my master's thesis, where I compare the scheduling overhead of some real-time operating systems. Every number I report is a difference of two DWT->CYCCNT reads, so the measurement has to be repeatable down to the single cycle. It is not, and after a fairly long investigation I have narrowed the cause down to one mechanism that I cannot confirm from the documentation. I would like to ask whether my explanation is actually correct, because if it is wrong then I have excluded everything else and have no candidate left.

The short version of the question: while a debug session is open, LinkServer periodically reads DHCSR and CPACR to find out whether the core has halted. Both of these are SCS registers inside the private peripheral bus, so the access is serviced inside the core and never appears on the code bus or the system bus. My assumption is that such an access nevertheless competes with the core's own instruction fetches and data accesses somewhere inside the core, and that this is what costs my measurement a few cycles every time a poll lands inside the measured window. Is that assumption right?

Setup

The board is an FRDM-MCXN947 and I am using core0 only, at 150 MHz for the real measurements and at 12 MHz for a control experiment described further down. I debug through the on-board MCU-Link with LinkServer, which speaks CMSIS-DAP v2 over USB bulk, and I have tried SWD wire clocks of both 50 and 500. The launch configuration passes --no-rtos and --semihost-port=-1, and liveWatch is disabled, so the IDE should not be reading anything from the target while the core runs.

The code under test executes from RAM at 0x2000_0000, which means it is fetched over the system bus. Caches and RAM ECC are off. There is a second, separate variation in how the image gets into RAM: for most of the work the probe simply loads it there, but for one experiment I stored the image in flash and copy it into RAM during startup, because an image that only exists in RAM cannot start without a probe to load it in the first place. That distinction matters only for that one experiment, and I mention it now so the setup is unambiguous later.

The thing being measured is an empty counting loop, executed a given number of times and timed by reading CYCCNT before and after. I repeat each measurement 64 times.

Problem

Running the identical measurement twice does not give the identical answer. On the empty loop the 64 values spread over three to six cycles, and on real application code roughly nine out of a hundred thousand iterations come out one to ten cycles above all the others. The deviations only ever go upward, never downward, and the minimum value is perfectly stable and reproducible. What was most informative is that the spread grows with the length of the measurement window rather than being a fixed cost per measurement call, which already rules out a constant overhead somewhere in my instrumentation.

Here is the full sweep. Each cell is 64 measurements of the empty loop; "outliers" counts how many of the 64 came out above the first one, and min and max are given as cycles above the loop's clean value.

Core Loop count SWD 50: outliers min max SWD 500: outliers min max

Core F	loops	outliers per 64 MEasurements	min jitter count	max jitter count	outliers/64mea.	min jitter	max jitter
12 MHz	100	0	+13	+13	0	+13	+13
	1 000	4	+13	+15	7	+13	+16
	10 000	24	+13	+16	18	+13	+17
	100 000	55	+13	+23	57	+13	+22
	1 000 000	56	+40	+69	59	+45	+102
150 MHz	100	0	+13	+13	0	+13	+13
	1 000	0	+13	+13	1	+13	+15
	10 000	4	+13	+15	3	+13	+15
	100 000	13	+13	+15	16	+13	+17
	1 000 000	53	+13	+20	60	+13	+25

Three things come out of this table. First, the constant +13 is not part of the problem. It shows up as the minimum in every cell that has not saturated, at both core clocks and at both wire speeds, and it is simply the loop prologue plus the CYCCNT read itself. Because it is independent of both the core clock and the probe, it is a fixed bias that I subtract, and everything above +13 is what I am actually chasing.

Second, and this is the important one, the disturbance follows wall-clock time and not the number of cycles executed. If you compare the two clocks at the same loop count, there are consistently more outliers at 12 MHz than at 150 MHz: four against zero and seven against one at a thousand iterations, twenty-four against four and eighteen against three at ten thousand, fifty-five against thirteen and fifty-seven against sixteen at a hundred thousand. The instruction stream is identical in both cases and therefore executes the same number of cycles, and the only thing that differs is that the 12 MHz run takes 12.5 times longer in real time.

Third, the SWD wire clock makes no difference at all. Across the ten matched pairs above the counts scatter in both directions and stay within counting noise, so a tenfold change in wire rate leaves them alone. Whatever sets the pace, it is not the wire.

Excluded as root cause

I want to be upfront that a lot of this was ruled out by actually reading the relevant register back from the target with the session open, rather than by assuming that the reset default still held.

On the SoC side, CPU1, eDMA0, eDMA1 and the SmartDMA are all disabled, RAM ECC is off and the code cache is off, so nothing else is contending for memory. On the trace side, ITM is disabled, no stimulus port is enabled, timestamps are off, the stall-for-trace bit is clear and ETM is off, which means nothing is generating trace packets and therefore nothing can be stalling in order to deliver them. Within the DWT, PC sampling is off, exception trace is off, all the event counters are off, the watchpoint comparators are unused, and the performance monitor is off. TRCENA is of course set, but it has to be set in both the disturbed and the undisturbed case because CYCCNT depends on it, so it cannot be the difference between them.

Interrupts and exceptions are out on grounds of size alone. Any exception on this core costs somewhere between twenty-five and fifty cycles, and my deviations are one to ten. There is no small version of an exception, so the entire class is excluded regardless of what the interrupt masks say. Breakpoints are out for a similar reason: an FPB comparator costs nothing until it matches, and when it matches it halts the core rather than delaying it slightly.

The code under test is out because of a control experiment. An empty loop contains no data-dependent work whatsoever and therefore cannot vary on its own, and it varies anyway. So the cause is in the platform, not in what I am measuring.

That left debugger memory reads as by far the most promising explanation, and I would like to explain why I dropped it, because it is the mechanism most people would reasonably suspect. A debug read of SRAM does leave the core and would genuinely fight the core for the bus port, so the mechanism would work perfectly. The trouble is that it does not happen. Every read the IDE performs travels as a GDB m packet, and in the gdbserver log there is not a single packet between the $vCont;c that starts execution and the host interrupt thirty-two seconds later. Every memory read in that log sits after the halt, which is just the IDE filling its views once the core has stopped. The same argument disposes of the even more attractive theory that the debugger is reading CYCCNT itself, which would contend with my measurement directly: that would also appear as an m packet, and it does not, and liveWatch is disabled anyway.

What it seems to be

To separate the probe from the session I ran the identical binary with the probe physically plugged in
but with no debug session open. This is the experiment that needed the flash-to-RAM copy at startup I
mentioned earlier, since a RAM-only image has no way to get loaded without a probe. The probe stayed
connected for both runs, so the only difference between them was whether a session was attached.

Without a debug session the jitter is gone. Same binary, same clock, same boot path. So it is not the
probe being connected, and it is not the build: a probe that is merely attached but issuing no
transactions costs nothing, and attaching a session brings the jitter back.

At that point I had a disturbance that is definitely caused by the session, that is paced by real time
rather than by the core clock, and for which I had eliminated every mechanism I could think of.

To find out what LinkServer sends while the core is running, I captured the MCU-Link's CMSIS-DAP bulk endpoints with USBPcap and Wireshark. Two DAP_Transfer requests repeat continuously, roughly every fifty milliseconds.

I should immediately qualify that number. The polling does not look like a fixed timer, because the host appears to send the next request only after the previous response has come back, which makes the interval at least partly event-driven — host turnaround plus USB latency plus whatever delay the host inserts of its own accord. So fifty milliseconds is an order-of-magnitude figure for how often it repeats, not a period I would want to quote, and it also means I cannot cleanly reason backwards from the rate.

The first request reads DHCSR and CPACR together. On the wire it is 05 00 06 08 00 00 00 00 05 f0 ed 00 e0 0f 08 00 00 00 00 05 88 ed 00 e0 0f, which decodes as:

# Req Decode Data Target access

1	08	DP write SELECT	0x00000000	none (DP-internal)
2	05	AP write TAR	0xE000EDF0 (DHCSR)	none (AP-internal)
3	0f	AP read DRW	—	1 read, PPB
4	08	DP write SELECT	0x00000000	none (DP-internal)
5	05	AP write TAR	0xE000ED88 (CPACR)	none (AP-internal)
6	0f	AP read DRW	—	1 read, PPB

The answers were DHCSR = 0x01100001, meaning the core is running with secure debug enabled and instructions retiring, and CPACR = 0x00F00003, meaning the FPU is fully accessible. The second request is 05 00 03 08 00 00 00 00 05 f0 ed 00 e0 0f, which is just the first half of the above and asks only whether the core has halted, without the FPU check.

So the entire traffic reaching the target during my measurement window is two PPB reads, or one in the case of the shorter request. The SELECT and TAR writes produce no target access at all, since they are internal to the DP and the AP respectively, and only the two DRW reads actually go anywhere. I checked the transfer count against the data-word count and confirmed a direct value-to-register mapping with no posted-read shift on this adapter, so I am confident about which value belongs to which address.

Both addresses are SCS registers inside the PPB, and neither appears on the SoC bus matrix. There is no SRAM access anywhere in this, and there is no access to the core register file either, since nothing goes through the DCRSR/DCRDR keyhole, which would require the core to be halted first. For completeness: a separate seventy-one-transfer dump of S0 to S31 and FPSCR does appear in the capture, but only at a moment when the core was halted at a stopAtSymbol breakpoint, not during the run. My guess is that CPACR is bundled with DHCSR precisely so that the adapter already knows whether the floating-point registers are live at the instant the core does halt.

Assumption to check:

What I currently believe is happening is this. The request enters the core through the D-AHB debug
port and is then arbitrated by the core's internal interconnect against the core's own instruction
fetches and data accesses. Being resolved inside the core does not put the access on a path that is
disjoint from execution; it still meets execution at a shared internal stage. The core holds priority,
so in the normal case the debug access waits and the core carries on unaffected. The residual cost
appears only when a debug beat is already in flight and cannot be preempted, in which case the core
has to wait for that beat to finish. That wait would be exactly what I observe: a few cycles, always
upward, and only on the fraction of measurement windows that a poll happens to fall inside.

Questions:

Is the mechanism I described above actually correct? Does a DAP-originated read of a PPB or SCS register such as DHCSR or CPACR, arriving over the D-AHB debug port, contend with the running core's own accesses inside the M33 in a way that can stall it by a small number of cycles? Or is the debug port genuinely disjoint from the execution path for PPB targets, in which case I have excluded everything and am still missing the real cause?
Is this documented anywhere? I have the Cortex-M33 TRM (100230_0100_07_en) and can see the D-AHB and the internal interconnect in it, but I have not found any statement about what a debug access to core-internal PPB space costs while the core is running. A pointer to the right section, or to anything MCXN947-specific about how the DAP is integrated on this part, would help me a great deal.
Would CYCCNT even see such a stall? In other words, is the core genuinely held up, so that these are real execution cycles that CYCCNT counts, rather than CYCCNT being perturbed by some other route?
Is there any way to slow down or disable the halt-state poll in LinkServer? That would give me the clean confirmation I am after: change the poll rate and see whether the outlier rate follows. I have not been able to find a setting for it, and anything that reduces or stops the periodic DHCSR read while the core runs would work for this test.
If the mechanism is real, is measuring with no session attached simply the correct practice for cycle-accurate work on this part? That is what I do now, and where a session is unavoidable I fall back on taking the minimum across many windows and reporting how many deviated and by how much. I would like to know whether that is the accepted answer, or whether there is a supported way to measure cleanly with a live session.

Core and Memory

Development Board

Re: MCU-Link - Linkserver - Injection of unwanted cyclecounts at performance tests

Hi @hms-isyu

Thank you for your detailed analysis.

I'm sorry, but this question is beyond the typical scope of NXP MCU application support. I can help with MCXN947, MCU-Link, SDK, and device usage questions. However, the behavior you are investigating involves Cortex-M33 debug architecture and cycle-accurate performance analysis, which is outside my area of expertise.

For questions related to the injection of additional cycle counts during performance measurements, I would recommend contacting Arm support for further guidance.

Thank you for your understanding.

BR

Alice