Synchronization of ethernet and audio clock via PTP in IMX8MP

[MatejI]

Dear Support,

I would like to ask about PTP clock synchronization in IMX8MP. We are using Toradex Mallow board V1.1 and IMX8MP.

My goal is to synchronize audio clock (derived from audio_pll1) via PTP with ethernet clock. I tried several approaches but none worked. I derived my custom_audio_clock from audio_pll1 and used it for the audio codec. Everything worked fine. Then I tried to run:

phc2sys -s /dev/ptp0 -c /dev/custom_audio_clock -w

to synchronize ethernet clock with my custom_clock, but the custom_clock (or any other clock than ethernet) was not exposed to the /dev. When I use compatible = "fixed-clock" or "fixed-factor-clock", it is recognized by imx8mp clock driver, I can see the clock within clk_summary, but it is not exposed and linuxptp does not see it. I am not sure, but I guess the clock itself should have separate driver so it is compatible with PTP, is that correct?

I tried to create custom clock driver with custom compatible property, but for some reason, even though the driver is compiled and compatible matches the one in device tree, it is never even loaded. I would like to ask you, if there is way to expose the clock to /dev so I can use phc2sys functionality or is it possible to synchronize a clock with ethernet clock via PTP just by modifying some registers?

Here I attach my approach and changes to device tree:
&clk {
/* Define custom_clock as a fixed-factor clock derived from audio_pll1 */
custom_audio_clock: custom_audio_clock {
compatible = "fixed-factor-clock";
clocks = <&audio_blk_ctrl IMX8MP_CLK_AUDIO_BLK_CTRL_SAI1_MCLK1>;
clock-names = "custom_audio_clock";
clock-div = <1>; /* need to achieve 12288000 Hz! */
clock-mult = <1>;
#clock-cells = <0>
};
};

&fec {
phy-supply = <&reg_eth2phy>;
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_fec>;
phy-mode = "rgmii-id";
phy-handle = <&ethphy0>;
fsl,magic-packet;
fsl,ptp-clock-source = <1>; /* Use custom_clock as the PTP clock source */
fsl,ptp-timer-period = <81380>; /* 12288000 Hz = 81.38 ns period */
};

&flexspi {
status = "okay";
ak4621: audio-codec@0 { 

compatible = "asahi-kasei,ak4621";
reg = <0>;
clocks = <&custom_audio_clock>;
clock-names = "mclk";
#sound-dai-cells = <0>;
};
};

And attached, I also provide my custom clock driver (which is probably incorrect). Is there a way to synchronize audio_pll1 clock (or some children of it) with ptp0 (ethernet clock)?

Thank you.

Best regards

Matej I.

[MatejI]

Is it possible to solve this by routing ENET_AXI_CLK_ROOT as master of AUDIO_PLL1_CLK? In IMX8MP reference manual, page 231, change ENET_AXI_CLK_ROOT as master of AUDIO_PLL1_CLK by modifying 0x8880101?

Best regards

Matej I.

[Zhiming_Liu]

Hello,

The PLLs are generated from 24MHz, you could adjust the PLLs rate, but can't modify it's source.

In i.MX8MP, the available rate tables is in struct imx_pll14xx_rate_table imx_pll1443x_tbl drivers/clk/imx/clk-pll14xx.c. If your audio clock is divided from AUDIO_PLL1_CLK, the audio clock rate you used should be able to get from  these rates in imx_pll1443x_tbl.

For example, the micfil use IMX8MP_AUDIO_PLL1_OUT as source of  "pll8k", and IMX8MP_AUDIO_PLL2_OUT as source of "pll11k"

micfil: micfil@30ca0000 {
					compatible = "fsl,imx8mp-micfil";
					reg = <0x30ca0000 0x10000>;
					interrupts = <GIC_SPI 109 IRQ_TYPE_LEVEL_HIGH>,
						     <GIC_SPI 110 IRQ_TYPE_LEVEL_HIGH>,
						     <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>,
						     <GIC_SPI 45 IRQ_TYPE_LEVEL_HIGH>;
					clocks = <&audio_blk_ctrl IMX8MP_CLK_AUDIOMIX_PDM_IPG>,
						 <&audio_blk_ctrl IMX8MP_CLK_AUDIOMIX_PDM_ROOT>,
						 <&clk IMX8MP_AUDIO_PLL1_OUT>,
						 <&clk IMX8MP_AUDIO_PLL2_OUT>,
						 <&clk IMX8MP_CLK_EXT3>;
					clock-names = "ipg_clk", "ipg_clk_app",
						      "pll8k", "pll11k", "clkext3";
					dmas = <&sdma2 24 25 0x80000000>;
					dma-names = "rx";
					power-domains = <&audiomix_pd>;
					status = "disabled";
				};

These two clocks are handled in sound/soc/fsl/fsl_utils.c. In a way, these speeds are not freely settable, there are certain bindings that depend on whether the PLLs can support divide to get these speeds. You can check the clocks setting in sound/soc/fsl/fsl_micfil.c and sound/soc/fsl/fsl_sai.c.

/**
 * fsl_asoc_get_pll_clocks - get two PLL clock source
 *
 * @Dev: device pointer
 * @pll8k_clk: PLL clock pointer for 8kHz
 * @pll11k_clk: PLL clock pointer for 11kHz
 *
 * This function get two PLL clock source
 */
void fsl_asoc_get_pll_clocks(struct device *dev, struct clk **pll8k_clk,
			     struct clk **pll11k_clk)
{
	*pll8k_clk = devm_clk_get(dev, "pll8k");
	if (IS_ERR(*pll8k_clk))
		*pll8k_clk = NULL;

	*pll11k_clk = devm_clk_get(dev, "pll11k");
	if (IS_ERR(*pll11k_clk))
		*pll11k_clk = NULL;
}
EXPORT_SYMBOL(fsl_asoc_get_pll_clocks);

Best Regards,
Zhiming

[MatejI]

Dear Zhiming,

Thank you for the information. I am not sure I made myself clear. I would like to reroute the source of SAI (or one of its parents) to ethernet clock instead of the default PLL. I need to have a clock of 12288000 Hz (as long as there is a possible division between this frequency and the source clock, it should be fine) as audio clock and its parent has to be either:

-- ethernet clock

-- or a clock that is synchronized with ethernet clock.

So I guess my options would be to derive custom clock directly from ethernet clock or change the source of audio_pll1 via registers to ethernet clock. Based on the information from IMX8MP full reference manual on page 231, is it possible to modify ENET_AXI_CLK_ROOT to AUDIO_PLL1_CLK by 0x8880101? If I understand correctly, I should be able to feed ethernet clock to the audio_pll1, right?

Thank you.

Matej I.

[Zhiming_Liu]

Hello,

Yes, you can choose AUDIO_PLL1_CLK as ENET_AXI_CLK_ROOT. It's very simple, modify the clock parent in enet, replace the IMX8MP_SYS_PLL1_266M to IMX8MP_AUDIO_PLL1_OUT. Then the enet clock will divide from IMX8MP_AUDIO_PLL1_OUT. But you need take care if you could use the adjustable ENET_AXI_CLK_ROOT in your application. If you have confirmed that this will not affect your ethernet application, please ignore last sentence.

Best Regards,
Zhiming

[MatejI]

I misunderstood the table I refered to. I mean do the opposite thing – making audio_pll or its children children of ethernet clock. However, I may find a workaround by deriving custom_audio_clock from ethernet clock itself. If the PTP ethernet clock is by default synchronized via PTP, the derived custom clock could be synchronized as well. Is that correct?

Best regards

Matej I.

[Zhiming_Liu]

Hello,

I mean do the opposite thing – making audio_pll or its children children of ethernet clock

-->You can choose source of ethernet clock,  the source is flexible, but the final clock output node of ethernet is fixed.

deriving custom_audio_clock from ethernet clock itself

-->With a driver? The clocks you want to synchronize is two hardware clocks, but how to connect them with software? I mean how to affect another hardware clock? Write the registers of another hardware clock?

Best Regards,
Zhiming

[MatejI]

What I mean is to ask, if there is a way to get a custom clock (by custom I mean a software clock with 12288000 Hz frequency derived from one of the hardware clocks) to be synchronized via PTP.
For instance, I have two devices on the same network and I call PTP synchronization (via linuxptp package) to synchronize ethernet clocks of both devices. This way, ethernet clocks of both devices are synchronized. But I also want to have custom audio clock that is synchronized as well. How to do this?

First I though I would create a new clock derived from audio_pll1 and synchronize it with ethernet clock but I encountered problems with clock driver configuration in imx8mp. Then I changed the strategy and tried to derived custom clock from ethernet clock itself. For instance, here is the snippet of the device tree:

&clk {
enet_audio_clock: enet_audio_clock {
compatible = "fixed-clock";
clocks = <&clk IMX8MP_CLK_ENET_QOS_TIMER>; // PTP clock?
clock-frequency = <12288000>; // 12.288 MHz
#clock-cells = <0>;
};
};

/* Verdin SPI_1 */
&ecspi1 {
fsl,spi-num-chipselects = <1>;
status = "okay";
ak4621: audio-codec@0 {
compatible = "asahi-kasei,ak4621";
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_sai1>;
spi-max-frequency = <1000000>;
reg = <0>;
clocks = <&enet_audio_clock>;
#sound-dai-cells = <0>;
};
};

So here I have my custom audio codec and its clock refers directly to enet_audio_clock and enet_audio_clock is derived from  <&clk IMX8MP_CLK_ENET_QOS_TIMER> which if I understand correctly, is the PTP clock. Now if I synchronize ethernet clocks of two separate devices, is the custom codec gonna be synchronized as well?

Thank you.

Matej I.

[Zhiming_Liu]

 Hello,

Now I understand the architecture you are trying to achieve. From the hardware configuration, it is the pps signal from the Ethernet output that you can externally manipulate, have you connected the pps to the audio clock input? The Ethernet clock is fixed in there and can't be freely distributed as your software expresses. I have a couple questions.
1. you defined enet_audio_clock as a fixed rate clock, how do you implement dynamic modification of this clock? This is a fixed rate clock.
2. if you use pps signals to clock different rate clock signals, first you need to control the pps output to achieve the speed you want, then your slave device driver uses clk_set_rate() to correct the audio speed of the other board based on the clock offset of the userspace interface /sys/class/ptp/ptpX/.

Best Regards,
Zhiming

[MatejI]

Okay, thank you. Is there any other way how to make this setup work? For instance, is it possible to derive a new custom clock from ENET_QOS_TIMER_CLK_ROOT (<&clk IMX8MP_CLK_ENET_QOS_TIMER> in device tree) and feed this into audio codec? This way, the clock for audio codec would be PTP synchronized when ENET_QOS_TIMER_CLK_ROOT is synchronized. The problem is that ENET_QOS_TIMER_CLK_ROOT is 100 MHz, which is not divisble to 12288000 we need. Furthermore, I am not even sure this scenario could work.

Do you think the only way to achieve this is to make custom software clock with custom driver that is visible to phc2sys and could be synchronized as slave directly from ethernet ptp0 master? I tried to implement custom PTP clock driver but so far I was not successful.

Thank you.

Matej I.

[Zhiming_Liu]

Hello,

The problem is that ENET_QOS_TIMER_CLK_ROOT is 100 MHz, which is not divisble to 12288000 we need. 

-->Yes, this is a hardware limitation. What's your final application?

If you just want to synchronize the timestamps of the audio data stream with the PTP network, you can do that with ALSA's timestamp + PTP synchronization, it's not necessary to synchronize the physical clock of the codec.

If you do need precise audio hardware synchronization (TSN, AVB, etc. scenarios), the TSN or AVB has already be supported on i.MX8MP BSP.

Best Regards,
Zhiming

[MatejI]

My application is multiple devices in a same network capturing audio via microphones. What I need to achieve is that I need to make sure the recording samples are aligned (synchronized) as precisely as possible. That is why I want to use PTP to drive (or help with) audio codec's master clock so I know the audio samples I am recording are aligned.

"you can do that with ALSA's timestamp + PTP synchronization, it's not necessary to synchronize the physical clock of the codec."

--> You mean derive ALSA's timestamp from system clock that is synchronized via PTP with ethernet? Would that be precise enough?

"If you do need precise audio hardware synchronization (TSN, AVB, etc. scenarios), the TSN or AVB has already be supported on i.MX8MP BSP"

--> How exactly could I leverage this in my scenario?

Thank you for your time and information.

Matej I.

[Zhiming_Liu]

Hello,

You mean derive ALSA's timestamp from system clock that is synchronized via PTP with ethernet? Would that be precise enough?

-->If you need to record samples are aligned (synchronized) as precisely as possible, this is not perfect way. As the standard kernel is not real time kernel.

For the TSN/AVB, you can refer below pages.

Real time edge website:

https://www.nxp.com/design/design-center/software/development-software/real-time-edge-software:REALT...

Real time edge user guide:

https://www.nxp.com.cn/docs/en/user-guide/REALTIMEEDGEUG.pdf

Real time edge Yocto layer:

https://github.com/nxp-real-time-edge-sw/meta-real-time-edge

GenAVB/TSN Stack Evaluation User Guide[important]:

https://www.nxp.com/docs/en/user-guide/GENAVBTSNUG.pdf

Best Regards,
Zhiming

[MatejI]

Dear @Zhiming_Liu ,

Thank you, I will look at the documentations but I am not sure we would go this way with Mallow board V1.1. 

One more thing, maybe we could do it with custom modifications. Would it be possible to get a PTP-synchronized signal from one of the pins on IMX8MP? How to configure that on X1 pin 133 or 135 (SAI2_TXFS or SAI2_RXD0), which in ALT2 mode is ENET_QOS_1588_EVENT3_OUT and ENET_QOS_1588_EVENT2_OUT, we get a signal between 50 Hz and 30 MHz that is synchronized via PTP? Is PPS signal coming from ENET_QOS_1588_EVENTs?

Thank you.

Matej I.