AN12149 IEEE 1588 Sync Error

[steven_keeter]

Hello,

I have two i.MXRT1052 EVK boards and am running the firmware from AN12149.  I have the hardware configured just like the app note.  The PPS signals are on a scope and the offset is about 65nS and dithering approximately 50nS.  I have not edited any of the AN configured constants.  So, why do you think the AN12149 has a delta t of about 11nS and mine is a lot larger?  What would you recommend adjusting?  I'm still learning the finer details of the note but hoped NXP could offer suggestions quicker.

Thank you.

[xabiermarquiegu]

Hello everyone,

I would like to test AN12149. Could any of you please, share the source code, or point me to where I can find it?

Thank you very much.

[steven_keeter]

Another bit of information: 

I increased the #define ENET_1588TIME_DELAY_COUNT 600U  to #define ENET_1588TIME_DELAY_COUNT 6000U, to see if longer packet timing captures would help avoid big jumps due to a few long packet times, and cause the PTP state machine to leave the fine correction and jump back to coarse corrections.  So far this change seems to help.  I am logging a test over the weekend to see if the large jumps are gone.  BTW, the Grandmaster is connected to a switch with several devices attached.  Comments about this constant counter adjustment and network stability would be appreciated.

[jimmychan]

The AN12149 IEEE 1588 demo is based on the PTPd with some features tailoring, only supports two devices connection directly. Please don't use router or switch to connect devices.

[steven_keeter]

Hi Jimmy,  That's not very helpful advice.  I have tracked the issue down to two things.  One is that the constant called Nibble transport should have been set to 0x00 instead of 0x80.  You can read about this here

https://sourceforge.net/p/ptpd/bugs/25/

The second issue is more serious, and it is related to the delay response timestamp not getting received or converted correctly, and causing a too large of delay and therefore no adjustment happens.  The error is either in the PTPd or with Spectracom Securesync formatting.  I will find out shortly.  BTW, if the PTPd modifications made were so limiting to actual use then AN12149 should have pointed this out.  In general the code is working well with the Securesync now, but it is not compensating for the switch delay in the master to slave direction.  This is causing a 13uS offset, even though the PTPd thinks it is sync'd to +- 0.1uS.  I am close to resolving and can let you know the cause.

Thanks,

-Steve

[steven_keeter]

Hi Jimmy,  I have found and corrected the PTPd stack errors.  The two fixes are as follows:

1.  The Nibble transport 0x80; needed to change to Nibble transport 0x00;  This avoids getting 0 for nanoseconds from master, as the master thinks a transparent device is going to add it.

2.  Within the servo.c file mean path delay was only using the received timestamp from the buffer without converting from TAI time to local UTC time.  Basically, the servo function had master to slave delay errors from not correcting for leap seconds using the utc offset from the master.  Because of this the calculated delay ( M2S ) was considered too large and it just aborted any further adjustment.  This is why adding delays from switches in between was not getting accounted for.

[jimmychan]

Hi Steven,

Thanks for your finding. Howard is the owner of AN12149. For further investigation, would you mind sharing the servo.c with us?

Thanks and have a nice day.

Best regards,

Jimmy

[steven_keeter]

Hello,  I changed from servo.c file to lower level protocol.c to handle here instead.

1.  Within protocol.c (handleDelayResp function ):  Add the following code before timestamps assigned.

// Adjust for UTC offset
TimeInternal TAI_Offset;

if(ptpClock->currentUtcOffsetValid)
      TAI_Offset.seconds = ptpClock->currentUtcOffset;

subTime(&requestReceiptTimestamp,&requestReceiptTimestamp,&TAI_Offset);

ptpClock->delay_req_receive_time.seconds =
requestReceiptTimestamp.seconds;
ptpClock->delay_req_receive_time.nanoseconds =
requestReceiptTimestamp.nanoseconds;

2.  Within msg.c add the following code to properly handle transport flag for unicast or multicast messages:

/*Pack header message into OUT buffer of ptpClock*/
void  msgPackHeader(Octet * buf, PtpClock * ptpClock)
{
Nibble transport = 0x00;

if (ptpClock->netPath.unicastAddr)
transport = 0x80;

/* (spec annex D) */
*(UInteger8 *) (buf + 0) = transport;
*(UInteger4 *) (buf + 1) = ptpClock->versionNumber;
*(UInteger8 *) (buf + 4) = ptpClock->domainNumber;

[steven_keeter]

BTW, The code changes above fix the multicast mode only.  Unicast and minicast (hybrid) modes need to be similarly evaluated and corrected as well.  I have not done that yet.  If any of you get these two modes fixed please share.

Thanks,

-Steve

[howard_liu]

Hi Steve,

Thanks for your sharing. We'll test it. As for the further fix for other modes, we'll do it later if possible. I have no time to do it at the present. The demo uses the ptpd Ver2.2.2 source code. Actually it's better to upgrade to the latest pdpd source code.

Regards,

Howard

[steven_keeter]

Hi Howard,

I have been looking at the latest PTPd version and the changes seem substantial.  I'm not sure what is the best method to try to update without massive efforts.  Do you have a good strategy  you can share to avoid dealing too much with massive reconstruction to cleanup compiler errors?  I was trying to update the hybrid mode to use the master IP address for the delay requests but it always uses the multicast address and the master will not respond to this when in hybrid mode.  Can you take a look at this to see why this request is not working?

/*Pack and send on event multicast ip adress a DelayReq message*/
void
issueDelayReq(RunTimeOpts *rtOpts,PtpClock *ptpClock)
{
Timestamp originTimestamp;
TimeInternal internalTime;

DBG("==> Issue DelayReq (%d)\n", ptpClock->sentDelayReqSequenceId );

/* call GTOD. This time is later replaced on handle_delayreq, to get the actual send timestamp from the OS */
getTime(&internalTime);
fromInternalTime(&internalTime,&originTimestamp);

// uses current sentDelayReqSequenceId
msgPackDelayReq(ptpClock->msgObuf,&originTimestamp,ptpClock);

Integer32 dst = 0;
#ifdef PTP_EXPERIMENTAL
if (rtOpts->do_hybrid_mode) {
dst = ptpClock->MasterAddr;
}
#endif

[howard_liu]

Hi steven.keeter@te.com,

Can you share the modification for servo.c file with us? Thanks.

Regards,

Howard

[Cozmo]

Hi Howard, is there any code for AN12149 with MCUXpresso IDE v11.8.0 and the latest SDK (i.MXRT1050EVK)?
I wanted to test out PTP 1588 with two i.MX RT1050 EVK boards?
If you can not send buildable source code could you send a binary for AN12149?

Many thanks
Colin

[Cozmo]

I attach some demo binarys for the MX1050 EVK (nor spi flash) that are the compiled application note AN12149  for your delight. So about aprox Sync 80ns with a crossover cable and 150ns via a switch without PTP support ( one hop).

[steven_keeter]

Hello,

I have additional information about AN12149.  Using the exact i.MXRT1052 EVK as above and the code from AN12149, I have now taken the slave EVK into the lab and connected it to a Spectracom SECURESYNC PTPv2 Grand Master with a 1 PPS output.  When I compare the PPS output of the slave EVK to the Spectracom instrument the EVK signal is delayed by 37mS, even though the debug print points indicate the offset is less than 100nS.  When I use the slave EVK with the master EVK their PPS output match within 100nS.  Therefore, I believe there is a bug in the EVK source related to the timer2 event compare output IRQ.  Because the same firmware is used in the slave and master EVK it makes since that they would match, and that it hides this severe delay outputting the PPS signal.  Can the author of AN12149 or someone else check this and please provide the corrected code?  Additionally, I think the overflow on the nanosecond register is also suspect of not being handled properly, as after a little while the offset instantly jumps from less than 100nS to almost the full size of the 32-bit nanosecond register.  Please also comment and suggest how to correct this.  In summary, there appears to be two significant bugs within AN12149 code:  delayed PPS out by 37mS, and potentially nanosecond rollover ( or other causing severe loss of sync ). 

Thank you

[xabiermarquiegu]

I am no longer working on this, but it may be interesting to link related work being done for this hw platform on other SW projects:

https://github.com/zephyrproject-rtos/zephyr/pull/60855#discussion_r1281347749