Exporting Persistent Data on JN516x to Host

nguyenanhgiau10 · ‎08-25-2019

Hi all,

As I know, the ZigBee Control Bridge node by default uses the internal EEPROM to hold persisted data. This is about 4KByte in JN5169 device and can restrict network size.

To overcome this, we need to export the data persistence to the host device.

How do I enable this feature?

p/s: I am using JN-AN-1216-ZigBee-3-0-IoT-ControlBridge

Thank you!

jonathan_thomps · ‎03-02-2020

Just wanted to follow up on chrisgray‌'s extremely helpful comments from above - for anyone who happens upon this thread hoping to use this functionality, I've implemented it here: Implemented message types 0x02000-0x0208 and 0x8200-0x8208 by schrodingersket · Pull Request #281 · ...

You'll want to note that I changed a few of the messages and their formats to accommodate the API exposed by the JN-SW-4170 SDK API, so you'll need to account for that in your host code.

mario_castaneda · ‎08-26-2019

Hi Giau,

I am not sure what is the data that you need.

If you want to get the list of neighbouring nodes, you could send to the router a ZPS_eAplZdpMgmtLqiRequest, the coordinator will return the list of devices depending on the index that you send.

Also, you could create a serial command, a new case switch case calling the iAHI_ReadDataFromEEPROMsegment and send through UART.

Regards,

Mario

nguyenanhgiau10 · ‎08-26-2019

Hi mario_castaneda,

I see in the document JN-AN-1216-ZigBee-3-0-ControlBridge, section B.4. Exporting Persistent Data to Host.

"The ZigBee Control Bridge node by default uses the internal EEPROM to hold persisted data.
This is about 4Kbytes on a JN5169 device and can restrict network size. To overcome this it
is possible to export the data persistence to the host device. This requires a binary with this
feature turned “ON”."

How do I turn on this feature in firmware? And how do I export these data to host device (such as SoC or other MCU)?

Thank you so much,

mario_castaneda · ‎08-29-2019

Hi Giau,

Please look at the JN-AN-1223 and the pdm_host.c files will help you with the exporting persistent data to host.

"C:\NXP\bstudio_nxp\workspace\JN-AN-1223-ZigBee-IoT-Gateway-Control-Bridge-v1014\Source\ZigbeeNodeControlBridge\pdm_host.c"

Regards,

Mario

jonathan_thomps · ‎01-17-2020

Hi mario_castaneda,

I have a quick follow-up to this question, as I'm looking to implement the same feature.

You indicated that we should look to JN-AN-1223 for pdm_host.c - JN-AN-1223 is however the HA 1.2 Application Note; nguyenanhgiau1008@gmail.com‌ and I are working with JN-AN-1216, which is the Zigbee PRO 3.0 Application Note. Can the code in JN-AN-1223 that's related to exporting PDM data to the host OS simply be copied over to the code in JN-AN-1216 and be expected to work?

The primary reason I ask is that the SDK for JN-AN-1216 is JN-SW-4170 (I'm using the latest build: 1840), and the SDK for JN-AN-1223 is JN-SW-4168. The documentation for JN-AN-1216 (as quoted above by nguyenanhgiau1008@gmail.com‌ indicates that a build parameter can be set that allows JN-AN-1216 to export PDM data to the host. In my code, I've set the PDM_BUILD_TYPE variable to _NONE, which results in undefined reference errors with respect to PDM_* functions.

Is the JN-AN-1216 documentation in error with respect to the B.4. Exporting Persistent Data to Host section?

Thanks in advance,

-Jonathan T.

nguyenanhgiau10 · ‎02-01-2020

Hi chrisgray, jonathan.thompson@nimbusnine.co, mario_castaneda
I was save from node to host successful, but I'm facing a problem when load from host to node with size greater 128 byte (with size < 128, it load successful).

On host, I use sqlite3 for storing.

On node (JN5169), implement pdm_host.c from JN-AN-1223.

PUBLIC PDM_teStatus PDM_eReadDataFromRecord( uint16 u16IdValue,
void *pvDataBuffer,
uint16 u16DataBufferLength,
uint16 *pu16DataBytesRead)
{

/* send LoadRecordRequest to host */
  DBG_vPrintf(TRACE_PDM_HOST, "E_SL_MSG_LOAD_PDM_RECORD_REQUEST\n");
  uint8 au8Buffer[MAX_PACKET_SIZE], *pu8Buffer, *pdmBuffer;
  uint32 u32Length = 0;
  uint32 u32Size,u32TotalBlocks=0,u32BlockId=0,u32BlockSize=0;
  uint16 u16RecordId,u16MessageType;
  bool_t bNoMoreData = FALSE;

  PDM_tsRecordDescriptor *psDesc;
  psDesc->u16UserId = u16IdValue;
  psDesc->pu8Data = (uint8*)pvDataBuffer;
  psDesc->u32DataSize = u16DataBufferLength;
  psDesc->u16Id = PDM_INVALID_ID;
  /* set security level of the record */
  psDesc->eState = PDM_RECOVERY_STATE_NONE;

  if(!bCheckPdmRecordsList(psDesc))
  {
      psDesc->psNext = pHead;
      pHead = psDesc;
  }
  /* flush hardware buffer */
  UART_vOverrideInterrupt(FALSE);

  pu8Buffer = au8Buffer;
  memcpy(pu8Buffer, &u16IdValue,sizeof(uint16));
  u32Length = sizeof(uint16);
  pu8Buffer += sizeof(uint16);
  u16MessageType = E_SL_MSG_LOAD_PDM_RECORD_REQUEST;
  vUart_WriteMessage(u16MessageType, u32Length, au8Buffer);
  pdmBuffer = psDesc->pu8Data;
  do
  {
      u8QueueByte = 0xff;
      if(UART_bGetRxData((uint8*)&u8QueueByte) ){
      if(TRUE==bSL_ReadMessage(&u16PacketType,&u16PacketLength,MAX_PACKET_SIZE,au8LinkRxBuffer,u8QueueByte))
      {
          switch(u16PacketType)
          {
              case(E_SL_MSG_LOAD_PDM_RECORD_RESPONSE):
              {
              DBG_vPrintf(TRACE_PDM_HOST, "E_SL_MSG_LOAD_PDM_RECORD_RESPONSE\n");
              uint8 u8Status = au8LinkRxBuffer[0];
              memcpy(&u16RecordId, &au8LinkRxBuffer[1], sizeof(uint16));
              DBG_vPrintf(TRACE_PDM_HOST, "u16RecordId; %x\n", u16RecordId);
              memcpy(&u32Size,&au8LinkRxBuffer[3], sizeof(uint32));
              DBG_vPrintf(TRACE_PDM_HOST, "u32Size; %x\n", u32Size);
              memcpy(&u32TotalBlocks, &au8LinkRxBuffer[7], sizeof(uint32));
              DBG_vPrintf(TRACE_PDM_HOST, "u32TotalBlocks; %x\n", u32TotalBlocks);
              memcpy(&u32BlockId, &au8LinkRxBuffer[11],sizeof(uint32));
              DBG_vPrintf(TRACE_PDM_HOST, "u32BlockId; %x\n", u32BlockId);
              memcpy(&u32BlockSize, &au8LinkRxBuffer[15], sizeof(uint32));
              uint16 u16BlockSize = (uint16)u32BlockSize;
              DBG_vPrintf(TRACE_PDM_HOST, "u16BlockSize; %x\n", u16BlockSize);
              memcpy(pu16DataBytesRead, &u16BlockSize, sizeof(uint16)); /*save length data was read*/
              if((u8Status != PDM_RECOVERY_STATE_NONE) && (u16RecordId == psDesc->u16UserId))
              {
                  memcpy(pdmBuffer, &au8LinkRxBuffer[19], u16BlockSize);
                  /* show data received */
                  uint32 i;
                  DBG_vPrintf(TRACE_PDM_HOST, "Data: ");
                  for (i=0; i<u16BlockSize; i++) {
                      DBG_vPrintf(TRACE_PDM_HOST, "%02x", pdmBuffer[i]);
                  }
                  DBG_vPrintf(TRACE_PDM_HOST, "\n");

                  pdmBuffer += u16BlockSize;
                  if(u32TotalBlocks == u32BlockId)
                  {
                      DBG_vPrintf(TRACE_PDM_HOST, "PDM_RECOVERY_STATE_RECOVERED\n");
                      bNoMoreData = TRUE;
                      psDesc->eState = PDM_RECOVERY_STATE_RECOVERED;
                  }
              }
              if(u8Status == PDM_RECOVERY_STATE_NONE)
              {
                  DBG_vPrintf(TRACE_PDM_HOST, "PDM_RECOVERY_STATE_NONE\n");
                  bNoMoreData = TRUE;
                  psDesc->eState = PDM_RECOVERY_STATE_NONE;
              }

                 u8Status = 0;
                 memcpy(au8Buffer, &u8Status, sizeof(uint8));
                 memcpy(&au8Buffer[1], &u8Status, sizeof(uint8));
                 memcpy(&au8Buffer[2], &u16MessageType, sizeof(uint16));
                 /* show data received */
                 uint32 i;
                 DBG_vPrintf(TRACE_PDM_HOST, "Data: ");
                 for (i=0; i<4; i++) {
                     DBG_vPrintf(TRACE_PDM_HOST, "%02x", au8Buffer[i]);
                 }
                 vUart_WriteMessage(E_SL_MSG_STATUS, 4, au8Buffer);
                 DBG_vPrintf(TRACE_PDM_HOST, "\nNext block\n\n");

          }
          break;
          default:
          break;
          }
      }
  }
}
while(!bNoMoreData);

UART_vOverrideInterrupt(TRUE);
return PDM_E_STATUS_OK;
}

chrisgray · ‎01-27-2020

Hello,

I had an email conversation with Jonathan about this and I am updating this thread with the current status, in case anyone else comes across the problem.

As you’ve seen, the Zigbee HA control bridge (JN-AN-1223) implements the host PDM option but the Zigbee 3.0 version (JN-AN-1216) does not. JN-AN-1223 is actually an older version of the stack and components, despite the higher AN number, and the host PDM option was valuable because it was proposed to store Zigbee private keys in the PDM.

When JN-AN-1216/JN-SW-4170 were being developed, there were two changes that are of importance to this discussion:

Zigbee private keys were stored in flash instead of PDM
The PDM was simplified and made more efficient by replacing the descriptors (each record had a block of memory to contain state info) with IDs, which changed the PDM APIs as it was a new way of using the PDM

Hence the host PDM host option was no longer required for the standard release and, because the PDM APIs had changed, the existing pdm_port.c was no longer suitable. Rather than update it, the decision was made to leave it out of the release. Hence simply copying pdm_port.c from JN-AN-1223 to JN-AN-1216 won’t build: the stack and application are trying to call out to a different set of APIs. Unfortunately, it seems that the documentation was not updated with the correct information.

You can modify pdm_port.c to implement the functions needed by JN-AN-1216, instead of the functions needed by JN-AN-1223. These functions seem to be:
PDM_eCreateBitmap, PDM_eDeleteBitmap, PDM_eGetBitmap, PDM_eIncrementBitmap, PDM_eSaveRecordData, PDM_vDeleteAllDataRecords, PDM_eInitialise, PDM_eReadDataFromRecord, PDM_vDeleteDataRecord. The protoypes are in PDM.h and the functionality is described in https://www.nxp.com/docs/en/user-guide/JN-UG-3116.pdf. Starting from the existing pdm_port.c you can discard the descriptors completely, and the code reduces to building up the serial data streams to send commands for each API to the host, and reading the data streams coming back. You’ll have to implement the host side that this interacts with anyway, so the extra effort probably isn’t that great and the exact format of the UART message payloads can be entirely up to you.

Best regards,

Chris.

nguyenanhgiau10 · ‎09-17-2019

Hi Mario,

I'm using JN-AN-1216 application and export PDM to host.

Just the last step.

I added pdm_host.c file to project

in Makefile:

PDM_BUILD_TYPE = _NONE

ifeq ($(PDM_BUILD_TYPE), _NONE)
APPSRC += pdm_host.c
endif

---------------------------------

in app_start.c

vLog_Printf ( TRACE_APPSTART,LOG_DEBUG, "\n\nInitialising \n" );
#ifdef PDM_NONE
PDM_vWaitHost();
#endif

PRIVATE void vInitialiseApp ( void )
{

...

#ifdef PDM_EEPROM
PDM_eInitialise ( 63 );
#else
/* USING HOST TO SAVE PERSISTENT DATA */
#endif

...

----------------------------------

I have to define a few function to building get success (they are in pdm_host.c)

PDM_eSaveRecordData

PDM_eReadDataFromRecord

PDM_vDeleteDataRecord

PDM_vDeleteAllDataRecords

PDM_teStatus PDM_eGetBitmap

In fact, these function look like:

PUBLIC PDM_teStatus PDM_eSaveRecordData(
      uint16       u16IdValue,
      void          *pvDataBuffer,
      uint16       u16Datalength)
{

return PDM_E_STATUS_OK;
}

It is empty. What do I need to fill?

Thanks

mario_castaneda · ‎09-17-2019

Hi Giau,

This is part of the libraries that we provide.

For example, if you want to save the Group Table, the API will show like this.

    PDM_eSaveRecordData ( PDM_ID_APP_GROUP_TABLE,
                          &sGroupTable,
                          sizeof ( tsZllGroupInfoTable ) );‍‍‍

Regards,

Mario

nguyenanhgiau10 · ‎09-03-2019

Thanks Mario

Exporting Persistent Data on JN516x to Host

Exporting Persistent Data on JN516x to Host

JN

ZigBee Software