NXP Tech Blog

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

NXP Tech Blog

gaurav_sharma
NXP Employee
NXP Employee

Hey everyone! This blog will cover the following: -

 

  1. System Overview
  2. Use-case
  3. What are Outbound and Inbound windows in PCIe and how do they work?
  4. What is ATU and why is it important in PCIe?
  5. How to configure the PCIe windows in LS1028 and iMX8QXP
  6. Code walkthrough
  7. Running the test case



System Overview

1.png

As depicted in the illustration above, the system has 2 main blocks: -

a. iMX8QXP [configured as PCIe Root Complex]
b. LS1028 [ configured as PCIe Endpoint]


Software components: -

iMX8QXP - Linux Factory 6.6.36
LS1028ARDB - LSDK-18.09

Hardware components: -

iMX8QXP MEK Board
LS1028ARDB Board
M.2 Key E PCIe Bridge


The root complex and endpoint are connected via a PCIe bridge on M.2 Key E connector of both the boards.
Reference clock used for both PCIe RC and EP - 100 MHz 




PCIe Bridge with M.2 Key E interfacePCIe Bridge with M.2 Key E interface

M.2 Key E PCIe Bridge


Use-case

 

There was a customer requirement wherein a software program was needed at Uboot to benchmark the PCIe to DDR transfers, involving cacheable and non-cacheable DRAM memory regions. 
 

The benchmark software periodically realizes several DMA transfers from PCIe space to DDR in the following way: -

 

1) Start PCIe to DDR DMA transfer (no descriptor). Data are transferred from PCIe to a non-cacheable buffer (A).

2) Wait for the DMA transfer to finish.

3) Copy received data from non-cacheable buffer (A) to a cacheable buffer(B).

 

4)  Check if there's a data mismatch and reports accordingly.


Note: -
There are 2 types of DMA transfers - Descriptor and Non-descriptor.
In Descriptor based DMA, the CPU sets up a list or chain of 'descriptors' in memory. Each descriptor is a data structure containing info such as source, destination, transfer size etc.

In Non-descriptor DMA, the CPU directly configures the DMA controller's registers to define parameters for a single, specific DMA transfer.

This use-case employs non-descriptor DMA transfers.

RC and EP will exercise the above use-case at Uboot itself. iMX8QXP by default acts as a RC. However, to configure LS1028 PCIe2 as EP, we have to configure the RCW[HOST_AGT_PEX] = 1 and then boot LS1028 with the modified RCW flashed onto the board.


To implement this use-case: -

On the iMX8QXP we will be adding our own functionality to the 'pci' utility by modifying the cmd/pci.c in the Uboot source code. After applying this patch, we can execute the test case at Uboot console by just executing this command - "pci p"

On LS1028A, we will simply be writing to the PCIe controller registers using the 'mw' command to set up the EP for the use-case.
 

What are Outbound and Inbound transactions in PCIe?

To explain the concept of outbound and inbound transactions, there are 2 entities that we need to keep in mind w.r.t the transactions that flow in the PCIe fabric:-

Initiator - component that initiates the transaction by sending a request. Example- Memory Read request sends the request for data.

Completer -  component that receives the request and eventually sends a response. Example - After receiving Memory Read request sent by the initiator to it, it sends the completion TLP containing the requested data back to the initiator.

 

3_memrd_write.png

 

Transactions in PCIe can be categorized into 2 parts, depending on the direction of data flow relative to a device [Root complex or End-point]  :-

Inbound - Data is coming into the device from the PCIe bus. From the device's perspective, the device is the completer of this request which was initiated by some other device. Example - Root complex[initiator] sending memory write TLP to an End-point[completer]. In this case EP will have inbound transactions coming from RC.

Outbound - Data is going out from the device to the PCIe bus. From the device's perspective, the device is the initiator of this request which will be completed by some other device. Example - End-point[Initiator] sending memory read TLP to a Root-complex[completer]. In this case, EP will have outbound transactions going towards RC.
 

What is ATU and why is it important in PCIe?

 

PCIe TLP transactions use PCIe addresses. These addresses are different from local internal bus addresses [like AXI, AHB] which are used in on-chip communication between CPU, memory and peripherals. So, we need an entity that maps the addresses from PCIe to local internal bus. Here comes the ATU: -

ATU is an Address Translation Unit that is responsible for address mapping between the device's address space and host's address space. It enables devices to access the host system's memory or other resources. Have a look at the following scenarios where ATU is needed: -

1. A PCIe device (such as network card, storage controller) needs to access the host's memory via DMA. So, the addresses that are issued by the device need to be translated via ATU so that the host can recognize and process these addresses.

 

2. A PCIe device wants to access a memory-mapped IO address. To do this, ATU is needed for translating device's requested address into an address that the host understands.

 

So, whenever a PCIe device initiates an access request like DMA memory read/write, the ATU translates the address issued by the device into the address of the host system as illustrated below: -

ATU TranslationATU Translation

 

Now that we know what Inbound/Outbound transactions are and how does ATU work in PCIe, we will now discuss how to enable the devices [RC/EP] to carry out these transactions. To achieve this, we configure Inbound and Outbound windows in the PCIe controller of a device by setting up ATU translation.
 

 

How to configure PCIe Outbound and Inbound windows in iMX8QXP & LS1028 respectively?



5_ib_ob.png

For our use-case, we will be configuring 1 outbound window on iMX8QXP [Root Complex] and 1 Inbound window on LS1028A[Endpoint].
 
a. To set up an Outbound window on iMX8QXP, we configure the following registers:-

iATU Index register
- defines which region is being accessed and its direction[Inbound/Outbound].


iATU Region Control 1 Register - To configure Max ATU Region size and other TLP attributes.


iATU Lower Base Address Register

                   &
iATU Upper Base Address Register


- These registers configure the start address of a window before the translation

 

iATU Limit Address Register - To configure the End-address of the window before the translation


iATU Lower Target Address Register
                    &

iATU Upper Target Address Register

- These registers configure the start and end addresses after the translation.
 

iATU Region Control 2 Register - To enable the REGION_EN bit and select match mode.

 

 

Outbound WindowOutbound Window

 

b. To set up an Inbound window on LS1028A, we configure the same set of registers :-

iATU Index Register
iATU Region Control 1 Register

iATU Lower Base Address Register

iATU Upper Base Address Register

iATU Limit Address Register

iATU Lower Target Address Register

iATU Upper Target Address Register

iATU Region Control 2 Register
 

Additionally, we will be configuring

PCI Express Command Register - to enable Bus master bit
PEX PF0 CONFIG Register - to set the Config Ready bit. Used by EP to indicate the RC that the controller has done its initialization.


 

The reason why we need to configure 2 additional registers on the EP side is that LS1028[EP] will be at Uboot console the whole time we are exercising our use-case. If it was in Linux, the PCIe drivers would have taken care of this.
 

 

Inbound WindowInbound Window

Code-walkthrough


There's a patch attached with this blog so that the readers can go through it and use it if required.

Note:-  In no way this claims itself to be a production level code, so readers are encouraged to only take it as a reference.
 

The Uboot patch achieves the following:-

 

  1. Configure MMU to make the DRAM range 80020000-88000000 non-cacheable. We are using the DRAM range 0x92000000-FE000000 as cacheable memory for our test case.  
  2. Creates an outbound window on iMX8QXP[connected to LS1028 End-point via PCIe bridge]. To complement this, an Inbound window will be configured on the LS1028.
  3. In a loop:-
    1. zeroize the cacheable and non-cacheable 1MB memory region.
    2. flushing the data cache for the cacheable 1MB region.
    3. configure and trigger DMA read to transfer 1MB of data from End-point’s DDR to RC’s non-cacheable memory region.
    4. copy 1MB data from non-cacheable to cacheable memory.
    5. compares the cacheable 1MB memory with the expected data. If mismatch occurs, we error out and break out of loop. Otherwise the   looping continues.


 

Executing ‘pci p’ command at U-boot console starts the above sequence.

 

After applying the patch in Uboot source, build the Uboot binary, then build SPL. Flash the SPL to iMX8QXP.

 

Running the test case


a. Connect RC [iMX8QXP] and EP[LS1028ARDB] via M.2 Key.E bridge, boot RC and EP till Uboot.


b. At Uboot console of LS1028, execute the following commands to configure the inbound window: -


 

mw.l 0x3500900 0x80000001

mw.l 0x3500904 0x0

mw.l 0x350090c 0x0

mw.l 0x3500910 0x0

mw.l 0x3500914 0x07ffff

 

mw.l 0x3500918 0xA0000000

mw.l 0x350091c 0x0

mw.l 0x3500908 0xC0000000

mw.b 0x3500004 0x06

mw.l  0x35C0014 0x00001001


The above registers are part of PCIe ATU configuration that was mentioned in the section -

How to configure PCIe Outbound and Inbound windows in iMX8QXP & LS1028 respectively

Please feel free to cross-check the Reference manual of LS1028A to verify the register addresses and their significance.
 

 c. On RC, execute 'pci p' -- this will trigger the DMA transfer use-case. 

 

This blog merely scratched the surface of PCIe inbound and outbound transactions. However it aimed at giving a raw view of how memory transfer can be triggered using DMA. For any queries, feel free to DM or send your questions in the comments section.

 

 

 

 

 



More
0 0 173
KosukeSogawa
NXP Employee
NXP Employee

NXP車載向けマイコンの機能安全対応について、通常の(機能安全非対応)マイコンと何が違うのか説明します。

(読了:5分)

Read more...

More
1 0 550
Keita_Nagashima
NXP Employee
NXP Employee

プロセス・テクノロジーの微細化に伴い、28nm以降の世代では、従来からあるFlashメモリをマイコンへ搭載することが難しくなり、各社次世代マイコンに対してFlashメモリに代わる不揮発性メモリ(NVM)を検討しています。その中のひとつがMRAMです。

本記事では、MRAMの特徴を説明します。

(読了:5分)

Read more...

More
1 0 776
YuseiUegama
NXP Employee
NXP Employee

組み込みデバイスの限られたリソースの中でLLMを使用するためには、RAGが必要となります。

本稿では、NXPの LLM + RAG ソリューション 「eIQ GenAI Flow」についてハンズオン形式で紹介します。

Read more...

More
2 0 2,094
Keita_Nagashima
NXP Employee
NXP Employee

eIQ Time Series Studio(時系列モデル開発ツール)の概要から、プロジェクトの作成 → データセット → トレーニング → エミュレーション → デプロイメント → データロギングと、使い方の流れを記載しています。

本記事は、eIQ Time Series Studio v1.0.4を起動させた際に表示される"Documentation"メニューに記載されている資料を翻訳したものです。必ず最新の英語版をご確認いただき、本記事はあくまでも補足的な内容としてご利用ください。

Read more...

More
1 0 3,838
Keita_Nagashima
NXP Employee
NXP Employee

CANバス/プロトコルの概要から特長について説明します。

NXPは古くからCANに携わっており、マイコン、プロセッサ、トランシーバと様々なCAN製品を提供しています。

Read more...

More
1 0 2,406
YuseiUegama
NXP Employee
NXP Employee

組込み機器へのAI/ML導入における課題に対して、NXPのAI/MLソリューション eIQを活用するメリットをご紹介します

-LLM+RAGソリューションをリリース予定

-時系列AI、画像処理AIモデルを開発

-Convert機能により、既存のAI/ML資産をスムーズに移行

Read more...

More
3 0 2,101