Hello to you all! In this blog, we will focus on low power management in PCIe. PCIe Power management aims at reducing power consumption in PCIe devices by putting them to low power states when they are idle and bringing them back to full power state when required.

Why does it matter ?  Because PCIe devices in the market such as graphic cards, storage devices, Network Interface Cards can consume significant power when they are active. Power management allows them to scale down their power consumption during idle or periods of low activity.

 The scope of PCIe power management is comparatively huge and the readers are encouraged to go through the PCIe specification document in case they are looking for every possible detail.

We are going to touch upon some important concepts that are just enough to get started. So this blog will stick to the following agenda:-
 

1. Introduction to PCIe power management
2. What is PCIe LTSSM ?
3. What are L1 sub-states and where do they fall inside the LTSSM?
4. How does a PCIe device enters and exit the ASPM L1 Sub-states?
5. How is ASPM exercised in linux kernel?
6. Quick look at the PCIe analyzer traces

Introduction to PCIe power management

In the grand scheme of things, PCIe power management is exercised at 2 levels:-
 
    a.  Link

Link power management conserves power on the serial link within a PCIe fabric when there's no active data transfer. This can happen even when the device is fully powered on. ASPM[Active State Power Management] is responsible for managing the power consumption of this link. Link States such as L0,L0s[Standby], L1, L1ss[Sub-states], L2 and L3 fall under this category.

LTSSM

    b. Device

Device power management aims at putting the entire device into low-power state when not in use. PCI-PM is responsible for managing the power states of the entire device. States such as D0,D1,D2.D3Hot and D3Cold fall under this category. The action of changing the device state indirectly causes a change in the Link state. 

 

This blog will focus on the first category i.e Link Power Management using ASPM. We will take up PCI-PM in another blog.

What is PCIe LTSSM?

LTSSM acronym for Link Training and Status State Machine, is a finite state machine within  the PCIe physical layer that ensures a link between two devices[Root Complex and Endpoint] is properly established and maintained.

It is responsible for managing the link initialization, link configuration, link recovery and power states among other things. Since it is a FSM, it operates through some defined states. It transitions between these states based on various conditions.

Some of the major states in the LTSSM:

• Detect

• The device checks for the presence of another PCIe device.

• Polling

• Devices exchange training sequences to align link parameters.

• Configuration

• Link width and speed are negotiated.

• L0 (Active State)

• Normal data transmission occurs.

• L0s / L1 / L2 (Low Power States)

• Power-saving modes when the link is idle or in standby.

• Recovery

• Used to recover from errors or renegotiate link parameters.

• Disabled / Hot Reset / Loopback

• Special states for testing, error handling, or reset.

The arrow in the above illustration denotes that one state can be transitioned to another under certain conditions. Please do not stress too much about every state in the diagram as this blog will only touch upon the ones that are needed to understand the L1 sub-states.
 

What are L1 sub-states and where do they fall inside the LTSSM?

To give you some context, L0 > L0s > L1 > L2 > L3 is the sequence of link states from the one that consumes most power[L0] to the one that consumes the least[L3]. It also means that the time to resume from low-power to active state increases in the reverse order. Meaning that the time taken by a PCIe device to go from L3 to L0 is greater than L1 to L0 transition.
In the previous section, we had mentioned the following link states:-

L0 - Active state in which data transmission happens.

L0s [Standby] - Link logically remains at L0 but physical signaling is reduced. No data transfer occurs in this state. Just minimal activity to keep the link going.
 

L1 - State providing greater power savings than L0s but with a cost of greater recovery latency. Also known as L1.0, It results in the deactivation of 'Link and Transaction' layer within each device.

To further reduce link power consumption, optionally L1 can have sub-states - L1.1 and L1.2. This also means that the recovery time latency increases from these low-power states to the active state.

L2 - In this state power can be aggressively conserved. Most of Transmitter and Receiver may be shut off. Main power and clock are not guaranteed but Auxiliary power is available. L2 support is dependent on the availability of Aux power on the board.
Note:- iMX95EVK does not support L2 state as it does not have an Auxiliary power.

L3 - When no power is present, the component is said to be in L3 state. This means that the link and device are completely inactive as there's no power.

Let's try to dig deeper into the L1 Sub-states of PCIe:-

Below is a state diagram that depicts the transitions into/out of L1 sub-states. It starts with L1.0 which is same as L1 explained previously.  This also shows that in order to Enter L1.1/L1.2, the link state should go to L1.0 first and then these states may be entered. The link does not transition directly from L1.1 to L1.2 without entering L1.0 first.


 

           L1.1

• In the L1.1 link state, the PLL,RX/TX circuits are OFF but Link common mode voltages are maintained to allow faster recovery from low-power state.
• Power savings are moderate.
• A bi-directional open-drain clock request signal i.e. CLKREQ# is used for entering and exit from this state.
• The Upstream and Downstream ports are not required to be enabled to detect Electrical Idle.
  
  
  L1.2
   
• In the L1.2 link state, the PLL,RX/TX circuits are OFF and Link common mode voltages are not required to be maintained.
• Power savings are more than L1.1 but higher exit latency than L1.1
• A bi-directional open-drain clock request signal i.e. CLKREQ# is used for entering and exit from this state.
• The Upstream and Downstream ports are not required to be enabled to detect Electrical Idle.

From the above discussion on L1.1 and L1.2, it is important to understand the role of CLKREQ# signal in ASPM. In L1.2 for example, the reference clock is cut-off, so in order for the device to request the clock back, this signal is used. When the device wants to exit the L1.2 low power state, It asserts the CLKREQ# low.

Because CLKREQ# is a bi-directional open-drain signal, both host and device can potentially drive this signal.

How does a PCIe device enters and exit the ASPM L1 Sub-states?

The discussion in this section will assume that we have the below setup:-

Root-Complex                              End-point        
iMX95EVK           <-------->      AW693 Wi-Fi module

                               M.2 Key.E slot

BSP used -  6.6.52 Linux Factory
 
The PCIe device [if supported] should advertise that it supports ASPM L1.1 or L1.2 by setting ASPM L1.1 and ASPM L1.2 bits of L1 PM Substates Extended Capability Register. This setting of bits must be performed at the downstream port before it is done at the upstream port.

 

To determine if the PCIe device has ASPM enabled

The Link capability register specifies a device's support for ASPM. Moreover, software can enable and disable ASPM via 'Active State PM Control' field of the 'Link Control Register'. This is what Linux PCI drivers use to enable/disable ASPM.

On linux, you can execute "lspci -vvv" and check the output for your device. Example :-

 

In the snippet above, we observe that ASPM is enabled for this device.



Transition to low-power states

L0 is the active link state where data transfer happens. After the link has been trained and established, it is said to be in L0.  The link enters and exits the L1 from/to L0 state.
The specification suggests that the condition for L0 to enter L1 is that both directions of the link have entered L0s and have been in this state for a preset amount of time. 

After reaching L1.0, the controller is capable of automatically moving to deeper power-saving L1 sub-states i.e L1.1 and L1.2 can be entered. This happens in response to the de-assertion of the CLKREQ# signal, which indicates that the reference clock should be shut off.

To initiate or indicate transitions in these power management states, the DLLP layer send/receive Power Management messages. As part of the protocol, these messages are exchanged between Root-Complex and End-point to ensure that both the link partners are prepared for the transition. At the end of the message exchanges, the PCIe controller moves to a state machine[L1.0 or L2/L3] in which clock can be cut off.

We can understand this using the following scenario: -

*** Consider a message exchange required to Enter L1 ASPM Sub-states from L0 ***

The downstream component sends PM_Active_State_Request_L1 DLLP repeatedly to the Upstream component. After receiving this, the upstream component can either accept the request by sending PM_Request_Ack DLLP or reject by sending PM_Active_State_Nak TLP

Upon getting PM_Request_Ack DLLP, the downstream port stops sending the request DLLP and disables the DLLP & TLP transmission, places its transmit lanes into Electrical Idle state.
 

When the upstream component receives an Electrical Idle ordered set on its Receive Lanes, it stops sending the PM Request Ack DLLP, disables DLLP & TLP transmission, placing its transmit lanes into Electrical Idle state.

At this point, the link is in L1 state.

Further, if the PCIe device supports L1 sub-states and it is advertising the same in its link capabilities register, it can de-assert CLKREQ# signal to indicate that it does not need clock. The system then turns off the clock. The link goes to L1 substates.
 

How is ASPM exercised in linux kernel?

To configure PCIe ASPM in linux kernel, ensure that the following CONFIGS are enabled in the defconfig:-

CONFIG_PCIEASPM=y
CONFIG_PCIEASPM_POWER_SUPERSAVE=y

Build the kernel and then deploy it on the board. 

At linux console, execute the following :-

echo "powersupersave" > /sys/module/pcie_aspm/parameters/policy

Writing to the above file instructs the kernel to change the ASPM policy to the one with most aggressive power-saving. That's all you need to do in the software. The rest will be taken care by the kernel driver and the pcie controller hardware state machine. That means if both the link partner supports L1 ASPM and there are no pending transactions, the link will automatically go to L1 sub-states.


Please have a look at the L1 Substates Entry and Exit flows below to understand what happens inside the linux kernel after enabling ASPM :-
 

-- L0 to L1 Sub-states Entry

-- L1 Sub-states to L0 Exit

The Exit from L1ss to L0 can be initiated by either Upstream or Downstream port. We have shown below how downstream port initiates the exit: -

Quick look at the PCIe analyzer traces

In this section, we will have a quick look at the PCIe analyzer traces to give you a real sense of PCIe world. These traces were captured via Teledyne Lecroy PCIe Analyzer. It has proven to be extremely useful for the TLP and DLLP level system troubleshooting/debugging. These traces help us to verify if the link indeed goes to L1 sub-states.

-- LTSSM Flow depicting L0 state at packet 1204

-- LTSSM Flow depicting L1 state at packet 1231

-- L0 state depicting memory TLP transactions on the analyzer

-- L1 State depicted at packet 1231

-- PM Active state request sent from Downstream to upstream

-- CLKREQ Deasserted for L1.1/L1.2 at packet 2822

Hey everyone! This blog will cover the following: -

1. System Overview
2. Use-case
3. What are Outbound and Inbound windows in PCIe and how do they work?
4. What is ATU and why is it important in PCIe?
5. How to configure the PCIe windows in LS1028 and iMX8QXP
6. Code walkthrough
7. Running the test case
   
   
   
   

System Overview

As depicted in the illustration above, the system has 2 main blocks: -

a. iMX8QXP [configured as PCIe Root Complex]
b. LS1028 [ configured as PCIe Endpoint]

Software components: -

iMX8QXP - Linux Factory 6.6.36
LS1028ARDB - LSDK-18.09

Hardware components: -

iMX8QXP MEK Board
LS1028ARDB Board
M.2 Key E PCIe Bridge

The root complex and endpoint are connected via a PCIe bridge on M.2 Key E connector of both the boards.
Reference clock used for both PCIe RC and EP - 100 MHz 

PCIe Bridge with M.2 Key E interface

M.2 Key E PCIe Bridge

Use-case

There was a customer requirement wherein a software program was needed at Uboot to benchmark the PCIe to DDR transfers, involving cacheable and non-cacheable DRAM memory regions. 
 

The benchmark software periodically realizes several DMA transfers from PCIe space to DDR in the following way: -

 

1) Start PCIe to DDR DMA transfer (no descriptor). Data are transferred from PCIe to a non-cacheable buffer (A).

2) Wait for the DMA transfer to finish.

3) Copy received data from non-cacheable buffer (A) to a cacheable buffer(B).

4)  Check if there's a data mismatch and reports accordingly.

Note: -
There are 2 types of DMA transfers - Descriptor and Non-descriptor.

In Descriptor based DMA, the CPU sets up a list or chain of 'descriptors' in memory. Each descriptor is a data structure containing info such as source, destination, transfer size etc.

In Non-descriptor DMA, the CPU directly configures the DMA controller's registers to define parameters for a single, specific DMA transfer.

This use-case employs non-descriptor DMA transfers.

RC and EP will exercise the above use-case at Uboot itself. iMX8QXP by default acts as a RC. However, to configure LS1028 PCIe2 as EP, we have to configure the RCW[HOST_AGT_PEX] = 1 and then boot LS1028 with the modified RCW flashed onto the board.

To implement this use-case: -

On the iMX8QXP we will be adding our own functionality to the 'pci' utility by modifying the cmd/pci.c in the Uboot source code. After applying this patch, we can execute the test case at Uboot console by just executing this command - "pci p"

On LS1028A, we will simply be writing to the PCIe controller registers using the 'mw' command to set up the EP for the use-case.
 

What are Outbound and Inbound transactions in PCIe?

To explain the concept of outbound and inbound transactions, there are 2 entities that we need to keep in mind w.r.t the transactions that flow in the PCIe fabric:-

Initiator - component that initiates the transaction by sending a request. Example- Memory Read request sends the request for data.

Completer -  component that receives the request and eventually sends a response. Example - After receiving Memory Read request sent by the initiator to it, it sends the completion TLP containing the requested data back to the initiator.

Transactions in PCIe can be categorized into 2 parts, depending on the direction of data flow relative to a device [Root complex or End-point]  :-

Inbound - Data is coming into the device from the PCIe bus. From the device's perspective, the device is the completer of this request which was initiated by some other device. Example - Root complex[initiator] sending memory write TLP to an End-point[completer]. In this case EP will have inbound transactions coming from RC.

Outbound - Data is going out from the device to the PCIe bus. From the device's perspective, the device is the initiator of this request which will be completed by some other device. Example - End-point[Initiator] sending memory read TLP to a Root-complex[completer]. In this case, EP will have outbound transactions going towards RC.
 

What is ATU and why is it important in PCIe?

PCIe TLP transactions use PCIe addresses. These addresses are different from local internal bus addresses [like AXI, AHB] which are used in on-chip communication between CPU, memory and peripherals. So, we need an entity that maps the addresses from PCIe to local internal bus. Here comes the ATU: -

ATU is an Address Translation Unit that is responsible for address mapping between the device's address space and host's address space. It enables devices to access the host system's memory or other resources. Have a look at the following scenarios where ATU is needed: -

1. A PCIe device (such as network card, storage controller) needs to access the host's memory via DMA. So, the addresses that are issued by the device need to be translated via ATU so that the host can recognize and process these addresses.

2. A PCIe device wants to access a memory-mapped IO address. To do this, ATU is needed for translating device's requested address into an address that the host understands.

So, whenever a PCIe device initiates an access request like DMA memory read/write, the ATU translates the address issued by the device into the address of the host system as illustrated below: -

ATU Translation

Now that we know what Inbound/Outbound transactions are and how does ATU work in PCIe, we will now discuss how to enable the devices [RC/EP] to carry out these transactions. To achieve this, we configure Inbound and Outbound windows in the PCIe controller of a device by setting up ATU translation.
 

How to configure PCIe Outbound and Inbound windows in iMX8QXP & LS1028 respectively?

For our use-case, we will be configuring 1 outbound window on iMX8QXP [Root Complex] and 1 Inbound window on LS1028A[Endpoint].
 
a. To set up an Outbound window on iMX8QXP, we configure the following registers:-

iATU Index register - defines which region is being accessed and its direction[Inbound/Outbound].

iATU Region Control 1 Register - To configure Max ATU Region size and other TLP attributes.

iATU Lower Base Address Register

                   &
iATU Upper Base Address Register

- These registers configure the start address of a window before the translation

iATU Limit Address Register - To configure the End-address of the window before the translation

iATU Lower Target Address Register
                    &

iATU Upper Target Address Register

- These registers configure the start and end addresses after the translation.
 

iATU Region Control 2 Register - To enable the REGION_EN bit and select match mode.

Outbound Window

b. To set up an Inbound window on LS1028A, we configure the same set of registers :-

iATU Index Register
iATU Region Control 1 Register

iATU Lower Base Address Register

iATU Upper Base Address Register

iATU Limit Address Register

iATU Lower Target Address Register

iATU Upper Target Address Register

iATU Region Control 2 Register
 

Additionally, we will be configuring

PCI Express Command Register - to enable Bus master bit
PEX PF0 CONFIG Register - to set the Config Ready bit. Used by EP to indicate the RC that the controller has done its initialization.

The reason why we need to configure 2 additional registers on the EP side is that LS1028[EP] will be at Uboot console the whole time we are exercising our use-case. If it was in Linux, the PCIe drivers would have taken care of this.
 

Inbound Window

Code-walkthrough

There's a patch attached with this blog so that the readers can go through it and use it if required.

Note:-  In no way this claims itself to be a production level code, so readers are encouraged to only take it as a reference.
 

The Uboot patch achieves the following:-

1. Configure MMU to make the DRAM range 80020000-88000000 non-cacheable. We are using the DRAM range 0x92000000-FE000000 as cacheable memory for our test case.  
2. Creates an outbound window on iMX8QXP[connected to LS1028 End-point via PCIe bridge]. To complement this, an Inbound window will be configured on the LS1028.
3. In a loop:-
   1. zeroize the cacheable and non-cacheable 1MB memory region.
   2. flushing the data cache for the cacheable 1MB region.
   3. configure and trigger DMA read to transfer 1MB of data from End-point’s DDR to RC’s non-cacheable memory region.
   4. copy 1MB data from non-cacheable to cacheable memory.
   5. compares the cacheable 1MB memory with the expected data. If mismatch occurs, we error out and break out of loop. Otherwise the   looping continues.

Executing ‘pci p’ command at U-boot console starts the above sequence.

After applying the patch in Uboot source, build the Uboot binary, then build SPL. Flash the SPL to iMX8QXP.

Running the test case

a. Connect RC [iMX8QXP] and EP[LS1028ARDB] via M.2 Key.E bridge, boot RC and EP till Uboot.

b. At Uboot console of LS1028, execute the following commands to configure the inbound window: -

 

mw.l 0x3500900 0x80000001

mw.l 0x3500904 0x0

mw.l 0x350090c 0x0

mw.l 0x3500910 0x0

mw.l 0x3500914 0x07ffff

mw.l 0x3500918 0xA0000000

mw.l 0x350091c 0x0

mw.l 0x3500908 0xC0000000

mw.b 0x3500004 0x06

mw.l  0x35C0014 0x00001001

The above registers are part of PCIe ATU configuration that was mentioned in the section -

How to configure PCIe Outbound and Inbound windows in iMX8QXP & LS1028 respectively

Please feel free to cross-check the Reference manual of LS1028A to verify the register addresses and their significance.
 

 c. On RC, execute 'pci p' -- this will trigger the DMA transfer use-case. 

This blog merely scratched the surface of PCIe inbound and outbound transactions. However it aimed at giving a raw view of how memory transfer can be triggered using DMA. For any queries, feel free to DM or send your questions in the comments section.

NXP車載向けマイコンの機能安全対応について、通常の(機能安全非対応)マイコンと何が違うのか説明します。

(読了：5分)

プロセス・テクノロジーの微細化に伴い、28nm以降の世代では、従来からあるFlashメモリをマイコンへ搭載することが難しくなり、各社次世代マイコンに対してFlashメモリに代わる不揮発性メモリ（NVM）を検討しています。その中のひとつがMRAMです。

本記事では、MRAMの特徴を説明します。

(読了：5分)

組み込みデバイスの限られたリソースの中でLLMを使用するためには、RAGが必要となります。

本稿では、NXPの LLM ＋ RAG ソリューション 「eIQ GenAI Flow」についてハンズオン形式で紹介します。

eIQ Time Series Studio(時系列モデル開発ツール)の概要から、プロジェクトの作成 → データセット → トレーニング → エミュレーション → デプロイメント → データロギングと、使い方の流れを記載しています。

本記事は、eIQ Time Series Studio v1.0.4を起動させた際に表示される"Documentation"メニューに記載されている資料を翻訳したものです。必ず最新の英語版をご確認いただき、本記事はあくまでも補足的な内容としてご利用ください。

CANバス/プロトコルの概要から特長について説明します。

NXPは古くからCANに携わっており、マイコン、プロセッサ、トランシーバと様々なCAN製品を提供しています。

組込み機器へのAI/ML導入における課題に対して、NXPのAI/MLソリューション eIQを活用するメリットをご紹介します

-LLM+RAGソリューションをリリース予定

-時系列AI、画像処理AIモデルを開発

-Convert機能により、既存のAI/ML資産をスムーズに移行