Issue with Jailhouse hypervisor Inmate cell Creation for i.MX8MQ

[sumiteca09]

Dear Sir,

Trying to create an custom (E-call) Inmate linux demo cell via Jailhouse in NXP i.MX 8MQ EVK Board.

All steps followed are added in attached 'NXP_Jailhouse_Ticket.txt' file.

Issue Facing:-
We are facing issue with compilation, 
how to compile the linux-inmate-demo.c file and generate .o and .cell file?

How to create proper .bb file for compiling linux-inmate-demo.c file and generate .o and .cell file and adding to final image.?

Please provide us any Application note you have for creating Custom Jailhouse inmate cell creation
Or
Any link to previous NXP community ticket having the same solution?

Also please correct us about above steps and provide solution.

[Bio_TICFSL]

Hello,

You can follow similar approach as NXP Harpoon Jailhouse integration trough a bbappend recipe.

See examples: meta-nxp-harpoon/recipes-extended/jailhouse/jailhouse-imx_%.bbappend at imx-linux-scarthgap · NXP/me...

User guide: https://www.nxp.com/docs/en/user-guide/Harpoon_USERS_GUIDE.pdf

Regards

[sumiteca09]

Hi Bio_TICFSL  

Thanks a lot for quick reply.

Since we are using i.MX NXP 8MQ EVK board 'i.MX 8MQuad', 
With Part Number "TDA6637".
So currently I have flashed Linux Yocto 5.15 Kirkstone '5.15.71-2.2.0'.

But in your reply for 'NXP Harpoon Jailhouse integration' we need 'i.MX Linux Yocto Project BSP 6.6.36-2.1.0 Harpoon v3 Release',.
For i.MX 8MQ board it support till 5.15 Kirkstone Image, and in your Harpoon User Guide 'UG10170' and 
Github link 'https://github.com/NXP/meta-nxp-harpoon/tree/imx-linux-scarthgap'

The following boards were tested in this release:

• NXP i.MX 8MMini LPDDR4 EVK (imx8mm-lpddr4-evk)
• NXP i.MX 8MNano LPDDR4 EVK (imx8mn-lpddr4-evk)
• NXP i.MX 8MPlus LPDDR4 EVK (imx8mp-lpddr4-evk)

No mention of i.MX 8MQ, can you provide a specific Jailhouse solution for our board, or suggest some other possibility to create inmate cell?

Thanks a lot for support.

Thanks,
Sumit

[Bio_TICFSL]

Hello,

There is no jailhouse for MX8MQ, but is the same family of the MX8M you can take as base the github or take the 6.6.36 BSP.

Regards

[sumiteca09]

Hi Bio_TICFSL,

In your github link, we found some link for 5.15 Kirkstone:

https://github.com/NXP/meta-nxp-harpoon/blob/imx-linux-kirkstone/recipes-extended/jailhouse/jailhous...

https://github.com/NXP/meta-nxp-harpoon/tree/imx-linux-kirkstone

Will it help for our NXP i.MX8MQ board?

Thanks,

Sumit

[Peng_Fan]

Let me give some tips:

1. Documentation/inter-cell-communication.md describes how the memory is arranged.

Each cell needs the follow regions for ivshmem:

- read-only region to hold state table, generally one page large
- one region that is read/writable for all peers
- one output region per peer that is only read-writeable for one of them

2. To i.MX8MQ

/* IVHSMEM shared memory region for 00:00.0 */ {
.phys_start = 0xbfdf0000,
.virt_start = 0xbfdf0000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ,    ===> This is state table.
},
{
.phys_start = 0xbfdf1000,
.virt_start = 0xbfdf1000,
.size = 0x9000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE ,     ===> This is RW section. You could use this for communication. Or enlarge this.

},
{
.phys_start = 0xbfdfa000,
.virt_start = 0xbfdfa000,
.size = 0x2000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE ,   ==> This is root input/output
},
{
.phys_start = 0xbfdfc000,
.virt_start = 0xbfdfc000,
.size = 0x2000,
.flags = JAILHOUSE_MEM_READ,           ==> This is root input
},
{
.phys_start = 0xbfdfe000,
.virt_start = 0xbfdfe000,
.size = 0x2000,
.flags = JAILHOUSE_MEM_READ,      ==> This is root input

This is similar layout in imx8mq-inmate-linux.c.

You could configure RW and INPUT/OUTPUT SECTION, do not touch State section.

For INPUT/OUTPUT, if you define multiple regions, each region should be same size.

Regards

Peng.

[sumiteca09]

Dear Peng Fan,

Thank you for your valuable guidance on IVSHMEM memory regions for Jailhouse on the i.MX 8MQ, as provided in your response regarding the state table, RW section, and input/output sections. We have incorporated your suggestions into our cell configuration, but we are still facing a persistent compilation error when trying to create a custom E-Call inmate cell using Yocto 5.15 Kirkstone (Linux 5.15.71-2.2.0) on the i.MX 8MQ EVK board. Below is a detailed summary of the issue, our attempts to resolve it, and the current status. We would greatly appreciate any specific references, application notes, or further guidance you can provide to help us overcome this issue.

Issue Summary
We are attempting to compile a custom E-Call inmate cell configuration (initially based on `imx8mq-linux-demo.c`, now using a simplified `ecall-inmate-linux-demo.c`) using the `jailhouse-cell-linux` tool. The compilation fails during the `do_compile` task of our `jailhouse-ecall.bb` recipe with the following error:
"
ERROR: jailhouse-ecall-1.0-r0 do_compile: ExecutionError('/home/automotive/yocto_imx_linux/build/tmp/work/armv8a-poky-linux/jailhouse-ecall/1.0-r0/temp/run.do_compile.225085', 1, None, None)
Log data follows:
DEBUG: Executing shell function do_compile
Not a cell configuration: /home/automotive/yocto_imx_linux/build/tmp/work/armv8a-poky-linux/jailhouse-ecall/1.0-r0/imx8mq-linux-demo.c
WARNING: exit code 1 from a shell command.
ERROR: Task (/home/automotive/yocto_imx_linux/sources/meta-ecall/recipes-jailhouse/jailhouse-cells/jailhouse-ecall.bb:do_compile) failed with exit code '1'
"

This error indicates that `jailhouse-cell-linux` does not recognize the cell configuration file as valid. Despite multiple attempts to resolve this, the issue persists.

Details of Attempts to Resolve
We have made several updates to our Yocto recipes, cell configuration, and environment based on community feedback and trial-and-error. Below is a summary of the key efforts:

1. Initial Setup and Error:
- Started with `jailhouse-ecall.bb` using `imx8mq-linux-demo.c` and `jailhouse-native.bb` based on the Siemens Jailhouse fork (`git://github.com/siemens/jailhouse.git;branch=master;SRCREV=e57d1eff`).
- Encountered errors related to missing `pyjailhouse` modules and incorrect `jailhouse-cell-linux` command syntax (e.g., misinterpreting `-I` as `CELLCONFIG`).

2. Recipe Updates:
- jailhouse-native.bb:
- Initially lacked header installation (`jailhouse/types.h`, `jailhouse/cell-config.h`). Updated to include:
"bitbake
install -d ${D}${includedir}/jailhouse
cp -r ${S}/include/jailhouse/*.h ${D}${includedir}/jailhouse/
"
- Switched to the NXP Jailhouse fork (`git://source.codeaurora.org/external/imx/imx-jailhouse.git;branch=imx_5.4.47_2.2.0;SRCREV=8bbe203`) to align with `jailhouse_0.12.bb` for i.MX 8MQ compatibility.
- Ensured `pyjailhouse` installation and `python3-native` dependency.
- jailhouse-ecall.bb:
- Corrected the 'jailhouse-cell-linux' command by removing an invalid '/dev/null' argument and using proper syntax:
"bitbake
${STAGING_BINDIR_NATIVE}/jailhouse-cell-linux \
${S}/ecall-inmate-linux-demo.c \
-o ${S}/ecall-inmate-linux-demo.cell
"
- Updated 'SRC_URI' to use `ecall-inmate-linux-demo.c` instead of `imx8mq-linux-demo.c` to focus on the E-Call use case.
- Added 'export C_INCLUDE_PATH' and 'PYTHONPATH' to ensure headers and Python modules are found.

3. Cell Configuration Updates:
- Simplified 'ecall-inmate-linux-demo.c' based on 'imx8mq-linux-demo.c' and your IVSHMEM guidance:
- State table: `0xbfdf0000`, 4 KB, read-only.
- RW section: `0xbfdf1000`, 64 KB, read/write.
- Root output: `0xbfdfa000`, 8 KB, read/write.
- Root input: `0xbfdfc000`, 8 KB, read-only.
- Included UART1 (`0x30860000`), RAM (`0xc0000000`, 256 MB), and communication region (`0x80000000`).
- Removed complex macros (e.g., `JAILHOUSE_SHMEM_NET_REGIONS`) to avoid parsing issues.
- Example configuration snippet:
".c
.mem_regions = {
/* IVSHMEM state table */
{
.phys_start = 0xbfdf0000,
.virt_start = 0xbfdf0000,
.size = 0x1000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_ROOTSHARED,
},
/* IVSHMEM RW section */
{
.phys_start = 0xbfdf1000,
.virt_start = 0xbfdf1000,
.size = 0x10000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,
},
/* IVSHMEM root output */
{
.phys_start = 0xbfdfa000,
.virt_start = 0xbfdfa000,
.size = 0x2000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_WRITE | JAILHOUSE_MEM_ROOTSHARED,
},
/* IVSHMEM root input */
{
.phys_start = 0xbfdfc000,
.virt_start = 0xbfdfc000,
.size = 0x2000,
.flags = JAILHOUSE_MEM_READ | JAILHOUSE_MEM_ROOTSHARED,
},
...
}
"

4. Dependency and Layer Configuration:
- Added `meta-ecall` and `meta-freescale` to `bblayers.conf`.
- Updated `local.conf`:
"conf
IMAGE_INSTALL:append = " jailhouse jailhouse-ecall python3"
DISTRO_FEATURES:append = " jailhouse virtualization"
"
- Verified `jailhouse_0.12.bb` uses the NXP fork for target deployment.
- Ensured `python3-native`, `pyjailhouse`, and headers are installed in the native sysroot (`${STAGING_DIR_NATIVE}/usr/share/jailhouse/tools/pyjailhouse/`, `${STAGING_DIR_NATIVE}/usr/include/jailhouse/`).

5. Manual Testing:
- Manually tested `jailhouse-cell-linux`:
"bash
export PYTHONPATH=/home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/share/jailhouse/tools
export C_INCLUDE_PATH=/home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/include/jailhouse
/home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/bin/jailhouse-cell-linux \
/home/automotive/yocto_imx_linux/build/tmp/work/armv8a-poky-linux/jailhouse-ecall/1.0-r0/ecall-inmate-linux-demo.c \
-o ecall-inmate-linux-demo.cell
"
- Same error: `Not a cell configuration`.
- Verified sysroot contents: `pyjailhouse` modules, headers, and `jailhouse-cell-linux` are present.

Current Status
- The error `Not a cell configuration` persists despite:
- Correcting `jailhouse-cell-linux` syntax.
- Simplifying the cell configuration.
- Aligning `jailhouse-native.bb` with the NXP fork.
- Ensuring headers and `pyjailhouse` are installed.
- Other recipes (`jailhouse-native`, `jailhouse`) compile successfully, indicating the issue is specific to `jailhouse-ecall.bb` and the cell configuration parsing.

Questions and Request for Guidance
1. Could the error be due to a version mismatch between the NXP Jailhouse fork (`imx_5.4.47_2.2.0`) and our cell configuration? Are there specific requirements for `jailhouse-cell-linux` in this fork?
2. Is there an application note or user guide for creating custom Jailhouse inmate cells on i.MX 8MQ with Yocto 5.15 Kirkstone? We couldn’t find detailed documentation for this platform.
3. Are there known issues with `jailhouse-cell-linux` parsing complex configurations (e.g., IVSHMEM regions)? Should we further simplify the configuration?
4. Can you provide a reference `jailhouse-ecall.bb` recipe or example for compiling a `.cell` file for i.MX 8MQ?
5. Are there specific U-Boot or kernel configurations required for Jailhouse on i.MX 8MQ that we might be missing?

6. How to resolve the Jailhouse dependencies issue, for custom recipes/bb layers?

Attached Files (for Reference)
- `jailhouse-ecall.bb`: Updated recipe for E-Call cell.
- `jailhouse-native.bb`: Updated to NXP fork with headers.
- `ecall-inmate-linux-demo.c`: Simplified configuration with IVSHMEM regions.
- `imx8mq-linux-demo.c`: Original configuration (for reference).
- Verbose compilation log (if needed, please confirm how to share).

We are eager to resolve this issue to proceed with deploying the E-Call inmate cell and testing shared memory communication. Any insights, references, or corrections to our approach would be greatly appreciated.

Thank you for your support!

Best regards,
Sumit

Note:-Let me know if you need help refining the response or generating the verbose log!

[Peng_Fan]

I have little knowledge on Yocto stuff. Just share my build command:

. /opt/fsl-imx-internal-xwayland/6.12-styhead/environment-setup-armv8a-poky-linux

make KDIR=$HOME/linux/linux-lts-nxp CC="aarch64-poky-linux-gcc --sysroot=$SDKTARGETSYSROOT" ARCH=arm64 CROSS_COMPILE=aarch64-poky-linux- V=1

Then the cells will be built out.

5.10 should be similar.

Please also share your compiled out cell file, let me have a look on why jailhouse-cell-linux report error on the cell.

[Peng_Fan]

BTW:

This is wrong

     "bash

     export PYTHONPATH=/home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/share/jailhouse/tools

     export C_INCLUDE_PATH=/home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/include/jailhouse

     /home/automotive/yocto_imx_linux/build/tmp/work/x86_64-linux/jailhouse-native/0.12-r0/image/usr/bin/jailhouse-cell-linux \

     /home/automotive/yocto_imx_linux/build/tmp/work/armv8a-poky-linux/jailhouse-ecall/1.0-r0/ecall-inmate-linux-demo.c \

     -o ecall-inmate-linux-demo.cell

cells are compiled out by GCC, not jailhouse-cell-linux.

[sumiteca09]

Dear PENG,

Thanks a lot your suggestions.
We are still facing issues, details are below:

Flow followed for Creating .cell File (Jailhouse Inmate Binary), SDK compilation commands:

Tool Chain Enviornment Setup:

 source /opt/fsl-imx-wayland/5.15-kirkstone/environment-setup-armv8a-poky-linux

PATH
bash: /opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/usr/bin:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/usr/sbin:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/bin:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/sbin:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/usr/bin/../x86_64-pokysdk-linux/bin:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux:/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/x86_64-pokysdk-linux/usr/bin/aarch64-poky-linux-musl:/home/automotive/.local/bin:/home/automotive/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin: No such file or directory

aarch64-poky-linux-gcc -c -I/home/automotive/jailhouse/hypervisor/include -I/home/automotive/jailhouse/hypervisor/arch/arm64/include -I/home/automotive/jailhouse/include -nostdlib -nostartfiles -ffreestanding  ecall-inmate-linux-demo.c -o ecall-inmate-linux-demo.o

NOTE:-Jailhouse header files are not part of tool-chain, so we separately downloaded it in Jailhouse folder from Seimens (github version). We used header file for compiling, inmate-linux-demo.cell. We didn't do any modification in .c file.
For testing purpose we used default one.
 
Step 1: Link the .o to produce a flat ELF

aarch64-poky-linux-ld \
  -Ttext=0x0 \
  -nostdlib \
  -o ecall-inmate-linux-demo.elf \
  ecall-inmate-linux-demo.o
-Ttext=0x0 puts the .text section at address 0x0 for flat binary layout.

-nostdlib prevents linking standard C library (inmates don't need it).

Step 2: Convert ELF to raw binary .cell

aarch64-poky-linux-objcopy \
  -O binary \
  ecall-inmate-linux-demo.elf \
  ecall-inmate-linux-demo.cell

Verify the result:

file ecall-inmate-linux-demo.cell

Printed message:

ecall-inmate-linux-demo.cell: data

Removed the default imx8mq.cell and imx8mq-linux-demo.cell

Generated 'imx8mq.cell and imx8mq-linux-demo.cell' file is pushed to Target folder: '/usr/share/jailhouse/cells' 

After that enable Jailhouse with generated imx8mq.cell , and facing below error

/ jailhouse enable /usr/share/jailhouse/cells/imx8mq.cell
[ 2533.385877] jailhouse: Not a system configuration
JAILHOUSE_ENABLE: Invalid argument

 
Please provide guidance and if possible some USerGuide/Application note or Makefile/Proper GCC command, to resolve the issue.

Attaching makefile (it has all commands as mentioned above)

Compilation, 
 make
aarch64-poky-linux-gcc -nostdlib -nostartfiles -ffreestanding -I/home/automotive/jailhouse/hypervisor/include -I/home/automotive/jailhouse/hypervisor/arch/arm64/include -I/home/automotive/jailhouse/include -c imx8mq.c -o imx8mq.o
aarch64-poky-linux-ld -Ttext=0x0 -nostdlib -o imx8mq.elf imx8mq.o
aarch64-poky-linux-ld: warning: cannot find entry symbol _start; defaulting to 0000000000000000
aarch64-poky-linux-objcopy -O binary imx8mq.elf imx8mq.cell

It generated .elf , .cell and .o.
Verified file format with following commands:

file imx8mq.elf 
imx8mq.elf: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, not stripped
file imx8mq.o 
imx8mq.o: ELF 64-bit LSB relocatable, ARM aarch64, version 1 (SYSV), not stripped

file imx8mq.cell 
imx8mq.cell: data

Thanks for support.

Sumit

[Peng_Fan]

So my understanding is you have compliation issue.

NXP has a repo: https://github.com/nxp-imx/imx-jailhouse

I not understand why you are trying to self build cell files. It is quite simple to build jailhouse and all cell files just using make. Even if you are using Simense repo, the docmumentation also says:

make [KDIR=/path/to/kernel/objects]

And my command to build all:

. /opt/fsl-imx-internal-xwayland/6.12-styhead/environment-setup-armv8a-poky-linux

export CROSS_COMPILE=aarch64-poky-linux-

unset CFLAGS

unset LDFLAGS

make KDIR=$HOME/linux/linux-lts-nxp CC="aarch64-poky-linux-gcc --sysroot=$SDKTARGETSYSROOT" ARCH=arm64 CROSS_COMPILE=aarch64-poky-linux-

[sumiteca09]

Hi Peng,

Thanks a lot for response, now we are able to compile cell.c code and generated .cell file.
"
Custom ecall-inmate-linux-demo cell created successfully, and the ecall-inmate-linux-demo guest is up and running.

Linux Host

root imx8mqevk:~jailhouse cell list

ID      Name                    State             Assigned CPUs           Failed CPUs

0       imx8mq                  running           0-1

1       ecall-inmate-linux-demo running           2-3

ecall-inmate-linux-demo guest

"
We are using /dev/uio0 device node for communication.
Tried some python code, write/send is successful on Root cell. At the same time, not able to receive anything to Linux demo inmate cell.
Later we moved to C programming code, attaching 'ivshmem_common.h', 'ivshmem_inmate.c', 'ivshmem_root.c', and Makefile.

Used below command for generating binaries
source /opt/fsl-imx-wayland/5.15-kirkstone/environment-setup-armv8a-poky-linux 
export CROSS_COMPILE=aarch64-poky-linux-
make KDIR=/home/automotive/yocto_imx_linux/build/tmp/work-shared/imx8mqevk/kernel-build-artifacts CC="aarch64-poky-linux-gcc --sysroot=$SDKTARGETSYSROOT" ARCH=arm64 CROSS_COMPILE=aarch64-poky-linux-
aarch64-poky-linux-gcc --sysroot=/opt/fsl-imx-wayland/5.15-kirkstone/sysroots/armv8a-poky-linux -Wall -O2 -o root ivshmem_root.c

We compiled using imx8mq tool chain, with above mentioned commands, on desktop side.
We want to copy generated binary to the Linux inmate demo cell.
We copied to root cell side successfully, but we don't know steps to copy or Compile on Inmate Linux demo cell?

Can you provide me how to do the same?
I am referring to NXP document 'UG10163' name 'iMX_Linux_user_guide_LF6.12.20_2.0.0'.

I want some working sample code and user guide to use this Linux demo inmate cell , so transmitting (some bundled data like JSON or String/Binary) data from Root cell and vis versa receiving from Linux Inmate demo cell.

Thanks,

Sumit

[Peng_Fan]

Executing your 'root' or 'inmate' in linux will trigger segment fault. 

To test ivshmem between two linux, you could do below:

in root cell: ivshmem-demo -d /dev/uio0 -t 2

in inmate cell: ivshmem-demo -d /dev/uio0 -t 0

Then you will see communication between the two cells.

The source code of ivshmem-demo is in imx-jailhouse/tools/demos/ivshmem-demo.c

Regarding how to copy files to inmate cell, after inmate linux starts up,

in inmate cell: ifconfig eth0 192.168.0.4

in root cell: ifconfig eth0 192.168.0.3

Then you could use scp to copy files from root to inmate.

Or you could configure ip forward in root cell to let inmate access external network.

[sumiteca09]

Hi Peng,

Thanks a lot for support, We are able to copy 'scp' from root cell to inmate cell.

Basically we want to keep our E-call application with MQTT on Inmate cell side.

We are able to share read/write Shared memory from root to inmate cell. Also, we are able to give static IP to root/inmate cell as proposed by you.

Now we need an internet connection on inmate cell, for transmitting received data from root cell, to cloud.

We connected Quectel EC25 module to NXP i.MX8MQ board via USB port.

IFCONFIG on Root cell data:

"

 ifconfig

eth0: flags=-28669<UP,BROADCAST,MULTICAST,DYNAMIC>  mtu 1500

        ether 00:04:9f:05:a5:9c  txqueuelen 1000  (Ethernet)

        RX packets 0  bytes 0 (0.0 B)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 0  bytes 0 (0.0 B)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=-28605<UP,BROADCAST,RUNNING,MULTICAST,DYNAMIC>  mtu 16384

        inet 192.168.0.3  netmask 255.255.255.0  broadcast 192.168.0.255

        inet6 fe80::88f0:a7ff:fe40:2407  prefixlen 64  scopeid 0x20<link>

        ether 8a:f0:a7:40:24:07  txqueuelen 1000  (Ethernet)

        RX packets 41  bytes 7030 (6.8 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 71  bytes 18812 (18.3 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536

        inet 127.0.0.1  netmask 255.0.0.0

        inet6 ::1  prefixlen 128  scopeid 0x10<host>

        loop  txqueuelen 1000  (Local Loopback)

        RX packets 103  bytes 9268 (9.0 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 103  bytes 9268 (9.0 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

usb0: flags=-28605<UP,BROADCAST,RUNNING,MULTICAST,DYNAMIC>  mtu 1500

        inet 192.168.225.39  netmask 255.255.255.0  broadcast 192.168.225.255

        inet6 2401:4900:4bb5:e7a7:609f:c3ff:fe2c:903b  prefixlen 64  scopeid 0x0<global>

        inet6 fe80::609f:c3ff:fe2c:903b  prefixlen 64  scopeid 0x20<link>

        ether 62:9f:c3:2c:90:3b  txqueuelen 1000  (Ethernet)

        RX packets 84  bytes 7614 (7.4 KiB)

        RX errors 0  dropped 0  overruns 0  frame 0

        TX packets 117  bytes 13860 (13.5 KiB)

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

 "

As you can see Internet connection is established on ROOT cell side.

Below is PING command output

"

root@imx8mqevk:/home ping 8.8.8.8

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=193 ms

64 bytes from 8.8.8.8: icmp_seq=2 ttl=114 time=68.9 ms

64 bytes from 8.8.8.8: icmp_seq=3 ttl=114 time=62.8 ms

64 bytes from 8.8.8.8: icmp_seq=4 ttl=114 time=68.5 ms

64 bytes from 8.8.8.8: icmp_seq=5 ttl=114 time=66.5 ms

64 bytes from 8.8.8.8: icmp_seq=6 ttl=114 time=78.6 ms

64 bytes from 8.8.8.8: icmp_seq=7 ttl=114 time=71.0 ms

"

Now I am facing internet issue on Inmate cell, below if IFCONFIG log:

"

root@imx8mqevk:~ ifconfig

eth0      Link encap:Ethernet  HWaddr 6E:58:98:59:01:A0  

          inet addr:192.168.0.4  Bcast:192.168.0.255  Mask:255.255.255.0

          inet6 addr: fe80::6c58:98ff:fe59:1a0/64 Scope:Link

          UP BROADCAST RUNNING MULTICAST  MTU:16384  Metric:1

          RX packets:184 errors:0 dropped:0 overruns:0 frame:0

          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:78814 (76.9 KiB)  TX bytes:14002 (13.6 KiB)

lo        Link encap:Local Loopback  

          inet addr:127.0.0.1  Mask:255.0.0.0

          inet6 addr: ::1/128 Scope:Host

          UP LOOPBACK RUNNING  MTU:65536  Metric:1

          RX packets:22 errors:0 dropped:0 overruns:0 frame:0

          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0

          collisions:0 txqueuelen:1000 

          RX bytes:3897 (3.8 KiB)  TX bytes:3897 (3.8 KiB)

"

ALSO we tried :IP Forwarding on ROOT cell side:

To allow the inmate cell having external network access:

 In root cell

echo 1 > /proc/sys/net/ipv4/ip_forward

iptables -A FORWARD -i eth1 -o usb0 -j ACCEPT

iptables -A FORWARD -i usb0 -o eth1 -j ACCEPT

iptables -t nat -A POSTROUTING -o usb0 -j MASQUERADE

"

When I run above commands, I get below logs:

"

root@imx8mqevk:/home iptables -t nat -A POSTROUTING -o usb0 -j MASQUERADE

[ 1869.738194] audit: type=1325 audit(1755848821.593:11): table=nat family=2 entries=7 op=xt_replace pid=713 comm="iptables"

root@imx8mqevk:/home [ 1869.749806] audit: type=1300 audit(1755848821.593:11): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaafd66a400 items=0 ppid=607 pid=713 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[ 1869.779649] audit: type=1327 audit(1755848821.593:11): proctitle=69707461626C6573002D74006E6174002D4100504F5354524F5554494E47002D6F0075736230002D6A004D415351554552414445"  

"

Also putting Dmesg log from root cell

"

[   71.141601] Created Jailhouse cell "ecall-linux-inmate-demo"

[   77.822386] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready

[  206.451703] audit: type=1325 audit(1755847158.198:4): table=nat family=2 entries=0 op=xt_register pid=651 comm="iptables"

[  206.463131] audit: type=1300 audit(1755847158.198:4): arch=c00000b7 syscall=209 success=yes exit=0 a0=4 a1=0 a2=40 a3=ffffe78d01c0 items=0 ppid=607 pid=651 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  206.493176] audit: type=1327 audit(1755847158.198:4): proctitle=69707461626C6573002D74006E6174002D4100504F5354524F5554494E47002D6F0065746830002D6A004D415351554552414445

[  206.508268] audit: type=1325 audit(1755847158.202:5): table=nat family=2 entries=5 op=xt_replace pid=651 comm="iptables"

[  206.519201] audit: type=1300 audit(1755847158.202:5): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaabb51c230 items=0 ppid=607 pid=651 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  206.547353] audit: type=1327 audit(1755847158.202:5): proctitle=69707461626C6573002D74006E6174002D4100504F5354524F5554494E47002D6F0065746830002D6A004D415351554552414445

[  217.506636] audit: type=1325 audit(1755847169.257:6): table=nat family=2 entries=6 op=xt_replace pid=657 comm="iptables"

[  217.517791] audit: type=1300 audit(1755847169.257:6): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaaec783230 items=0 ppid=607 pid=657 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  217.547639] audit: type=1327 audit(1755847169.257:6): proctitle=69707461626C6573002D74006E6174002D4100504F5354524F5554494E47002D6F0075736230002D6A004D415351554552414445

[  294.389911] audit: type=1325 audit(1755847246.146:7): table=filter family=2 entries=0 op=xt_register pid=659 comm="iptables"

[  294.401564] audit: type=1300 audit(1755847246.146:7): arch=c00000b7 syscall=209 success=yes exit=0 a0=4 a1=0 a2=40 a3=fffff5707410 items=0 ppid=607 pid=659 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  294.431369] audit: type=1327 audit(1755847246.146:7): proctitle=69707461626C6573002D4100464F5257415244002D690065746831002D6F0075736230002D6A00414343455054

[  294.445250] audit: type=1325 audit(1755847246.150:8): table=filter family=2 entries=4 op=xt_replace pid=659 comm="iptables"

[  294.456430] audit: type=1300 audit(1755847246.150:8): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaaeb8d3950 items=0 ppid=607 pid=659 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  294.484541] audit: type=1327 audit(1755847246.150:8): proctitle=69707461626C6573002D4100464F5257415244002D690065746831002D6F0075736230002D6A00414343455054

[  314.598783] audit: type=1325 audit(1755847266.267:9): table=filter family=2 entries=5 op=xt_replace pid=662 comm="iptables"

[  314.610726] audit: type=1300 audit(1755847266.267:9): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaac05eaac0 items=0 ppid=607 pid=662 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[  314.640305] audit: type=1327 audit(1755847266.267:9): proctitle=69707461626C6573002D4100464F5257415244002D690075736230002D6F0065746831002D6A00414343455054

[  849.488680] audit: type=1006 audit(1755847801.267:10): pid=671 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=3 res=1

[  849.501588] audit: type=1300 audit(1755847801.267:10): arch=c00000b7 syscall=64 success=yes exit=1 a0=3 a1=ffffc0aa76a0 a2=1 a3=ffff83653020 items=0 ppid=238 pid=671 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="crond" exe="/usr/sbin/crond" key=(null)

[  849.527425] audit: type=1327 audit(1755847801.267:10): proctitle=2F7573722F7362696E2F63726F6E64002D6E

[ 1869.738194] audit: type=1325 audit(1755848821.593:11): table=nat family=2 entries=7 op=xt_replace pid=713 comm="iptables"

[ 1869.749806] audit: type=1300 audit(1755848821.593:11): arch=c00000b7 syscall=208 success=yes exit=0 a0=4 a1=0 a2=40 a3=aaaafd66a400 items=0 ppid=607 pid=713 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=ttymxc0 ses=4294967295 comm="iptables" exe="/usr/sbin/xtables-legacy-multi" key=(null)

[ 1869.779649] audit: type=1327 audit(1755848821.593:11): proctitle=69707461626C6573002D74006E6174002D4100504F5354524F5554494E47002D6F0075736230002D6A004D415351554552414445

 "

CAN YOU help us in resolving above Internet issue on Inmate cell?

[Peng_Fan]

Regarding how to setup network to allow inmate access internet, refer as below, you need to update to use usb net or eth net in your set.

1. How to setup network for 2nd linux using network
   1. After first linux boots up

      1. sysctl -w net.ipv4.ip_forward=1
         sysctl -p /etc/sysctl.conf

   2. After root cell enabled
      1. iptables -A FORWARD -i eth1 -j ACCEPT

         iptables -A FORWARD -o eth1 -j ACCEPT
         iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

   3. After 2nd linux boots up

      1. ifconfig eth1 or 2 192.168.1.4 for the 1st linux root cell

      2. ifconfig eth0 192.168.1.5 for the 2nd linux inmate cell

      3. "ip route add default via 192.168.1.4 dev eth0" for the 2nd linux
         "ip route add 10.193.100.0/24 via 192.168.1.4 dev eth0" for 2nd linux

      4. Now you could "mount -t nfs 10.193.108.xx:/home/xxx/nfs /mnt"
      5. Update 10.193.xx.xx to yours according to your network, in step 3 and 4

Regards

Peng.

[sumiteca09]

@Peng 
peng.fan@nxp.com


Please suggest some solution for Custom Jailhouse cell creation issue we are facing, as explained in above ticket?
Thanks for support.
Do let us know any further details required.