Bitbake full image failed by armnn-onnx and onnxruntime

cancel
Showing results for 
Search instead for 
Did you mean: 

Bitbake full image failed by armnn-onnx and onnxruntime

Jump to solution
1,179 Views
khang_letruong
Senior Contributor II

Dear Community,

I'm bitbaking full image for imx8mpevk with BSP-5.4.70-2.3.2 and struggling with following error :

ERROR: onnxruntime-1.5.3-r0 do_unpack: gitsm: submodule unpack failed: UnpackError Unpack failure for URL: 'gitsm://github.com/onnx/onnx;protocol=https;name=cmake/external/onnx;subpath=cmake/external/onnx;bareclone=1;nobranch=1'. No up to date source found: clone directory not available or not up to date: /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/../../NXP/Boards/Common/Yocto/yocto_dl/v5.4//git2/github.com.onnx.onnx; shallow clone not enabled
ERROR: onnxruntime-1.5.3-r0 do_unpack: Unpack failure for URL: 'gitsm://github.com/onnx/onnx;protocol=https;name=cmake/external/onnx;subpath=cmake/external/onnx;bareclone=1;nobranch=1'. No up to date source found: clone directory not available or not up to date: /workdir/Dynimlabs/spark-yocto-bsp-5_4_70-2_3_2/../../NXP/Boards/Common/Yocto/yocto_dl/v5.4//git2/github.com.onnx.onnx; shallow clone not enabled
ERROR: Logfile of failure stored in: /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/build-xwayland/tmp/work/aarch64-poky-linux/onnxruntime/1.5.3-r0/temp/log.do_unpack.16020
ERROR: Task (/workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/onnxruntime/onnxruntime_1.5.3.bb:do_unpack) failed with exit code '1'
WARNING: armnn-onnx-1.3.0-r0 do_fetch: Failed to fetch URL git://github.com/onnx/onnx.git, attempting MIRRORS if available
ERROR: armnn-onnx-1.3.0-r0 do_fetch: Fetcher failure: Unable to find revision bae6333e149a59a3faa9c4d9c44974373dcf5256 in branch master even from upstream
ERROR: armnn-onnx-1.3.0-r0 do_fetch: Fetcher failure for URL: 'git://github.com/onnx/onnx.git'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/build-xwayland/tmp/work/aarch64-poky-linux/armnn-onnx/1.3.0-r0/temp/log.do_fetch.16006
ERROR: Task (/workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/armnn/armnn-onnx_1.3.0.bb:do_fetch) failed with exit code '1'

Meanwhile I cloned above repositories and checkouted the commits found in related recipes without problem.

Have you ever seen this issue? 

My host is Ubuntu-1804 (docker container).

 

Thanks and best regards,

Khang.

 

 

 

1 Solution
1,163 Views
sanjaikumar
Contributor I

Hi Khang,

           The Problem is because of the branch name. Branch name should the main not master (Since Github default branch name changed to main), if not mentioned it will look for master.

So change the SRC_URI in /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/armnn/armnn-onnx_1.3.0.bb as below.

SRC_URI = "git://github.com/onnx/onnx.git;branch=main"

Regards,

Sanjaikumar

View solution in original post

22 Replies
1,016 Views
josephzhou1
Contributor V

yes i do L5.10.35 ok

Best Regards,
Joseph Zhou Jianhui / Senior Embedded Software Engineer, Singapore
0 Kudos
1,013 Views
khang_letruong
Senior Contributor II

i @josephzhou1 ,

Thanks for your answer but I'm looking for the solution of BSP-5.4.70-2.3.2 that we are using. Anyway, I could compare and hopefully I could port the changes from your version back to that BSP version.

Regards,
K.

 

1,101 Views
sanjaikumar
Contributor I

Hi Khang,

              You need to specify the branch main in onnxruntime_1.5.3.bb . From the above error logs,it looks like trying to download subpath from the same repository above mentioned.

 

Regards,

Sanjaikumar

 

1,023 Views
khang_letruong
Senior Contributor II

Hi @sanjaikumar ,

As I mentioned earlier, the URI of onnxruntime is : https://source.codeaurora.org/external/imx/onnxruntime-imx/

ONNXRUNTIME_SRC ?= "gitsm://source.codeaurora.org/external/imx/onnxruntime-imx.git;protocol=https"
SRCBRANCH = "imx_5.4.70_2.3.2"

There's no branch main of that package's repository. And the branch imx_5.4.70_2.3.2 well exists.

The onnxruntime repository composes of different submodules, including onnx as below :

[submodule "cmake/external/onnx"]
	path = cmake/external/onnx
	url = https://github.com/onnx/onnx

I would say the issue is from above. I tried to put branch = main in /workdir/L4ES/imx-yocto-bsp-5_4_70-2_3_2/build-xwayland/tmp/work/aarch64-poky-linux/onnxruntime/1.5.3-r0/git/.gitmodules like following :

[submodule "cmake/external/onnx"]
	path = cmake/external/onnx
	url = https://github.com/onnx/onnx
	branch = main

but the .gitmodules is ovveridden each time I continue bitbaking.

Best Regards,
K.

 

 

1,091 Views
khang_letruong
Senior Contributor II

Hi @sanjaikumar ,

I do not found the relevant info in onnxruntime_1.5.3.bb.There's only following SRC_URI :

ONNXRUNTIME_SRC ?= "gitsm://source.codeaurora.org/external/imx/onnxruntime-imx.git;protocol=https"
SRCBRANCH = "imx_5.4.70_2.3.2"

 

I believe that it comes from below : https://source.codeaurora.org/external/imx/onnxruntime-imx/tree/.gitmodules?h=imx_5.4.70_2.3.2

Do you kow how to patch it ?

Thanks,

K.

 

1,164 Views
sanjaikumar
Contributor I

Hi Khang,

           The Problem is because of the branch name. Branch name should the main not master (Since Github default branch name changed to main), if not mentioned it will look for master.

So change the SRC_URI in /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/armnn/armnn-onnx_1.3.0.bb as below.

SRC_URI = "git://github.com/onnx/onnx.git;branch=main"

Regards,

Sanjaikumar

1,107 Views
khang_letruong
Senior Contributor II

Hi @sanjaikumar ,

Thanks for your support. Then it works for armnn-onnx but I still faced with following issue of onnxruntime :

ERROR: onnxruntime-1.5.3-r0 do_unpack: gitsm: submodule unpack failed: UnpackError Unpack failure for URL: 'gitsm://github.com/onnx/onnx;protocol=https;name=cmake/external/onnx;subpath=cmake/external/onnx;bareclone=1;nobranch=1'. No up to date source found: clone directory not available or not up to date: /workdir/Dynimlabs/spark-yocto-bsp-5_4_70-2_3_2/../../NXP/Boards/Common/Yocto/yocto_dl/v5.4//git2/github.com.onnx.onnx; shallow clone not enabled
ERROR: onnxruntime-1.5.3-r0 do_unpack: Unpack failure for URL: 'gitsm://github.com/onnx/onnx;protocol=https;name=cmake/external/onnx;subpath=cmake/external/onnx;bareclone=1;nobranch=1'. No up to date source found: clone directory not available or not up to date: /workdir/Dynimlabs/spark-yocto-bsp-5_4_70-2_3_2/../../NXP/Boards/Common/Yocto/yocto_dl/v5.4//git2/github.com.onnx.onnx; shallow clone not enabled
ERROR: Logfile of failure stored in: /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/build-xwayland/tmp/work/aarch64-poky-linux/onnxruntime/1.5.3-r0/temp/log.do_unpack.10853
ERROR: Task (/workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/onnxruntime/onnxruntime_1.5.3.bb:do_unpack) failed with exit code '1'

 

Do you know how to resolve, please ?

Khang

1,048 Views
joanxie
NXP TechSupport
NXP TechSupport

Indeed this issue is fixed since 5.10.35-2.0.0 where the branch name is specified.

On 5.4.70-2.3.0 (and earlier than 5.10.35-2.0.0) you need to modify SRC_URI in the recipe:

SRC_URI = "git://github.com/onnx/onnx.git;branch=main"

 

 

1,043 Views
khang_letruong
Senior Contributor II

Hi @joanxie ,

So change the SRC_URI in /workdir/L4ES/spark-yocto-bsp-5_4_70-2_3_2/sources/meta-imx/meta-ml/recipes-libraries/armnn/armnn-onnx_1.3.0.bb as below.

SRC_URI = "git://github.com/onnx/onnx.git;branch=main"

For armnn-onnx, I did modify above recipe (just after the issue had happened), then I got issue with onnxruntime. I think I will try bitbaking from scratch after above modification.

Best Regards,
K.

1,027 Views
khang_letruong
Senior Contributor II

Hi @sanjaikumar and @joanxie ,

I do not know if there's still something missing but unfortunately your suggestion seems not enough to resolve the issue of bitbaking onnxruntime.

Best Regards,
K.

996 Views
joanxie
NXP TechSupport
NXP TechSupport

I suggest that you delete these packages and build again, I don't find this issue before, I don't know if you change something

992 Views
khang_letruong
Senior Contributor II

Hi @joanxie ,

I did remove the whole build-xwayland folder as well. I might try from repo syncing but I do not see any difference aside from a clean sources folder.

Best Regards,
K.

985 Views
joanxie
NXP TechSupport
NXP TechSupport

give me all of your build steps, let me reproduce this on my side

971 Views
khang_letruong
Senior Contributor II

Hi @joanxie ,

Step #1 : Create bsp folder and move to inside

mkdir imx-yocto-bsp && cd imx-yocto-bsp

Step #2 : Sync the bsp source code

repo init -u https://source.codeaurora.org/external/imx/imx-manifest -b imx-linux-zeus -m imx-5.4.70-2.3.2.xml

Step #3 : Setup the build environment

DISTRO=fsl-imx-xwayland MACHINE=imx8mpevk source imx-setup-release.sh -b build-xwayland

Step #4 : Add branch=main into armnn-onnx recipe

vi ../sources/meta-imx/meta-ml/recipes-libraries/armnn/armnn-onnx_1.3.0.bb
...
SRC_URI = "git://github.com/onnx/onnx.git;branch=main"
...

Step #5 : Bitbake full image

bitbake imx-image-full

 

Best Regards,
K.

959 Views
josephzhou1
Contributor V

hello,

 

step#2  should have 1 more command 

repo sync

 

please try it out , if OK please set as solution, thank you

Best Regards,
Joseph Zhou Jianhui / Senior Embedded Software Engineer, Singapore
927 Views
khang_letruong
Senior Contributor II

Hi @josephzhou1 ,

That was my typo. Of course step #2 includes repo sync. Without it, it is impossible to bitbake. So your suggestion is unfortunately NOT a solution, sorry. But thank you anyway for pointing out the missing command.

Regards,
K.

 

 

 

900 Views
joanxie
NXP TechSupport
NXP TechSupport

I have built the bsp successfully

joanxie_0-1643194102981.png

only one error for the onnxruntime, after I change the SRC_URI, I don't get other errors

 

886 Views
khang_letruong
Senior Contributor II

Hi @joanxie ,

Do you confirm that you changed the SRC_URI of armnn-onnx recipe or onnxruntime recipe, please ?

Best Regards,

K

 

878 Views
joanxie
NXP TechSupport
NXP TechSupport

change this:

armnn-onnx_1.3.0.bb

 

847 Views
khang_letruong
Senior Contributor II

Hi @joanxie ,

Finally I figure out the issue: even-though I re-do everything from scratch as per steps I provided you,  armnn-onnx and onnxruntime packages are still unchanged in separate download folder :

 

DL_DIR ?= "/workdir/NXP/Boards/Common/Yocto/yocto_dl/v5.4/"

 

Together with the appending of "branch=main" into SRC_URI within armnx-onnx recipe, removing them from the separate download folder to force a new download of both packages solves the issue.

Thanks for your support!

K.