AnsweredAssumed Answered

Questions/thoughts on the download (fetch) system

Question asked by Yvon Grall on Feb 24, 2017
Latest reply on Mar 9, 2017 by gusarambula

Hi all,

 

I'm building the imx6sabresd machine with the Linux 4.1.15_1.2.0 BSP. Unfortunately, the network access to freescale/NXP servers (git://git.freescale.com) is erratic : I had to re-run several times the build (bitbake core-image-minimal) when it comes to fetch u-boot. So I'm trying to understand the download (fetch) system.

 

When it fails, we can see this in the corresponding log.do_fetch file :

DEBUG: Trying PREMIRRORS
[...]
DEBUG: For url git://git.freescale.com/imx/uboot-imx.git;protocol=git;branch=imx_v2015.04_4.1.15_1.0.0_ga returning http://downloads.yoctoproject.org/mirror/sources/git2_git.freescale.com.imx.uboot-imx.git.tar.gz
[...]
DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
http://downloads.yoctoproject.org/mirror/sources/git2_git.freescale.com.imx.uboot-imx.git.tar.gz:
2017-02-23 12:00:25 ERROR 404: Not Found.
[...]
DEBUG: Trying Upstream

DEBUG: Fetcher failure: Fetch command failed with exit code 128, output:
Cloning into bare repository '/home/yon/fsl-nxp-1.2.0/downloads//git2/git.freescale.com.imx.uboot-imx.git'...
fatal: read error: Connection reset by peer
[...]
DEBUG: Trying MIRRORS
[...]
DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
http://sources.openembedded.org/git2_git.freescale.com.imx.uboot-imx.git.tar.gz:
2017-02-23 12:00:29 ERROR 404: Not Found.
ERROR: Fetcher failure: Fetch command failed with exit code 128, output:
Cloning into bare repository '/home/yon/fsl-nxp-1.2.0/downloads//git2/git.freescale.com.imx.uboot-imx.git'...
fatal: read error: Connection reset by peer
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished
ERROR: Function failed: Fetcher failure for URL: 'git://git.freescale.com/imx/uboot-imx.git;protocol=git;branch=imx_v2015.04_4.1.15_1.0.0_ga'. Unable to fetch URL from any source.
  1. The build system maps git://git.freescale.com/imx/uboot-imx.git to http://downloads.yoctoproject.org/mirror/sources/git2_git.freescale.com.imx.uboot-imx.git.tar.gz, or later to http://sources.openembedded.org/git2_git.freescale.com.imx.uboot-imx.git.tar.gz.
    Q : where are these mappings defined (in which config/include/recipe/anything_else file) ?
  2. Some archives on the mirrors are large (e.g. 1 GB for git2_git.freescale.com.imx.linux-2.6-imx.git.tar.gz). So in case when they are fetched, it may take a long time to download.
    Q : Why this huge size, is it because an archive contains all branches/tags of a package ?
  3. On the other hand, in case when the build system is able to connect to the git repository, I guess it only downloads the desired branch/tag, which is much faster.
    Q : shouldn't NXP provide public git clones as fallbacks instead of compressed archives on http/ftp mirrors ?
  4. Assuming the DEBUG lines were printed in chronological order, the build system seems to try to fetch at first on the mirror, before attempting to fetch on the primary source.
    Q1 : shouldn't the primary source be tried at first ?
    Q2 : how to disable mirrors (and only use the freescale git repository instead) ?
  5. At the time of writing, there's indeed no git2_git.freescale.com.imx.uboot-imx.git.tar.gz within the mirrors. Searching in the NXP community forums, we can see that many users encountered similar issues for some other packages : this issue (of non up-to-date mirrors) seems quite frequent.
    Q : would there be any acceptable generic workaround to that problem ? e.g. such as git-cloning the repositories, and then pointing SRC_URI to that local clone via file://....

 

 

Thanks in advance for your help,

-Yvon

Outcomes