Yocto Project Package Management - smart

12 Kudos

Introduction

The "smart" package management system is available in Yocto Project for managing packages on a target machine. A host is configured as a server for the packages and on the target the "package-management" feature is enabled for working with the packages. The steps for setup and usage are described below.

Resources

The Yocto Project package management system will work with many hosts and targets. The following were used for creating this document:

Host: Ubuntu 14.04 64-bit
Target: MCIMX6Q-SDP
Freescale Yocto Project Release Documentation: Linux 3.14.38_6ul BSP & Multimedia Codecs Documentation (fsl-yocto-L3.14.38_6ul-ga.tar.gz)

Host

You have successfully installed a Freescale Yocto Project release. (Refer to Freescale Yocto Project Release Documentation). There are two steps for adding package management and then building:

1. Modify conf/local.conf

EXTRA_IMAGE_FEATURES = "debug-tweaks package-management"
‍

2. Build the image:

bitbake core-image-minimal
‍

The core-image-minimal recipe provides an image enabling the target board to boot and support a serial console.

3. Create SDCARD:

$ cd <build>/tmp/deploy/images/imx6qsabresd
$ sudo dd if=core-image-minimal-imx6qsabresd.sdcard of=/dev/sdb bs=4M && sync
‍‍

Note - verify location of SDCARD on your host, /dev/sdb in this example. Examine 'cat /proc/partitions'

4. Setup web server and add link to rpm packages

A web server, lighttpd, is installed.

$ sudo apt-get install lighttpd
‍

Provide user write capability in /var/www

$ sudo chmod 777 /var/www
‍

Create a soft link in the default web server directory to the rpm directory from the build. Note: Please update $HOME/<build> to your actual location:

$ ln -s $HOME/<build>/tmp/deploy/rpm /var/www/imx6qsd
‍‍‍

Target

Insert the SDCARD created from step 3 above, connect power and console cable and power on the MCIMX6Q-SDP. Login using the "root" id, no password required.

The /usr/bin/smart application is now used to setup the channels and perform package commands. For all smart options:

smart --help
‍

1. Add channels

To add the packages from the host to your target, the smart channel --add is used:

Please enter the IP adress of your server, replacing SERVERIP below:

smart channel --add all type=rpm-md name=all baseurl=
http://SERVERIP/imx6qsd/all

smart channel --add cortexa9hf_vfp_neon type=rpm-md name=cortexa9hf_vfp_neon baseurl=
http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon

smart channel --add imx6qsabresd type=rpm-md name=imx6qsabresd baseurl=
http://SERVERIP/imx6qsd/imx6qsabresd

‍‍‍‍‍‍‍‍‍

Check the added channels:

root@imx6qsabresd:~# smart channel --list
all
imx6qsabresd
rpmsys
cortexa9hf_vfp_neon
‍‍‍‍‍

2. Update local package cache

Once the chanels have been added, the local package cache is updated.

Note SERVERIP below will be the host IP address in your network.

root@imx6qsabresd:~# smart update

Loading cache...

Updating cache... ######################################## [100%]

Fetching information for 'all'...

-> http://SERVERIP/imx6qsd/all/repodata/repomd.xml

repomd.xml ######################################## [ 16%]

Fetching information for 'imx6qsabresd'...

-> http://SERVERIP/imx6qsd/imx6qsabresd/repodata/repomd.xml

repomd.xml ######################################## [ 41%]

Fetching information for 'cortexa9hf_vfp_neon'...

-> http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon/repodata/repomd.xml

repomd.xml ######################################## [ 66%]

Updating cache... ######################################## [100%]

Channels have no new packages.

3. Searching for packages

Let us look at all packages containing the string client

root@imx6qsabresd:~# smart search client*
Loading cache...
Updating cache...               ######################################## [100%]

libice-dbg - ICE: Inter-Client Exchange library - Debugging files
libice-dev - ICE: Inter-Client Exchange library - Development files
libice-doc - ICE: Inter-Client Exchange library - Documentation files
libice-staticdev - ICE: Inter-Client Exchange library - Development files (Static Libraries)
libice6 - ICE: Inter-Client Exchange library
libsm-dbg - SM: Session Management library - Debugging files
libsm-dev - SM: Session Management library - Development files
libsm-doc - SM: Session Management library - Documentation files
libsm-staticdev - SM: Session Management library - Development files (Static Libraries)
libsm6 - SM: Session Management library
libx11-6 - Xlib: C Language X Interface library
libx11-dbg - Xlib: C Language X Interface library - Debugging files
libx11-dev - Xlib: C Language X Interface library - Development files
libx11-doc - Xlib: C Language X Interface library - Documentation files
libx11-locale - Xlib: C Language X Interface library
libx11-staticdev - Xlib: C Language X Interface library - Development files (Static Libraries)
libx11-xcb1 - Xlib: C Language X Interface library
libxau-dbg - Xau: X Authority Database library - Debugging files
libxau-dev - Xau: X Authority Database library - Development files
libxau-doc - Xau: X Authority Database library - Documentation files
libxau-staticdev - Xau: X Authority Database library - Development files (Static Libraries)
libxau6 - Xau: X Authority Database library
python-netclient - Python Internet Protocol clients
xtrans-dbg - XTrans: X Transport library - Debugging files
xtrans-dev - XTrans: X Transport library - Development files
xtrans-doc - XTrans: X Transport library - Documentation files
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Adding openssh client to core-image minimal

The core-image-minimal does not provide openssh client applications like ssh or scp. Let's add them on the host then update the target cache of packages and then install.

Host

Run bitbake to exercise all the tasks for packagegroup-core-ssh-openssh

$ bitbake packagegroup-core-ssh-openssh
‍

After building a package individually, always update the package-index

$ bitbake package-index
‍

Target

Run smart to update the local cache which will pickup the new packages from the bake above.

root@imx6qsabresd:~# smart update
Loading cache...
Updating cache...               ######################################## [100%]


Fetching information for 'all'...                                              
-> 
http://SERVERIP/imx6qsd/all/repodata/repomd.xml
                          

repomd.xml                      ######################################## [ 16%]
-> 
http://SERVERIP/imx6qsd/all/repodata/primary.xml.gz
                      

primary.xml.gz                  ######################################## [ 25%]
-> 
http://SERVERIP/imx6qsd/all/repodata/filelists.xml.gz
                    

filelists.xml.gz                ######################################## [ 33%]
                                                                               
Fetching information for 'imx6qsabresd'...
-> 
http://SERVERIP/imx6qsd/imx6qsabresd/repodata/repomd.xml
                 

repomd.xml                      ######################################## [ 50%]
-> 
http://SERVERIP/imx6qsd/imx6qsabresd/repodata/primary.xml.gz
             

-> 
http://SERVERIP/imx6qsd/imx6qsabresd/repodata/filelists.xml.gz
           

filelists.xml.gz                ######################################## [ 58%]
primary.xml.gz                  ######################################## [ 66%]
                                                                               
Fetching information for 'cortexa9hf_vfp_neon'...
-> 
http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon/repodata/repomd.xml
          

repomd.xml                      ######################################## [ 83%]
-> 
http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon/repodata/primary.xml.gz
      

primary.xml.gz                  ######################################## [ 91%]
-> 
http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon/repodata/filelists.xml.gz
    

filelists.xml.gz                ######################################## [100%]


Updating cache...               ######################################## [100%]


Channels have 15 new packages.
Saving cache...
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Examine information about local cache:

root@imx6qsabresd:~# smart stats
Loading cache...
Updating cache...               ######################################## [100%]


Installed Packages: 80
Total Packages: 3586
Total Provides: 6580
Total Requires: 1611
Total Upgrades: 3565
Total Conflicts: 25
‍‍‍‍‍‍‍‍‍‍‍

See what ssh packages are now available:

root@imx6qsabresd:~# smart search *ssh*
Loading cache...
Updating cache...               ######################################## [100%]


openssh - Secure rlogin/rsh/rcp/telnet replacement
openssh-dbg - Secure rlogin/rsh/rcp/telnet replacement - Debugging files
openssh-dev - Secure rlogin/rsh/rcp/telnet replacement - Development files
openssh-doc - Secure rlogin/rsh/rcp/telnet replacement - Documentation files
openssh-keygen - Secure rlogin/rsh/rcp/telnet replacement
openssh-misc - Secure rlogin/rsh/rcp/telnet replacement
openssh-ptest - Secure rlogin/rsh/rcp/telnet replacement - Package test files
openssh-scp - Secure rlogin/rsh/rcp/telnet replacement
openssh-sftp - Secure rlogin/rsh/rcp/telnet replacement
openssh-sftp-server - Secure rlogin/rsh/rcp/telnet replacement
openssh-ssh - Secure rlogin/rsh/rcp/telnet replacement
openssh-sshd - Secure rlogin/rsh/rcp/telnet replacement
packagegroup-core-ssh-openssh - OpenSSH SSH client/server
packagegroup-core-ssh-openssh-dbg - OpenSSH SSH client/server - Debugging files
packagegroup-core-ssh-openssh-dev - OpenSSH SSH client/server - Development files
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Install openssh

root@imx6qsabresd:~# smart install openssh
Loading cache...
Updating cache...               ######################################## [100%]


Computing transaction...


Installing packages (9):
  openssh-6.7p1-r0@cortexa9hf_vfp_neon                                          
  openssh-keygen-6.7p1-r0@cortexa9hf_vfp_neon                                   
  openssh-scp-6.7p1-r0@cortexa9hf_vfp_neon                                      
  openssh-ssh-6.7p1-r0@cortexa9hf_vfp_neon                                      
  openssh-sshd-6.7p1-r0@cortexa9hf_vfp_neon                                     
  shadow-4.2.1-r0@cortexa9hf_vfp_neon                                           
  shadow-base-4.2.1-r0@cortexa9hf_vfp_neon                                      
  shadow-securetty-4.2.1-r3@imx6qsabresd                                        
  util-linux-sulogin-2.25.2-r1@cortexa9hf_vfp_neon                              


1.4MB of package files are needed. 3.2MB will be used.


Confirm changes? (Y/n): y


Fetching packages...                                                           
-> 
http://SERVERIP/imx6qsd/.../openssh-6.7p1-r0.cortexa9hf_vfp_neon.rpm
     

-> 
http://SERVERIP/imx6qsd/.../shadow-securetty-4.2.1-r3.imx6qsabresd.rpm
   

shadow-securetty-4.2.1-r3.imx.. ######################################## [ 11%]
-> 
http://SERVERIP/imx6qsd/.../openssh-scp-6.7p1-r0.cortexa9hf_vfp_neon.rpm
 

openssh-scp-6.7p1-r0.cortexa9.. ######################################## [ 22%]
openssh-6.7p1-r0.cortexa9hf_v.. ######################################## [ 33%]
-> 
http://SERVERIP/imx6qsd/.../openssh-sshd-6.7p1-r0.cortexa9hf_vfp_neon.rpm

openssh-sshd-6.7p1-r0.cortexa.. ######################################## [ 44%]
-> 
http://SERVERIP/imx6qsd/.../shadow-4.2.1-r0.cortexa9hf_vfp_neon.rpm
      

-> 
http://SERVERIP/imx6qsd/.../openssh-ssh-6.7p1-r0.cortexa9hf_vfp_neon.rpm
 

openssh-ssh-6.7p1-r0.cortexa9.. ######################################## [ 55%]
-> 
http://SERVERIP/imx6qsd/.../shadow-base-4.2.1-r0.cortexa9hf_vfp_neon.rpm
 

shadow-base-4.2.1-r0.cortexa9.. ######################################## [ 66%]
shadow-4.2.1-r0.cortexa9hf_vf.. ######################################## [ 77%]
-> 
http://SERVERIP/.../util-linux-sulogin-2.25.2-r1.cortexa9hf_vfp_neon.rpm
 

util-linux-sulogin-2.25.2-r1... ######################################## [ 88%]
-> 
http://SERVERIP/.../openssh-keygen-6.7p1-r0.cortexa9hf_vfp_neon.rpm
      

openssh-keygen-6.7p1-r0.corte.. ######################################## [100%]


                                                                               
Committing transaction...
Preparing...                    ######################################## [  0%]
   1:Installing openssh-ssh     ######################################## [ 11%]
Output from openssh-ssh-6.7p1-r0@cortexa9hf_vfp_neon:                          
update-alternatives: Linking /usr/bin/ssh to /usr/bin/ssh.openssh              
   2:Installing openssh-scp     ######################################## [ 22%]
Output from openssh-scp-6.7p1-r0@cortexa9hf_vfp_neon:                          
update-alternatives: Linking /usr/bin/scp to /usr/bin/scp.openssh              
   3:Installing shadow-secure.. ######################################## [ 33%]
   4:Installing shadow-base     ######################################## [ 44%]
Output from shadow-base-4.2.1-r0@cortexa9hf_vfp_neon:                          
update-alternatives: Linking /usr/bin/newgrp to /usr/bin/newgrp.shadow         
update-alternatives: Linking /usr/bin/groups to /usr/bin/groups.shadow
update-alternatives: Linking /bin/login to /bin/login.shadow
update-alternatives: Linking /bin/su to /bin/su.shadow
   5:Installing util-linux-su.. ######################################## [ 55%]
Output from util-linux-sulogin-2.25.2-r1@cortexa9hf_vfp_neon:                  
update-alternatives: Linking /sbin/sulogin to /sbin/sulogin.util-linux         
   6:Installing openssh-keygen  ######################################## [ 66%]
   7:Installing shadow          ######################################## [ 77%]
Output from shadow-4.2.1-r0@cortexa9hf_vfp_neon:                               
update-alternatives: Linking /usr/bin/passwd to /usr/bin/passwd.shadow         
update-alternatives: Linking /usr/bin/chfn to /usr/bin/chfn.shadow
update-alternatives: Linking /usr/bin/chsh to /usr/bin/chsh.shadow
update-alternatives: Linking /usr/sbin/chpasswd to /usr/sbin/chpasswd.shadow
update-alternatives: Linking /sbin/vipw to /sbin/vipw.shadow
update-alternatives: Linking /sbin/vigr to /sbin/vigr.shadow
Output from openssh-sshd-6.7p1-r0@cortexa9hf_vfp_neon:                         
 Removing any system startup links for sshd ...                                
Running useradd commands...
NOTE: Performing useradd with [ --system --no-create-home --home-dir /var/run/sshd --shell /bin/false --user-group sshd] and 10 times of retry
   8:Installing openssh-sshd    ######################################## [ 88%]
 Adding system startup for /etc/init.d/sshd.                                   
Starting OpenBSD Secure Shell server: sshd
  generating ssh RSA key...
  generating ssh ECDSA key...
  generating ssh DSA key...
  generating ssh ED25519 key...
done.
   9:Installing openssh         ######################################## [100%]
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

Check for the scp command:

root@imx6qsabresd:~# which scp
/usr/bin/scp
‍‍

Summary

To add a new package on the server host, run bitbake <recipe> then bitbake package-index to update the rpm tracking information. On the target board, run smart update and then smart install <package>. Use smart search <regular expression string> to hunt for a package to install.

1990rahulreddy · ‎10-10-2015

I have followed this document but, got following errors. Please, help as early as you can

After bitbake fsl-image-gui, got following errors

WARNING: Failed to fetch URL http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz, attempting MIRRORS if available

WARNING: Failed to fetch URL http://ftp.gnu.org/gnu/libtool/libtool-2.4.6.tar.gz, attempting MIRRORS if available

WARNING: Failed to fetch URL http://tukaani.org/xz/xz-5.2.0.tar.gz, attempting MIRRORS if available

WARNING: Failed to fetch URL https://gmplib.org/download/gmp/gmp-6.0.0a.tar.bz2, attempting MIRRORS if available

WARNING: Failed to fetch URL http://download.savannah.gnu.org/releases/attr/attr-2.4.47.src.tar.gz, attempting MIRRORS if available

WARNING: Failed to fetch URL http://downloads.sourceforge.net/flex/flex-2.5.39.tar.bz2, attempting MIRRORS if available

WARNING: Failed to fetch URL http://ftp.gnu.org/gnu/binutils/binutils-2.24.tar.bz2, attempting MIRRORS if available

WARNING: Failed to fetch URL http://www.multiprecision.org/mpc/download/mpc-1.0.2.tar.gz, attempting MIRRORS if available

ERROR: Fetcher failure: Fetch command failed with exit code 4, output:

wget: unable to resolve host address 'www.multiprecision.org'

ERROR: Function failed: Fetcher failure for URL: 'http://www.multiprecision.org/mpc/download/mpc-1.0.2.tar.gz'. Unable to fetch URL from any source.

ERROR: Logfile of failure stored in: /home/swapnil/fsl-release-bsp/build-x11/tmp/work/x86_64-linux/libmpc-native/1.0.2-r0/temp/log.do_fetch.25793

ERROR: Task 3407 (virtual:native:/home/swapnil/fsl-release-bsp/sources/poky/meta/recipes-support/libmpc/libmpc_1.0.2.bb, do_fetch) failed with exit code '1'

pfc8pfc8 · ‎05-31-2017

Hello,

Very usefull tutorial, thanks!!

I want to point something that may need a correction in the code you give to us CurtisWald.

The name of the folder in the host computer does not match in the next two steps, and it gives an error.

Step1:

Create a soft link in the default web server directory to the rpm directory from the build. Note: Please update $HOME/<build> to your actual location:

$ ln -s $HOME/<build>/tmp/deploy/rpm /var/www/imx6q

Step2:

1. Add channels
To add the packages from the host to your target, the smart channel --add is used:
Please enter the IP adress of your server, replacing SERVERIP below:

smart channel --add all type=rpm-md name=all baseurl=http://SERVERIP/imx6qsd/all
smart channel --add cortexa9hf_vfp_neon type=rpm-md name=cortexa9hf_vfp_neon baseurl=http://SERVERIP/imx6qsd/cortexa9hf_vfp_neon
smart channel --add imx6qsabresd type=rpm-md name=imx6qsabresd baseurl=http://SERVERIP/imx6qsd/imx6qsabresd

Cheers.

CurtisWald · ‎05-31-2017

Updated initial post, thanks.

rans · ‎11-15-2017

Thank you for this post!

But Isn't there a web server "for everyone" (in world wide web...) to use rpm's for imx6 with smart (just like with normally use "apt-get" in ubunbtu) ?

Thanks,

Ran

yihan_cheng · ‎02-17-2020

Hello,

I would like to ask if the SMART package management support the S32XX Platform?
Since i'm using the s32g Platform and I do excaly the same steps as the tutorial said. I didn't got any error, however when I successfully boot the SD card and insert it into the slot of my board. In the rootfs, I can't find the smart excuable file and the konsol said there is no command called smart.

Thanks,

Yihan

CurtisWald · ‎02-17-2020

Hello,

Yocto Project 2.4 Rocko release smart has been replaced with dnf:

https://www.yoctoproject.org/docs/2.4/mega-manual/mega-manual.html

24.11.8. Package Management Changes¶<https://www.yoctoproject.org/docs/2.4/mega-manual/mega-manual.html#migration-2.3-package-management-changes>

The following package management changes took place:

Smart package manager is replaced by DNF package manager. Smart has become unmaintained upstream, is not ported to Python 3.x. Consequently, Smart needed to be replaced. DNF is the only feasible candidate.

If the Yocto Project you are using is 2.4 or later, recommend using dnf.

yihan_cheng · ‎02-18-2020

Hello Curtis,

Thank you for the Rely. I got some error after use the command "bitbake package-index" in the <build-directory>

ERROR: package-index-1.0-r0 do_package_index: Error executing a python function in exec_python_func() autogenerated:

.

Subprocess output:
Temporary repodata directory /home/cheng/fsl-auto-yocto-bsp/build_s32g274aevb/tmp/deploy/rpm/.repodata/ already exists! (Another createrepo process is running?)

ERROR: package-index-1.0-r0 do_package_index: Function failed: do_package_index
ERROR: Logfile of failure stored in: /home/cheng/fsl-auto-yocto-bsp/build_s32g274aevb/tmp/work/aarch64-fsl-linux/package-index/1.0-r0/temp/log.do_package_index.39272
ERROR: Task (/home/cheng/fsl-auto-yocto-bsp/sources/poky/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'

Summary: 1 task failed:
/home/cheng/fsl-auto-yocto-bsp/sources/poky/meta/recipes-core/meta/package-index.bb:do_package_index
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

Do you know how to solve it?

Thanks!

Yihan

CurtisWald · ‎02-18-2020

Hello Yihan,

From the info you shared:

directory /home/cheng/fsl-auto-yocto-bsp/build_s32g274aevb/tmp/deploy/rpm/.repodata/

already exists!

Remove that directory and try again?

Maybe something like:

$ rm -fr /home/cheng/fsl-auto-yocto-bsp/build_s32g274aevb/tmp/deploy/rpm/.repodata/

$ cd $BUILDDIR

$ bitbake package-index

I have not worked with S32 much, but for i.MX I can share some steps that worked for me:

On Ubuntu 64-bit host (18.04 currently)

Web Server – there are a few out there, this one worked for me:

$ sudo apt-get install python-twisted-web

Build your image, for example on i.MX this recipe:

$ cd $BUILDDIR

$ bitbake imx-image-full

Create the meta data for dnf:

$ bitbake package-index

Start web server: (Ubuntu host IP address 10.0.0.100, used on development system below)

$ twistd web –patch tmp/deploy/rpm –port tcp:8080

At this point, web server started with recipes from imx-image-full available for dnf.

On development system:

The /etc/yum.repos.d directory is where dnf default looks for information that contain the repositories.

I copy and paste the following in the console of the development system to setup dnf. You can modify for your setup:

mkdir /etc/yum.repos.d

cd /etc/yum.repos.d

cat > imx.repo << END

name=i.MX rpm repository

baseurl=http://10.0.0.100:8080

enabled=1

metadata_expire=0

gpgcheck=0

END

Last setup step is to create the local cache on the development system:

$ dnf makecache

Now you can “dnf search package” and “dnf install package”.

If on your development system you want to add something the overall flow is:

On Host:

$ bitbake newRecipe

$ bitbake package-index - need to update the meta data for dnf

On development system:

$ dnf update or dnf makecache

$ dnf install newRecipe

That flow had worked for me, good luck.

-Curtis

Yocto Project Package Management - smart

Yocto Project Package Management - smart