AI and Machine Learning Course #1: Hardware and Software Setup

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

AI and Machine Learning Course #1: Hardware and Software Setup

No ratings

AI and Machine Learning Course #1: Hardware and Software Setup

1. INTRODUCTION

In this article we are going to discuss about the HW and SW prerequisites needed to complete this course successful. At the end of this session the hardware setup and software environment should be ready for running a deep learning examples in simulations and on the real target represented by the NXP S32V234 Vision Processor

This article explains the following topics:

  1. How to configure the SBC-S32V234 Evaluation Board for Machine Vision and Machine Learning applications;
  2. How to install the NXP Vision Toolbox from MATLAB Add-on Explorer via MathWorks File Exchange;
  3. How to generate a valid license from NXP website, free-of-cost, to activate the NXP Vision Toolbox into MATLAB;
  4. How to install the NXP Vision SDK package that contains GCC and APU Compilers, optimized kernels and functions used for HW acceleration on S32V processors;
  5. How to setup your PC environment to perform a successful cross compilation of the code generated from MATLAB;
  6. How to prepare an SD-CARD with NXP official pre-built u-boot and Linux images required to boot up the HW platform using the Vision SDK for S32V
  7. How to install and configure additional MATLAB toolboxes what are required for this course

Before we start with specific details, please make yourself familiar with the main features supported by NXP Vision ToolboxThe NXP Vision Toolbox for S32V234 is an NXP proprietary tool that is designed to helps you:

  • Test vision algorithms using NXP Vision SDK functions in the MATLAB environment for a complete development, simulation and execution on the NXP targets by generating the C++ code directly from m-scripts using nxpvt_codegen()
  • Program the NXP APEX cores directly from MATLAB environment using Apex Core Framework graphs
  • Configure the NXP S32V Targets to enable code deployment directly from MATLAB environment and execute vision algorithm on NXP S32V Evaluation Boards
  • Fast evaluation of NXP solutions using ready-to-run examples derived from MATLAB Computer Vision System Toolbox and Deep Learning Toolbox
  • Build examples using pretrained/retrained MATLAB Convolutional Neural Networks in and deploy them on the NXP S32V234 boards with just a few lines of code

2. SBC-S32V234 Hardware Overview

 

Throughout this course we are going to use the SBC type of S32V evaluation boards. Anyhow if you have the more expensive version of the S32V234EVB2 you can still follow up these articles, the only differences being the HW setup which can be found in the toolbox documentation and help: the Quick Start Guide that is also attached here for your reference.

The SBC-S32V234 is a cost-competitive evaluation board and development platform engineered for high-performance, safe computation-intensive front vision, surround vision, and sensor fusion applications. Developed in partnership with MicroSys and based on the Arm® Cortex®-A53 based S32V processors, it has an efficient form factor while covering most of the uses cases available for the S32V234. The SBC-S32V234 is the recommended starting evaluation board for S32V.

The SBC-S32V234 main features are:

  • Video input: 2 x MIPI-CSI2 and Video output: RGB to HDMI converter
  • Communication: Gigabit Ethernet, 1x PCIE 2.0, 2 x CAN, 1 x LIN and 1 x UART
  • 2 GiB DDR Memory plus SD card slot and 16 GiB EMMC for NVM
  • 10 pin JTAG debug connector
  • 12V Power Supply connector

 

 pastedImage_1.png

For a more comprehensive view and a more detailed description regarding hardware specifications please check the SBC User Manual or visit this NXP webpage for the S32V Data Sheet. For a Quick Start Guide for the S32V234-SBC board please check this link: https://www.nxp.com/docs/en/quick-reference-guide/Quick-Start-Guide-SBC-S32V234.pdf

 

For this course on Machine Learning we are going to make use of the following peripherals and components hence make sure you have all of them available and you are familiar with their intended scope:

  1. At least 4GB class 10 micro SD-card that will be configured in the next section to boot up and initialize the platform. This needs to be inserted into the micro SD card slot on the SBC. In addition, for initial configuration you need an SD-card reader for connecting it with your PC;
  2. S32V-SONYCAM camera is used for capturing the video frames used for computer vision processing and deep learning applications. The camera needs to be inserted into the MIPI-A port on the SBC board;
  3. A CAT-5 ethernet cable will be used for downloading the application via TCP/IP or for getting video frames from the S32V on board camera to be processed in MATLAB;
  4. A microUSB cable that will connect the SBC-S32V234 with your Host PC will be used for finding the IP of the board and other verifications over the UART terminal;
  5. Connect a LCD monitor via HDMI cable with S32V234 SBC. The monitor will be used to display the computation results.
  6. 12v power supply

For more details please review the SBC-S32V User Manual.

 

2. Software Overview

 

Please read carefully this section since it contains all sort of tips and trick to have working setup for being able to generate the code from MATLAB scripts for NXP S32V and also to allow you perform various interactions with the real target. Due to the complexity of the software used to configure and control this processor we have to deal with 3 types of deliveries:

  1. Software delivered by MathWorks:
    • MATLAB (we assume you have already installed and configure this). As a hint, please make sure the MATLAB is installed in a path with empty spaces;
    • MATLAB Coder is the key component that allows us to generate the C++ code that will be cross-compiled to be executed on the target;
    • Image Processing
    • Computer Vision System allows us to perform various interactions with the data captured from the camera;
    • Deep Learning Toolbox allows us to use pre-trained networks or re-train and re-purpose them for other scenarios;
    • and various other support packages that are detailed in the next chapters;
  2. Software delivered by NXP:
    • NXP Vision Toolbox as S32V embedded target support and plug in for MATLAB environment to allow code generation and deployment. Make sure you install this toolbox in MATLAB 2018b and in a path without spaces.
    • NXP Vision SDK as primary source of optimized function, kernels, libraries and cross compilers for S32V. Make sure you install this package in a path without spaces otherwise the cross compilation will fail.
  3. Open Source Software
    • ARM COMPUTE library
    • various programs required to perform generic tasks: putty UART terminal, SD card formater, etc

I think now you get a better picture why some manual steps are needed to configure host PC to address all software dependencies. Hence, let's start the process...

2.1 Installation and Configuration for NXP Support Package for S32V234

For convenience a step-by-step installer guide is available on MathWorks’s File Exchange website. Open MATLAB and select Get Add-Ons:
pastedImage_16.png

  1. Once the Add-On Explorer window opens, search fornxp vision toolbox s32v
    pastedImage_18.png
  2. Select the NXP Support Package for S32V234 and click on Add button to start the installation of the installer guide into your MATLAB instance.
    pastedImage_19.png
  3. Wait until the toolbox is installed and then click on Open Folder button.
    pastedImage_21.png
  4. Run the NXP_Support_Package_S32V234 command in your MATLAB console to start the Installer Guide.
    pastedImage_22.png
  5. The NXP Support Package for S32V234 - Installer Guide User Interface is started
    pastedImage_23.png

The Installer Guide contains instructions for downloading, installing and verification of all software components required for being able to develop vision application with MATLAB for NXP S32V234 automotive vision processors:

  • Steps to download, install and verification of the NXP Vision Toolbox for S32V234
  • Steps to generate, activate and verification of the license for NXP Vision Toolbox for S32V234
  • Steps to download and install NXP Vision SDK package
  • Steps to configure the software environment for code generation
  • Steps to download additional software

 

There are 2 main advantages of using this Installer Guide:

  • Each step completion is automatically checked by the tool. If the action is completed successfully, then the tool is going to mark it as green. If a particular step cannot be verified, then the tool will issue a warning or error and is going to highlight in red that particular step that needs more attention for user side.
  • Future updates will be made available via this online toolbox. In case you wish to keep your software up to date, then please install this into your MATLAB Add-ons and once a new update will be available your MATLAB instance will notify you.

The next screen capture shows how the Installer Guide notify user of successful or failed actions. At the end of installation all push buttons should be green.

pastedImage_24.png

 

 You can obtain the NXP Vision Toolbox for S32V234 by:

  1. Using the Installer guideGo To NXP Download Sitebutton
    pastedImage_26.png
  2. Go directly into your NXP Software Account and download the toolbox using this link

No matter which option is used, the NXP Vision Toolbox for S32V234 installation steps are similar: once you have the toolbox on your PC, double click on the *.mltbx file to start the MATLAB Add-ons installer that will automatically start the installation process.

You will be prompted with the following options:

  1. The NXP’s Vision Toolbox Installation Wizard dialog will appear. ClickInstallto proceed.
    pastedImage_27.png
  2. Indicate acceptance of the NXP Software License Agreement by selectingI agree to the terms of the licenseto proceed.
    pastedImage_29.png
  3. ClickOKto start the MATLAB installation process. The rest of the process is silent and under MATLAB control. All the files will be automatically copied into default Add-Ons folder within MATLAB
  4. The default location can be changed prior to installation by changing the Add-Ons path from MATLAB Preferences
    pastedImage_30.png
    After a couple of seconds, the NXP’s Vision Toolbox should be visible as a new Add-ons. pastedImage_31.png
    pastedImage_33.png
  5. More details about the NXP’s Vision Toolbox can be found by clicking on View Details
    pastedImage_34.png
  6. NXP Vision Toolbox documentation, help and examples are fully integrated with MATLAB development environment. Get more details by accessing the standard Help and Supplemental Software section 
  7. In case you are using the Installer Guide, then you have the option to check if the NXP Vision Toolbox is installed correctly on your MATLAB environment by simply clicking onVerify Vision Toolbox Installationbutton
    pastedImage_39.png
  8. After this step you should see all button related with Vision Toolbox Step 1, green

pastedImage_41.png

2.2 License Generation and Activation

The NXP Vision Toolbox for S32V234 is available free of charge, however, a valid license is required. You can obtain the NXP Vision Toolbox for S32V234 license free of charge by:

  1. Using the Installer guide “Generate License File” button
    pastedImage_46.png
  2. Go directly into your NXP Software Account and Generate the license using this link

Perform the following steps to obtain the NXP Vision Toolbox for S32V234 license. For the first-time log-in, theSoftware Terms and Conditionspage will be displayed. Click onI agreebutton to consent to the software license agreement.

In this section we presume, you already logged into your NXP account to download the toolbox prior to license generation step

 pastedImage_50.png

  1. Click onLicense KeystabpastedImage_51.png
  2. Verify if the correct tool and version are identified and then check the box and click onGeneratepastedImage_54.png
  3. Select Disk Serial Number or Ethernet address as theNode Host ID”. If you do not know your Disk Serial Number nor the Ethernet address then check the link available on this page with details about License Generation. Enter a name for license to help managing them in case you need to use the Vision Toolbox on multiple computers. (Optional)pastedImage_58.png
  4. Click on “Generatebutton to get the license. Verify if the information is correct: Toolbox version, expiration date, Node Host IDpastedImage_59.png
  5. Either click on “Save All” or copy and paste the file into a text editor, and save the file as “license.dat” into the “Vision Toolbox installed directory\license” folder.
    pastedImage_60.png

In case you are using the Installer Guide, then you can save the license file anywhere and use the “Activate NXP Vision Toolbox” option to make sure the license is copied correctly in the appropriate toolbox location. Check if the license file is installed correctly by using the “Verify Vision Toolbox License” button. If everything is ok, then the Installer Guide will confirm the action.

pastedImage_61.png

Alternatively, you can check from command line is the license for NXP Vision Toolbox is activated. Run the command nxpvt_license_check. If there are issues with the license, this command will return the root-cause.

pastedImage_62.png

 

2.3 Installation of NXP Vision SDK and Build Tools

All the code generated by NXP Vision Toolbox is based on S32V234 Vision SDK package. This software package is also free of charge and apart of optimized kernels and libraries for the S32V automotive vision processors, it also contains the build tools to cross-compile the MATLAB generated code to ARM A53 and APEX cores. You can obtain the S32V234 Vision SDK free of charge by:

  1. Using the Installer guideGo To VSDK Download Sitebutton
    pastedImage_63.png
  2. Go directly to NXP website

 

Perform the following steps to obtain and install the S32V234 Vision SDK and NXP Build Tools:

  1. Download the Vision SDK RTM v1.3.0 on your PC. Due to the size of the package this might take a while.pastedImage_64.png
  2. Once the VisionSDK_S32V2_RTM_1_3_0.exe download is finished, selectInstall VSDK and A53/APU Compilersoption in the Installer Guide UI.
  3. Select the exe file and wait for the Vision SDK Install Anywhere to start. pastedImage_66.png
  4. Make sure you follow all the steps and install the:
    • NXP APU Compiler v1.0 – used to compile the generated code for APEX Vision Accelerator
    • NXP ARM GNU Compilers – used to compile the generated code for ARM A53
    • MSYS2 – used to configure the bootable Linux image and to download the actual vision application to the S32V234 Evaluation BoardpastedImage_67.png

2.4 Environment Setup

The last step required for software configuration is to set two system or user environmental variables APU_TOOLS and S32V234_SDK_ROOT that points to:

APU_TOOLS= C:/NXP/APU_Compiler_v1.0

S32V234_SDK_ROOT = C:/NXP/VisionSDK_S32V2_RTM_1_3_0/s32v234_sdk

Ensure system or user environment variables, corresponding to the compiler(s) you have installed, are defined to compiler path value as shown below:

pastedImage_68.png

pastedImage_69.png

Paths shown are for illustration, your installation path may be different. Once environmental variables are setup you will need to restart MATLAB to use these variables.

An alternative for setting the system paths manually is the “Set the environment variables” option from the NXP Vision Toolbox support package installer:

pastedImage_70.png

 

If the MATLAB is open with Administrator rights, then the “Set system wide” can be used to set the system variables. Othervise (most of the cases) use “Set user wide” to setup the environment variables

In order to use the Convolution Neural Networks the ARM_COMPUTELIB variable should also be set to point to the top of the arm_compute installation. More on the ARM Compute library installation in the following chapter.

2.5 SD-Card Configuration

The entire procedure for configuration and booting up the platform is described in the Vision SDK manuals. Unfortunately, not everyone has access to a Host PC with Linux OS to configure an SD card (formatting, uboot, filesystem, linux image copy).  To do this, you must log into your NXP account at www.nxp.com and download the SD card images:

pastedImage_2.png

  1. From the My account page go to “Software Licensing and Support”, then click on “View accounts” from the Software accounts panelpastedImage_3.png
  2. Click on the “Automotive SW – Vision Software” link:pastedImage_5.png
  3. Select the latest SDK (at the being SW32V23-VSDK001-RTM-1.3.0 pastedImage_6.png
  4. Agree with the terms and conditions:pastedImage_7.png
  5. Then click on the SD card image based on Yocto:pastedImage_9.png

Unpack the downloaded file and the build_content/v234_linux_build/s32v234sbc/’ folder will contain the sbc SD-card image which can be written directly from MATLAB.

Follow the next steps to create a bootable SD card for S32V234 SBC evaluation board:

  1. Begin by inserting a microSD card with at least 4GB capacity in your Host PC running Windows OS. The Windows OS should be able to recognize the SD card and assign a drive letter (e.g.: “D:”)pastedImage_4.png
  2. From MATLAB command window run the command: 
      nxpvt_create_target('sdcard-sbc.tar.bz2', 'D:');

    This example assumes you have untar the SD Card archive downloaded from the NXP website and you run the nxpvt_create_target command from the same directory as sdcard-sbc.tar.bz2 image

    This command will format the card and then it is going to copy all the required files from the *.bz2 image to the SD Card for booting up the Linux on S32V234 SBC.
  3. The copying process might take a while depending on the SD Card class type. During the process the following message will be shown on the screen.Wait until the copying process is finished and the “Image writing done” message is displayed on the MATLAB command prompt
    pastedImage_9.png
    pastedImage_11.png
    pastedImage_12.png
  4. After the copying process is completed, you should be able to see an additional drive mapped on your system (e.g. E) that cannot be accessed since it is an ext3 file system type.
    pastedImage_14.png
  5. Check that the initial mapped drive (e.g. D) contains: Image and s32v234-sbc file
    pastedImage_15.png
  6. Remove the SD card from the Host PC and check the next section for details on how to bootup the S32V234 SBC Evaluation Board the S32V234 Evaluation Board Configuration

 

Before running any example on the S32V234 SBC you need to perform the following steps:

  1. Insert the micro SD-card that has been configured in the previous section into the micro SD card slot
  2. Insert the Sony camera into the MIPI-A port. The Sony camera is used for capturing the video frames used for computer vision processing
  3. Insert an Ethernet cable in the ETH port. This will be used for downloading the application via TCP/IP
  4. Connect the S32V234 SBC via a microUSB cable with your Host PC. This is used for finding the IP of the board.
  5. Connect a LCD monitor via HDMI cable with S32V234 SBC
  6. Power on the board

2.6 Setting Up Additional Toolboxes and Utilities

The ARM Compute Library is a collection of low-level functions optimized for Arm CPU and GPU architectures targeted at image processing, computer vision, and machine learning. It is available free of charge under a permissive MIT open source license. The library’s collection of functions includes:

  • Basic arithmetic, mathematical, and binary operator functions
  • Color manipulation (conversion, channel extraction, and more)
  • Convolution filters (Sobel, Gaussian, and more)
  • Canny Edge, Harris corners, optical flow, and more
  • Pyramids (such as Laplacians)
  • HOG (Histogram of Oriented Gradients)
  • SVM (Support Vector Machines)
  • H/SGEMM (Half and Single precision General Matrix Multiply)
  • Convolutional Neural Networks building blocks (Activation, Convolution, Fully connected, Locally connected, Normalization, Pooling, Soft-max)

Download ARM Compute by going to this site:

https://github.com/ARM-software/ComputeLibrary

The toolbox examples were built using version 18.03, so download this one to avoid any backward or forward compatibility issues - scroll down until you find Binaries section like in the image below:

pastedImage_71.png

After downloading and unpacking the ARM Compute image the ARM_COMPUTELIB should point to the top of the installation folder:

pastedImage_72.png
pastedImage_73.png

The ARM Compute library should contain the linux-arm-v8a-neon with the correct libraries:

pastedImage_74.png

To be able to run the CNN examples in the toolbox, the following MATLAB Add-Ons should be installed :

  1. Deep Learning Toolbox
  2. Deep Learning Toolbox™ Model for GoogLeNet Network
  3. Deep Learning Toolbox™ Model for AlexNet Network
  4. Deep Learning Toolbox™ Model for SqueezeNet Network
  5. MATLAB Coder Interface for Deep Learning Libraries

pastedImage_75.png

3. Conclusions

 

At this point you should be able to run all examples in the NXP Vision toolbox, including the ones containing Convolutional Neural Networks

Your setup should now be configured with:

  • NXP Vision SDK package (libraries and compilers)
  • NXP Vision Toolbox MATLAB Add-on for S32V processor
  • MATLAB environment ready for CNN simulation and code generation
  • SBC-S32V234 Evaluation Board ready to run applications from MATLAB
Labels (1)
Attachments
Comments

Hi,

it says I shall use the SD card image based on Yocto. Can I use the Debian image instead? I tried the Debian image (bsp19-debian-stretch-s32v234sbc.tar.bz2) but I can't login with "root". Do you know the correct user name and password for the pre-built SD card image based on Debian?

Capture.JPG

And there is another question. Is it possible to run ROS on this board? 

Regards,

Hongjun Wang

Hi, hongjun.wang@mathworks.cn

You can use the debian version as well, but you have to perform a few extra steps and some of the features of the NXP Vision Toolbox, like automatically deploying on target and running bash commands from MATLAB will not be available for the moment. We will add support for debian as well in the near future. Nevertheless, you can still just copy the .elf resulted from codegen and the required extra files. I think it would be best to get the entire vision_toolbox/examples/data folder and have the whole folder in the same location as the .elf you would like to run. 

The user and password for the debian distribution is vision and vis, respectively. You then need to enable networking:

Select, if you are using static or dhcp

/etc/network/interfaces
auto eth0
# DHCP
iface eth0 inet dhcp
# Static
#iface eth0 inet static
# address 10.0.0.2
# netmask 255.255.255.0
#network 10.170.79.0
#broadcast 10.170.79.255
#gateway 10.170.79.254
#dns-* options are implemented by the resolvconf package, if installed
#dns-nameservers 10.170.72.1 10.210.0.3
#dns-search freescale.net

Now start your interface

sudo ifup eth0
sudo dhclient

Run the dhclient command just to be sure. Now try to ping some public domain and your network should be available. 

Then you should try to do:

sudo apt-get install vsdk-demos

It should install the dependencies and get things running. If you encounter any issues, please reply.

Thank you,

Vlad Dascau

 

Hi hongjun.wang@mathworks.cn‌,

Regarding installing ROS, we haven't tried that yet, but being that this is a Debian Stretch distribution it might be feasible. We will try it as well.

Thank you,

Vlad Dascau

Hi,

According to the above steps to make SD card, putty cannot input and has no information.Can you help me find out the problem? Thank you

pastedImage_1.png

pastedImage_3.png

pastedImage_2.png

Hi 751301185@qq.com‌,

Are you using the SBC board or the EVB board ? The steps you've taken are correct and it should work. Can you please check if maybe there's something wrong with the serial connection ? We've experienced something similar that was fixed by changing the USB port and MicroUSB cable. Also the connection in PuTTy needs to be opened with 115200 speed, not just the configuration in Device Manager: 

pastedImage_1.png

Please let me know if any of these work. If not, we will try to investigate it further.

Thank you,

Vlad Dascau

Hi,Vlad Dascau

I use EVB board.  When I follow this link to make a card, it can start working.

HOWTO: Prepare A SD Card For Linux Boot Of S32V234-EVB Using BSP From VSDK 

pastedImage_4.pngpastedImage_5.png

When I create a bootable SD card for S32V234 EVB evaluation board based on Yocto,Through the following methods:

AI and Machine Learning Course #1: Hardware and Software Setup 

PuTTY  has no input.

pastedImage_6.pngpastedImage_7.png

Is this related to USB port and MicroUSB cable?

Thank you,

     lilei

No, it is not related to USB port and MicroUSB cable if you are saying that you've got it working with the alternative procedure. nxpvt_create_target is writing the card with a tool that is similar to the dd command in linux. The sizes of the image file and dtb you've attached are the same as the ones I have on my working card so I assume the image was written correctly. Are you using the same card/card reader/usb port that worked with the other method ? 

Thanks,

Vlad Dascau

Hi,Vlad Dascau

I use the same SD card, the same USB  port

Well, for the moment I would suggest using the card that you have created with the other method, since this should work fine. Otherwise, we can continue to investigate it together if you would like that. 

Thank you,

Vlad Dascau

The program compiled by MATLAB, when I run the application program on the s32v board, the following errors occur:

./face_detection.elf: error while loading shared libraries: libopencv_calib3d.so.3.4: cannot open shared object file: No such file or directory

Iulian Bulancea suggested that I use the Yocto SD card image provided with Vision SDK 1.4.0. So there are some problems mentioned above.

pastedImage_2.png

pastedImage_7.png

Kind regards,

    lilei

The two ways of writing an image should be equivalent. Did you use the same yocto version from sdk 1.4.0 when writing the card with the virtual machine ? If so, it should work. Can you please do a ls in the lib folder and a uname -a in the command line:

pastedImage_1.png

Thank you,

Vlad Dascau

Hello Vlad Dascau,

It is mentioned in this document that at least 4GB class 10 micro SD-card is needed to configure the bootup and initialize the platform of SBC-S32V234. My question is: What is the maximum size of the SD card I can use on the SBC-S32V234 board? The NXP provided SD card is 4 GB in size and its almost full in our case. Therefore I want to replace this 4 GB micro SD card with 64 GB or 128 GB. 

Best Regards

Md Anayatullah

Hello md.anayatullah@stud.hs-kempten.de‌,

Yes, you should be able to create an image with a 128GB SD-Card. If you need any assistance, please let us know.

Thank you,

Vlad Dascau

Hi md.anayatullah@stud.hs-kempten.de‌,

I'll paste here the answer that I got from brent_wylie‌:

On closer inspection of the device reference manual, the S32V2 supports both SD High Capacity (SDHC) and SD Extended Capacity (SDXC) cards:

Compatible with the SD Memory Card Specification version 3.0 and supports the Extended Capacity SD Memory Card

 

SDHC cards typically have a capacity from 4GB to 32GB, and SDXC cards have a capacity starting at 32GB with a theoretical maximum of 2TB.

Both are supported by the controller.

Best regards,

Paul

Version history
Last update:
‎09-10-2020 03:35 AM
Updated by: