1. INTRODUCTION
In this article we are going to discuss about the HW and SW prerequisites needed to complete this course successful. At the end of this session the hardware setup and software environment should be ready for running a deep learning examples in simulations and on the real target represented by the NXP S32V234 Vision Processor.
This article explains the following topics:
- How to configure the SBC-S32V234 Evaluation Board for Machine Vision and Machine Learning applications;
- How to install the NXP Vision Toolbox from MATLAB Add-on Explorer via MathWorks File Exchange;
- How to generate a valid license from NXP website, free-of-cost, to activate the NXP Vision Toolbox into MATLAB;
- How to install the NXP Vision SDK package that contains GCC and APU Compilers, optimized kernels and functions used for HW acceleration on S32V processors;
- How to setup your PC environment to perform a successful cross compilation of the code generated from MATLAB;
- How to prepare an SD-CARD with NXP official pre-built u-boot and Linux images required to boot up the HW platform using the Vision SDK for S32V
- How to install and configure additional MATLAB toolboxes what are required for this course
Before we start with specific details, please make yourself familiar with the main features supported by NXP Vision Toolbox. The NXP Vision Toolbox for S32V234 is an NXP proprietary tool that is designed to helps you:
- Test vision algorithms using NXP Vision SDK functions in the MATLAB environment for a complete development, simulation and execution on the NXP targets by generating the C++ code directly from m-scripts using nxpvt_codegen()
- Program the NXP APEX cores directly from MATLAB environment using Apex Core Framework graphs
- Configure the NXP S32V Targets to enable code deployment directly from MATLAB environment and execute vision algorithm on NXP S32V Evaluation Boards
- Fast evaluation of NXP solutions using ready-to-run examples derived from MATLAB Computer Vision System Toolbox and Deep Learning Toolbox
- Build examples using pretrained/retrained MATLAB Convolutional Neural Networks in and deploy them on the NXP S32V234 boards with just a few lines of code
2. SBC-S32V234 Hardware Overview
Throughout this course we are going to use the SBC type of S32V evaluation boards. Anyhow if you have the more expensive version of the S32V234EVB2 you can still follow up these articles, the only differences being the HW setup which can be found in the toolbox documentation and help: the Quick Start Guide that is also attached here for your reference.
The SBC-S32V234 is a cost-competitive evaluation board and development platform engineered for high-performance, safe computation-intensive front vision, surround vision, and sensor fusion applications. Developed in partnership with MicroSys and based on the Arm® Cortex®-A53 based S32V processors, it has an efficient form factor while covering most of the uses cases available for the S32V234. The SBC-S32V234 is the recommended starting evaluation board for S32V.
The SBC-S32V234 main features are:
- Video input: 2 x MIPI-CSI2 and Video output: RGB to HDMI converter
- Communication: Gigabit Ethernet, 1x PCIE 2.0, 2 x CAN, 1 x LIN and 1 x UART
- 2 GiB DDR Memory plus SD card slot and 16 GiB EMMC for NVM
- 10 pin JTAG debug connector
- 12V Power Supply connector
For a more comprehensive view and a more detailed description regarding hardware specifications please check the SBC User Manual or visit this NXP webpage for the S32V Data Sheet. For a Quick Start Guide for the S32V234-SBC board please check this link: https://www.nxp.com/docs/en/quick-reference-guide/Quick-Start-Guide-SBC-S32V234.pdf
For this course on Machine Learning we are going to make use of the following peripherals and components hence make sure you have all of them available and you are familiar with their intended scope:
- At least 4GB class 10 micro SD-card that will be configured in the next section to boot up and initialize the platform. This needs to be inserted into the micro SD card slot on the SBC. In addition, for initial configuration you need an SD-card reader for connecting it with your PC;
- S32V-SONYCAM camera is used for capturing the video frames used for computer vision processing and deep learning applications. The camera needs to be inserted into the MIPI-A port on the SBC board;
- A CAT-5 ethernet cable will be used for downloading the application via TCP/IP or for getting video frames from the S32V on board camera to be processed in MATLAB;
- A microUSB cable that will connect the SBC-S32V234 with your Host PC will be used for finding the IP of the board and other verifications over the UART terminal;
- Connect a LCD monitor via HDMI cable with S32V234 SBC. The monitor will be used to display the computation results.
- 12v power supply
For more details please review the SBC-S32V User Manual.
2. Software Overview
Please read carefully this section since it contains all sort of tips and trick to have working setup for being able to generate the code from MATLAB scripts for NXP S32V and also to allow you perform various interactions with the real target. Due to the complexity of the software used to configure and control this processor we have to deal with 3 types of deliveries:
- Software delivered by MathWorks:
- MATLAB (we assume you have already installed and configure this). As a hint, please make sure the MATLAB is installed in a path with empty spaces;
- MATLAB Coder is the key component that allows us to generate the C++ code that will be cross-compiled to be executed on the target;
- Image Processing
- Computer Vision System allows us to perform various interactions with the data captured from the camera;
- Deep Learning Toolbox allows us to use pre-trained networks or re-train and re-purpose them for other scenarios;
- and various other support packages that are detailed in the next chapters;
- Software delivered by NXP:
- NXP Vision Toolbox as S32V embedded target support and plug in for MATLAB environment to allow code generation and deployment. Make sure you install this toolbox in MATLAB 2018b and in a path without spaces.
- NXP Vision SDK as primary source of optimized function, kernels, libraries and cross compilers for S32V. Make sure you install this package in a path without spaces otherwise the cross compilation will fail.
- Open Source Software
- ARM COMPUTE library
- various programs required to perform generic tasks: putty UART terminal, SD card formater, etc
I think now you get a better picture why some manual steps are needed to configure host PC to address all software dependencies. Hence, let's start the process...
2.1 Installation and Configuration for NXP Support Package for S32V234
For convenience a step-by-step installer guide is available on MathWorks’s File Exchange website. Open MATLAB and select Get Add-Ons:
- Once the Add-On Explorer window opens, search for “nxp vision toolbox s32v”
- Select the NXP Support Package for S32V234 and click on Add button to start the installation of the installer guide into your MATLAB instance.
- Wait until the toolbox is installed and then click on Open Folder button.
- Run the NXP_Support_Package_S32V234 command in your MATLAB console to start the Installer Guide.
- The NXP Support Package for S32V234 - Installer Guide User Interface is started
The Installer Guide contains instructions for downloading, installing and verification of all software components required for being able to develop vision application with MATLAB for NXP S32V234 automotive vision processors:
- Steps to download, install and verification of the NXP Vision Toolbox for S32V234
- Steps to generate, activate and verification of the license for NXP Vision Toolbox for S32V234
- Steps to download and install NXP Vision SDK package
- Steps to configure the software environment for code generation
- Steps to download additional software
There are 2 main advantages of using this Installer Guide:
- Each step completion is automatically checked by the tool. If the action is completed successfully, then the tool is going to mark it as green. If a particular step cannot be verified, then the tool will issue a warning or error and is going to highlight in red that particular step that needs more attention for user side.
- Future updates will be made available via this online toolbox. In case you wish to keep your software up to date, then please install this into your MATLAB Add-ons and once a new update will be available your MATLAB instance will notify you.
The next screen capture shows how the Installer Guide notify user of successful or failed actions. At the end of installation all push buttons should be green.
You can obtain the NXP Vision Toolbox for S32V234 by:
- Using the Installer guide “Go To NXP Download Site” button
- Go directly into your NXP Software Account and download the toolbox using this link
No matter which option is used, the NXP Vision Toolbox for S32V234 installation steps are similar: once you have the toolbox on your PC, double click on the *.mltbx file to start the MATLAB Add-ons installer that will automatically start the installation process.
You will be prompted with the following options:
- The NXP’s Vision Toolbox Installation Wizard dialog will appear. Click “Install” to proceed.
- Indicate acceptance of the NXP Software License Agreement by selecting “I agree to the terms of the license” to proceed.
- Click “OK” to start the MATLAB installation process. The rest of the process is silent and under MATLAB control. All the files will be automatically copied into default Add-Ons folder within MATLAB
- The default location can be changed prior to installation by changing the Add-Ons path from MATLAB Preferences
After a couple of seconds, the NXP’s Vision Toolbox should be visible as a new Add-ons. - More details about the NXP’s Vision Toolbox can be found by clicking on View Details
- NXP Vision Toolbox documentation, help and examples are fully integrated with MATLAB development environment. Get more details by accessing the standard Help and Supplemental Software section
- In case you are using the Installer Guide, then you have the option to check if the NXP Vision Toolbox is installed correctly on your MATLAB environment by simply clicking on “Verify Vision Toolbox Installation” button
- After this step you should see all button related with Vision Toolbox Step 1, green
2.2 License Generation and Activation
The NXP Vision Toolbox for S32V234 is available free of charge, however, a valid license is required. You can obtain the NXP Vision Toolbox for S32V234 license free of charge by:
- Using the Installer guide “Generate License File” button
- Go directly into your NXP Software Account and Generate the license using this link
Perform the following steps to obtain the NXP Vision Toolbox for S32V234 license. For the first-time log-in, the “Software Terms and Conditions” page will be displayed. Click on “I agree” button to consent to the software license agreement.
In this section we presume, you already logged into your NXP account to download the toolbox prior to license generation step
- Click on “License Keys” tab
- Verify if the correct tool and version are identified and then check the box and click on “Generate”
- Select Disk Serial Number or Ethernet address as the “Node Host ID”. If you do not know your Disk Serial Number nor the Ethernet address then check the link available on this page with details about License Generation. Enter a name for license to help managing them in case you need to use the Vision Toolbox on multiple computers. (Optional)
- Click on “Generate” button to get the license. Verify if the information is correct: Toolbox version, expiration date, Node Host ID
- Either click on “Save All” or copy and paste the file into a text editor, and save the file as “license.dat” into the “Vision Toolbox installed directory\license” folder.
In case you are using the Installer Guide, then you can save the license file anywhere and use the “Activate NXP Vision Toolbox” option to make sure the license is copied correctly in the appropriate toolbox location. Check if the license file is installed correctly by using the “Verify Vision Toolbox License” button. If everything is ok, then the Installer Guide will confirm the action.
Alternatively, you can check from command line is the license for NXP Vision Toolbox is activated. Run the command nxpvt_license_check. If there are issues with the license, this command will return the root-cause.
2.3 Installation of NXP Vision SDK and Build Tools
All the code generated by NXP Vision Toolbox is based on S32V234 Vision SDK package. This software package is also free of charge and apart of optimized kernels and libraries for the S32V automotive vision processors, it also contains the build tools to cross-compile the MATLAB generated code to ARM A53 and APEX cores. You can obtain the S32V234 Vision SDK free of charge by:
- Using the Installer guide “Go To VSDK Download Site” button
- Go directly to NXP website
Perform the following steps to obtain and install the S32V234 Vision SDK and NXP Build Tools:
- Download the Vision SDK RTM v1.3.0 on your PC. Due to the size of the package this might take a while.
- Once the VisionSDK_S32V2_RTM_1_3_0.exe download is finished, select “Install VSDK and A53/APU Compilers” option in the Installer Guide UI.
- Select the exe file and wait for the Vision SDK Install Anywhere to start.
- Make sure you follow all the steps and install the:
- NXP APU Compiler v1.0 – used to compile the generated code for APEX Vision Accelerator
- NXP ARM GNU Compilers – used to compile the generated code for ARM A53
- MSYS2 – used to configure the bootable Linux image and to download the actual vision application to the S32V234 Evaluation Board
2.4 Environment Setup
The last step required for software configuration is to set two system or user environmental variables APU_TOOLS and S32V234_SDK_ROOT that points to:
APU_TOOLS= C:/NXP/APU_Compiler_v1.0
S32V234_SDK_ROOT = C:/NXP/VisionSDK_S32V2_RTM_1_3_0/s32v234_sdk
Ensure system or user environment variables, corresponding to the compiler(s) you have installed, are defined to compiler path value as shown below:
Paths shown are for illustration, your installation path may be different. Once environmental variables are setup you will need to restart MATLAB to use these variables.
An alternative for setting the system paths manually is the “Set the environment variables” option from the NXP Vision Toolbox support package installer:
If the MATLAB is open with Administrator rights, then the “Set system wide” can be used to set the system variables. Othervise (most of the cases) use “Set user wide” to setup the environment variables
In order to use the Convolution Neural Networks the ARM_COMPUTELIB variable should also be set to point to the top of the arm_compute installation. More on the ARM Compute library installation in the following chapter.
2.5 SD-Card Configuration
The entire procedure for configuration and booting up the platform is described in the Vision SDK manuals. Unfortunately, not everyone has access to a Host PC with Linux OS to configure an SD card (formatting, uboot, filesystem, linux image copy). To do this, you must log into your NXP account at www.nxp.com and download the SD card images:
- From the My account page go to “Software Licensing and Support”, then click on “View accounts” from the Software accounts panel
- Click on the “Automotive SW – Vision Software” link:
- Select the latest SDK (at the being SW32V23-VSDK001-RTM-1.3.0
- Agree with the terms and conditions:
- Then click on the SD card image based on Yocto:
Unpack the downloaded file and the ‘build_content/v234_linux_build/s32v234sbc/’ folder will contain the sbc SD-card image which can be written directly from MATLAB.
Follow the next steps to create a bootable SD card for S32V234 SBC evaluation board:
- Begin by inserting a microSD card with at least 4GB capacity in your Host PC running Windows OS. The Windows OS should be able to recognize the SD card and assign a drive letter (e.g.: “D:”)
- From MATLAB command window run the command:
nxpvt_create_target('sdcard-sbc.tar.bz2', 'D:');
This command will format the card and then it is going to copy all the required files from the *.bz2 image to the SD Card for booting up the Linux on S32V234 SBC.This example assumes you have untar the SD Card archive downloaded from the NXP website and you run the nxpvt_create_target command from the same directory as sdcard-sbc.tar.bz2 image
- The copying process might take a while depending on the SD Card class type. During the process the following message will be shown on the screen.Wait until the copying process is finished and the “Image writing done” message is displayed on the MATLAB command prompt
- After the copying process is completed, you should be able to see an additional drive mapped on your system (e.g. E) that cannot be accessed since it is an ext3 file system type.
- Check that the initial mapped drive (e.g. D) contains: Image and s32v234-sbc file
- Remove the SD card from the Host PC and check the next section for details on how to bootup the S32V234 SBC Evaluation Board the S32V234 Evaluation Board Configuration
Before running any example on the S32V234 SBC you need to perform the following steps:
- Insert the micro SD-card that has been configured in the previous section into the micro SD card slot
- Insert the Sony camera into the MIPI-A port. The Sony camera is used for capturing the video frames used for computer vision processing
- Insert an Ethernet cable in the ETH port. This will be used for downloading the application via TCP/IP
- Connect the S32V234 SBC via a microUSB cable with your Host PC. This is used for finding the IP of the board.
- Connect a LCD monitor via HDMI cable with S32V234 SBC
- Power on the board
2.6 Setting Up Additional Toolboxes and Utilities
The ARM Compute Library is a collection of low-level functions optimized for Arm CPU and GPU architectures targeted at image processing, computer vision, and machine learning. It is available free of charge under a permissive MIT open source license. The library’s collection of functions includes:
- Basic arithmetic, mathematical, and binary operator functions
- Color manipulation (conversion, channel extraction, and more)
- Convolution filters (Sobel, Gaussian, and more)
- Canny Edge, Harris corners, optical flow, and more
- Pyramids (such as Laplacians)
- HOG (Histogram of Oriented Gradients)
- SVM (Support Vector Machines)
- H/SGEMM (Half and Single precision General Matrix Multiply)
- Convolutional Neural Networks building blocks (Activation, Convolution, Fully connected, Locally connected, Normalization, Pooling, Soft-max)
Download ARM Compute by going to this site:
https://github.com/ARM-software/ComputeLibrary
The toolbox examples were built using version 18.03, so download this one to avoid any backward or forward compatibility issues - scroll down until you find Binaries section like in the image below:
After downloading and unpacking the ARM Compute image the ARM_COMPUTELIB should point to the top of the installation folder:
The ARM Compute library should contain the linux-arm-v8a-neon with the correct libraries:
To be able to run the CNN examples in the toolbox, the following MATLAB Add-Ons should be installed :
- Deep Learning Toolbox
- Deep Learning Toolbox™ Model for GoogLeNet Network
- Deep Learning Toolbox™ Model for AlexNet Network
- Deep Learning Toolbox™ Model for SqueezeNet Network
- MATLAB Coder Interface for Deep Learning Libraries
3. Conclusions
At this point you should be able to run all examples in the NXP Vision toolbox, including the ones containing Convolutional Neural Networks.
Your setup should now be configured with:
- NXP Vision SDK package (libraries and compilers)
- NXP Vision Toolbox MATLAB Add-on for S32V processor
- MATLAB environment ready for CNN simulation and code generation
- SBC-S32V234 Evaluation Board ready to run applications from MATLAB
Hi,
it says I shall use the SD card image based on Yocto. Can I use the Debian image instead? I tried the Debian image (bsp19-debian-stretch-s32v234sbc.tar.bz2) but I can't login with "root". Do you know the correct user name and password for the pre-built SD card image based on Debian?
And there is another question. Is it possible to run ROS on this board?
Regards,
Hongjun Wang
Hi,
According to the above steps to make SD card, putty cannot input and has no information.Can you help me find out the problem? Thank you
Hi 751301185@qq.com,
Are you using the SBC board or the EVB board ? The steps you've taken are correct and it should work. Can you please check if maybe there's something wrong with the serial connection ? We've experienced something similar that was fixed by changing the USB port and MicroUSB cable. Also the connection in PuTTy needs to be opened with 115200 speed, not just the configuration in Device Manager:
Please let me know if any of these work. If not, we will try to investigate it further.
Thank you,
Vlad Dascau
Hi,Vlad Dascau
I use EVB board. When I follow this link to make a card, it can start working.
HOWTO: Prepare A SD Card For Linux Boot Of S32V234-EVB Using BSP From VSDK
When I create a bootable SD card for S32V234 EVB evaluation board based on Yocto,Through the following methods：
AI and Machine Learning Course #1: Hardware and Software Setup
PuTTY has no input.
Is this related to USB port and MicroUSB cable？
Thank you,
lilei
The program compiled by MATLAB, when I run the application program on the s32v board, the following errors occur：
./face_detection.elf: error while loading shared libraries: libopencv_calib3d.so.3.4: cannot open shared object file: No such file or directory
Iulian Bulancea suggested that I use the Yocto SD card image provided with Vision SDK 1.4.0. So there are some problems mentioned above.
Kind regards,
lilei
The two ways of writing an image should be equivalent. Did you use the same yocto version from sdk 1.4.0 when writing the card with the virtual machine ? If so, it should work. Can you please do a ls in the lib folder and a uname -a in the command line:
Thank you,
Vlad Dascau
