This article presents a set of APIs that can be used for the hardware management of the NXP Cup kit. It contains an API for communication with Pixy2, an API for controlling the rear and servo motors as well as some errors management. These APIs can be further modified to fit your hardware and software requirements easily, in case you will choose a different board or need to get more from what Pixy can offer.
If you do not want to worry about the hardware management of the kit, you might want to try this Model-based design[12] solution that offers you all the functions to control the hardware easily, without having to worry about what is under the hood. Also, keep in mind that you have access to the implementation, so you can further improve the API to tailor your needs.
OVERVIEW
Hardware
-
Components
-
Connections
-
Motor duty cycles
Software
Description
-
Pixy API
-
Motor control API
-
Error Displays
Design
-
Pixy API
-
Motor control API
-
Error Displays
Implementation details
-
Motor control
-
Pixy
Configurations
-
Board configs
-
LPSPI configs
-
PWM configs
Testing the API
-
Vector localisation
-
FreeMaster view of the lines
Build and run
References
HARDWARE
Components
-
Pixy2[1]
Pixy2 is a chip with a camera integrated that has line tracking and object recognition functions implemented. By connecting this to your board, your application will gain an “eye” that it can use for directing the car. Your board can communicate with Pixy to gain information about the surroundings seen by it.
-
DFRobot Kit[3]
This is a kit that can be used in the competition which contains a buildable chassis, 2 rear motors for controlling the speed of the car and a front servo motor. The rear motors are controlled via ESCs(Electronic speed controllers) and the servo-motor can be controlled directly through the cable it offers.
-
S32K146[4]
This is a microcontroller(MCU) produced by NXP from the series S32K1. The MCU will be the “brain” of your car where the algorithm is implemented and where the control of all the hardware is managed.
-
Battery
A 7.2V LiPo battery that is used to power all the hardware – MCU, motors and Pixy. Some of the hardware will be connected directly to this battery, while the rest will be connected through the MCU.
Connections
This subchapter will cover aspects about the physical connections between hardware components – what connections need to be made and how to adapt them to the MCU you will be using.
-
Pixy
Pixy will be connected to the board’s SPI through jumper cables. The pins that need to be connected from the board depend on the SPI module that you choose and the pins from Pixy that need to be connected can be found on their website[9]. Pixy is also connected to power through the board’s 5V and GND pins, but it can also be connected directly to the battery separately using the analog power pin on Pixy.
-
Rear Motors
The rear motors are connected through ESCs to the board. The ESCs are commanded by the board through the PWM modules. The ESCs are also used to power the motors, these ones being connected directly to the battery.
-
Servo Motor
The servo motor has a cable that can be used to connect to the board. That cable will offer both the connection to power through the 5V and a GND pins, as well as a PWM signal.
-
Board
The board is connected to the battery, therefore the jumper J107 will have to be put on the pins 1 and 2[4].
-
Battery
All direct connections to the battery are multiplexed with the use of a cable splitter.
Motor duty cycles
The duty cycles that a motor works within may vary from motor to motor. Although we control the motor using the duty cycle, that is not the aspect that determines the force or speed. What actually matters is the duration of the pulse that is sent in PWM. By combining different duty cycles with different frequencies we can obtain different pulse durations as well as different durations between the end of a pulse and the start of the next. The proper way to see how these affect your motors is to do some testing in order to see exactly what duty cycles to use for your motors and what the duration between 2 pulses should be so that your motor will be able to process the signal.
Servo-motor
The servo-motor translates durations of pulses into angles by using a torque. Therefore, for a given frequency of the signal, the more you increase the duty cycle of your PWM, the more it will turn right or left. The servo-motor used in this project turns to right for higher PWM duty cycles. These values are:
- 50Hz chosen frequency
- 0.03 minimum duty cycle: the wheels are in the leftmost position
- 0.075 medium duty cycle: the wheels are pointing forward
- 0.11 maximum duty cycle: the wheels are in the rightmost position
Rear motor
The rear motor translates PWM duration pulses into RPM(speed). Therefore, for a given frequency, when you increase the duty cycle of your PWM signal, the car wheels will start spinning faster. The values used are:
- 100Hz chosen frequency
- 0.1 minimum duty cycle: the car will go backwards on full speed
- 0.15 medium duty cycle: the car will stand still
- 0.2 maximum duty cycle: the car will go forward on full speed
SOFTWARE
For this project, the programming will occur on MATLAB Simulink using the following add-ons:
-
MATLAB Coder
-
Simulink Coder
-
Embedded Coder
-
Embedded Coder Support Package for ARM Cortex-M Processors
-
Stateflow
-
NXP Model-Based Design Toolbox for S32K1xx (or the desired target)
-
[optional] FreeMaster
DESCRIPTION
Pixy API
For a better understanding about how Pixy2 works, take a look at their official website[2] and for a better description of the API, check the official API for Arduino[5]. Also, if more functions need to be implemented from that API, you might want to check the packet reference of Pixy2 communication protocol[6]. Any other needed information about Pixy can be found on their website.
The API implemented offers the following functions:
- init(): used to check that the communication with Pixy can be established
- getFeatures(): used to get the features from the Line Tracking mode which are lines, intersections and barcodes – this one includes both getMainFeatures and getAllFeatures
- setLamp(): sets the lamp(LEDs) for a better illumination
- setMode(): sets the operation mode for the line tracking algorithm
- setDefaultTurn(): sets the angle for the default turn in an intersection
- setNextTurn(): sets the angle for the turn in the next intersection
- setVector(): sets the vector that Pixy will consider the main vector in the future
- reverseVector(): reverses the orientation of the main vector tracked
Note that this API is implemented for using the line tracking program that Pixy2 offers[8].
Motor Control API
For understanding how the motors can be controlled, try doing a quick research about Electronic Speed Controllers[7].
The API offers the following functions:
- set_speed(): sets the speed of one of the rear motors using percentages
- set_angle(): sets the turning angle of the car through the servo-motor using degrees
Error Displays
While the program is running, the green LED will blink to mark this event. Also, when the developer wants to mark that there was an error while talking to Pixy, they can use set_pixy_error() function to mark this event by turning the LED purple. This section can be further completed with other types of errors such as an error that marks when the motors are not connected to power.
Note: The constants of the model are declared in Model Workspace which can be found at MODELING > Model Workspace. Here you will see all constants used in the model along with their descriptions.
DESIGN
Pixy2 API
-
Data Section
There are 2 buffers used for communicating through SPI: one for sending data and one for receiving. The information will be stored in 6 variables, 3 arrays of data (vectors, barcodes and intersections) and 3 integers storing the counts of each of these elements. The arrays are filled with 0s in positions where there is no information, thus the previously mentioned counts are needed to mark the ends where information can be found. The information is stored similarly to how Pixy sends it.
- Vectors are represented as an array of 6 character(uint8) strings
- positions of the tail and head of the vector represented through x and y
- an identifier for the vector(index), in case there is more than one vector sent
- some flags
- Barcodes are represented as an array of 4 character(uint8) strings
- Position of the barcode represented through x and y
- Some flags
- The meaning of the barcode (code)
- Intersections
- Position of the intersection represented through x and y
- Number of lines found in the intersection
- One byte padding – this byte is ignored
- 6 lines represented in a contiguous memory area – each line has 4 bytes of data
- An identifier (index)
- 1 byte of padding
- The angle represented on 2 bytes as a signed integer in little endian byte ordering
-
Outputs section
The data received from Pixy is represented at the output of the Pixy API block.
-
Common functions
There are 3 functions used as helper functions in the API:
- cs_check(): checks the checksum validity in conformity with the serial protocol[6]
- talk_to_pixy(): communicates through SPI – sends a request and receives a reply from Pixy
- shift_recv_buffer(): since Pixy first needs to receive the request and process it before sending a response, the receive buffer will be filled with “garbage” information at the beginning. Thus, this functions seeks the sync bytes and shifts the whole buffer so that it will start with the beginning of the message sent by Pixy.
-
API functions
Each function is described at the beginning of this article.
Motor control API
-
Duty cycles
The functions in this API will compute duty cycles for the pwm modules that control each motor.
- DTCSVM – duty cycle for servo-motor
- DTC(L/R)M – duty cycle for the rear left and right motors
-
PWM Modules
The PWM modules control the PWM signals sent to the motors. The duty cycles are given as input.
-
API Functions
The functions are described at the beginning of this article.
Error displays
IMPLEMENTATION DETAILS
Motor Control
Since we have established the duty cycles for the motors, we can discuss about how they are used in the model:
-
Rear Motor
The value of the PWM for 0 speed will be summed with the value of the desired speed divided by a constant that converts speed in percentage to PWM duty cycle. This can be done because the speed of the motors is proportional to the variation in duty cycle.
-
Servo Motor
For the servo motor, the same principle applies. Note that if the servo motor is not symmetrical from right to left, this part of the program should be duplicated, to separate the calculation for right and left turns. The only difference will be in the constant used, since the variations in duty cycles will be different for the right and the left turns. For each turn – right or left – the turning angle can be assumed to be proportional to the angle of the turn.
Pixy
-
talk_to_pixy()
What is of interest here is the fact that we use a blocking function since we need to make sure that the receive buffer is filled before reading from it. In case you need it to be non-blocking, you can modify the SPI block from above, but this will require a use of interrupts to manage the end of the transmission.
-
getFeatures()
The API functions follow mostly the same route. Thus I will explain getFeatures() in order to understand how these functions work.
-
get_features_wr()
This function simply writes the request message to the send buffer. The details for each field are described in the communication protocol[6].
-
get_features_rd()
This function reads the buffer to find all the elements sent by Pixy. These include vectors, barcodes and intersections. The reading is done using this chart that first reads the type of an element, the count for them and then reads the elements in a for loop.
-
Main flow
As can be seen in the chart above, the flow will first check the requirements for a packet to be considered valid such as finding the sync sequence, getting the right response from Pixy(checking the type of the packet), having a correct checksum, etc.
CONFIGURATIONS
Board configs
If you are going to use another K1 board, you can keep the block and simply choose from the target drop-down list. For other boards, a different config block should be used.
LPSPI configs
This module is used for talking to Pixy. Based on their documentation[9] the required configurations for this block are the following:
From this block you can also configure the module of the SPI used, the pins you will connect the board through and the baud rate of the clock.
To initiate data transfer you need the following block:
You can configure the instance of SPI used, the selected chip and the type of transfer: blocking or non-blocking.
PWM configs
This block is used to configure the PWM modules and to give them the duty cycles desired.
The frequency chosen for each module is based on the types of motors controlled by each one.
TESTING THE API
Vector localisation
The model contains an example program that uses Pixy to localize the main vector on the ground. This program can be used when the car is placed on the track to compute the position of the line that Pixy establishes is the main vector in its view.
Furthermore, the program will run the car at a small speed of 5% in circles with a turning angle of 20 degrees to the left. Note that if Pixy fails to communicate in any way, the car will stop its engines and the turning angle will be set to 0.
Pos1 and pos2 are the resulting positions of the tail and the head of the vector. These are tridimensional vectors corresponding to coordinates in real world. Z(third dimension) represents the direction forward, X(first dimension) is the direction to the right and Y(second dimension) represents the direction upward.
For understanding the algorithm, I recommend reading about the pinhole camera model[10].
Below is an image where the geometry behind the algorithm can be found.
- OY and OZ will be the upward and forward directions of our space.
- C is the position of the camera
- S is the position of the object
- CAB is the camera’s field of view
- A1B1 is the projection plane where the images are formed (see the pinhole camera model) – in 2D it looks like a segment
- S1 is the position of the object in the projection plane
- F is where the focal center can be found on the projection plane
Note that this is a 2D model. The 3D model can be obtained by adding OX as a direction and by representing the horizontal field of view of the camera as well as the vertical one.
In order to get the vector OS – the position of our object in the real world, we can see that we need to compute CS – which can be obtained by using OC as a vertical projection of this segment and the vector CS1.To get CS1, it is needed to find FS1 which will be done by using the following method:
This is a model of the camera view. O1Px and O1Py will be vectors in the real world, that can be found on the projection plane. These vectors will be of length 1, their sole purpose being to give a meaning in the real world to the positions of objects found in the camera view.
By getting the coordinates of an object in the camera view, we can convert each coordinate from pixels to meters using proportionality. We know the horizontal and vertical fields of view of the camera, as well as the number of pixels it has on both coordinates. Thus, by multiplying each vector to the coordinates in meters - relative to the focal center of the camera - at which the object can be found in this plane, we get the position in real world of S1.
FreeMaster - view of the lines
FreeMaster view of the lines
This program will receive lines and intersections from Pixy and then the information will be read using FreeMaster. If you want to get a visual representation of the vectors and intersections that Pixy sends to your target, this program will render those lines and intersections for you. You need to build and run the model on your target, then open the FreeMaster project and connect it to the board. For more details about how to use FreeMaster, check this link[11].
The picture above presents the view in FreeMaster on the left in comparison to the view from PixyMon v2[13] on the right.
All you have to do for this part is to open the .pmpx file in FreeMaster and then press the green GO button on the upper left corner of the window.
BUILD AND RUN
For building and running any of the models presented in this article, you can follow the next steps:
-
Open the desired model in Matlab – it will automatically open Simulink for you
-
Connect the board through the USB cable to your computer
-
Go to the Apps tab in Simulink and click on Embedded Coder
-
Go to the C Code Tab and click on the drop-down arrow of Generate Code/Build
-
Press Build for building the application or Generate Code for simply getting the C Code for your application
If you pressed build, your application will be built on your computer and the executable will be stored on the board. Once the process is done, the application will start running right away on your microcontroller.
Tip: For a faster build, you can press crtl+B on your keyboard.
REFERENCES
[1]: https://pixycam.com/pixy2/
[2]: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:overview
[3]: https://eu.mouser.com/new/dfrobot/dfrobot-rob0170-racing-car/
[5]: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:general_api
[6]: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:protocol_reference
[7]: https://en.wikipedia.org/wiki/Electronic_speed_control
[8]: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:line_api
[9]: https://docs.pixycam.com/wiki/doku.php?id=wiki:v2:porting_guide
[10] : https://towardsdatascience.com/image-formation-and-pinhole-model-of-the-camera-53872ee4ee92
[11] : https://community.nxp.com/t5/NXP-Model-Based-Design-Tools/FreeMASTER-How-To/ta-p/1115477