Hi,
As per the CAN FD Specification, Classic CAN has a limit of 8 bytes per frame, as shown below:

For messages with more than 8 bytes per frame, CAN FD will be required. As for your error, can you share your CAN configurations under Config Tools? To understand what the error could be.
Also, are you using any example? From which RTD version? Which IDE version are you using?
Please, let us know.