Description of problem
The audio data received when operating the NXP RT1050 as a USB Host with the Audio class does not represent the audio data spoken into a connected USB microphone device.
This issue exists not with just a single microphone or device, as I have tested three different types of USB microphones. I have independently tested the microphones with a PC to confirm they work correctly.
The data corruption has been confirmed by taking traces of the USB bus and comparing the data received on the RT1050 with the data seen on the bus.
Prior Requisite Details
I fixed a bug in the USB stack that misreports the received data length. The details of that problem and solution can be found in the USB Host Audio Stream Receive Data Length Misreporting thread.
Setup Details
My RT1050 is setup as a USB Audio Host. I created the USB Audio setup based off of the USB HID Mouse example. I included the state machine below that sets the control and streaming interfaces.
I'm able to receive audio data from a connected Full-Speed USB microphone by repeatedly calling USB_HostAudioStreamRecv.
Test Setup and Results
I have results from two different ways of using the USBHostAudioStreamRecv function that both result in receiving unexpected audio data. In these tests, I'm speaking into the microphone and collecting three seconds of audio data into a RAM buffer on the RT1050. At the same time, I'm collecting a trace of the USB communication using a Teledyne Advisor T3 USB Analyzer.
/* USB Audio Buffer */
USB_DMA_NONINIT_DATA_ALIGN(USB_DATA_ALIGN_SIZE) static uint8_t _AudioBuffer[AUDIO_BUFFER_SIZE];
/* audioInstance->audioBuffer points to _AudioBuffer */
status = USB_HostAudioStreamRecv(audioInstance->classHandle,
audioInstance->audioBuffer,
AUDIO_BUFFER_SIZE,
_HostAudioInCallback,
audioInstance);
Here are descriptions of the two modes of operation. The audio bin files are 16-bit, 16kHz PCM audio, which can be played using Audacity.
Mode 1: More than one packet (greater than 34 bytes) is requested for transfer at a time
USB Trace of Audio: microphone_4k_read_usb_trace.bin.zip
Received data on RT1050 read from RAM buffer: microphone_4k_read_ram_buffer.bin.zip
a) AUDIO_BUFFER_SIZE = 34
b) The received audio’s speech is audible, but there is significant noise
c) In this mode, the audio data on the bus is equivalent to the audio data of the source (microphone)
d) Comparing the received data to the source data, bytes are rearranged and values are different
Mode 2: Exactly one packet (34 bytes) is requested for transfer at a time
USB Trace of Audio: microphone_34_read_usb_trace.bin.zip
Received data on RT1050 read from RAM buffer: microphone_34_read_ram_buffer.bin.zip
a) AUDIO_BUFFER_SIZE = 4096
b) The received audio does not resemble speech
c) The audio data recorded on the bus does not match the audio data of the source (microphone)!
d) One isochronous transaction occurs every 3ms, instead of the expected 1ms (Full Speed)
e) It seems there’s a delay in the stack that is preventing the reads from happening fast enough
Code and Operation Description
Below are snippets of my software that implement the USB Audio Host, which I based off of the USB HID mouse example.
There are two tasks: one that calls the USB_HostEhciTaskFunction, and another that manages the USB application state. The application state task (_MicrophoneApplicationTask) blocks on a FreeRTOS Event Group bit, which is set in _HostAudioControlCallback and _HostAudioInCallback. This allows the _MicrophoneApplicationTask to immediately react to events and handle any state changes.
/*! @brief USB host audio instance structure */
typedef struct USBHostAudioInstance
{
usb_host_configuration_handle configHandle; /*!< the audio configuration handle */
usb_device_handle deviceHandle; /*!< the audio device handle */
usb_host_class_handle classHandle; /*!< the audio class handle */
usb_host_interface_handle streamingInterfaceHandle; /*!< the audio streaming interface handle */
usb_host_interface_handle controlInterfaceHandle; /*!< the audio control interface handle */
uint8_t deviceState; /*!< device attach/detach status */
uint8_t prevState; /*!< device attach/detach previous status */
uint8_t runState; /*!< audio application run status */
uint8_t runWaitState; /*!< audio application wait status, go to next run status when the wait status success */
uint16_t maxPacketSize; /*!< Interrupt in max packet size */
uint8_t *audioBuffer; /*!< use to receive data */
} USBHostAudioInstance_t;
/*! @brief host app run status */
typedef enum USBHostAudioRunState
{
USBHostAudioRunState_Idle = 0, /*!< idle */
USBHostAudioRunState_SetControlInterface, /*!< execute set interface code */
USBHostAudioRunState_WaitSetControlInterface, /*!< wait set interface done */
USBHostAudioRunState_SetStreamingInterface, /*!< execute set interface code */
USBHostAudioRunState_WaitSetStreamingInterface, /*!< wait set interface done */
USBHostAudioRunState_ReceiveStream, /*!< set interface is done, start streaming */
USBHostAudioRunState_WaitReceiveStream /*!< wait for streaming callback */
} USBHostAudioRunState_t;
static void _HostAudioControlCallback(
void *param,
uint8_t *data,
uint32_t dataLength,
usb_status_t status)
{
UNUSED_PARAMETER(data);
UNUSED_PARAMETER(dataLength);
UNUSED_PARAMETER(status);
USBHostAudioInstance_t *audioInstance = (USBHostAudioInstance_t *)param;
if (audioInstance->runWaitState == USBHostAudioRunState_WaitSetControlInterface)
{
audioInstance->runState = USBHostAudioRunState_SetStreamingInterface;
}
else if (audioInstance->runWaitState == USBHostAudioRunState_WaitSetStreamingInterface)
{
audioInstance->runState = USBHostAudioRunState_GetSamplingFrequency;
}
else if (audioInstance->runWaitState == USBHostAudioRunState_WaitGetSamplingFrequency)
{
audioInstance->runState = USBHostAudioRunState_ReceiveStream;
}
else
{
DEBUG_USB("%s: Unhandled runWaitState\r\n", __func__);
}
xEventGroupSetBits(_USBEventGroup, /* The event group being updated. */
BIT_0); /* The bits being set. */
}
static void _HostAudioInCallback(
void *param,
uint8_t *data,
uint32_t dataLength,
usb_status_t status)
{
USBHostAudioInstance_t *audioInstance = (USBHostAudioInstance_t *)param;
if (audioInstance->deviceState == kStatus_DEV_Attached)
{
if (status == kStatus_USB_Success)
{
//DEBUG_USB("%s: Successfully received %u bytes\r\n", __func__, dataLength);
VoiceProcessor_ReceivedDataCallback(data, dataLength);
if (audioInstance->runWaitState == USBHostAudioRunState_WaitReceiveStream)
{
audioInstance->runState = USBHostAudioRunState_ReceiveStream;
}
else
{
/* unhandled condition */
ASSERT_FAILURE();
}
}
else
{
ASSERT_FAILURE();
DEBUG_USB("%s: Receive unsuccessful\r\n", __func__);
}
}
else
{
ASSERT_FAILURE();
DEBUG_USB("%s: Device not attached\r\n", __func__);
}
xEventGroupSetBits(_USBEventGroup, /* The event group being updated. */
BIT_0); /* The bits being set. */
}
/***************************************************************************//**
USB Host Task
\param[in] param - Task input parameter
*//****************************************************************************/
static void _USBHostTask(void *param)
{
while (1)
{
USB_HostEhciTaskFunction(param);
}
}
/***************************************************************************//**
USB Host Application Task
\param[in] param - Task input parameter
*//****************************************************************************/
static void _MicrophoneApplicationTask(void *param)
{
EventBits_t uxBits;
while (1)
{
uxBits = xEventGroupWaitBits(_USBEventGroup, /* The event group being tested. */
BIT_USBEvent, /* The bits within the event group to wait for. */
pdTRUE, /* Cleared bits before returning. */
pdFALSE, /* Don't wait for both bits, either bit will do. */
portMAX_DELAY);/* Wait forever. */
_HostAudioStateManager(param);
}
}
/***************************************************************************//**
Task to handle the current device state
\param[in] param - the device instance
*//****************************************************************************/
static void _HostAudioStateManager(void *param)
{
usb_status_t status;
USBHostAudioInstance_t *audioInstance = (USBHostAudioInstance_t *)param;
/* Process device state changes (process once for each state) */
if (audioInstance->deviceState != audioInstance->prevState)
{
audioInstance->prevState = audioInstance->deviceState;
switch (audioInstance->deviceState)
{
case kStatus_DEV_Idle:
break;
/* Device is attached and enumeration is done */
case kStatus_DEV_Attached:
DEBUG_USB("Mic device attached\r\n");
audioInstance->runState = USBHostAudioRunState_SetControlInterface;
status = USB_HostAudioInit(audioInstance->deviceHandle, &audioInstance->classHandle);
if (status != kStatus_USB_Success)
{
DEBUG_USB("USB Host Audio class initialize failure\r\n");
ASSERT_FAILURE();
}
break;
/* Device is detached */
case kStatus_DEV_Detached:
audioInstance->deviceState = kStatus_DEV_Idle;
audioInstance->runState = USBHostAudioRunState_Idle;
/* Audio class de-initialization */
USB_HostAudioDeinit(audioInstance->deviceHandle,
audioInstance->classHandle);
audioInstance->classHandle = NULL;
DEBUG_USB("USB Audio device detached\r\n");
break;
default:
break;
}
}
/* Process the run state */
switch (audioInstance->runState)
{
case USBHostAudioRunState_Idle:
break;
/* Set the audio control interface */
case USBHostAudioRunState_SetControlInterface:
audioInstance->runWaitState = USBHostAudioRunState_WaitSetControlInterface;
audioInstance->runState = USBHostAudioRunState_Idle;
status = USB_HostAudioControlSetInterface(audioInstance->classHandle,
audioInstance->controlInterfaceHandle,
DEVICE_CONTROL_ALTERNATE_SETTING,
_HostAudioControlCallback,
audioInstance);
ASSERT(status == kStatus_USB_Success);
DEBUG_USB("Set Control Interface Complete\r\n");
break;
/* Set the audio streaming interface */
case USBHostAudioRunState_SetStreamingInterface:
audioInstance->runWaitState = USBHostAudioRunState_WaitSetStreamingInterface;
audioInstance->runState = USBHostAudioRunState_Idle;
status = USB_HostAudioStreamSetInterface(audioInstance->classHandle,
audioInstance->streamingInterfaceHandle,
DEVICE_MIC_ALTERNATE_SETTING,
_HostAudioControlCallback,
audioInstance);
ASSERT(status == kStatus_USB_Success);
DEBUG_USB("Set Streaming Interface Complete\r\n");
break;
case USBHostAudioRunState_ReceiveStream:
audioInstance->runWaitState = USBHostAudioRunState_WaitReceiveStream;
audioInstance->runState = USBHostAudioRunState_Idle;
uint32_t maxPacketSize = USB_HostAudioPacketSize(audioInstance->classHandle,
USB_ENDPOINT_ISOCHRONOUS,
USB_IN);
ASSERT(AUDIO_BUFFER_SIZE >= maxPacketSize);
status = USB_HostAudioStreamRecv(audioInstance->classHandle,
audioInstance->audioBuffer,
AUDIO_BUFFER_SIZE,
_HostAudioInCallback,
audioInstance);
ASSERT(status == kStatus_USB_Success);
break;
default:
break;
}
}
Hi! I'm finding this thread very helpful. I've tried the evk-usb-host-audio_v1.zip file and have it working. But it says "device not supported" for each of the 3 devices (Logitech speakers, HyperX and MPOW adapters for headset) I'm trying, whose vendor and product IDs are 0951 & 16A4, 1130 & 1620, and 0D8C & 0014. Is there any guidance on what devices would be supported by this application?
Thanks!
-Allan
See the comment above under Software Setup > Download and configure the project for a list of supported devices and the procedure to add another device.
Hello,
Please refer to the information as the below:
I've checked the code that you provide, i'm sure that the main problem is the timing of the API USB_HostAudioStreamRecv calls. Please try to call USB_HostAudioStreamRecv in _HostAudioInCallback and set a buff that the size is at least three times packet length and call USB_HostAudioStreamRecv more than three times to ensure that there is enough callback process time.Actually, the _HostAudioInCallback of customer's code take too much time that more than 1ms, so it appears that the packet time is too long. And that it is not recommended to do operation that cost to much time in the callback of isochronous transfer.
The images referenced will be in the next reply post.
Hi fangfang,
The issue does not necessarily exist with the RT1050 operating as a USB Device. The issue does exist with the RT1050 operating as a USB Host. Please use the example application I provided to reproduce the issue. Otherwise, create another USB Host Audio application to reproduce the issue.
Thank you
I'm continuing to investigate this further, but could really use some help with this. jeremyzhou jingpan jorge_a_vazquez
I have been looking at the initialization of the siTD linked list, and I have some questions regarding the memory alignment requirement outlined by the reference manual.
Can anyone explain the significance of the 4K (page) aligned memory addresses that need to be supplied to each siTD structure?
In the case of Mode 1 (More than one packet (greater than 34 bytes) is requested for transfer at a time) with a requested transfer of 4K and the logic in the USB_HostEhciSitdArrayInit function, 121 siTD structures are required.
/* Calculate the number of siTD structures required */
sitdNumber = ((transfer->transferLength - 1 + (ehciPipePointer->pipeCommon.maxPacketSize)) /
(ehciPipePointer->pipeCommon.maxPacketSize));
Does the 4K (page) aligned memory addresses specified in the reference manual mean that each of the siTD structures must point to a buffer that is 4K (page) aligned?
Based on the existing code, it is clear that each of these addresses is not 4K (page) aligned.
For each siTD, the buffer address is set as follows:
sitdLength = dataLength;
if (sitdLength > ehciPipePointer->pipeCommon.maxPacketSize)
{
sitdLength = ehciPipePointer->pipeCommon.maxPacketSize;
}
dataBufferValue = (uint32_t)(transfer->transferBuffer + (transfer->transferLength - dataLength));
dataLength -= sitdLength;
sitdPointer->transferResults[1] = dataBufferValue;
sitdPointer->transferResults[2] = ((dataBufferValue + 4 * 1024) & 0xFFFFF000U);
Hello
I get the feedback from the expert team as the below:
"We do not have USB host audio mic demo, where is your code come from? And what is the device connected to host?Is it a special USB MIC? And there is a very long statement for the issue description.Could you please kindly summarize it in a short expression?I see one issue you report is USB HOST missed some package in some frame.Any other issue you want to report?I do not have a USB mic device on hand, I have a USB mic+speaker device, but it is not recognized by the USB HOST demo. Anyway, I'll check if I can do some work and make a quick test on it.If any progress, I'll share to you."
Thanks for your patience.
Description of problem: The audio data received when operating the NXP RT1050 as a USB Host with the Audio class does not represent the audio data spoken into a connected USB microphone device. This is true regardless of the USB device (microphone) that is used.
Identified Issues: Depending on how the USB Stream Recieve API is used, either packets of data are missing, or the data that is received is corrupt. See details in first post.
USB Devices used: I have used three different microphones with the USB Audio Host. All three lead to the RT1050 Host exhibiting the above problems. Two of the devices are the ReSpeaker Mic Array v2.0 and the Labtec Verse 704.
I included the main pieces of the code required for setting up the USB Audio host in the first post.
If there's anything I can do to assist in your testing and debugging, please let me know.
Thank you for getting back to me.
Hello Austin.
The question will may take some time. Thanks for your patience.
Hello Austin.
Thanks for your updated information. I will let you know if has any update.
Hello, Austin
I apologize for the late response.I forwarded the question to the expert team.
Due to technical details which may take some times. Let us wait for the expert's reply.
Sorry for the inconvenience may cause. Have a nice day.
Best Regards
Update:
I was able to confirm that even when operating in Mode 1 (More than one packet is requested for transfer at a time), there is still always a 3ms delay between consecutive transfers.
I also confirmed with a spare GPIO and an oscilloscope that there is no delay between the _HostAudioInCallback being called and the state manager calling the USB_HostAudioStreamRecv function again. I believe this definitely points to there being a delay somewhere in the USB stack itself.