This design enable i.MX processor as a USB Playback/Capture device on one USB OTG/Device port. USB host that has usb gadget audio driver can playback PCM through this device and capture PCM also through this device as the same time.
This design is based on i.MX28 EVK board, and code is based on i.MX28 L2.6.35_10.12.01_ER_source. Although it was only be tested on i.MX28 EVK, I think this design is compatible with all SOC that has USB device function.
I attach the kernel patch for this design and demo application code.
Kernel patch file:L2.6.35_10.12.01_ER_source_Kernel.patch
Demo application code:testapp.tar.bz2
Design Block Diagram:
Driver user space interface:
As file /dev/gadget/g_audio
file system : gadgetaudiofs_type
usage: mount -t gadgetaudiofs path /dev/gadget /* if /dev/gadget is not exist, create it manually */
r/w interface open read write close
ssize_t read(int fd, void *buf, size_t count);
Notice:
Attempts to read up to count bytes from file descriptor fd into the buffer starting at buf.
On success, the number of bytes read is returned.
This call may be blocked if device can't give enough data.
Only if usb out pipe be broken(host stop audio player), return value is still positive and less than count.
Error:
Count must align with PCM audio frame size. If not -EFAULT as errorno.
Notes:
Audio frame size = audio sample size x audio channels.
ssize_t write(int fd, void *buf, size_t count);
Notice:
Writes up to count bytes from the buffer pointed buf to the file referred to by the file descriptor fd.
On success, the number of bytes written is returned.
This call will never be blocked, even if the internal ring buffer dose not have enough space to write.
A successful return from write() that return value equal to count does not make any guarantee that data all have been put to internal ring buffer. For example, if ring buffer is empty and has 1K bytes space, while write count is 3K bytes, only the last 1K bytes will be put to the ring buffer!
Key attribute for g_audio driver.
USB out pipe parameter:
Sample width hard code to 16bits.
static int out_sample_rate = 48000;
static int out_channel = 2;
#define OUT_EP_ALIGN (2 * out_channel) // 2 mean 2 bytes, sample width 16bits
OUT_EP_ALIGN mean “read ring buffer” write/read align, if read/write length not align this value, throw out error. Read mean user space read system call, write mean driver internal copy req->buf to “read ring buffer”.
#define OUT_EP_MAX_PACKET_SIZE (192) // out pipe max packet size, it based on out_sample_rate and out_channel. 192 = (48KHZ / 1000) * out_channel * sample_width(as bytes)
192 = (48000 / 1000 ) * 2 * 2;
Hear 1000 based on:
as_out_ep_desc.bInterval = 4;
/* 4 in hi speed as 2 exp (4 -1) = 8 uframe time, 8 uframe time is 8 * 125 us = 1ms 1000 = 1s / 1ms */
static int out_req_count = 256; /* out pipe queue count, this value dose not introduce extra audio latency ! */
#define AUDIO_READ_RINGBUF_LEN (23 * OUT_EP_MAX_PACKET_SIZE)
/* 4416 bytes, max valid 23 * 192, about 23 ms at 48KHZ, this value determine out pipe max audio latency */
If “read ring buffer” full, how to handle continue write: discard the 1/3 oldest ring buffer.
See function
int f_audio_ringbuffer_write
if(ringbuf->len - ringbuf->actual < alignLen)
{
ringbuf->rp += (alignLen * (ringbuf->len /(alignLen * 3))); /* if cache up read pointer, discard 1/3 ring buf */
...
}
USB in pipe parameter:
Sample width hard code to 16bits.
static int in_sample_rate = 8000;
static int in_channel = 2;
#define IN_EP_ALIGN (2 * in_channel) // 2 mean 2 bytes, sample width 16bits
IN_EP_ALIGN mean “write ring buffer” write/read align, if read/write length not align this value, throw out error. Write mean user space write system call, read mean driver internal copy “write ring buffer” to req->buf.
#define IN_EP_MAX_PACKET_SIZE (32) // in pipe max packet size, it based on in_sample_rate and in_channel. 32 = (8KHZ / 1000) * in_channel * sample_width(as bytes)
32 = (8000 / 1000 ) * 2 * 2;
Hear 1000 based on:
as_in_ep_desc.bInterval = 4;
/* 4 in hi speed as 2 exp (4 -1) = 8 uframe time, 8 uframe time is 8 * 125 us = 1ms 1000 = 1s / 1ms */
static int in_req_count = 32; /* in pipe queue count, this value dose introduce extra audio latency ! Latency = 32ms */
#define AUDIO_WRITE_RINGBUF_LEN (32 * OUT_EP_MAX_PACKET_SIZE)
/* 1024 bytes, max valid 32 * 32, about 32 ms at 8KHZ, this value and in_req_count determine in pipe max audio latency 32 + 32 = 64 ms*/
If “write ring buffer” full, how to handle continue write: discard the 1/3 oldest ring buffer.
See function
int f_audio_ringbuffer_write
if(ringbuf->len - ringbuf->actual < alignLen)
{
ringbuf->rp += (alignLen * (ringbuf->len /(alignLen * 3))); /* if cache up read pointer, discard 1/3 ring buf */
...
}
Test
Environment.
Ubuntu 10.0.4 LTS Kernel 3.0.0-15 64bit.
Ubuntu 10.0.4 LTS Kernel 2.6.32-42 64bit.
Test application.
Test user space application based on http://www.rosoo.net/a/201107/14725.html
I will attach modified code.
Test procedure.
/* I.MX28 EVK board audio ADC default input is MIC, so, set it to LINE IN */
amixer sset 'ADC Mux' 'LINE_IN'
/* insmod g_audio driver and create directory for gadgetaudiofs */
modprobe g_audio && mkdir /dev/gadget
/* mount gadgetaudiofs */
mount -t gadgetaudiofs path /dev/gadget
/* start read g_audio device application.
It read PCM data from g_audio_device and put it to alsa playback device.
It read from g_audio_device per read system call per period_size (300 bytes).
See alsa PCM playback configuration
stream : PLAYBACK
access : RW_INTERLEAVED
format : S16_LE
subformat : STD
channels : 2
rate : 48000
exact rate : 48000 (48000/1)
msbits : 16
buffer_size : 1200
period_size : 300
period_time : 6250
*/
./lplay /dev/gadget/g_audio
/* start write g_audio device application.
It read PCM data from alsa capture device and put it to g_audio_device.
It write to g_audio_device per write system call per period_size (50 bytes).
See alsa PCM capture configuration
stream : CAPTURE
access : RW_INTERLEAVED
format : S16_LE
subformat : STD
channels : 2
rate : 8000
exact rate : 8000 (8000/1)
msbits : 16
buffer_size : 200
period_size : 50
period_time : 6250
*/
./lrecord /dev/gadget/g_audio
The two processes CPU utilization on I.MX28 EVK board is about 5~15%, if you change per g_audio_device read/write system call size more larger, the more smaller CPU utilization you will get, but at the same time the more audio latency you will get.
Please attention per read/write system call size should has relation will “read/write ring buffer” size in g_audio driver. For example, if per write system call size is larger than “write ring buffer” size, then every write system call will discard some part of write buffer data.
During 15 hours lplay and lrecord long test, driver and application both use default configuration as upper description, summarily 271 times driver internal buffer full appear.
In another test, driver keep default configuration, new test set microphone ALSA capture buffer to 250ms, ALSA capture read as unblock mode, other configuration as lrecord application, block read USB gadget device 200 x 192 bytes(need waiting 200ms), then unblock read whole ALSA capture buffer, found about every 1.5s, ALSA buffer will less then 16bytes (4 sample) compare to 200 x 32 bytes.
Clock Sync issue of echo cancellation based on this implementation:
Worry about：
USB clock domain different with play back and microphone, some buffer will be discard by USB audio driver.
See this diagram:
1: Echo cancellation application will try best to read “USB Output Buffer”, so no buffer will be discarded from output. ( Application input and output based on the same clock).
2: “Host playback buffer” maybe overrun because:
A: Playback source unstable and host playback buffer not enough larger.
B: Clock(playback) quick than Clock(USB).
3: Echo cancellation application will try best to read “ALSA Buffer”, so no buffer will be discarded from “ALSA buffer”. (Application discard some samples of “ALSA Buffer”)
3: Echo cancellation application handle “USB Output Buffer” and “ALSA Buffer” based on “USB Output Buffer” that same time mean based on Clock(USB). We assume Echo cancellation application will insert or discard some samples of “ALSA Buffer” based on “USB Output Buffer”.
4: Echo cancellation application will send processed buffer to USB audio driver based on Clock(USB), so no buffer will be discard from “USB Input Buffer”.
If Echo cancellation application said “ I don't have the ability to insert or discard some samples of “ALSA Buffer”, we need adjust the Clock(MIC) based the internal buffer level of Echo cancellation application. But I think the “internal buffer level” will be influenced by difference of the clocks, the buffer input and output task runtime loading, so it may not be reality to implement this, need do more test on this!
Add USB get output/input buffer length interface.
Add USB SAIF(only for i.mx 28) set clock interface.
For i.mx28 SAIF clock based on 480MHz.
The fraction divider is 16bit, that mean the mini step is 0x 0.0001.
0x 0.0001 * 480MHz = 7324.21875Hz.
If master clock as 512x frame rate, the mini step of frame rate is 14.3Hz
USB get output/input buffer length interface:
IOCTL CMD:
#define USBAUDIO_BUFFER_STATUS_GET \
_IOR('g', 200, struct usbaudio_buffer_status)
structure:
struct usbaudio_buffer_status{
/* all as bytes */
__u32 playbackBufferTotalLen;
__u32 playbackBufferCurrentLen;
__u32 microphoneBufferTotalLen;
__u32 microphoneBufferCurrentLen;
};
usage:
struct usbaudio_buffer_status bufferStatus;
ioctl(fd, USBAUDIO_BUFFER_STATUS_GET, &bufferStatus);
USB SAIF(only for i.mx 28) set clock interface.
IOCTL CMD:
#define USBAUDIO_SAIF_CLOCK_CONTROL \
_IOWR('g', 201, struct usbaudio_saif_clock_control)
structure:
struct usbaudio_saif_clock_control{
/* all as HZ -1 as invalid */
__u32 saifCurrentClock; /* read */
__u32 saifNextClock; /* write */
};
usage:
struct usbaudio_saif_clock_control saifClkCtl;
saifClkCtl.saifNextClock = -1;
ioctl(fd, USBAUDIO_SAIF_CLOCK_CONTROL, &saifClkCtl);
saifClkCtl.saifNextClock = saifClkCtl.saifCurrentClock + step;
ioctl(fd, USBAUDIO_SAIF_CLOCK_CONTROL, &saifClkCtl);
Compile sample:
/opt/freescale/usr/local/gcc-4.4.4-glibc-2.11.1-multilib-1.0/arm-fsl-linux-gnueabi/bin/arm-linux-gcc -I /home/haidong/Work/Mx28/L2.6.35_10.12.01_ER_source/LTIB/ltib/rootfs/usr/include/ -I /home/haidong/Work/Mx28/L2.6.35_10.12.01_ER_source/LTIB/Kernel/linux-2.6.35.3/include -L /home/haidong/Work/Mx28/L2.6.35_10.12.01_ER_source/LTIB/ltib/rootfs/usr/lib/ lrecord.c -o lrecord -lasound
Original Attachment has been moved to: 2-testapp.tar.bz2
Original Attachment has been moved to: L2.6.35_10.12.01_ER_source_Kernel.patch.zip
