Android 9.0 AudioRecord代碼分析

簡介

打算開始寫一些Android代碼的流程分析,加深一些記憶,理清一些細節,如果剛好能幫助到你,那就更加好了。手頭有的最新的Android代碼,就是9.0,所以以此爲基準。
爲什麼從AudioRecord開始?
一是這個接口的修改新舊Android版本的差別不大。
二是它要相對簡單一些。
所以作爲一個開始來分析代碼,其中也有一些細節,比如IBinder/Sharememory等,後面再單獨開一篇文章分析代碼細節。

Java 接口及用法

構造函數:

   /**
     * Class constructor.
     * Though some invalid parameters will result in an {@link IllegalArgumentException} exception,
     * other errors do not.  Thus you should call {@link #getState()} immediately after construction
     * to confirm that the object is usable.
     * @param audioSource the recording source.
     *   See {@link MediaRecorder.AudioSource} for the recording source definitions.
     * @param sampleRateInHz the sample rate expressed in Hertz. 44100Hz is currently the only
     *   rate that is guaranteed to work on all devices, but other rates such as 22050,
     *   16000, and 11025 may work on some devices.
     *   {@link AudioFormat#SAMPLE_RATE_UNSPECIFIED} means to use a route-dependent value
     *   which is usually the sample rate of the source.
     *   {@link #getSampleRate()} can be used to retrieve the actual sample rate chosen.
     * @param channelConfig describes the configuration of the audio channels.
     *   See {@link AudioFormat#CHANNEL_IN_MONO} and
     *   {@link AudioFormat#CHANNEL_IN_STEREO}.  {@link AudioFormat#CHANNEL_IN_MONO} is guaranteed
     *   to work on all devices.
     * @param audioFormat the format in which the audio data is to be returned.
     *   See {@link AudioFormat#ENCODING_PCM_8BIT}, {@link AudioFormat#ENCODING_PCM_16BIT},
     *   and {@link AudioFormat#ENCODING_PCM_FLOAT}.
     * @param bufferSizeInBytes the total size (in bytes) of the buffer where audio data is written
     *   to during the recording. New audio data can be read from this buffer in smaller chunks
     *   than this size. See {@link #getMinBufferSize(int, int, int)} to determine the minimum
     *   required buffer size for the successful creation of an AudioRecord instance. Using values
     *   smaller than getMinBufferSize() will result in an initialization failure.
     * @throws java.lang.IllegalArgumentException
     */
    public AudioRecord(int audioSource, int sampleRateInHz, int channelConfig, int audioFormat,
            int bufferSizeInBytes)

其中

  1. audioSource,定義在MediaRecorder.AudioSource,有很多種不同的source,底層的AudioFlinger會根據不同的source做一些不同的處理,比如VOICE_COMMUNICATION,需要進行回聲消除的處理,一般只是想錄音的話,選擇AudioSource.MIC即可,或者選擇AudioSource.DEFAULT,底層一般也是MIC。
  2. sampleRateInHz,音頻採樣率,設置一個需要的值,看抓到的音頻數據要做什麼使用,44100一般用作高品質錄音,常規設置16000就可以滿足使用,當然後續讀取PCM數據量的大小與這裏設置的採樣率大小是正相關的
  3. channelConfig: AudioFormat.CHANNEL_IN_STEREO 和 AudioFormat.CHANNEL_IN_MONO,就是雙聲道和單聲道,看需要設置
  4. audioFormat:表示一個PCM採樣點的位數,8位/16位/32位,ENCODING_PCM_8BIT/ENCODING_PCM_16BIT/ENCODING_PCM_FLOAT,一般使用16位
  5. bufferSizeInBytes:設置底層obtainBuffer/releaseBuffer的緩衝區大小,作爲從應用的進程和AudioServer的進程之間共享內存的大小,分配的函數在: AudioFlinger::ThreadBase::TrackBase::TrackBase,一般讀取AudioRecord.getMinBufferSize,設置爲最小buffersize即可,這個minbuffersize已經包含了兩個buffer的pingpong機制,可以滿足要求。
    在這裏插入圖片描述
    跟着Android的代碼進去看了一下這個bufferSizeInBytes的用法,需要收回之前AudioRecord比較簡單是說法了,代碼也是極其複雜的。

read函數

這裏我們重點分析一下read函數,看下數據是如何從驅動,一路上到APK的,流程會很長。

    public int read(@NonNull byte[] audioData, int offsetInBytes, int sizeInBytes) {
        return read(audioData, offsetInBytes, sizeInBytes, READ_BLOCKING);
    }
READ_BLOCKING/READ_NON_BLOCKING 這裏可以選擇block或者nonbblock的方式,大部分我想應該是BLOCK的,應用方便處理,而且音頻是流式的,理論上不會阻塞。
    private native final int native_read_in_byte_array(byte[] audioData,
            int offsetInBytes, int sizeInBytes, boolean isBlocking);

上面還是在java裏面,下面開始進到native代碼裏面,這裏欠一篇java和jni的調用的分析

frameworks/base/core/jni/android_media_AudioRecord.cpp
template <typename T>
static jint android_media_AudioRecord_readInArray(JNIEnv *env,  jobject thiz,
                                                  T javaAudioData,
                                                  jint offsetInSamples, jint sizeInSamples,
                                                  jboolean isReadBlocking) {

frameworks/av/media/libaudioclient/AudioRecord.cpp
ssize_t AudioRecord::read(void* buffer, size_t userSize, bool blocking)

...... 省略部分不重要的代碼
        status_t err = obtainBuffer(&audioBuffer,
                blocking ? &ClientProxy::kForever : &ClientProxy::kNonBlocking);
......
        size_t bytesRead = audioBuffer.size;
        memcpy(buffer, audioBuffer.i8, bytesRead);
......
這裏就是從audioBuffer裏面將數據複製到用戶的buffer裏面,等下返回給APK,那麼audioBuffer又是什麼呢?

status_t AudioRecord::obtainBuffer(Buffer* audioBuffer, const struct timespec *requested,
        struct timespec *elapsed, size_t *nonContig)
                status = proxy->obtainBuffer(&buffer, requested, elapsed);
                audioBuffer->raw = buffer.mRaw;

在這裏插入圖片描述
注意這裏Buffer的定義比較亂,很多結構體裏都有這個定義,Google也發現了這個問題,要統一起來,而且還有一個union在裏面,爲了支持8bit/16bit錄音的使用,所以這個audioBuffer->raw與前面的audioBuffer->i8實際上是一個地址。

所以上面的問題pcm地址audioBuffer->raw = buffer.mRaw;
其中buffer是Proxy::Buffer buffer;
sp<AudioRecordClientProxy> proxy = mProxy;
status = proxy->obtainBuffer(&buffer, requested, elapsed);
變成了這個buffer是怎麼來的,這裏要返回去看這個mProxy是什麼?
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
其中比較重要的參數是cblk和buffers,,再繼續往前追一下,這兩個變量的出處。

    iMemPointer = output.cblk ->pointer();
    cblk = static_cast<audio_track_cblk_t*>(iMemPointer);
    buffers = output.buffers->pointer();
    mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
可以看出這個AudioRecordClientProxy,只是方便調用cblk和buffers的Proxy,提供了obtainBuffer的接口,來從真正的共享buffer裏面將獲得數據,這裏面的細節不再追進去了,實際上是通過audio_track_cblk_t的一個共享內存,採用read/write指針的方法(mFront/mRear)來看有多少avail的數據,然後獲取指針出來。

重點是output的出處
status_t AudioRecord::createRecord_l(const Modulo<uint32_t> &epoch, const String16& opPackageName)
record = audioFlinger->createRecord(input,
                                                              output,
                                                              &status);
input裏面包括的是我們想設置的參數,output裏面是AudioServer實際設置的參數,已經分配的共享內存。

問題的重點是前面的cblk和buffers是什麼???

上面還是在APK的進程裏面,下面開始進到AudioServer的進程裏面了,尷尬不,這裏又欠一篇IBinder跨進程調用的分析

const sp<IAudioFlinger>& audioFlinger = AudioSystem::get_audio_flinger();

class IAudioFlinger : public IInterface
virtual sp<media::IAudioRecord> createRecord(const CreateRecordInput& input,
                                        CreateRecordOutput& output,
                                        status_t *status) = 0;

sp<media::IAudioRecord> AudioFlinger::createRecord(const CreateRecordInput& input,
                                                   CreateRecordOutput& output,
                                                   status_t *status)
	//上來先把我們最重要的變量clear一下
    output.cblk.clear();
    output.buffers.clear();
    //從recordTrack當中得到這個共享內存
    output.cblk = recordTrack->getCblk();
    output.buffers = recordTrack->getBuffers();

再看這個recordTrack,是一個RecordThread創建出來的
recordTrack = thread->createRecordTrack_l(client, input.attr, &output.sampleRate,
                                                  input.config.format, input.config.channel_mask,
                                                  &output.frameCount, sessionId,
                                                  &output.notificationFrameCount,
                                                  clientUid, &output.flags,
                                                  input.clientInfo.clientTid,
                                                  &lStatus, portId);
傳了一大堆參數進去,細節不深究,那麼getCblk得到了什麼?
class RecordTrack : public TrackBase {
class TrackBase : public ExtendedAudioBufferProvider, public RefBase {
            sp<IMemory> getCblk() const { return mCblkMemory; }
        mCblkMemory = client->heap()->allocate(size);

sp<MemoryDealer> AudioFlinger::Client::heap() const
{
    return mMemoryDealer;
}

mMemoryDealer = new MemoryDealer(
            audioFlinger->getClientSharedHeapSize(),
            (std::string("AudioFlinger::Client(") + std::to_string(pid) + ")").c_str());
getCblk實際得到了一個heap的內存,這個內存分配在AudioServer裏面,write在AudioServer裏面,read在AudioRecord裏面,也就是在應用APK的進程空間。

下面要看下,數據如何從HAL層到mMemoryDealer。

RecordThread -> RecordTrack
bool AudioFlinger::RecordThread::threadLoop()
	status_t result = mInput->stream->read(
                    (uint8_t*)mRsmpInBuffer + rear * mFrameSize, mBufferSize, &bytesRead);
	(void)posix_memalign(&mRsmpInBuffer, 32, mRsmpInFramesOA * mFrameSize);
	
	從HAL層裏面將數據讀到這個mRsmpInBuffer,名義上看是resample用的,因爲HAL層出來的採樣率與上層需要的不一定一樣,所以需要一個resample的過程,這個buffer裏面有可能是44100hz的pcm數據。
            // loop over each active track
        for (size_t i = 0; i < size; i++) {
            activeTrack = activeTracks[i];
            status_t status = activeTrack->getNextBuffer(&activeTrack->mSink);
一個RecordThread裏面可以有幾個active的RecordTrack,將mRsmpInBuffer的數據針對不同track的需要resample成不同的PCM數據。

status_t AudioFlinger::RecordThread::RecordTrack::getNextBuffer(AudioBufferProvider::Buffer* buffer)
{
    ServerProxy::Buffer buf;
    buf.mFrameCount = buffer->frameCount;
    status_t status = mServerProxy->obtainBuffer(&buf);  //還記得前面說的ClientProxy了嗎,這裏對應到了ServerProxy,是同一個buffer沒錯了。
    buffer->frameCount = buf.mFrameCount;
    buffer->raw = buf.mRaw;
    if (buf.mFrameCount == 0) {
        // FIXME also wake futex so that overrun is noticed more quickly
        (void) android_atomic_or(CBLK_OVERRUN, &mCblk->mFlags);
    }
    return status;
}

好了,這裏徹底對上了,將mResamplerBufferProvider裏面的原始pcm數據,通過mRecordBufferConverter::convert轉換成需要的samplerate。
framesOut = activeTrack->mRecordBufferConverter->convert(
       activeTrack->mSink.raw, activeTrack->mResamplerBufferProvider, framesOut);

再看一下這個converter
    mRecordBufferConverter = new RecordBufferConverter(
            thread->mChannelMask, thread->mFormat, thread->mSampleRate,
            channelMask, format, sampleRate);
很清楚,就是從RecordThread的格式轉換爲RecordTrack的格式,細節不再進去追了,只是一些實現細節。

到這裏以後,從AudioServer到APK的數據流已經清楚了,就是從HAL層到這個mMemoryDealer的共享內存,然後通過mServerProxy和AudioRecordClientProxy對這個共享內存的跨進程同步,複製到AudioRecord的audioBuffer當中,然後複製到Java層。

其他接口

AudioRecord.startRecording();
AudioRecord.stop();
控制開始和結束AudioRecord的API,必須使用

AudioRecord.setNotificationMarkerPosition
AudioRecord.setPositionNotificationPeriod
AudioRecord.setRecordPositionUpdateListener
看起來是可以週期性的notify一下app,不確定什麼場景,難道APK自己計數一下read到的數據長度不行?非得回調?

AudioRecord.getTimestamp
獲取當前錄製音頻的時間戳

AudioRecord用法samplecode

從Android的cts裏面複製出來的代碼,cts是一個理解API很好的代碼庫

mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, mHz,
                            AudioFormat.CHANNEL_CONFIGURATION_MONO,
                            AudioFormat.ENCODING_PCM_16BIT,
                            AudioRecord.getMinBufferSize(mHz,
                                    AudioFormat.CHANNEL_CONFIGURATION_MONO,
                                    AudioFormat.ENCODING_PCM_16BIT) * 10);
mAudioRecord.startRecording();
while (System.currentTimeMillis() - time < RECORD_TIME) {
    Thread.sleep(SLEEP_TIME);
     mAudioRecord.read(byteData, 0, BUFFER_SIZE);   //只是samplecode, 可以讀到數據,一般應用不會這樣寫
}
mAudioRecord.stop();

後記

大致分析了一下AudioRecord的數據流程,不涉及到HAL層的代碼,因爲各家的實現會有不同,裏面的細節很多,包括IBinder調用/cblk的共享內存/JNI的調用都還沒有詳細分析,留作後面的功課。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章