簡介
打算開始寫一些Android代碼的流程分析,加深一些記憶,理清一些細節,如果剛好能幫助到你,那就更加好了。手頭有的最新的Android代碼,就是9.0,所以以此爲基準。
爲什麼從AudioRecord開始?
一是這個接口的修改新舊Android版本的差別不大。
二是它要相對簡單一些。
所以作爲一個開始來分析代碼,其中也有一些細節,比如IBinder/Sharememory等,後面再單獨開一篇文章分析代碼細節。
Java 接口及用法
構造函數:
/**
* Class constructor.
* Though some invalid parameters will result in an {@link IllegalArgumentException} exception,
* other errors do not. Thus you should call {@link #getState()} immediately after construction
* to confirm that the object is usable.
* @param audioSource the recording source.
* See {@link MediaRecorder.AudioSource} for the recording source definitions.
* @param sampleRateInHz the sample rate expressed in Hertz. 44100Hz is currently the only
* rate that is guaranteed to work on all devices, but other rates such as 22050,
* 16000, and 11025 may work on some devices.
* {@link AudioFormat#SAMPLE_RATE_UNSPECIFIED} means to use a route-dependent value
* which is usually the sample rate of the source.
* {@link #getSampleRate()} can be used to retrieve the actual sample rate chosen.
* @param channelConfig describes the configuration of the audio channels.
* See {@link AudioFormat#CHANNEL_IN_MONO} and
* {@link AudioFormat#CHANNEL_IN_STEREO}. {@link AudioFormat#CHANNEL_IN_MONO} is guaranteed
* to work on all devices.
* @param audioFormat the format in which the audio data is to be returned.
* See {@link AudioFormat#ENCODING_PCM_8BIT}, {@link AudioFormat#ENCODING_PCM_16BIT},
* and {@link AudioFormat#ENCODING_PCM_FLOAT}.
* @param bufferSizeInBytes the total size (in bytes) of the buffer where audio data is written
* to during the recording. New audio data can be read from this buffer in smaller chunks
* than this size. See {@link #getMinBufferSize(int, int, int)} to determine the minimum
* required buffer size for the successful creation of an AudioRecord instance. Using values
* smaller than getMinBufferSize() will result in an initialization failure.
* @throws java.lang.IllegalArgumentException
*/
public AudioRecord(int audioSource, int sampleRateInHz, int channelConfig, int audioFormat,
int bufferSizeInBytes)
其中
- audioSource,定義在MediaRecorder.AudioSource,有很多種不同的source,底層的AudioFlinger會根據不同的source做一些不同的處理,比如VOICE_COMMUNICATION,需要進行回聲消除的處理,一般只是想錄音的話,選擇AudioSource.MIC即可,或者選擇AudioSource.DEFAULT,底層一般也是MIC。
- sampleRateInHz,音頻採樣率,設置一個需要的值,看抓到的音頻數據要做什麼使用,44100一般用作高品質錄音,常規設置16000就可以滿足使用,當然後續讀取PCM數據量的大小與這裏設置的採樣率大小是正相關的
- channelConfig: AudioFormat.CHANNEL_IN_STEREO 和 AudioFormat.CHANNEL_IN_MONO,就是雙聲道和單聲道,看需要設置
- audioFormat:表示一個PCM採樣點的位數,8位/16位/32位,ENCODING_PCM_8BIT/ENCODING_PCM_16BIT/ENCODING_PCM_FLOAT,一般使用16位
- bufferSizeInBytes:設置底層obtainBuffer/releaseBuffer的緩衝區大小,作爲從應用的進程和AudioServer的進程之間共享內存的大小,分配的函數在: AudioFlinger::ThreadBase::TrackBase::TrackBase,一般讀取AudioRecord.getMinBufferSize,設置爲最小buffersize即可,這個minbuffersize已經包含了兩個buffer的pingpong機制,可以滿足要求。
跟着Android的代碼進去看了一下這個bufferSizeInBytes的用法,需要收回之前AudioRecord比較簡單是說法了,代碼也是極其複雜的。
read函數
這裏我們重點分析一下read函數,看下數據是如何從驅動,一路上到APK的,流程會很長。
public int read(@NonNull byte[] audioData, int offsetInBytes, int sizeInBytes) {
return read(audioData, offsetInBytes, sizeInBytes, READ_BLOCKING);
}
READ_BLOCKING/READ_NON_BLOCKING 這裏可以選擇block或者nonbblock的方式,大部分我想應該是BLOCK的,應用方便處理,而且音頻是流式的,理論上不會阻塞。
private native final int native_read_in_byte_array(byte[] audioData,
int offsetInBytes, int sizeInBytes, boolean isBlocking);
上面還是在java裏面,下面開始進到native代碼裏面,這裏欠一篇java和jni的調用的分析
frameworks/base/core/jni/android_media_AudioRecord.cpp
template <typename T>
static jint android_media_AudioRecord_readInArray(JNIEnv *env, jobject thiz,
T javaAudioData,
jint offsetInSamples, jint sizeInSamples,
jboolean isReadBlocking) {
frameworks/av/media/libaudioclient/AudioRecord.cpp
ssize_t AudioRecord::read(void* buffer, size_t userSize, bool blocking)
...... 省略部分不重要的代碼
status_t err = obtainBuffer(&audioBuffer,
blocking ? &ClientProxy::kForever : &ClientProxy::kNonBlocking);
......
size_t bytesRead = audioBuffer.size;
memcpy(buffer, audioBuffer.i8, bytesRead);
......
這裏就是從audioBuffer裏面將數據複製到用戶的buffer裏面,等下返回給APK,那麼audioBuffer又是什麼呢?
status_t AudioRecord::obtainBuffer(Buffer* audioBuffer, const struct timespec *requested,
struct timespec *elapsed, size_t *nonContig)
status = proxy->obtainBuffer(&buffer, requested, elapsed);
audioBuffer->raw = buffer.mRaw;
注意這裏Buffer的定義比較亂,很多結構體裏都有這個定義,Google也發現了這個問題,要統一起來,而且還有一個union在裏面,爲了支持8bit/16bit錄音的使用,所以這個audioBuffer->raw與前面的audioBuffer->i8實際上是一個地址。
所以上面的問題pcm地址audioBuffer->raw = buffer.mRaw;
其中buffer是Proxy::Buffer buffer;
sp<AudioRecordClientProxy> proxy = mProxy;
status = proxy->obtainBuffer(&buffer, requested, elapsed);
變成了這個buffer是怎麼來的,這裏要返回去看這個mProxy是什麼?
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
其中比較重要的參數是cblk和buffers,,再繼續往前追一下,這兩個變量的出處。
iMemPointer = output.cblk ->pointer();
cblk = static_cast<audio_track_cblk_t*>(iMemPointer);
buffers = output.buffers->pointer();
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
可以看出這個AudioRecordClientProxy,只是方便調用cblk和buffers的Proxy,提供了obtainBuffer的接口,來從真正的共享buffer裏面將獲得數據,這裏面的細節不再追進去了,實際上是通過audio_track_cblk_t的一個共享內存,採用read/write指針的方法(mFront/mRear)來看有多少avail的數據,然後獲取指針出來。
重點是output的出處
status_t AudioRecord::createRecord_l(const Modulo<uint32_t> &epoch, const String16& opPackageName)
record = audioFlinger->createRecord(input,
output,
&status);
input裏面包括的是我們想設置的參數,output裏面是AudioServer實際設置的參數,已經分配的共享內存。
問題的重點是前面的cblk和buffers是什麼???
上面還是在APK的進程裏面,下面開始進到AudioServer的進程裏面了,尷尬不,這裏又欠一篇IBinder跨進程調用的分析
const sp<IAudioFlinger>& audioFlinger = AudioSystem::get_audio_flinger();
class IAudioFlinger : public IInterface
virtual sp<media::IAudioRecord> createRecord(const CreateRecordInput& input,
CreateRecordOutput& output,
status_t *status) = 0;
sp<media::IAudioRecord> AudioFlinger::createRecord(const CreateRecordInput& input,
CreateRecordOutput& output,
status_t *status)
//上來先把我們最重要的變量clear一下
output.cblk.clear();
output.buffers.clear();
//從recordTrack當中得到這個共享內存
output.cblk = recordTrack->getCblk();
output.buffers = recordTrack->getBuffers();
再看這個recordTrack,是一個RecordThread創建出來的
recordTrack = thread->createRecordTrack_l(client, input.attr, &output.sampleRate,
input.config.format, input.config.channel_mask,
&output.frameCount, sessionId,
&output.notificationFrameCount,
clientUid, &output.flags,
input.clientInfo.clientTid,
&lStatus, portId);
傳了一大堆參數進去,細節不深究,那麼getCblk得到了什麼?
class RecordTrack : public TrackBase {
class TrackBase : public ExtendedAudioBufferProvider, public RefBase {
sp<IMemory> getCblk() const { return mCblkMemory; }
mCblkMemory = client->heap()->allocate(size);
sp<MemoryDealer> AudioFlinger::Client::heap() const
{
return mMemoryDealer;
}
mMemoryDealer = new MemoryDealer(
audioFlinger->getClientSharedHeapSize(),
(std::string("AudioFlinger::Client(") + std::to_string(pid) + ")").c_str());
getCblk實際得到了一個heap的內存,這個內存分配在AudioServer裏面,write在AudioServer裏面,read在AudioRecord裏面,也就是在應用APK的進程空間。
下面要看下,數據如何從HAL層到mMemoryDealer。
RecordThread -> RecordTrack
bool AudioFlinger::RecordThread::threadLoop()
status_t result = mInput->stream->read(
(uint8_t*)mRsmpInBuffer + rear * mFrameSize, mBufferSize, &bytesRead);
(void)posix_memalign(&mRsmpInBuffer, 32, mRsmpInFramesOA * mFrameSize);
從HAL層裏面將數據讀到這個mRsmpInBuffer,名義上看是resample用的,因爲HAL層出來的採樣率與上層需要的不一定一樣,所以需要一個resample的過程,這個buffer裏面有可能是44100hz的pcm數據。
// loop over each active track
for (size_t i = 0; i < size; i++) {
activeTrack = activeTracks[i];
status_t status = activeTrack->getNextBuffer(&activeTrack->mSink);
一個RecordThread裏面可以有幾個active的RecordTrack,將mRsmpInBuffer的數據針對不同track的需要resample成不同的PCM數據。
status_t AudioFlinger::RecordThread::RecordTrack::getNextBuffer(AudioBufferProvider::Buffer* buffer)
{
ServerProxy::Buffer buf;
buf.mFrameCount = buffer->frameCount;
status_t status = mServerProxy->obtainBuffer(&buf); //還記得前面說的ClientProxy了嗎,這裏對應到了ServerProxy,是同一個buffer沒錯了。
buffer->frameCount = buf.mFrameCount;
buffer->raw = buf.mRaw;
if (buf.mFrameCount == 0) {
// FIXME also wake futex so that overrun is noticed more quickly
(void) android_atomic_or(CBLK_OVERRUN, &mCblk->mFlags);
}
return status;
}
好了,這裏徹底對上了,將mResamplerBufferProvider裏面的原始pcm數據,通過mRecordBufferConverter::convert轉換成需要的samplerate。
framesOut = activeTrack->mRecordBufferConverter->convert(
activeTrack->mSink.raw, activeTrack->mResamplerBufferProvider, framesOut);
再看一下這個converter
mRecordBufferConverter = new RecordBufferConverter(
thread->mChannelMask, thread->mFormat, thread->mSampleRate,
channelMask, format, sampleRate);
很清楚,就是從RecordThread的格式轉換爲RecordTrack的格式,細節不再進去追了,只是一些實現細節。
到這裏以後,從AudioServer到APK的數據流已經清楚了,就是從HAL層到這個mMemoryDealer的共享內存,然後通過mServerProxy和AudioRecordClientProxy對這個共享內存的跨進程同步,複製到AudioRecord的audioBuffer當中,然後複製到Java層。
其他接口
AudioRecord.startRecording();
AudioRecord.stop();
控制開始和結束AudioRecord的API,必須使用
AudioRecord.setNotificationMarkerPosition
AudioRecord.setPositionNotificationPeriod
AudioRecord.setRecordPositionUpdateListener
看起來是可以週期性的notify一下app,不確定什麼場景,難道APK自己計數一下read到的數據長度不行?非得回調?
AudioRecord.getTimestamp
獲取當前錄製音頻的時間戳
AudioRecord用法samplecode
從Android的cts裏面複製出來的代碼,cts是一個理解API很好的代碼庫
mAudioRecord = new AudioRecord(MediaRecorder.AudioSource.DEFAULT, mHz,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT,
AudioRecord.getMinBufferSize(mHz,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT) * 10);
mAudioRecord.startRecording();
while (System.currentTimeMillis() - time < RECORD_TIME) {
Thread.sleep(SLEEP_TIME);
mAudioRecord.read(byteData, 0, BUFFER_SIZE); //只是samplecode, 可以讀到數據,一般應用不會這樣寫
}
mAudioRecord.stop();
後記
大致分析了一下AudioRecord的數據流程,不涉及到HAL層的代碼,因爲各家的實現會有不同,裏面的細節很多,包括IBinder調用/cblk的共享內存/JNI的調用都還沒有詳細分析,留作後面的功課。