Android OMAP4 Ducati 和 OpenMax IL介紹

Android OMAP4 Ducati 和 OpenMax IL介紹

Update
TI 發佈的這個文章，個人感覺特別好，想研究的朋友可以看一下：
http://processors.wiki.ti.com/index.php/OMX_EZSDK_Examples
轉載一篇很好的文章，地址：http://book.51cto.com/art/201101/243147.htm，感謝分享
文章主要個大家介紹OMAP平臺OpenMax IL的硬件實現過程。其中包括TI OpenMax IL實現的結構和機制，TI OpenMax IL的核心和公共內容，一個TI OpenMax IL組件的實現三個部分。
來源於一本書：Android系統級深入開發--移植與調試 

這裏說說TI 的OMAP4 Ducati
轉載一篇介紹Ducati 的文章，寫的也不錯：http://www.61ic.com/Article/DaVinci/OMAP3x/201111/39340.html
同樣感謝分享，共同進步

當然TI官網資料是最好的瞭解Ducati 的資料：http://omappedia.com/wiki/Ducati_For_Dummies

我想着重看的在這裏：

ISS Overview

The Imaging Sub System (ISS) deals with the processing of pixel data coming from an external image sensor or from memory. ISS as a component forms part of still image capture, camera viewfinder and video record use-cases. Figure below shows the Imaging subsystem constituents.

ISS subparts include:

• Two CSI2: Receives data from sensor.
• CCP2: Receives data from sensor or reads from memory.
• ISP: Pre/post processing operations on pixel data (received from sensor or from memory).
• SIMCOP: Imaging accelerator.
• BTE: Burst Translation Engine. Converts from raster to 2D tiled order and vice versa, for data write and data read respectively.
• CBUFF: Circular Buffer for linear space, physically located in memory.

Among these, ISP and SIMCOP are the two major processing blocks.

IPC on Ducati

DOMX
it

Distributed OpenMAX? Integration Layer (IL) framework (DOMX) is the multimedia framework used for Inter Processor Communication (IPC) on OMAP4 to enable easy and transparent use of OpenMAX IL (OMX) components, on remote processing cores like Ducati. It is built upon Remote Procedure Call (RPC) concepts.

On OMAP4, many of the (high performance) video/imaging codecs and algorithms execute on the hardware accelerator subsystem (Ducati), but the application/client which invokes these codecs and algorithms executes on the host processor (Cortex A9). The IPC between Cortex A9 and Ducati is handled by SysLink. The only interface that Ducati exposes to Cortex A9 is OMX.

DOMX framework provides a seamless abstraction for Cortex A9 and Ducati to communicate via SysLink. This framework abstracts all details like buffer address translations, maintaining cache-external memory coherence, exact mechanisms used to communicate to remote cores, etc.

IPC Overview

Based on the functionalities involved, DOMX is layered into OMX-Proxy, OMX-RPC (OpenMAX API specific Remote Procedure Call) and RCM (Remote Command Message is a generic RPC service maintained by SysLink).

All IPC between Cortex A9 and Ducati has to pass through these layers. Figure below shows a block diagram of communication between these two processing cores by means of DOMX. Note that the Host processor is Cortex A9 and Remote processor is Ducati sub system. There are proxies for each component (H.264 encoder, MPEG-4 Visual decoder, JPEG decoder, etc.) The code common to all these proxies is extracted into Proxy Common block, with only the component specific parts in the individual proxies.

A Proxy is a thin component that provides OpenMAX interface APIs for peer OpenMAX components and clients. A proxy component as the name suggests is a ‘Proxy’ on local core that corresponds to a real component implemented on a remote core. It is a thin layer exporting the OpenMAX interfaces.

The proxy component’s principal functionality is to forward calls to remote component using RPC Stub functions and RCM Client-Server infrastructure. Proxies also forward client call backs to the IL Client for calls that originated on the remote core. All buffer address mapping, maintenance, etc. are the responsibility of the proxy component.

OMX-RPC layer implements Stubs (packing the information of an API call) and Skeletons (unpacking the information of an API call) for all the OpenMAX APIs. For calls originating on local core RPC Stub allocates message, packs the arguments and invokes appropriate remote function based on OpenMAX API using RCM layer.

The RPC Skeleton (on the remote core) receives the packed data, unpacks it and invokes appropriate remote function with the unpacked arguments. RPC Skeleton is also responsible for packing any return values associated with the remote call and sending it back to the local core. On the return path, the RPC Stub extracts the remote invocation ‘Return’ value, frees the message and returns the remote invocation ‘Return’ value to the caller.

The RCM layer is provided by SysLink and abstracts core specific IPC and transport layers and manages the messaging between processing cores. RCM is implemented on Client-Server paradigm. RCM client implements the transport mechanism to send a message to a given server. The server is identified by a name that is unique across the entire system. On reception of the message the RCM Server extracts the function index from it and invokes the associated remote function. To enable index based remote invocation the server maintains an Index-Remote Function table that is populated at initialization or by registration. The client will discover the index for a given remote function by querying the server by the remote function name.

All communication from host to remote OMX component goes via a proxy which exposes the same OMX interface as the real component. The proxy sends message to RPC layer which packs data and uses the RCM client-server model to send this data to remote core via SysLink. On remote core, the RPC layer unpacks the message received and sends it to the real OMX component. Callbacks if any, follow a similar path in reverse direction.

Thus the DOMX layer ensures that OMX components and the clients which invoke these components remain unaware of any IPC involved. As far as the components and their clients are concerned there is no difference between a local and remote call. DOMX framework ensures that a remote call is executed in the same way as a local call.

Here is a sequence of all the OMX API calls that the hello_video sample program makes (from raspberry pi raspbian sample image).

1. OMX_Init()
2. OMX_GetHandle(pHandleDecode, "OMX.broadcom.video_decode", pAppData, &callbacks)
3. OMX_GetComponentVersion(pHandleDecode, pszNameArray, pCompVersion, pSpecVersion, pUID) name: "OMX.broadcom.video_decode:22", version: 0's, spec version: 1.1.2, uid same as name
4. OMX_GetParameter(pHandleDecode, OMX_IndexParamAudioInit, &pPorts) (nPorts returned as 0)
5. OMX_GetParameter(pHandleDecode, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 2) (ports.nStartPortNumber returned 130)
6. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 130, NULL)
7. (waits for event callback to fire in other thread to indicate that port has been disabled)
8. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 131, NULL)
9. (waits for event callback to fire in other thread to indicate that port has been disabled)
10. OMX_GetHandle(pHandleRender, "OMX.broadcom.video_render", pAppData, &callbacks)
11. OMX_GetComponentVersion(pHandleRender, ...)
12. OMX_GetParameter(pHandleRender, OMX_IndexParamAudioInit, &pPorts) (nPorts returned as 0)
13. OMX_GetParameter(pHandleRender, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 1) (ports.nStartPortNumber returned 90)
14. OMX_GetHandle(pHandleClock, "OMX.broadcom.clock", ...)
15. OMX_GetParameter(pHandleClock, OMX_IndexParamOtherInit, ...) (nPorts returned as 6) (ports.nStartPortNumber returned UNKNOWN, FIND OUT)
16. OMX_SetParameter(pHandleClock?, OMX_IndexConfigTimeClockState, &cState)
17. OMX_GetHandle(pHandleSched, "OMX.broadcom.video_scheduler", ...)
18. OMX_GetParameter(pHandleSched, OMX_IndexParamVideoInit, &pPorts) (nPorts returned 2) (ports.nStartPortNumber returned 10)
19. OMX_GetParameter(pHandleSched, OMX_IndexParamAudioInit, &pPorts) (nPorts returned 1) (ports.nStartPortNumber returned 12)
20. OMX_GetState(pHandleClock, &state) (returns state of OMX_StateLoaded)
21. OMX_SendCommand(pHandleClock, OMX_CommandStateSet, OMX_StateIdle, NULL)
22. (waits for command to complete)
23. OMX_SendCommand(pHandleClock, OMX_CommandPortDisable, 80, NULL)
24. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 12, NULL)
25. (waits for both previous commands to finish)
26. OMX_GetParameter(pHandleClock, OMX_IndexParamNumAvailableStreams, &param) (returned an error because source port does not use streams)
27. OMX_SetupTunnel(pHandleClock, 80, pHandleSched, 12);
28. OMX_SendCommand(pHandleClock, OMX_CommandPortEnable, 80, NULL);
29. OMX_SendCommand(pHandleSched, OMX_CommandPortEnable, 12, NULL);
30. OMX_GetState(pHandleSched, &state) (returns OMX_StateLoaded)
31. (wait for enable tunnel command to complete for pHandeSched)
32. OMX_SendCommand(pHandleSched, OMX_CommandStateSet, OMX_StateIdle, NULL)
33. (wait for state change command to complete)
34. (wait for enable tunnel to complete for pHandleClock)
35. OMX_SendCommand(pHandleClock, OMX_CommandStateSet, OMX_StateExecute, NULL)
36. OMX_SendCommand(pHandleDecode, OMX_CommandStateSet, OMX_StateIdle, NULL)
37. OMX_SetParameter(pHandleDecode, OMX_IndexParamVideoPortFormat, &format)
38. OMX_GetParameter(pHandleDecode, OMX_IndexParamPortDefinition, &portdef) (index is 130)
39. (portdef.bEnabled == OMX_FALSE, portdef.nBufferCountActual == 20, portdef.nBufferSize == 81920, portdef.nBufferAlignment == 16, portdef.eDir == OMX_DirInput, portdef.eDomain == OMX_PortDomainVideo)
40. OMX_GetState(pHandleDecode, &state) (state is OMX_StateIdle)
41. OMX_SendCommand(pHandleDecode, OMX_CommandPortEnable, 130, NULL)
42. (allocates buffer of 81920 bytes in size)
43. OMX_UseBuffer(pHandleDecode, ?, 130, NULL, 81920, bufThatWasAllocated)
44. (this allocation of buffer followed by the OMX_UseBuffer call loops 20 times)
45. OMX_SendCommand(pHandleDecode, OMX_CommandStateSet, OMX_StateExecuting, NULL)
46. (get a unique buffer.. somehow.. lol)
47. (fread video stream into buffer)
48. (set buf->nFilledLen to the bytes we read, buf->nOffset = 0, buf->nFlags = OMX_BUFFERFLAG_STARTTIME)
49. OMX_EmptyThisBuffer(pHandleDecode, buf) where buf points to a OMX_BUFFERHEADTYPE struct
50. Keep doing the former starting with getting the unique buffer until we notice that port 131 has received a OMX_EventPortSettingsChanged event
51. begin one-time step
52. Set up tunnel from port 131 (decode) to port 10 (scheduler)
53. change video scheduler's state to executing
54. Set up tunnel from port 11 (scheduler) to port 90 (render)
55. change render state to executing
56. end of the one-time-step
57. continue looping (getting a unique buffer, etc) until fread gets end-of-file
58. OMX_EmptyThisBuffer(pHandleDecode, buf) where buf->nFlags = OMX_BUFFERFLAG_TIME_UNKNOWN | OMX_BUFFERFLAG_EOS and nFilledLen is 0
59. Wait for event from video render, where the event is OMX_EventBufferFlag of OMX_BUFFERFLAG_EOS
60. flush tunnels so that decoder can disable its input port by doing the following:
61. OMX_SendCommand(pHandleDecode, OMX_CommandFlush, 131, NULL)
62. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 10, NULL)
63. wait for two flush events to be received
64. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 11, NULL)
65. OMX_SendCommand(pHandleRender, OMX_CommandFlush, 90, NULL)
66. wait for two flush events to be received
67. OMX_SendCommand(pHandleClock, OMX_CommandFlush, 80, NULL)
68. OMX_SendCommand(pHandleSched, OMX_CommandFlush, 12, NULL)
69. wait for two flush events to be received
70. OMX_GetParameter(pHandleDecode, OMX_IndexParamPortDefinition, &portdef) (where port is 130)
71. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 130, NULL)
72. OMX_FreeBuffer(pHandleDecode, 130, ?) <-- loop 20 times for each buffer
73. wait for port disable to finish
74. OMX_SendCommand(pHandleDecode, OMX_CommandPortDisable, 131, NULL)
75. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 10, NULL)
76. wait for port disable to finish
77. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 11, NULL)
78. OMX_SendCommand(pHandleRender, OMX_CommandPortDisable, 90, NULL)
79. wait for port disable to finish
80. OMX_SendCommand(pHandleClock, OMX_CommandPortDisable, 80, NULL)
81. OMX_SendCommand(pHandleSched, OMX_CommandPortDisable, 12, NULL)
82. wait for port disable to finish
83. OMX_SetupTunnel(pHandleDecode, 131, NULL, 0)
84. OMX_SetupTunnel(pHandleSched, 10, NULL, 0)
85. OMX_SetupTunnel(pHandleSched, 11, NULL, 0)
86. OMX_SetupTunnel(pHandleRender, 90, NULL, 0)
87. OMX_SetupTunnel(pHandleClock, 80, NULL, 0)
88. OMX_SetupTunnel(pHandleSched, 12, NULL, 0)
89. change state of every handle to OMX_StateIdle (in random order for some unexplained reason)
90. change state of every handle to OMX_StateLoaded (in random order for some unexplained reason)
91. OMX_FreeHandle(pHandleDecode)
92. OMX_FreeHandle(pHandleRender)
93. OMX_FreeHandle(pHandleClock)
94. OMX_FreeHandle(pHandleSched)
95. OMX_Deinit()