Frame and Fields#

In Intel® VPL terminology, a frame (also referred to as frame surface) contains either a progressive frame or a complementary field pair. If the frame is a complementary field pair, the odd lines of the surface buffer store the top fields and the even lines of the surface buffer store the bottom fields.

Frame Surface Management#

During encoding, decoding, or video processing, cases arise that require reserving input or output frames for future use. For example, when decoding, a frame that is ready for output must remain as a reference frame until the current sequence pattern ends. The usual method to manage this is to cache the frames internally. This method requires a copy operation, which can significantly reduce performance.

Intel® VPL has two approaches to avoid the need for copy operations. The legacy approach uses a frame-locking mechanism that works as follows:

  1. The application allocates a pool of frame surfaces large enough to include Intel® VPL function I/O frame surfaces and internal cache needs. Each frame surface maintains a Locked counter, which is part of the mfxFrameData structure. The Locked counter is initially set to zero.

  2. The application calls a Intel® VPL function with frame surfaces from the pool whose Locked counter is set as appropriate for the desired operation. For decoding or video processing operations, where Intel® VPL uses the surfaces to write, the Locked counter should be equal to zero. If the Intel® VPL function needs to reserve any frame surface, the Intel® VPL function increases the Locked counter of the frame surface. A non-zero Locked counter indicates that the calling application must treat the frame surface as “in use.” When the frame surface is in use, the application can read but cannot alter, move, delete, or free the frame surface.

  3. In subsequent Intel® VPL executions, if the frame surface is no longer in use, Intel® VPL decreases the Locked counter. When the Locked counter reaches zero, the application is free to do as it wishes with the frame surface.

In general, the application should not increase or decrease the Locked counter since Intel® VPL manages this field. If, for some reason, the application needs to modify the Locked counter, the operation must be atomic to avoid a race condition.

Intel® VPL API version 2.0 introduces the mfxFrameSurfaceInterface structure which provides a set of callback functions for the mfxFrameSurface1 structure to work with frame surfaces. This interface defines mfxFrameSurface1 as a reference counted object which can be allocated by Intel® VPL or the application. The application must follow the general rules of operation with reference counted objects. For example, when surfaces are allocated by Intel® VPL during MFXVideoDECODE_DecodeFrameAsync() or with the help of MFXMemory_GetSurfaceForVPP() or MFXMemory_GetSurfaceForVPPOut() or MFXMemory_GetSurfaceForEncode(), the application must call the corresponding mfxFrameSurfaceInterface::Release function for the surfaces that are no longer in use.

Attention

Note that the Locked counter defines read/write access policies and the reference counter is responsible for managing a frame’s lifetime.

The second approach to avoid the need for copy operations is based on the mfxFrameSurfaceInterface and works as follows:

  1. Intel® VPL or the application allocates a frame surface and the application stores a value of reference counter obtained through mfxFrameSurfaceInterface::GetRefCounter.

  2. The application calls a Intel® VPL function with the frame surface. If Intel® VPL needs to reserve the frame surface it increments the reference counter through the mfxFrameSurfaceInterface::AddRef call. When the frame surface is no longer in use by the Intel® VPL it decrements reference counter through the mfxFrameSurfaceInterface::Release call which returns the reference counter to the original value.

  3. The application checks the reference counter of the frame surface and when it is equal to the original value after allocation, it can reuse the reference counter for subsequent operations.

Note

All mfxFrameSurface1 structures starting from mfxFrameSurface1::mfxStructVersion = {1,1} support the mfxFrameSurfaceInterface.