H264NvCodec Encoder Module Issues in Docker Environment

[kushaljain-apra]

Describe the bug
The H264NvCodec Encoder module becomes unstable when running in Docker, despite working correctly initially. When using MP4Writer as a sink, only the first output file is playable while subsequent files are corrupted. When using RTSP Pusher as a sink, the stream is only playable once and then becomes inaccessible. This issue affects long-running encoding sessions and multiple file output scenarios.

To Reproduce
Steps to reproduce the behavior:

1. Start a docker container with Aprapipe Framework installed.
2. Create a pipeline with one of these configurations:

For MP4Writer sink:

FileReader/MP4Reader -> Decoder -> H264NvCodec Encoder -> MP4Writer

For RTSP Pusher sink:

FileReader/MP4Reader -> Decoder -> H264NvCodec Encoder -> RTSP Pusher

3. Run the pipeline with the following encoder configuration:

auto cuContext = apracucontext_sp(new ApraCUcontext());
uint32_t gopLength = 30;
uint32_t bitRateKbps = 1000;
uint32_t frameRate = 30;
bool enableBFrames = true;
H264EncoderNVCodecProps::H264CodecProfile profile = H264EncoderNVCodecProps::HIGH;

auto encoder = boost::shared_ptr<Module>(new H264EncoderNVCodec(
    H264EncoderNVCodecProps(bitRateKbps, cuContext, gopLength, frameRate, profile, enableBFrames)
));

4. Observe the following issues:
   • For MP4Writer: Only the first output file is playable, subsequent files are corrupted
   • For RTSP Pusher: Stream is only playable once, then becomes inaccessible

Expected behavior

• For MP4Writer: All output files should be playable and contain valid H264 encoded video
• For RTSP Pusher: The stream should remain accessible and playable for multiple connections
• The encoder should maintain stable performance throughout the entire encoding session

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows (Docker Desktop Contianer)
• Cuda Version 11.8
• Architecture x86_64

Additional context

• The issue appears to be related to resource management in the Docker environment
• The encoder uses NVIDIA's NVENC API through the H264EncoderNVCodec module
• The problem occurs after some time of operation, suggesting potential resource leaks or buffer management issues
• The issue affects both Windows and Linux environments when running in Docker
• The encoder works correctly when running directly on the host system
• The problem is more pronounced in long-running encoding sessions
• Current workaround is to restart the pipeline periodically
• The issue might be related to how Docker handles GPU resources and how the NVENC API interacts with the containerized environment

Related Code
Key components involved:

• H264EncoderNVCodecHelper.cpp: Main encoder implementation
• H264EncoderNVCodec.cpp: Module interface
• NVCodecResources class: Resource management
• Buffer handling in Detail::encode() method

Would you like to work on the issue?
Please state if this issue should be assigned to you or who you think could help to solve this issue.