Introduction to Plugin Architecture
This guide explains the architectural patterns and data flow of Ant Media Server's plugin system. You'll learn about the different plugin use cases and how data flows between plugins and the server.
The Ant Media plug-in system allows developers to customize the video feed while keeping the server source codes untouched. The plug-in structure has various scenarios to address different developer needs. For example, MCU usage is developed as a plug-in by the Ant Media dev team, demonstrating the wide-range use cases for the plug-in architecture.
In the most basic explanation, a plug-in is added into the regular flow of AMS in ways that we will examine. Once you understand the structure and use cases, you can proceed to building your own plugin.
Here is what a plugin data flow looks like:
You can either get the decoded video frames( IFrameListener interface )or encoded video packets ( IPacketListener interface ) from Ant Media Server, then you can do whatever customization you require to do with them.
Use cases
Asynchronous case
Your plugin only gets the video frame and works on that video frame but doesn't feed the manipulated frame to the server again. In this case, the work should be in a different thread and onVideoFrame should return the same AVFrame that passed to the method.
For example: you may have such a plugin that only collects the frame for statistics
Synchronous case
Your plugin gets the video frame and works on that video frame and also feeds the manipulated frame to the server again. In this case, you should make the work in the caller thread and return the manipulated AVFrame.
For example: you may have such a plugin that adds watermark addition
Last point case
Your plugin gets the video frame and works on that video frame and there is no need to publish this AVFrame or feed this frame to other plugins. In this case, your plugin is the last point for the AVFrame in the execution. For this case, you should return a null value.
For example, you may have such a plugin that streams the AVFrame with a new protocol that isn't available in AMS or you may only record the frames.
First point case (custom broadcast)
Ant Media Server already implements many protocols like WebRTC, RTMP, SRT and stream source pulling. But you may want to implement other streaming protocols for publishing to your server. In this case, you can create a CustomBroadcast and feed it from your plugin.
For example, you may have a plugin that receives frames from an external source and feeds them to the CustomBroadcast. Then CustomBroadcast can publish this stream with WebRTC, HLS, or DASH.
For detailed implementation steps and code examples, see the Developing Plugins guide.