Source Code Organization

This discussion is based on OpenHarmony 3.2 Release source code from the player_framework module under the foundation directory. The source repository structure consists of two primary sections:

The frameworks directory contains two key components: a JavaScript/ArkTS binding layer (napi) that bridges application code with the native framework, and native interfaces that expose lower-level capabilities to the napi layer. The services directory houses multimedia system services corresponding to the media_service process in OpenHarmony, with the engine subdirectory containing the core implementation logic. Notably, services mirrors certain interfaces found in frameworks/native to enable client-server communication patterns.

Core Capabilities

The media subsystem provides URL-based playback of local audio and video content, but its scope extends considerably beyond this bassic functionality. It serves as the foundation for the entire multimedia subsystem. The architecture can be categorized into four primary functional areas:

• AVCodec: Encoding and decoding capabilities
• Player: Playback functionality
• Recorder: Recording capabilities
• AVMetadata: Media metadata extraction and management

These modules operate independently while sharing similar structural patterns. The general execution flow follows this sequence: OpenHarmony applications invoke Napi interfaces, which route calls to Client-side framework code, and subsequently communicate with Engine capabilities through inter-process communication mechanisms.

Engine Implementation Details

GStreamer Framework Integration

As illustrated in OpenHarmony's official architecture diagrams, various engines rely on GStreamer for core multimedia processsing. GStreamer is an open-source multimedia framework that handles demuxing of popular container formats including AVI, MP4, and Ogg, decodes video codecs such as H.264, H.265, and VPx, and supports streaming protocols including RTSP and RTMP. Its pipeline and plugin architecture provides substantial flexibility and operational control.

Pipeline and Plugin Architecture

Examining the media subsystem source code reveals that each Engine manages a Pipeline with key lifecycle methods including Start(), Prepare(), and Stop(). Consider the Recorder engine implementation, which follows this pattern.

Multimedia applications typically involve multiple processing stages. For a simple local Ogg file playback scenario, the workflow includes: demuxing the Ogg container, extracting encoded audio and video streams, decoding each stream separately, and delivering audio PCM frames to the speaker while rendering video frames to the display. In GStreamer terminology, the complete workflow constitutes a Pipeline, while each processing stage is an Element (or plugin).

In GStreamer, sink elements represent data entry points while src elements represent data exit points. Elements perform specific data transformations, and plugins serve as loadable code modules that contain one or more elements. According to the official GStreamer documentation: plugins are dynamically linked libraries that can be loaded at runtime, encapsulating one or more elements to extend framework functionality.

OpenHarmony leverages both pre-built GStreamer pipelines (such as playbin) and custom implementations (such as codecbin). The bin concept in GStreamer represents a higher-level abstraction than pipeline, essentially functioning as an executable pipeline. The lifecycle methods like Start() and Stop() serve as pipeline state controllers. After configuring pipeline parameters, Start() transitions the pipeline to an active state, allowing data to flow from input sources through processing elements to output destinations.

Following state configuration, the pipeline operates autonomously. Each element advertises its supported data types and parameters in advance, and pipeline structure often adapts dynamically based on input content characteristics. Developers primarily need to ensure the required plugins are registered with GStreamer or utilize its built-in plugin collection.

Implementation Example

Enabling RTMP Streaming Support

GStreamer provides comprehensive multimedia capabilities, yet OpenHarmony's native player lacks direct RTMP streaming support. Examining the player engine's BUILD.gn configuration reveals the plugin dependencies (located at foundation\multimedia\player_framework\services\engine\GStreamer\BUILD.gn).

The gstplugins_bad_packages dependency references compiled libraries from the GStreamer gstplugins_bad module, which contains the RTMP protocol plugin. Analysis of the BUILD.gn file shows that RTMP components are not included in the default build configuration for OpenHarmony 3.2. Enabling RTMP support requires adding the relevant entries to this build file.

The following configuration demonstrates the necessary modifications:

group("gstplugins_bad_packages") {
  deps = [
    ":gstcurl",
    ":gsthls",
    ":gstmpegtsdemux",
    ":gstvideoparsersbad",
    ":custom_rtmp_plugin"
  ]
}

The plugin source files must be defined in a corresponding source set. GStreamer plugin directories contain meson.build files specifying the required source files:

ohos_source_set("custom_rtmp_implementation") {
  sources = [
    "gst/rtmp2/gstrtmp2.c",
    "gst/rtmp2/gstrtmp2element.c",
    "gst/rtmp2/gstrtmp2locationhandler.c",
    "gst/rtmp2/gstrtmp2sink.c",
    "gst/rtmp2/gstrtmp2src.c",
    "gst/rtmp2/rtmp/amf.c",
    "gst/rtmp2/rtmp/rtmpchunkstream.c",
    "gst/rtmp2/rtmp/rtmpclient.c",
    "gst/rtmp2/rtmp/rtmpconnection.c",
    "gst/rtmp2/rtmp/rtmphandshake.c",
    "gst/rtmp2/rtmp/rtmpmessage.c",
    "gst/rtmp2/rtmp/rtmputils.c"
  ]
  configs = [ ":gst_plugins_config" ]
}

ohos_shared_library("custom_rtmp_plugin") {
  deps = [
    ":custom_rtmp_implementation",
    "//third_party/glib:glib",
    "//third_party/glib:gobject",
    "//third_party/GStreamer/gstplugins_bad:gstadaptivedemux",
    "//third_party/GStreamer/gstplugins_bad:gsturidownloader",
    "//third_party/GStreamer/gstplugins_base:gsttag",
    "//third_party/GStreamer/GStreamer:gstbase",
    "//third_party/GStreamer/GStreamer:GStreamer"
  ]
  relative_install_dir = "media/plugins"
  part_name = "multimedia_player_framework"
  subsystem_name = "multimedia"
}

After incorporating the RTMP plugin into the build system, the playbin pipeline automatically detects and incorporates it when constructing the playback pipeline for RTMP sources. The plugin discovery and pipeline assembly happen dynamically during playback initialization.