Competition Overview

This track requires participants to engineer a classifier capable of distinguishing between authentic and synthetically manipulated audiovisual sequences. Models must produce a continuous probability estimate reflecting the likelihood of adversarial alteration. The core challenge emphasizes architectural resilience against evolving synthesis pipelines and deployment variability.

Dataset Architecture & Evaluation Protocol

Data partitions are governed by structured label files corresponding to training and validation subsets. Each entry pairs an .mp4 filepath with a binary ground-truth indicator (1 for synthetic, 0 for real). Initial enumeration confirms a class distribution ratio of approximately 3:1 across splits.

Predictive performance is measured using standard forensic detection metrics:

• True Positive Rate (TPR): TP / (TP + FN)
• False Positive Rate (FPR): FP / (FP + TN)

Definitions: TP correctly identifies manipulated content, TN correctly validates authentic material, FP misclassifies genuine footage as synthetic, and FN fails to detect adversarial manipulation.

Baseline Implementation

Environment Verification

Confirm partition cardinality using shell directives within the notebook runtime:

!wc -l /kaggle/input/ffdv-sample-dataset/ffdv_phase1_sample/train_labels.txt
!wc -l /kaggle/input/ffdv-sample-dataset/ffdv_phase1_sample/val_labels.txt

Render media samples for quality assurance using IPython's native display engine:

from IPython.display import Video
Video("/kaggle/input/ffdv-sample-dataset/ffdv_phase1_sample/valset/00882a2832edbcab1d3dfc4cc62cfbb9.mp4", embed=True)

Dependency Management & Runtime Configuration

Initialize the workspace with the following dependencies:

!pip install moviepy librosa matplotlib numpy timm opencv-python-headless

Import core frameworks and adjust CUDA backend parameters. Enforcing deterministic operation fixes the Random Number Generator (RNG) state, guaranteeing gradient consistency at the cost of minor throughput reduction. Activating benchmark mode triggers an initial profiling phase where Convolutional Neural Network kernels select optimal implementation strategies, yielding significant acceleration during epoch loops.

import os
import glob
import time
import random
import numpy as np
import pandas as pd
import cv2
import torch
import timm
import librosa
import moviepy.editor as mp
from PIL import Image
import torch.nn as nn
import torch.optim as optim
import torchvision.transforms as T
import torchvision.datasets as D
import torch.utils.data as data_utils

# Fixed seeding for reproducible experiments
SEED = 42
torch.manual_seed(SEED)
torch.cuda.manual_seed_all(SEED)
np.random.seed(SEED)
random.seed(SEED)

# CUDA execution flags
torch.backends.cudnn.deterministic = True  # Locks RNG for consistent backward passes
torch.backends.cudnn.benchmark = True      # Profiles and caches peak convolution kernels

Spectral Feature Extraction Pipeline

Cross-modal forgery detection frequently exploits acoustic anomalies invisible to visual-only architectures. Converting extracted audio streams into Mel-spectrograms yields compact, frequency-domain representations suitable for CNN backbones. The following utility handles asynchronous demuxing and normalization:

def generate_spectral_map(video_source: str, band_count: int = 128, freq_cap: int = 8000, output_dims: tuple = (256, 256)) -> np.ndarray:
    """
    Transcodes a video file into a dimension-normalized Mel-spectrogram.
    
    Args:
        video_source: Absolute path to target .mp4 asset.
        band_count: Resolution along the frequency axis.
        freq_cap: Maximum audible threshold for filterbank construction.
        output_dims: Target resolution (height, width) for batch compatibility.
        
    Returns:
        uint8-encoded grayscale matrix.
    """
    scratch_audio = "interim_track.wav"
    
    try:
        # Stream extraction
        raw_clip = mp.VideoFileClip(video_source)
        raw_clip.audio.write_audiofile(scratch_audio, verbose=False, logger=None)
        
        # Waveform ingestion and feature mapping
        signal_tensor, sampling_rate = librosa.load(scratch_audio, sr=None)
        spec_matrix = librosa.feature.melspectrogram(y=signal_tensor, sr=sampling_rate, n_mels=band_count)
        
        # Dynamic range compression and scaling
        db_representation = librosa.power_to_db(spec_matrix, ref=np.max)
        scaled_matrix = cv2.normalize(db_representation, None, 0, 255, cv2.NORM_MINMAX).astype(np.uint8)
        
        # Spatial uniformity enforcement
        return cv2.resize(scaled_matrix, dsize=(output_dims[1], output_dims[0]), interpolation=cv2.INTER_LINEAR)
        
    finally:
        # Automated cleanup
        if os.path.isfile(scratch_audio):
            os.remove(scratch_audio)

This transformation isolates temporal-frequency patterns from the visual container, producing standardized tensors ready for multimodal fusion architectures.