VMAF Quality Analysis: Objective Video Quality Measurement

Beyond Bitrate: Measuring What Viewers See

Bitrate tells you how much data a stream uses. It tells you nothing about how good it looks.

A 10 Mbps stream of static slides looks perfect. A 10 Mbps stream of fast-motion sports may be full of artifacts. Two encoders at the same bitrate can produce dramatically different quality depending on their algorithms, presets, and tuning.

VMAF (Video Multi-Method Assessment Fusion) solves this. Developed by Netflix, VMAF is a machine-learning-based metric that predicts how a human viewer would rate video quality on a scale of 0 to 100. It correlates more accurately with subjective quality than traditional metrics like PSNR or SSIM.

Vajra Cast integrates VMAF scoring directly, giving you objective quality measurement on every route with a single click.

How VMAF Works

VMAF combines multiple elementary quality metrics using a support vector machine (SVM) trained on thousands of human quality ratings:

1. Visual Information Fidelity (VIF): measures how much visual information is preserved after encoding. Works at multiple spatial scales.
2. Detail Loss Metric (DLM): quantifies the loss of fine detail and texture.
3. Motion: accounts for temporal masking (viewers are less sensitive to artifacts during fast motion).

These metrics are fused into a single score:

For professional broadcasting, a VMAF score of 80+ is the typical target. Scores above 93 indicate diminishing returns. You could reduce bitrate and save bandwidth without perceptible quality loss.

VMAF in Vajra Cast

How It Works

Vajra Cast performs VMAF analysis by comparing the input (reference) to the output (distorted) of a route:

Input (original) ────────────────────┐
                                     ├→ VMAF comparison → Score
Output (encoded/transcoded) ─────────┘

This comparison is meaningful because:

• Passthrough routes should score 100 (no quality loss). Anything less indicates a problem
• Transcoded routes show the exact quality impact of your encoding settings
• Trend analysis reveals quality degradation over time (encoder drift, network issues)

Running a Quality Check

1. Open the route detail view
2. Click Quality Analysis
3. Vajra Cast captures a sample from both input and output
4. VMAF scoring runs on the sample
5. Results appear in the dashboard: VMAF score, PSNR, SSIM

The analysis runs without interrupting the live stream. It samples a configurable duration (default: 10 seconds) at the stream’s native resolution.

Automated Scheduled Analysis

For 24/7 routes, configure periodic quality checks:

curl -X POST https://your-server:3000/api/routes/route-47/quality-check \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "schedule": "*/30 * * * *",
    "duration": 10,
    "metrics": ["vmaf", "psnr", "ssim"]
  }'

This runs a 10-second quality check every 30 minutes and stores the results in the metrics history.

Setting Quality Thresholds

Why Thresholds Matter

VMAF scoring is most valuable when automated. Instead of manually checking quality, set thresholds and let the system alert you when quality drops below acceptable levels.

Configuring Thresholds

In the route settings, set:

• Minimum VMAF score: trigger an alert when the score falls below this value (e.g., 80)
• Minimum PSNR: secondary quality metric threshold (e.g., 35 dB)
• Alert action: log, webhook, or email notification

Threshold Recommendations by Use Case

VMAF for Transcoding Validation

When configuring hardware transcoding, VMAF helps you find the optimal quality-bitrate trade-off:

1. Start with your target bitrate (e.g., 6 Mbps for 1080p)
2. Run a VMAF analysis
3. If VMAF > 93: you can reduce bitrate (saving bandwidth) without perceptible quality loss
4. If VMAF < 80: increase bitrate or adjust encoding preset
5. Iterate until you find the sweet spot

This is more reliable than eyeballing quality or relying on bitrate alone.

Alerting on Quality Degradation

Prometheus Integration

VMAF scores are exposed via the Prometheus metrics endpoint:

vajracast_vmaf_score{route_id="route-47",route_name="Stadium Main"} 88.4
vajracast_psnr_db{route_id="route-47",route_name="Stadium Main"} 38.2
vajracast_ssim{route_id="route-47",route_name="Stadium Main"} 0.965

Grafana Alert Rule

alert: VajraCastQualityDegradation
expr: vajracast_vmaf_score < 80
for: 5m
labels:
  severity: warning
annotations:
  summary: "Video quality below threshold on {{ $labels.route_name }}"
  description: "VMAF score is {{ $value }} (threshold: 80). Check encoding settings or input quality."

The for: 5m clause prevents alerts on transient quality dips (e.g., scene changes with extreme motion). A sustained 5-minute drop below threshold indicates a real problem.

Quality Dashboard

Track VMAF scores over time in Grafana:

• VMAF timeline: plot scores over 24 hours to spot patterns (quality drops during peak hours, for example)
• Score distribution: histogram of VMAF scores across all routes
• Quality vs. bitrate scatter: identify routes that are over-bitrated (high VMAF, high bitrate) or under-bitrated (low VMAF, low bitrate)

VMAF vs. Other Quality Metrics

PSNR (Peak Signal-to-Noise Ratio)

• What it measures: Pixel-level difference between original and encoded frames
• Strengths: Fast to compute, well-understood
• Weaknesses: Does not correlate well with human perception. A blur and a sharp artifact can have the same PSNR but look very different to viewers.
• Vajra Cast: Available alongside VMAF for reference

SSIM (Structural Similarity Index)

• What it measures: Structural similarity: luminance, contrast, and structure comparison
• Strengths: Better perceptual correlation than PSNR
• Weaknesses: Less accurate than VMAF for video (originally designed for still images)
• Vajra Cast: Available alongside VMAF for reference

Why VMAF Wins

VMAF was trained specifically on video content with human quality ratings. It accounts for factors that PSNR and SSIM miss:

• Temporal masking: viewers tolerate more artifacts during fast motion
• Content complexity: complex scenes need more bits than simple ones
• Display resolution: artifacts matter more on large screens

Netflix, YouTube, and major broadcasters use VMAF as their primary quality metric. Vajra Cast exposes all three (VMAF, PSNR, SSIM) so you can cross-reference, but VMAF should be your primary decision-making metric.

Practical Workflows

Workflow 1: Encoding Preset Validation

Before going live with a new hardware transcoding configuration:

1. Set up the route with your intended encoding settings
2. Feed a representative test clip (sports footage, talking head, graphics, whatever matches your content)
3. Run VMAF analysis at each encoding preset (Fast, Balanced, Quality)
4. Compare VMAF scores against CPU/GPU usage
5. Select the preset that meets your quality threshold at the lowest resource cost

Workflow 2: Network Quality Monitoring

For routes receiving SRT streams over the internet:

1. Schedule VMAF checks every 15 minutes
2. Correlate VMAF scores with SRT packet loss metrics
3. When VMAF drops but packet loss is low: the problem is at the encoder
4. When VMAF drops and packet loss is high: the problem is the network
5. Use this correlation to direct troubleshooting to the right team

Workflow 3: Bitrate Optimization

For multi-output routes using zero-copy distribution with selective transcoding:

1. Run VMAF on each transcoded output
2. Identify outputs where VMAF > 93 (over-provisioned)
3. Reduce bitrate by 10-15% on those outputs
4. Re-run VMAF to confirm quality remains acceptable
5. Repeat until VMAF is in the 85-93 range (optimal quality-to-bitrate ratio)

This workflow can save 20-30% bandwidth across your infrastructure without perceptible quality impact.

Next Steps

• Configure hardware transcoding and validate quality with VMAF
• Set up Prometheus and Grafana monitoring for quality score dashboards
• Explore Docker and Kubernetes deployment for scalable quality analysis
• Return to the Broadcast Streaming Software guide for the complete platform overview