Methodology

How we measure our own predictions

Every number on this page comes from a frozen holdout, a named build, and a baseline it had to beat.

The first public scorecard publishes after the September 2026 run. Until then this page explains exactly what it will contain.

What each number means

MAPE

Mean absolute percent error between the modeled curve and the measured one. Reported as a median too, so one outlier cannot define the claim.

Drop timing hit rate

How often a flagged drop landed within 5% of the video duration of where it happened.

Average view duration error

How far the modeled average view duration lands from the measured one, in seconds and in percentage points.

End hold error

How far the modeled retention at the final point of the video lands from the measured retention there.

R squared

How much of the measured curve shape the modeled curve explains, point by point, across the whole video.

Spearman rho

Whether the model ranks the low points and high points in the same order the measured curve does.

95% confidence interval

The range a bootstrap resample of the same holdout would land in. One lucky month cannot read as a permanent number.

The baselines the model has to beat

The last three are reported by the training harness. They will appear here with numbers once a trained model is scored on this holdout.

This channel's usual shape

The channel median retention curve for videos of the same duration family. The only baseline in the production scorecard today.

Global median

The median retention curve across every channel in the holdout. Reported by the training harness for comparison.

Position only

A curve that only knows how far into the video a point is, with no knowledge of the content.

Loudness only

A curve built from the audio loudness track alone, with no knowledge of the visuals or the script.

The temporal position control

The temporal position control permutes the mapping between modeled and measured curves across videos, keeping each point in its original position. A model that only learned attention decays over time scores the same either way.

A published attention probe reached a correlation of 0.47 and collapsed under this exact control. The number looked real until position alone explained it.

A trained model ships only when its lift over the baseline is positive. Its 95% confidence interval must also exclude zero on this holdout.

Measured

History bands pulled directly from the platform
Published audience retention for the exact video
Fix outcomes: what happened after a creator re-screened a cut
Viewer panel response, when a panel has run

Modeled

The modeled retention curve before publish
Modeled drops and where they land
Modeled attention moments inside the video
Script lines the model flagged
The niche cohort a channel is compared against

Screen a rough cut against your own audience

Screen a rough cut See pricing

Methodology

How we measure our own predictions

Every number on this page comes from a frozen holdout, a named build, and a baseline it had to beat.

The first public scorecard publishes after the September 2026 run. Until then this page explains exactly what it will contain.

What each number means

MAPE

Mean absolute percent error between the modeled curve and the measured one. Reported as a median too, so one outlier cannot define the claim.

Drop timing hit rate

How often a flagged drop landed within 5% of the video duration of where it happened.

Average view duration error

How far the modeled average view duration lands from the measured one, in seconds and in percentage points.

End hold error

How far the modeled retention at the final point of the video lands from the measured retention there.

R squared

How much of the measured curve shape the modeled curve explains, point by point, across the whole video.

Spearman rho

Whether the model ranks the low points and high points in the same order the measured curve does.

95% confidence interval

The range a bootstrap resample of the same holdout would land in. One lucky month cannot read as a permanent number.

The baselines the model has to beat

The last three are reported by the training harness. They will appear here with numbers once a trained model is scored on this holdout.

This channel's usual shape

The channel median retention curve for videos of the same duration family. The only baseline in the production scorecard today.

Global median

The median retention curve across every channel in the holdout. Reported by the training harness for comparison.

Position only

A curve that only knows how far into the video a point is, with no knowledge of the content.

Loudness only

A curve built from the audio loudness track alone, with no knowledge of the visuals or the script.

The temporal position control

A published attention probe reached a correlation of 0.47 and collapsed under this exact control. The number looked real until position alone explained it.

A trained model ships only when its lift over the baseline is positive. Its 95% confidence interval must also exclude zero on this holdout.

Measured

History bands pulled directly from the platform
Published audience retention for the exact video
Fix outcomes: what happened after a creator re-screened a cut
Viewer panel response, when a panel has run

Modeled

The modeled retention curve before publish
Modeled drops and where they land
Modeled attention moments inside the video
Script lines the model flagged
The niche cohort a channel is compared against

Screen a rough cut against your own audience

Screen a rough cut See pricing

How we measure our own predictions

Latest scorecard

What each number means

MAPE

Drop timing hit rate

Average view duration error

End hold error

R squared

Spearman rho

95% confidence interval

The baselines the model has to beat

This channel's usual shape

Global median

Position only

Loudness only

The temporal position control

Measured versus modeled

Measured

Modeled

Screen a rough cut against your own audience

How we measure our own predictions

Latest scorecard

What each number means

MAPE

Drop timing hit rate

Average view duration error

End hold error

R squared

Spearman rho

95% confidence interval

The baselines the model has to beat

This channel's usual shape

Global median

Position only

Loudness only

The temporal position control

Measured versus modeled

Measured

Modeled

Screen a rough cut against your own audience