Meridian module for analyzing model fit in a Meridian model.
This module provides a ModelFitProcessor
, which assesses the goodness of fit
of a trained Meridian model. It compares the model's predictions against the
actual observed data, generating key performance metrics.
Key metrics generated include R-squared, MAPE, and Weighted MAPE. The output also includes timeseries data of actual values versus predicted values (with confidence intervals) and the predicted baseline.
The results are structured into a ModelFit
protobuf message.
Key Classes:
-
ModelFitSpec: Dataclass to specify parameters for the model fit analysis, such as whether to split by train/test sets and the confidence level for intervals. -
ModelFitProcessor: The processor class that performs the fit analysis.
Example Usage:
from
schema.processors
import
model_fit_processor
from
schema.processors
import
model_processor
# Assuming 'mmm' is a trained Meridian model object
trained_model
=
model_processor
.
TrainedModel
(
mmm
)
# Default spec: split results by train/test if holdout ID exists
spec
=
model_fit_processor
.
ModelFitSpec
()
processor
=
model_fit_processor
.
ModelFitProcessor
(
trained_model
)
# result is a model_fit_pb2.ModelFit proto
result
=
processor
.
execute
([
spec
])
print
(
"Model Fit Analysis Results:"
)
for
res
in
result
.
results
:
print
(
f
" Dataset:
{
res
.
name
}
"
)
print
(
f
" R-squared:
{
res
.
performance
.
r_squared
:
.3f
}
"
)
print
(
f
" MAPE:
{
res
.
performance
.
mape
:
.3f
}
"
)
print
(
f
" Weighted MAPE:
{
res
.
performance
.
weighted_mape
:
.3f
}
"
)
# Prediction data is available in res.predictions
# Each element in res.predictions corresponds to a time point.
# e.g., res.predictions[0].actual_value
# e.g., res.predictions[0].predicted_outcome.value
Classes
class ModelFitProcessor
: Generates a ModelFit proto for a given trained Meridian model.
class ModelFitSpec
: Stores parameters needed for generating ModelFit protos.
