Set custom ROI priors using past experiments

Meridian requires passing distributions for ROI calibration. Although setting custom priors using results from previous experiments is a sound approach, there are many nuances to consider before proceeding. For example:

• The timing of the experiment in relation to the MMM time window:If the experiment was conducted either before or after the MMM time window, the results might not be directly applicable.

• The duration of the experiment:An experiment of short duration might not effectively capture the long-term effects of the marketing effectiveness.

• The complexity of the experiment:If the experiment involved a mixture of channels, the results might not provide clear insights into the performance of individual channels.

• Estimand differences:The estimands used in experiments can differ from those used in the MMM. For example, the MMM counterfactual is zero spend, whereas some experiments might have a different counterfactual, such as reduced spend.

• Population differences:The population targeted in the experiment might not be the same as the population considered in the MMM.

We recommend setting the custom priors based on your belief in the effectiveness of a channel. A prior belief can be informed by many things, including experiments or other reliable analyses. Use the strength in that prior belief to inform the standard deviation of the prior:

• If you have a strong belief in the effectiveness of a channel, you can apply an adjustment factor to the standard deviation of the prior to reflect your confidence. For example, suppose you have conducted several experiments for a particular channel and all the experiments yielded similar ROI point estimates, or you have historical data from previous MMM analyses that support the effectiveness of this channel. In this case, you could set a smaller standard deviation for the prior so that the distribution won't vary widely. This tighter distribution indicates your strong confidence in the experimental results.

• Conversely, the experiment might not necessarily translate to the MMM, considering some of the nuances listed earlier. In this case, you might choose to apply an adjustment factor to standard deviation of the prior distribution. For example, you could set a larger standard deviation for the prior, depending on your level of skepticism.

You should consider using the roi_calibration_period argument in ModelSpec . For more information, see Set the ROI calibration period .

Define a lognormal prior from intuition or experiments

The lognormal distribution is a common distribution to use for an ROI prior. Meridian provides two helper functions to construct lognormal distributions from experiment results. prior_distribution.lognormal_dist_from_mean_std constructs a lognormal distribution with a mean and standard deviation provided by the user. For example, the chosen mean and standard deviation could be informed by the point estimate and standard errors of experiments, respectively. prior_distribution.lognormal_dist_from_range constructs a lognormal distribution so that a specified probability mass falls within a range. For example, the range could be informed by a 95% confidence interval from a previous experiment. When using either function to define a prior from experiments, keep the considerations discussed on this page in mind.

To construct a LogNormal distribution from a mean and standard deviation:

  import 
  
 numpy 
  
 as 
  
 np 
 from 
  
 meridian.model 
  
 import 
 prior_distribution 
 mean 
 = 
 [ 
 1.0 
 , 
 3.0 
 ] 
 # Point estimate 
 std 
 = 
 [ 
 0.3 
 , 
 2.0 
 ] 
 # Standard error 
 # Define the LogNormal distribution 
 lognormal_dist 
 = 
 prior_distribution 
 . 
 lognormal_dist_from_mean_std 
 ( 
 mean 
 , 
 std 
 ) 
 # Define the PriorDistribution object 
 prior 
 = 
 prior_distribution 
 . 
 PriorDistribution 
 ( 
 roi_m 
 = 
 lognormal_dist 
 ) 
 

To construct a LogNormal distribution from a range:

  import 
  
 numpy 
  
 as 
  
 np 
 from 
  
 meridian.model 
  
 import 
 prior_distribution 
 low 
 = 
 [ 
 0.1 
 , 
 0.5 
 ] 
 # Range lower bound 
 high 
 = 
 [ 
 2.0 
 , 
 10.0 
 ] 
 # Range upper bound 
 mass_percent 
 = 
 0.95 
 # Probability mass 
 # Define the LogNormal distribution 
 lognormal_dist 
 = 
 prior_distribution 
 . 
 lognormal_dist_from_range 
 ( 
 low 
 , 
 high 
 , 
 mass_percent 
 = 
 mass_percent 
 ) 
 # Define the PriorDistribution object 
 prior 
 = 
 prior_distribution 
 . 
 PriorDistribution 
 ( 
 roi_m 
 = 
 lognormal_dist 
 ) 
 

However, if the results from previous experiments are near zero, you should consider whether your prior beliefs are accurately represented by a non-negative distribution such as the LogNormal distribution. We highly recommend plotting the prior distribution to confirm it matches your prior intuitions before proceeding with the analysis.

It is strongly recommended that you plot and visualize custom priors, especially ROI priors. This can help you double check if the custom prior aligns with your prior expectations.

  from 
  
 matplotlib 
  
 import 
 pyplot 
 as 
 plt 
 from 
  
 meridian.model 
  
 import 
 prior_distribution 
 import 
  
 numpy 
  
 as 
  
 np 
 # Define the LogNormal distribution 
 lognormal_dist 
 = 
 prior_distribution 
 . 
 lognormal_dist_from_mean_std 
 ( 
 2.0 
 , 
 0.5 
 ) 
 # Plot a histogram of samples from the LogNormal distribution 
 plt 
 . 
 hist 
 ( 
 lognormal_dist 
 . 
 sample 
 ( 
 1000 
 ))