- Here is a wide variety of Marketing models, of which the most important and popular are those which link marketing efforts and their economical results – Marketing Mix Models (MMM). The main purpose of marketing mix is to estimate “the effectiviness coefficients”, indicating at what rate efforts are converted into sales volumetric results. In fact, modeling is done to estimate the Return on Marketing Investments (ROI or to be more exact ROMI) if it is considered in monetary form. However, pure ROI can be a short-sighted approach. It is particularly true when you consider long-term advertising effects.
- Marketing models have an intrinsic statistical character and should somehow combine statistical and marketing knowledge. The process of blending the intrinsic statistical character of marketing models with marketing knowledge often leads to serious problems in traditional MMM.
- The main problem with typical MMM is a strong misbalance between statistical and marketing components, i.e., the role of correct statistical analysis is underestimated while the role of “interpretability” is overestimated. It very often masks the fact that the model has a very weak statistical foundation. Voluntary interpretation helps a model survive, but it does not give you the real picture.
Below you can find the main weaknesses of the existing traditional MMM approach.
- It is dangerous to underestimate the importance of modern statistical knowledge;
the gap between the latest powerful approaches and the current level of methods being used is huge. A typical modeling company has neither the resources, the time, nor the willingness to even open statistical journals and newly published books;
AMModels has high level statistics professionals who are regularly involved in corporate and scientific projects. Collectively they have the arsenal of modern tools to succeed with any project.
- Common software packages
create an illusion of “deep analysis” without a real understanding of the methods used. The “point and click” or “cookie-cutter” approach generates a false impression of accumulated experience; in reality it is often just an accumulation of errors. It gives the wrong signal to a client who believes that everything was critically reconsidered based on regular scientific updating. The ease of use, or simplicity, of friendly software illuminates the necessity for a customized approach for different projects. Many companies use the same software, the same kind of models, and the same tools regardless of the nature of the business.
AMModels uses a combination of the latest version of traditional software packages and our original approaches for each separate project, together with our own software package. We are not a one-model-fits-all company.
- The typical MMM process is practically impossible to check.
A client cannot prove or disprove a modeler’s claims of effectiveness. This leaves a lot of room for plausible speculations and convenient interpretations. In particular, if economic-sounding conclusions are not directly derived from the model, they can be made up in the frame of “interpretability”. For instance, if advertising (GRP, TRP, adstock, or spending) enters into the model with a very low t-statistic, a statistic will not be reported, but the variable can be included anyway.
AMModels provides transparent results that allow the client to see clearly the quality and level of modeling, including the list of variables used in model. We encourage our client to be directly involved in every step of the process. This gives our clients confidence in their decision-making process.
- MMM mystifies itself and the process of modeling.
It is often said that a model is half science and half art, but the “artistic” part, unfortunately, is often just a convenient pretext for arbitrariness when the scientific part is much less than a half. For the general reasons explained above and below, a modeler’s energy is often spent building a shaky but plausible model that seems completely interpretable from a marketing prospective but may be extremely vulnerable from a statistical one. In such a model everything is truly “art” – if one takes out one variable, the other may change its sign (!); the signs of correlations may be different with signs of regression; contributions of different factors may vary several times depending on the way they are calculated; etc.
However, the presented results can be completely similar to those expected from the business side. This is a typical process of mystification, when reality is replaced by superficial plausibility.
AMModels insists that nothing may stop a researcher from honest and deep analysis on the way to the best results. Ultimately, it is a question of qualification: if you know what to do, you do it quickly and confidently using the proper tools and approaches. To make a decision how to improve the allocation of your marketing activities next fiscal year requires a real picture for the current year. Our advanced modeling gives you this picture.
The following list should make clear what kinds of problems are usually not addressed properly in traditional MMM. We recognize the problems and have appropriate solutions.
Many more details on the topic may be found in the article Statistical modeling and business expertise, or where is a truth? by I. Mandel.
|Business expectation from MMM
||Traditional MMM’s response and drawbacks
||Estimate of the effects of all possible factors, especially business (marketing) actions.
||Many important variables may not be included in a model for the following reasons:
- They have economically inappropriate signs of regression coefficients (advertising is expected to be positive and price negative, for instance). If a variable enters regression with the opposite sign, it is either not included in a model or made up somehow.
- The number of variables is too big to be estimated by standard regression techniques (sometimes more than the number of observed data points.
- Many variables enter the model with very low t-statistics and as a result are either rejected (the honest way) or left in the model.
||Reliable assessment of variables’ contribution. A client may not even consider that as a problem, because they believe in the modelers; so, it is a hidden business problem.
||Reliability generally cannot be guaranteed by the model:
- Multicollinearity (high mutual correlations between independent variables, which distort a correct solution of the regression problem) is very typical for marketing, but modern statistical approaches to counteract this are either not used at all or used incorrectly. Moreover, even the best advanced methods, if used, still do not guarantee a satisfactory solution in all cases. But the presence of multicollinear variables in practically all MMM makes regression coefficients unreliable.
- Even when variables are not collinear, estimation of parameters heavily depends on the composition of the model – the same variable will have one coefficient in one model and another coefficient in another Often, adding a new variable to a model changes the sign of the previously-included variable or even makes it “significant” (in t-statistics terms), but with the opposite sign. Such phenomena undermine the idea of correct reliable estimation (see CASE STUDY 2)
- Sparseness of original data is usually associated with weeks when advertising was not performed. Regression would consider those weeks as normal points with zero values, which may strongly deteriorate the results. In fact only “working weeks” matter – the concept could be captured within the paradigm of the yield analysis (CASE STUDY 1)
- There are many other more technical reasons, specific for time series (like presence of autocorrelations in errors, heteroskedasicity, etc.), which prevent most MMM from being stable in a statistical sense.
||Creation of a model working in normal business conditions; elimination of the random transient circumstances.
||MMM resolve this issue in a limited way, rarely looking into unusual observations and trying to use all data untouched. However, the restructuring of data may play a very serious role. Slight data change may improve a model without loss of generality, but it requires a lot of work and high statistical culture.
||Estimate of the synergetic effects of different kinds of advertising or other marketing actions.
||Very hard to break out synergetic effects, especially in a situation of many independent variables (inclusion of all product terms may complicate the model too much), high multicollinearity (see above) and sparseness of data (when just a few occurrences of some action took place).
||Estimate of dynamic of advertising or promotional effectiveness, i.e., to know that ROI of some channel goes up or down over time.
||Model cannot answer that question, because regression coefficients reflecting the effectiveness (or elasticity) are constant over time or space. In the new yield analysis approach it becomes possible (CASE STUDY1).
||Explanation of sales change (usually from one year to another) as sum of increments due to factors included in a model. Often it is considered as a main goal of MMM, its ultimate purpose.
||Usually a model answers this question improperly, because it automatically assigns effects, derived from the original data values (based on which a model was built), to the increments, which makes decomposition of sales change irrelevant to the level of data approximation in a model. It may well be that model has, say, 95% of determination (R2), but when sales change for two years is decomposed, the error of decomposition is 120% (CASE STUDY 3)
||Estimate of coefficients of the many models in a consistent way (like make one national and 50 DMA models), in order to differentiate the marketing strategies.
||Usually substitutes the problem in one of two improper ways:
- Makes all individual models without correspondence between them, which results in a strong inconsistency between individual models and those with national model;
- Pulls all individual data in one data set and makes just one model, which introduces severe heterogeneity into the analysis and makes many coefficients fictitious.
||Estimate of effects of difficult-to-measure or immeasurable factors, such as rumors, fads, word of mouth, previous experience, cultural differences, etc.
||Cannot answer those questions
||Estimate of long term effects, such as general market tendencies, brand value, accumulated effect of advertising, etc.
||Usually cannot answer these questions.
||Ongoing forecast reacting to the fast changes in environment.
||Current practice usually just makes a remodeling from time to time, but it often creates inconsistency with previous models (in other words, MMM typically does not belong to the class of adaptive models).
||Market share model which takes into account both volumes and shares.
||The best answer is to make a share model based on multinomial logit or Multiplicative Competitive Interaction (MCI) approaches, but volumes remain un-modeled. There is no commonly accepted methodology to comply with both.
||Simulate “what if” scenario for future planning under different conditions.
||Model answers that question superficially, because regression modeling allows estimating the outcome’s change due to change of given factor only under the assumption that other factors (as well as regression coefficients) remain the same in the future. But this is not what happens in real life.
||Estimation of the effects of new factors, like emerging competitors, or old factors, but in unusual scale, like doubling of the competitors’ advertising effects against the previous maximal level.
||Model either cannot answer these questions (for the new factors), or answers them with high error rates (for the unusual scale).
||Causal model of the process. This type of problem, as problem 2, is also hidden – a client always assumes that models are causal, otherwise they are meaningless in her eyes.
||MMMs do not specifically consider causality; models have pure statistical nature:
- Causality is a very specific topic, which recently attracted a new wave of attention in the statistical community. However, the problem of distinction between statistical and causal models is just ignored in MMM practice.
- As a result, MMMs may (and do) contain many fictitious regularities (in the sense that they capture “correlation, not causation”), which, combined with the problems described above, make traditional MMM not especially useful and/or actionable.
||Optimization of the marketing mix for the new fiscal year (half a year or quarter) based on MMM. Particularly, finding the best (optimal) level of spending on each marketing tactic, which yields the best efficiency.
||Model cannot assist with that problem reliably because of the combination of all of the difficulties listed above and because special optimization tools are either not used or do not guarantee the right solution. The problem of optimization by many parameters remains a complicated scientific problem, despite huge advances in that area in recent decades. AMModels has very effective algorithms of that type.