Primo et al.(2007) クラスター化標準誤差 vs マルチレベルモデリング
社会学や心理学でしばしば用いられるマルチレベルモデリングであるが，経済学者のいる研究会で「それって標準誤差をクラスター化すれば良いのでは？」と言われた経験のある人は多いのではないだろうか．このセリフを言ったことも言われたこともあるのだが，よくまとまっている論文とブログをみつけたのでメモしておく．
Primo, D. M., Jacobsmeier, M. L., & Milyo, J. 2007. "Estimating the Impact of State Policies and Institutions with Mixed-Level Data." State Politics & Policy Quarterly, 7(4), 446–459.
この論文を知ったきっかけはAndrew Gelmanのブログである．したがって，Gelmanのブログにも簡単な解説とコメントがのっている．
Primoらが推定しているのは以下の3モデル．
1) least-squares estimation ignoring state clustering,
(2) least squares estimation ignoring state clustering, with standard errors corrected using cluster information,
(3) multilevel modeling
Gelmanブログのコメントが秀逸なので彼の要約をそのままメモする(ちなみにGelmanは3)はstataでできないと書いているがそれは2007年当時のことであり現在はmixedコマンドで簡単に推定できる)．
1. One big advantage of multilevel modeling, beyond the cluster-standard-error approach recommended in this paper, is that it gives separate estimates for the individual states. Primo et al. minimize this issue by focusing on global questions–“Do voter registration laws affect turnout? Do legislators in states with term limits behave differently than legislators in states with no term limits”–and in their example they focus on p-values rather than point estimate or estimates of variation. Thus, in the examples they look at, multilevel modeling doesn’t have such a big comparative advantage.
2. Another advantage of multilevel modeling comes with unbalanced data–in their context, different sample sizes in different states.
3. I agree that it’s frustrating when software doesn’t work, and I agree with Primo et al. completely that it’s better to go with a reasonable method that runs, rather than trying to use a fancier approach that doesn’t work on your computer. That said, I think their abstract would’ve been clearer if they had simply said, “Stata couldn’t fit our multilevel model,” rather than vaguer claims about “large datasets or many cross-level interactions.”
4. I’d like to get their data and try to fit their model in R. It might very well crash in R also–we’ve had some difficulties with lmer()–in which case it would be useful to figure out what’s going on and how to get it to work.
5. I’d recommend displaying their Table 1 as a graph. (John K. also wrote a paper on this for political scientists.)
6. I completely disagree with their statement on page 456 that cluster-adjusted standard errors “requires fewer assumptions” than hierarchical linear modeling. As Tukey emphasized, methods are just methods. A method can be motivated by an assumption but it doesn’t “require” the assumption. For a simple example, least squares is maximum likelihood for a model with normally distributed errors. But if the errors have a different distribution, least squares is still least squares: it did not “require” the assumption. To go to the next step, classical least squares (which is what Primo et al. recommend for their point estimation) is simply multilevel modeling with group-level variance parameters set to zero. Thus, their estimate requires more assumptions than the multilevel estimate.
7. But, to conclude, I’m not criticizing their choice of clustered standard errors for their example. It’s not a bad idea to use a method that you’re comfortable with. Beyond that, it can be extremely helpful to fit complete-pooling and no-pooling models as a way of understanding multilevel data structures. (See here for more of my pluralistic thinking on this topic.) I hope that as more people read our book, they’ll become more comfortable with multilevel models. But what I really hope is that the software will improve (maybe I have to do some of the work on this) so we can actually fit such models, especially varying-intercept, varying-slope models with lots of predictors and nonnested levels.
