tag:blogger.com,1999:blog-738485353871431380.post5945544419804735246..comments2024-03-23T13:15:48.445-07:00Comments on Crystal Prison Zone: Comment on Data Colada [58]: Funnel plots, done correctly, are extremely usefulJoehttp://www.blogger.com/profile/10825531253125205466noreply@blogger.comBlogger2125tag:blogger.com,1999:blog-738485353871431380.post-36078034773221216732017-03-27T07:09:20.993-07:002017-03-27T07:09:20.993-07:00I'd agree, to some extent. I feel it is a comm...I'd agree, to some extent. I feel it is a common misconception that every systematic review need to include a meta-analysis, or that the core purpose of any meta-analysis is to provide a single effect size estimate. I think that there is room for systematic reviews to say "This literature, while rich with new ideas, is too heterogeneous to synthesize."Joehttps://www.blogger.com/profile/10825531253125205466noreply@blogger.comtag:blogger.com,1999:blog-738485353871431380.post-29279512757609748122017-03-25T14:03:21.344-07:002017-03-25T14:03:21.344-07:00Nice post. You say that
"There is a simple s...Nice post. You say that<br /><br /><i>"There is a simple solution... If your research literature is expected to contain some large and some small effects, and these are reflected by clear differences in experimental methodology and/or subject population, then analyze those separate methods and populations separately."</i><br /><br />I would say it may not be so simple to do this.<br /><br />We may find that many of the studies are idiosyncratic and don't fit into any groups, or that many of the groups are too small to meta-analyze (you can hardly interpret a funnel plot with only 3 studies.)<br /><br />You do note this issue but I don't think that we can always avoid it by not <i>"getting carried away chasing after perfectly homogeneous subgroups"</i>. In many cases the literature on a certain topic really is a mess with all kinds of different study designs and methods.<br /><br />I would say that in such a case, we shouldn't do a meta-analysis of that topic at all (rather we should use other approaches to summarize the literature).<br /><br />Even in cases where it is possible to define homogenous subgroups of studies, these subgroups may not be representative of all of the literature (consider for instance if there was one lab churning out studies using the same methodology, a methodology that no-one else in the field uses because it's flawed.)<br /><br />In such a case I would again say that meta-analysis is not the tool we're looking for to summarize the topic.Neuroskeptichttps://www.blogger.com/profile/06647064768789308157noreply@blogger.com