11/16/ · The authors present an overview of the meta-analytic approach and describe a meta-analysis from beginning to end. The example includes: problem specification; research questions; study retrieval and selection; coding procedure; calculating effect sizes; visual displays and summary statistics; conducting HLM analyses; and sensitivity blogger.com by: 11 Meta-analysis is an analytical tool that permits the evaluation of a diagnostic or therapeutic modality through the appropriate use of previously published smaller studies. 21 Meta-analysis is not the simple pooling of data reported in numerous small studies, a notion that has often caused investigational errors 11/3/ · Myung-Haeng Hur and his colleagues were able to utilize the meta-analysis systematic review approach in an effective manner. Utilizing a small population of sample for the study, this research involved particular patients who were depicted to have hypertension cases in the most constant occurrences
Meta Analysis, Meta Synthesis, Article Critique Example
The purpose of this article is to provide higher education researchers with an illustrative example of meta-analysis utilizing hierarchical linear modeling HLM. This article demonstrates the step-by-step process of meta-analysis using a recently-published study examining the effects of curricular and co-curricular diversity activities on racial bias in college students as an example Denson, Rev Educ Res —, The authors present an overview of the meta-analytic approach and describe a meta-analysis from beginning to end.
The example includes: problem specification; research questions; study retrieval and selection; coding procedure; calculating effect sizes; visual displays and summary statistics; conducting HLM analyses; and sensitivity analyses. The authors also offer guidelines and recommendations for improving the conduct and reporting of research which in turn can provide the information necessary for future and more comprehensive meta-analytic reviews. While meta-analytic techniques have been available for over a century see Cooper and Hedges for a history of meta-analysismeta-analysis as a method of inquiry for integrating quantitative results from a stream of research began to become popular approximately 30 years ago Glass While conducting a meta-analysis can be very time consuming when done properly, meta analysis article example, advances in computing and statistical software have facilitated conducting literature searches and applying meta-analytic procedures.
The purpose of this article is to provide higher education researchers with a pedagogical example of a meta-analysis utilizing hierarchical linear modeling HLM to encourage more widespread use of meta-analytic methods. In this article, we begin by first providing an overview of the meta-analytic approach.
Finally, we provide guidelines and recommendations for improving the conduct and reporting of future research. Put simply, meta-analysis is a statistical technique of combining the findings of a set of studies that address common research hypotheses Cooper and Hedges ; Lipsey and Wilson Conducting a meta-analysis is much like conducting a primary research study. In a meta-analysis, instead of students or participants being the unit of analysis, the primary studies themselves become the unit of analysis.
In meta-analysis, the researcher collects research studies from a particular domain e. As will be discussed in detail below, the key quantitative results of each study e. The effect sizes for a sample of studies become the dependent variable, and various study characteristics e. In this way, a meta-analysis provides an integrated review of the research base that enables us to calculate an overall summary of the effects of a program or policy of interest, and investigate various hypotheses concerning why the effects may be larger in some studies and smaller in others; that is, we can attempt to identify key factors meta analysis article example might magnify or dampen the effects of a program of interest.
Many extremely thoughtful and thorough qualitative research syntheses have been undertaken in education e. Meta-analysis complements qualitative literature reviews by integrating the findings in a systematic quantitative way, meta analysis article example, which we will meta analysis article example in detail below.
Some examples of meta-analyses in education include: coaching effects on SAT scores Messick and Jungeblutteacher expectancy effects on student IQ Raudenbushschool desegregation effects on African-American students Wells and Crainand effects of school reform programs on achievement Borman et al.
What is the magnitude of the overall effect size for a program or practice of interest? To what extent do the effect sizes of the studies in a sample vary around the overall effect size estimate? What are the reasons for the observed differences in effect sizes across studies?
The presence of heterogeneity provides an opportunity to ask additional questions, and to investigate more closely the reasons for the observed differences in effect size across studies. Such investigations have the potential to generate further insight into what conditions and for whom a program or practice of interest may be particularly beneficial. In the last 50 years, the rapidly changing demographics of the United States meta analysis article example seen a parallel shift in the demographic makeup of our meta analysis article example campuses.
Naturally, meta analysis article example, this change has been met with some resistance and racial tension. As a result, institutions have implemented diversity-related initiatives designed to promote positive intergroup relations. While there certainly have been many qualitative literature reviews on this topic e. What is the magnitude of the overall effect size of participation in curricular and co-curricular diversity activities on racial bias?
overall effect. Is there variation in the effect of curricular and co-curricular diversity activities on racial bias? heterogeneity in effect sizes. Which types of programs are most effective? Which types of studies show the most effective programs? factors that amplify or diminish the magnitude of effect sizes. One of the first steps to conducting a meta-analysis is study retrieval and selection Cooper This is an important step as a meta-analysis is only useful if it provides an accurate representation of the research at large.
So, a meta-analysis is only as good as the studies it represents. Thus, it is imperative to retrieve every relevant study possible. In addition to published research, meta analysis article example, a thorough search should also include unpublished studies such as dissertations and conference papers. The keywords and descriptors used as search terms here were: diversity, ethnic studies, women studies, bias, prejudice, stereotypes, discrimination, higher education, and college students.
Specifically, the search was implemented as follows:. In addition, the search was limited to sources written in English only, as well as to only studies based on students from four-year colleges or universities within the United States. These descriptors facilitated a computer-based search of three databases: Education Resources Information Center ERIC —PsycINFO —and ProQuest Digital Dissertations including Dissertation Abstracts International — through June In addition, a hand search was conducted over the last 10 years of the following top higher education journals: Journal of Higher EducationReview of Higher EducationResearch in Higher Educationmeta analysis article example, Journal of Social Issuesmeta analysis article example, and the Journal of College Student Development.
Finally, meta analysis article example were checked in each of the included studies as well as review articles for other relevant works. As in all meta-analyses, the present study required specific inclusion rules, meta analysis article example.
Table 1 lists the characteristics of the sample of coded studies and presents a brief overview of the distribution of studies. The next step is to code these studies for their study characteristics and effect sizes which will become the independent variables and dependent variable, respectively Lipsey and Wilson The study characteristics are possible moderators which may help explain the variation seen in effect sizes across studies.
It is likely that there are systematic differences across studies that may account for differences in effect sizes. Effect size estimates are standardized measures of an effect and comprise the key outcome measure in meta-analysis and enable the synthesizing and integrating of findings. The independent variables selected and coded for this meta analysis article example were included as possible covariates, and were categorized into three meta analysis article example study characteristics, student characteristics, meta analysis article example, and institutional characteristics.
The study characteristics consisted of three subgroups: study source characteristics a through cdiversity initiative characteristics d through iand methodological characteristics j through q. The study source characteristics included: a author identification, b year of publication, and c type of publication journal article, book, book chapter, conference paper, doctoral dissertation and other unpublished work.
The student characteristics included: a age, b gender, and c ethnicity. Finally, the institutional characteristics included: a number of institutions in study single, multipleb institution name, c institutional diversity meta analysis article example no, yesd type of institution public, privatee size of institution total number of full-time undergraduatesf structural diversity percentage of students of colorand g region of country Western, Southern, Eastern, Midwest.
All of the primary studies were coded according to these variables. In addition, meta analysis article example, because coding might be subject to human error, two coders were involved in the coding process and inter-rater reliability indices were calculated accordingly, meta analysis article example.
The dependent variable was effect size and was estimated as a group contrast in this meta-analysis. A group contrast compares an outcome variable across two or more groups of respondents. For example, meta analysis article example, the racial bias measures can be compared between students who had taken a required diversity course versus those who had not.
The group contrasts were represented by the standardized mean meta analysis article example because the operationalization of racial bias differed to some extent across the studies, and it is thus necessary to standardize the group contrasts so that the values can be meaningfully compared across the sample of studies. Where S G 1 is the standard deviation for Group 1 and S G 2 is the standard deviation for Group 2.
Meta analysis article example the case of small sample sizes, however, this effect size index has been shown to be upwardly biased, especially for sample sizes less than 20 Hedges As a result, a meta analysis article example correction was applied and all subsequent computations used this corrected unbiased effect size estimate as follows Hedges :.
Thus, the effect size estimate g and its standard error SE g were computed for each of the studies see Lipsey and Wilsonfor a more meta analysis article example explanation. The studies in the sample for this study were essentially quasi-experimental in that students selected into the various diversity programs, and in nearly all cases the authors of these studies meta analysis article example to statistically adjust for any pre-program differences in initial bias.
When possible, adjusted effect sizes were calculated from inferential statistics e. In these cases, the effect sizes can be viewed as being adjusted for differences in covariates e.
When insufficient information was reported such that it was not possible to obtain the adjusted effect size, the unadjusted effect size was calculated from the reported descriptive statistics e, meta analysis article example.
In the cases where there was insufficient detail provided in the article to calculate an effect size sthe author s were contacted to obtain the required information. Of the 27 studies eligible to be included in the meta-analysis, 11 studies were excluded due to insufficient information, meta analysis article example.
These authors were contacted via email, but either they did not respond or could not provide me with the requested information. Of the remaining 16 studies, two utilized the same sample, and thus the effect sizes were averaged. If a study included multiple outcomes of the same outcome type, then the effect sizes were averaged.
On the other hand, if a study examined multiple meta analysis article example types e. In total, 30 effect sizes were computed from the 16 studies, with each study contributing, on average, 1. Table 2 displays the 30 effect size estimates and their standard errors.
As can be seen the effect size estimates range in value from 0. To help understand the meaning of these quantities, consider, for example, the results for the Lopez et al. The effect size estimate for this study indicates that the outcome scores for the students in the treatment group were on average approximately 0.
According to Coheneffect size estimates around 0. Just as a treatment effect estimate provides us with an estimate of the true meta analysis article example of a program or treatment, an effect size estimate g j can be viewed as providing an estimate of the true effect size for a study δ j. Footnote 1 The standard error gives us a sense of how precise the effect size estimate is, and enables us to construct a confidence interval, which helps convey how large or small the true effect might be.
Thus, for the Lopez et al. study, adding and subtracting approximately 2 standard errors 2 × 0. Based on this interval, the notion that the true effect might be 0 is clearly not plausible; as can be seen in Fig. Note also that the interval excludes a value of 0. Next consider the effect size estimate from the Antonio study. The magnitude of the estimate is 0. Using the standard error of the estimate i. Just as in the case of the Lopez et al. study, the notion that the true effect size for this study is 0 is not very plausible.
But as can be seen the intervals for the Antonio and Lopez et al. studies do not overlap at all, i, meta analysis article example. This provides evidence meta analysis article example the true effect size for the Antonio study is likely appreciably smaller than the true effect size for the Lopez study. There are many types of different visual displays available to present and summarize information about magnitudes and distributions of effect sizes Kosslyn ; Tufte
Intro to Systematic Reviews \u0026 Meta-Analyses
, time: 16:57Examples of Meta-Analyses
Meta-analysis is an analytical tool that permits the evaluation of a diagnostic or therapeutic modality through the appropriate use of previously published smaller studies. 21 Meta-analysis is not the simple pooling of data reported in numerous small studies, a notion that has often caused investigational errors 11/3/ · Myung-Haeng Hur and his colleagues were able to utilize the meta-analysis systematic review approach in an effective manner. Utilizing a small population of sample for the study, this research involved particular patients who were depicted to have hypertension cases in the most constant occurrences 11/16/ · The authors present an overview of the meta-analytic approach and describe a meta-analysis from beginning to end. The example includes: problem specification; research questions; study retrieval and selection; coding procedure; calculating effect sizes; visual displays and summary statistics; conducting HLM analyses; and sensitivity blogger.com by: 11
No comments:
Post a Comment