Physical Activity over-estimated?

Exercise

We are told that phys ed in schools is a good idea.

This blog shares some recent research showing that there are indeed benefits.

But it is pretty useless when it comes to treating childhood obesity. I intentionally posted the complete research. Read the summary if you only want the bottom-line.

Here are the facts:

 Effectiveness of intervention on physical activity of

children: systematic review and meta-analysis of

controlled trials with objectively measured outcomes

(EarlyBird 54)

OPEN ACCESS

Brad Metcalf research fellow and statistician 1, William Henley professor of medical statistics 2,

Terence Wilkin professor of endocrinology and metabolism 1

1Department of Endocrinology and Metabolism, Peninsula College of Medicine and Dentistry, Plymouth University Campus, Plymouth, UK; 2Institute

of Health Services Research, Peninsula College of Medicine and Dentistry, University of Exeter Campus, Exeter, UK

Abstract

Objective To determine whether, and to what extent, physical activity

interventions affect the overall activity levels of children.

Design Systematic review and meta-analysis.

Data sources Electronic databases (Embase, Medline, PsycINFO,

SPORTDiscus) and reference lists of included studies and of relevant

review articles.

Study selection Design: randomised controlled trials or controlled clinical

trials (cluster and individual) published in peer reviewed journals.

Intervention: incorporated a component designed to increase the physical

activity of children/adolescents and was at least four weeks in duration.

Outcomes: measured whole day physical activity objectively with

accelerometers either before or immediately after the end of the

intervention period.

Data analysis Intervention effects (standardised mean differences) were

calculated for total physical activity, time spent in moderate or vigorous

physical activity, or both for each study and pooled using a weighted

random effects model. Meta-regression explored the heterogeneity of

intervention effects in relation to study participants, design, intervention

type, and methodological quality.

Results Thirty studies (involving 14 326 participants; 6153 with

accelerometer measured physical activity) met the inclusion criteria and

all were eligible for meta-analysis/meta-regression. The pooled

intervention effect across all studies was small to negligible for total

physical activity (standardised mean difference 0.12, 95% confidence

interval 0.04 to 0.20; P<0.01) and small for moderate or vigorous activity

(0.16, 0.08 to 0.24; P<0.001). Meta-regression indicated that the pooled

intervention effect did not differ significantly between any of the

subgroups (for example, for total physical activity, standardised mean

differences were 0.07 for age <10 years and 0.16 for ≥10 years, P=0.19;

0.07 for body mass index across the entire range and 0.22 for exclusively

overweight/obese children, P=0.07; 0.12 for study duration ≤6 months

and 0.09 for >6 months, P=0.71; 0.15 for home/family based intervention

and 0.10 for school based intervention, P=0.53; and 0.09 for higher

quality studies and 0.14 for lower quality studies, P=0.52).

Conclusions This review provides strong evidence that physical activity

interventions have had only a small effect (approximately 4 minutes

more walking or running per day) on children’s overall activity levels.

This finding may explain, in part, why such interventions have had limited

success in reducing the body mass index or body fat of children.

Introduction

Physical activity is associated with many health benefits,1-3 but

most children fail to meet national recommendations.4-6

Prevention of obesity in particular is thought to be one of the

benefits to being more active, and accordingly most

interventions aimed at reducing childhood obesity incorporate

a physical activity component. Observational studies consistently

show that greater activity is associated (r~−0.2) with lower

body mass index and girth,7-11 yet physical activity interventions

to date have been largely unsuccessful in improving the body

mass index or body composition of children,12-14 and

understanding the reasons for this is important. One explanation,

for which some evidence exists,15 is that more activity leads to

higher calorie consumption, offsetting any gains in energy

expenditure. Another, yet more fundamental, is that physical

activity interventions do not increase the activity of children

sufficiently for it to affect adiposity.

To date, all systematic reviews that have explored the

effectiveness of physical activity interventions on the activity

Correspondence to: B Metcalf brad.metcalf@nhs.net

Extra material supplied by the author (see http://www.bmj.com/content/345/bmj.e5888?tab=related#webextra)

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 1 of 11

Research

RESEARCH

levels of children have had important limitations. For example,

the reviews did not confine their analyses to whole day activity

or to objectively measured outcomes,12 16 17 although both may

be vital to their interpretation. These reviews included a large

proportion of studies that relied on questionnaires, which offer

a poor surrogate for objectively measured physical activity

(r~0.1-0.3).18-20 More importantly, questionnaires are liable to

bias through expectancy, raised awareness, or both.21

Accelerometers, by contrast, are considered the criterion method

for measuring free living activity and correspond well to activity

related energy expenditure.22 23 The existing reviews also include

studies that measured activity only during intervention specific

periods of the day (for example, physical education classes,

break/recess times, school hours only), and the effects observed

during such periods may not be representative of the

intervention’s effect on whole day physical activity.

The primary aim of this review was to determine whether, and

to what extent, physical activity interventions increase the

overall activity of children, using a meta-analysis approach. The

second aim was to examine the effects of the intervention in

relation to potential study level covariates by using

meta-regression. The review follows the PRISMA guidelines

for the preferred conduct and reporting of systematic reviews

and meta-analyses.24

Methods

Inclusion criteria

Studies had to meet all of the following criteria for inclusion in

this review.

Population—Participants had to be aged 16 years or younger.

Intervention—The intervention must have incorporated a

component that aimed to increase physical activity. The control

condition must not have incorporated an activity/exercise related

element of any kind. The intervention period needed to be at

least four weeks.

Outcomes—Physical activity must have been measured

objectively by using accelerometry in the same participants at

both baseline and follow-up. We included only studies that

measured “follow-up physical activity” before or immediately

after the end of the intervention period (within four weeks).

Whole day activity had to be reported as total physical activity

volume (usually expressed as “mean accelerometer

counts/minute”), time spent in moderate or vigorous intensity

activity (time spent in activities of at least the intensity of brisk

walking, usually expressed as “minutes/day” or “proportion of

day”), or both. We excluded studies that reported only “part

day” activity levels.

Study design—Studies had to be randomised controlled trials

or controlled clinical trials (cluster or individual).

Search strategy

We searched four electronic databases (Embase, Medline,

PsycINFO, and SPORTDiscus) for articles published in peer

reviewed journals from January 1990 to early March 2012. We

imposed no language restriction. We used search terms relevant

to childhood (child*, youth, adolescen*, juvenile, teen*, infant*,

boy*, girl*), an activity intervention (physical activity, activ*,

exercis*, training, sedentary, obesity, overweight, BMI), type

of trial (randomised, controlled, trial, RCT, intervention), and

an objective measure of activity (objective*, acceleromet*,

activity monitor, actigraph, MTI, CSA, actical, actiheart, tritrac,

unidimensional, triaxial, MVPA). Two reviewers independently

assessed the abstracts retrieved from this electronic search for

potentially eligible studies. When the abstract clearly showed

that the study was not suitable, the study was discarded. We

obtained full text for studies that seemed to be potentially

eligible and also for those that could not be excluded on the

basis of the abstract information alone. We compared the

abstracts selected for full text retrieval by each reviewer at this

stage and resolved any discrepancies by discussion. The same

two reviewers independently assessed the full text articles and

selected studies that met the inclusion criteria, again resolving

any discrepancies by discussion. We also searched the reference

lists of relevant review articles and of the articles that met the

inclusion criteria. Finally, we checked the articles that met the

inclusion criteria for duplicate reporting of the same data.

Data extraction

One reviewer extracted information on study level covariates

and the results of the trial, and a second reviewer checked it.

We extracted information on study participants (percentage of

girls, baseline age, body mass index, and activity level), size,

duration, setting of intervention (for example, home, school, or

community based), the physical activity component of the

intervention, the method for measuring activity (type and model

of accelerometer, requested number of days the accelerometers

were to be worn, and when follow-up measure was taken: during

or after the intervention), and dimensions of study quality/risk

of bias (randomisation, proportion lost to follow-up, analysed

with intention to treat approach) as potential effect modifiers.

We extracted result specific information for total physical

activity, moderate or vigorous physical activity, or both. The

type of data extracted differed according to the way in which

the results were reported. Where available, we extracted relevant

model statistics (F statistic, t statistic, or P value), “group ×

time” model coefficients, the between group mean difference

(difference in “activity change from baseline” or difference in

“follow-up activity” controlled for baseline activity), or the

within group means (mean change from baseline or raw means

at both baseline and follow-up). For model derived statistics or

coefficients, we extracted information on the covariates

wherever possible. We also extracted the corresponding

measures of precision (standard deviations, standard errors, or

95% confidence intervals) for all means and coefficients. Finally,

we extracted the total number of participants involved in each

study and the number who provided valid accelerometer data.

Calculating effect sizes

Not all studies used the same type of accelerometer. As a result,

the units for both outcomes (total physical activity and moderate

or vigorous physical activity) were not the same across all

studies, so we standardised them. As both outcome measures

were continuous variables, the intervention effects of each study

were represented by the standardised mean difference in

outcome, calculated for this review by dividing the between

group difference in mean activity change from baseline (or

follow-up activity controlled for baseline activity) by the pooled

standard deviation of activity change from baseline. We used

the “MAd” package for R statistical software (R.2.13.0) to

calculate the effect size (its variance and 95% confidence

interval) for each study by using Hedges’ g.25 When a trial

reported the within group means and standard deviations at

baseline and follow-up separately, we assumed a correlation of

r=0.54 26 27 between baseline and follow-up activity to estimate

the standard deviation for activity change from baseline. Where

studies reported only one of either total physical activity or

moderate or vigorous physical activity, we emailed the

corresponding author requesting the other. Where moderate

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 2 of 11

RESEARCH

intensity activity and vigorous intensity activity were reported

separately, we combined them to obtain measures of moderate

or vigorous activity.

Where studies reported multiple comparisons (such as for girls

and boys, for two time points, for groups receiving different

interventions, or for two different cohorts), we calculated a

standardised mean difference for each comparison separately.

Where a study did not report the combined effect and variance,

we calculated the weighted mean of the multiple effects as

described by Borenstein and colleagues.28 For comparisons that

were not independent of one another (for example, multiple

time points in the same cohort), we adjusted the combined

variance for the degree of correlation between the comparisons,

again assuming it to be r=0.5 if not stated.4 26 27 We extracted

and analysed only comparisons based on outcomes measured

during or immediately after the intervention; measures taken

more than one month after the end of the intervention period

were not included in the analysis.

Calculating an overall summary effect size

We combined the effect sizes of all the included studies to

estimate an overall summary effect size (and 95% confidence

interval) for both total physical activity and moderate or

vigorous physical activity by using a random effects

meta-analysis model weighted by the inverse of the effect

variances (MAd package in R.2.13.0). An overall standardised

mean difference of ~0.2 is considered small, ~0.5 is moderate,

and ~0.8 is large.29 We assessed heterogeneity among the study

effects by a visual inspection of the forest plots and by the I2

statistic. We used both to help to determine whether continuing

with our secondary aim—examining study level covariates as

possible sources of heterogeneity—was appropriate.

Effect sizes relative to study level covariates

For this analysis, the studies were not necessarily represented

by a single effect size. We did not pool multiple comparisons

within a study, considering them to be independent samples.

The notable exception was for multiple time points, for which

the two effects are clearly not independent of each other. We

pooled multiple time points for all analyses except that which

explored “duration” as a potential effect modifier.

We grouped all comparisons according to baseline

characteristics of the study population (sex: exclusively boys

and exclusively girls; age: <10 years and ≥10 years; body mass

index status: all categories and exclusively overweight/obese;

activity level: total physical activity <500 counts/minute and

≥500 counts/minute); study design (duration: ≤6 months and

>6 months; study size: n<200 and n≥200; methodological

quality: quality score ≤2/3 and quality score=3/3; timing of

follow-up: during intervention and after intervention); and the

type of intervention (setting: home/family based and school

based; activity sessions provided as part of intervention: yes/no).

We entered each of these dichotomous variables into a weighted

random effects meta-regression model separately (“metafor”

package in R.2.13.0). We then entered all variables into a

multivariate meta-regression model using a backward

elimination approach with a removal criterion of P>0.05. In

addition, we analysed the covariates that we extracted as

continuous variables to test whether the relation was linear and

consistent with the findings of the categorical analysis. We

calculated the proportion of total between study variance

explained by the model and reported it as R2.30

Sensitivity analysis

We used two recommended forms of sensitivity analysis to

verify the reliability of the meta-analysis results.31 Firstly, we

varied the assumed correlation of r=0.5 between baseline and

follow-up activity from r=0.3 to r=0.7 for all relevant studies

to see if this altered the overall summary estimates. Secondly,

we did quality specific subgroup analysis to assess whether the

overall intervention effect changed when studies deemed to be

of “lower” methodological quality were excluded.

Publication bias

We assessed publication bias by using two funnel plots, both

regarding total physical activity. The first plotted the

standardised mean differences, and the second plotted the

residuals produced from the meta-regression model containing

the associated study level covariates. We did the second one to

test whether any asymmetrical scatter of the mean differences,

indicative of publication bias, could alternatively be explained

by differences in studies’ characteristics.

Results

Literature search

The electronic search identified 344 potentially eligible reports

(fig 1⇓). We were able to exclude 286 of these on the basis of

the title or abstract alone. Of the 58 articles for which we

retrieved full text, a further 30 were excluded. We found two

additional eligible articles from searching the reference lists of

the 28 eligible articles and of relevant review articles, bringing

the total number of studies to 30.32-61

Studies’ characteristics

All eligible articles had been published between May 2003 and

December 2011. The studies varied in size, duration, and

intervention type (see web extra table). Study sizes ranged from

18 to 2840 participants (median 307, total 14 326), and the

number of children providing accelerometer based measures of

physical activity in each study ranged from 18 to 1138 (median

165, total 6153). The duration of follow-up ranged from 4 weeks

to 140 weeks (median 26 weeks); nine studies reported an

intervention effect at two time points (mid-intervention and end

of intervention). Seventeen of the trials were school based, 10

were home/family based, one was community centre based, one

was university gym based, and one was boy scout centre based.

Nineteen studies provided activity/exercise sessions as part of

the intervention, and 11 did not.

Participants’ characteristics

Of the 6153 children with appropriately measured activity, 3232

were girls and 2921 were boys. Two studies included girls

only,43 54 two included boys only,42 55 and 26 included both girls

and boys (the proportion of girls ranged from 27% to 64%,

median 51%), although only four of these reported sex specific

results.37 39 45 46 The mean age at baseline varied from 1.8 to 13.1

years (median 9.8 years). Eight studies involved exclusively

overweight/obese participants (n=691), and the rest involved

children recruited from all body mass index categories.

Types of outcome measures

Of the 30 eligible studies, 21 reported total physical activity,

23 reported moderate or vigorous physical activity, and 14

reported both. Where an article reported only one of these

outcomes, we emailed the corresponding author for the other.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 3 of 11

RESEARCH

Six authors responded, increasing the number of studies for

which we could analyse total physical activity to 25 (n=4386)

and moderate or vigorous physical activity to 25 (n=5001).

Twenty-six studies used uni-axial accelerometers to measure

activity (23 with the CSA/MTI Actigraph, two with the IM

Systems BioTrainer, and one with the Mini-Mitter Actical

uni-directional model), one study used a bi-axial accelerometer

(BodyMedia SenseWear Pro2 Armband), and three studies used

tri-axial accelerometers (two with the Hemokinetics

TriTrac-R3D and one with the Mini-Mitter Actical

omni-directional model). The accelerometer data collection

period varied according to study from one to 21 days. Sixteen

studies stipulated at least six days, which is the minimum

required to achieve at least 80% reliability in the measure of

physical activity.62

Methodological design and quality

We deemed 16 studies (n=3883) to be of “high” methodological

quality, as they scored positively on all three of the quality

criteria described above. Of the other 14 studies (n=2270), 12

scored positively on two of the quality criteria and two scored

positively on just one. Losses to follow-up ranged from 0% to

46% (median 11%); 20 studies reported less than 20% attrition.

All but three of the studies carried out intention to treat analysis.

Twenty-seven studies were randomised controlled trials; the

remaining three were controlled clinical trials. For two of the

controlled clinical trials, the intervention schools and control

schools were matched appropriately on age, ethnicity, and

socioeconomic status. The other trial selected schools from areas

that were “broadly comparable” on several socio-demographic

variables but did adjust for age, sex, and baseline activity in the

analyses. Fifteen of the included studies were cluster designs,

and all but two of these accounted for clustering in their

analyses. For these two studies, we multiplied the reported

variances by a “design factor” (1+((m−1)×ICC), where m is the

average number of participants in each cluster and ICC is the

intra-cluster correlation),63 which decreased their respective

weightings (towards the pooled effect) from 2.9% to 1.1% and

from 0.9% to 0.5%.

Overall summary estimate

The pooled analysis across all studies showed a statistically

significant effect in favour of the intervention group for both

total physical activity (standardised mean difference 0.12, 95%

confidence interval 0.04 to 0.20; P<0.01) and moderate or

vigorous physical activity (0.16, 0.08 to 0.24; P<0.001), although

both seemed to be of limited clinical significance according to

Cohen’s definitions.29 Hence, strong evidence exists from this

analysis to suggest that physical activity interventions have a

small to negligible effect on overall total physical activity and

a small effect on moderate or vigorous physical activity. Figure

2⇓ shows a forest plot of the effect sizes, confidence intervals,

and percentage weighting for both activity outcomes for the

individual studies and for all studies pooled. The largest

weighting of any individual study for total physical activity was

11.2%, and for moderate or vigorous physical activity it was

17.3%. A visual inspection of the forest plots suggests that a

degree of between study variation exists in terms of the effect

sizes. I2 values of 38% for total activity and 51% for moderate

or vigorous activity confirmed moderate levels of

heterogeneity.64

Effect sizes relative to study level covariates

We used 31 intervention effects (standardised mean differences)

for total physical activity and 33 for moderate or vigorous

physical activity in the analyses of all but three of the study

level covariates. The three exceptions were for duration (38 for

total physical activity and 42 for moderate or vigorous activity),

sex (9 and 13), and baseline activity (20 and 18). For all but

two of the covariates, the degree of heterogeneity did not differ

considerably between the subgroups. For study size and duration,

the I2 values were very different between the subgroups (for

example, for total physical activity, I2=0% for larger studies and

58% for smaller studies, and I2=0% for longer term studies and

50% for shorter term studies). In general, the subgroup specific

I2 values were slightly lower for total physical activity than for

moderate or vigorous physical activity (median I2=22% and

33% with interquartile ranges of 0-40% and 12-48%). Figure

3⇓ shows the pooled intervention effects (with 95% confidence

intervals) for the subgroups of each of the 10 study level

covariates analysed. None of the differences reached statistical

significance, although one was borderline. For total physical

activity, the standardised mean difference was 0.15 greater for

studies of overweight/obese only populations than for studies

with populations recruited from all body mass index categories

(P=0.07). Multivariate meta-regression confirmed that even

when considered simultaneously, none of the study level

covariates reached statistical significance (all P>0.05).

Figure 4⇓ shows the strength of linear association between the

intervention effects on total physical activity and each of the

continuous study level covariates (mean age, mean body mass

index, study size, and study duration). Body mass index z score

explained 7% of the variation in effect sizes (P=0.04) for which,

on average, the standardised mean difference was 0.074 greater

for each 1.0 increase in z score in mean body mass index.

However, this association did not exist across the range of mean

z scores—when we repeated the analysis without studies of

overweight/obese only participants (mean z score ≥2.4), the

association disappeared completely (β=0.008, P=0.69, R2=0.00).

This observation confirms the appropriateness of analysing body

mass index status as a categorical variable. None of the other

continuous covariates was significantly associated with the size

of the intervention effects (all P>0.18).

Sensitivity analysis

The findings of this meta-analysis were robust to the

assumptions that we made about the correlation between baseline

and follow-up activity. When the correlation was assumed to

be lower, at r=0.3, the resulting summary effects were

standardised mean differences of 0.11 for total physical activity

and 0.15 for moderate or vigorous physical activity. When the

correlation was assumed to be higher, at r=0.70, the summary

effects were standardised mean differences of 0.12 for total

physical activity and 0.17 for moderate or vigorous physical

activity. The meta-analysis results were also robust to the

inclusion of studies of lower methodological quality, as the

summary effects differed little when we excluded such studies

(based only on studies with a quality score of 3/3, standardised

mean differences were 0.09 for total physical activity and 0.11

for moderate or vigorous physical activity; based only on

randomised controlled trials, standardised mean differences

were 0.11 for total physical activity and 0.13 for moderate or

vigorous physical activity; based only on studies with <20%

loss to follow-up, standardised mean differences were 0.11 for

total physical activity and 0.15 for moderate or vigorous physical

activity; based only on studies that carried out intention to treat

analysis, standardised mean differences were 0.11 for total

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 4 of 11

RESEARCH

physical activity and 0.13 for moderate or vigorous physical

activity).

Publication bias

The funnel plots of the standardised mean differences and of

the residuals (produced from the meta-regression model

containing body mass index status) both showed a slightly

asymmetric scatter consistent with publication bias (fig 5⇓).

However, we cannot rule out the possibility of the “small study

effect,”65 as the asymmetry was attributable to the absence of

just two or three small studies producing negative effects.

Whichever is the case, the absence of some small negative

studies is likely to have had little effect on our findings, given

that the removal of small studies producing large positive effects

reduced the summary effect only from a standardised mean

difference of 0.12 to 0.09 for total physical activity and from

0.16 to 0.13 for moderate or vigorous physical activity.

Discussion

This systematic review found that physical activity interventions,

on average, achieved small to negligible increases in children’s

total activity volume, with small improvements in the time spent

in moderate or vigorous intensity activities (~4 minutes more

walking or running per day), the clinical effect of which is likely

to be minimal (for example, ~2 mm in waist circumference or

~0.06 mm Hg in systolic blood pressure).11 This review also

shows that the pooled intervention effect did not differ

significantly according to subgroups of study level

characteristics. Although trials of exclusively overweight/obese

participants tended to be slightly more effective at increasing

total activity than did those that recruited children from all body

mass index categories, the pooled intervention effect for this

group of studies was still small.

Possible explanations

Understanding why past attempts to increase children’s activity

have largely proved unsuccessful might help to improve future

attempts. Where interventions have failed to increase activity

or reduce body fat, authors have speculated about poor delivery

or poor uptake of the activity sessions,46 or suggested that the

physical activity component of the intervention was not

sufficiently intense.34 However, although these reasons are

intuitive and plausible, they are difficult to test. An alternative

explanation could be that the intervention specific exercise

sessions may simply be replacing periods of equally intense

activity. For example, after school activity clubs may simply

replace a period of time that children usually spend playing

outdoors or replace a time later in the day/week when the child

would usually be active.

Strengths and limitations

This review, uniquely we believe, focuses exclusively on

objectively measured, whole day physical activity.

Accelerometers are considered the criterion method for

measuring free living physical activity,22 23 and the avoidance

of studies using self reported measures meant avoidance of the

bias associated with them.21 Taber and colleagues showed that

for a given level of accelerometry derived activity, self reported

levels of activity were significantly higher in the intervention

group than in the control group.21 Although accelerometers are

accepted as being limited in their ability to measure water based

activities and cycling, such limitations are unlikely to have

biased the results given that none of the interventions was

specific to promoting swimming or cycling.

Our review also excludes studies that reported only activity

differences during intervention specific parts of the day, as effect

sizes derived from such short periods risk an over-estimation

of whole day effects. This can be demonstrated with the results

of the KISS study, in which the school time intervention effect

was more than four times greater than the whole day intervention

effect (total physical activity z scores 0.92 v 0.21).32 The reviews

by van Sluijs et al and Salmon et al incorporated studies that

reported any component of activity or inactivity, including those

that reported only “sedentary time” as their outcome.16 17 Our

review was specific to total physical activity and time spent in

moderate or vigorous activity, as these are strongly correlated

with activity related energy expenditure.22 23 We did not analyse

sedentary time, as this seems to be a poor surrogate measure of

total activity.66

Inevitably, some studies failed on just one of the inclusion

criteria but otherwise might have made a useful contribution to

the aim of this review. We excluded the TAAG trial because

the baseline and follow-up measures were not paired but were

collected from essentially independent samples.67 Nevertheless,

its largest intervention effect (+1.6 minutes/day of moderate or

vigorous physical activity, standardised mean difference ~0.06)

was comparable to the findings of our review.67 Three studies

that had accelerometer data at follow-up were excluded because

baseline activity was measured by questionnaire only.68-70 All

three produced effects sizes that were similar to the pooled

estimates obtained from this meta-analysis (the STOPP study:

standardised mean difference 0.1168; the LEAP and LEAP2

studies (both exclusive to overweight/obese children):

standardised mean difference ~0.269 70). We note these data to

show that the findings of our review would not have differed

had they been incorporated.

Many of the potential effect modifiers that we chose to explore

were evidence generated. For example, sex, age, and body mass

index have all been shown to be associated with objectively

measured physical activity.10 71 Duration, study size, and

mid-intervention outcomes were also positively associated with

greater reductions in sedentary time according to a previous

meta-analysis.12 The reporting of information on the dose,

frequency, and content of activity/exercise sessions varied so

much between trials that we were obliged to dichotomise it

simply as “provision of designated activity/exercise sessions:

yes/no” for the purpose of analysis, being aware nevertheless

that resolution of the measure might be compromised in the

process. We acknowledge that within study subgroup effects

would have provided a more powerful method of examining

subgroup differences (that is, testing whether the intervention

effect differed according to categories of risk—for example, by

baseline activity or body mass index status). However, of the

studies we inspected for this review, none reported such

“intervention × risk” analyses.

Comparisons with other recent reviews

The findings of this review are consistent with those of a

previous meta-analysis by Kamath and colleagues.12 That review

was primarily concerned with the effect of behavioural

interventions on the prevention of childhood obesity. It also

explored the effect of physical activity interventions on activity

levels, reporting an overall effect identical to the summary effect

calculated in our meta-analysis (standardised mean difference

0.12), despite only three studies being common to both. The

studies included in the review by Kamath et al were not specific

to whole day activity, nor to accelerometry, and half the studies

relied on questionnaire based physical activity. The authors

excluded studies in overweight/obese only populations,12 and,

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 5 of 11

RESEARCH

when such studies are removed from our meta-analysis, the

overall summary effect for total activity falls to a standardised

mean difference of 0.07. Two other systematic reviews

summarising the effectiveness of interventions on physical

activity were published in 2007 and reported similar findings

to one another.16 17 Salmon et al reviewed 76 studies and van

Sluijs et al reviewed 57, of which 40-50% were deemed to have

had a positive and statistically significant effect on activity.

Both reviews reported a higher proportion of positive

intervention effects in studies that measured activity objectively,

compared with those that measured it by questionnaire.

However, both considered “direct observation” to be an

objective measure, and this may have inflated the number of

positive results, given that direct observation is used only to

measure short, intervention specific periods of the day (such as

physical education classes or break/recess times).

Context, implications for health policy, and

future research

In the mind of the public, physical inactivity is a major cause

of childhood obesity, and the need to increase activity is

intuitive. However, the small increase in physical activity that

emerges from formal interventions seems insufficient to improve

the body mass/fat of children. Understanding why physical

activity interventions fail to increase activity sufficiently is

important, particularly as activity is associated with better

metabolic health independently of fatness.11 Accelerometers

relate minute by minute data to clock time, so future studies

could capture both whole day activity and the activity related

to intervention specific periods. Such an approach could not

only measure the real time activity response to an intervention

but could also establish whether the response was subsequently

subject to a degree of replacement or compensation.72 Also,

future physical activity intervention studies might usefully do

within study risk group analyses to examine whether the

intervention does indeed achieve the intended response in those

children who stand to benefit the most.

Systematic reviews and meta-analysis are not without their

critics, but they nevertheless represent the best available

evidence to government strategists. However counterintuitive

or discomforting it may be, strong evidence from this review

shows that physical activity interventions have little effect on

the overall activity of children. Organised physical activity may

nevertheless still offer benefits such as improved coordination

skills, greater self confidence, team participation, and social

inclusivity.

Conclusions

Physical activity interventions have little effect on the overall

activity levels of children, which may explain, at least in part,

why such interventions have had a limited effect on body mass

index or body fat. The outcome of this meta-analysis questions

the contribution of physical activity to the prevention of

childhood obesity.

Contributors: BM was involved in the design, research, analyses, and

writing of this article. WH was involved in the analysis. TW was involved

in the design, research, and writing. TW is the guarantor.

Funding: EarlyBird (BM and TW) is currently supported by the Bright

Future Trust, the Kirby Laing Foundation, the Peninsula Foundation,

and the EarlyBird Diabetes Trust. These funding sources had no role

in the study design, data collection, analysis, interpretation of results,

or the writing of the report. WH was supported by the National Institute

for Health Research (NIHR) Collaborations for Leadership in Applied

Health Research and Care (CLAHRC). The views expressed in this

publication are those of the authors and not necessarily those of the

NHS, the NIHR, or the Department of Health.

Competing interests: All authors have completed the Unified Competing

Interest form at www.icmje.org/coi_disclosure.pdf (available on request

from the corresponding author) and declare: no support from any

organisation for the submitted work; no financial relationships with any

organisations that might have an interest in the submitted work in the

previous three years; no other relationships or activities that could appear

to have influenced the submitted work.

Ethical approval: Not needed.

Data sharing: No additional data available.

1 Janssen I, Leblanc AG. Systematic review of the health benefits of physical activity and

fitness in school-aged children and youth. Int J Behav Nutr Phys Act 2010;7:40.

2 Guinhouya BC, Samouda H, Zitouni D, Vilhelm C, Hubert H. Evidence of the influence of

physical activity on the metabolic syndrome and/or on insulin resistance in pediatric

populations: a systematic review. Int J Pediatr Obes 2011;6:361-88.

3 Saavedra JM, Escalante Y, Garcia-Hermoso A. Improvement of aerobic fitness in obese

children: a meta-analysis. Int J Pediatr Obes 2011;6:169-77.

4 Metcalf BS, Voss LD, Hosking J, Jeffery AN, Wilkin TJ. Physical activity at the

government-recommended level and obesity-related health outcomes: a longitudinal study

(Early Bird 37). Arch Dis Child 2008;93:772-7.

5 Riddoch CJ, Mattocks C, Deere K, Saunders J, Kirkby J, Tilling K, et al. Objective

measurement of levels and patterns of physical activity. Arch Dis Child 2007;92:963-9.

6 Centers for Disease Control and Prevention (CDC). Physical activity levels of high school

students—United States, 2010. MMWR Morb Mortal Wkly Rep 2011;60:773-7.

7 Ekelund U, Sardinha LB, Anderssen SA, Harro M, Franks PW, Brage S, et al. Associations

between objectively assessed physical activity and indicators of body fatness in 9- to

10-y-old European children: a population-based study from 4 distinct regions in Europe

(the European Youth Heart Study). Am J Clin Nutr 2004;80:584-90.

8 Andersen LB, Harro M, Sardinha LB, Froberg K, Ekelund U, Brage S, et al. Physical

activity and clustered cardiovascular risk in children: a cross-sectional study (the European

Youth Heart Study). Lancet 2006;368:299-304.

9 Ness AR, Leary SD, Mattocks C, Blair SN, Reilly JJ, Wells J, et al. Objectively measured

physical activity and fat mass in a large cohort of children. PLoS Med 2007;4:e97.

10 Metcalf BS, Hosking J, Jeffery AN, Voss LD, Henley W, Wilkin TJ. Fatness leads to

inactivity, but inactivity does not lead to fatness: a longitudinal study in children (EarlyBird

45). Arch Dis Child 2011;96:942-7.

11 Ekelund U, Luan J, Sherar LB, Esliger DW, Griew P, Cooper A. Moderate to vigorous

physical activity and sedentary time and cardiometabolic risk factors in children and

adolescents. JAMA 2012;307:704-12.

12 Kamath CC, Vickers KS, Ehrlich A, McGovern L, Johnson J, Singhal V, et al. Behavioral

interventions to prevent childhood obesity: a systematic review and metaanalyses of

randomized trials. J Clin Endocrinol Metab 2008;93:4606-15.

13 Harris KC, Kuramoto LK, Schulzer M, Retallack JE. Effect of school-based physical activity

interventions on body mass index in children: a meta-analysis. CMAJ 2009;180:719-26.

14 Waters E, de Silva-Sanigorski A, Hall BJ, Brown T, Campbell KJ, Gao Y, et al. Interventions

for preventing obesity in children. Cochrane Database Syst Rev 2011;12:CD001871.

15 Westerterp KR. Physical activity, food intake, and body weight regulation: insights from

doubly labeled water studies. Nutr Rev 2010;68:148-54.

16 Van Sluijs EM, McMinn AM, Griffin SJ. Effectiveness of interventions to promote physical

activity in children and adolescents: systematic review of controlled trials. BMJ

2007;335:703.

17 Salmon J, Booth ML, Phongsavan P, Murphy N, Timperio A. Promoting physical activity

participation among children and adolescents. Epidemiol Rev 2007;29:144-59.

18 Sallis JF, Strikmiller PK, Harsha DW, Feldman HA, Ehlinger S, Stone EJ, et al. Validation

of interviewer- and self-administered physical activity checklists for fifth grade students.

Med Sci Sports Exerc 1996;28:840-51.

19 Treuth MS, Hou N, Young DR, Maynard LM. Validity and reliability of the Fels physical

activity questionnaire for children. Med Sci Sports Exerc 2005;37:488-95.

20 Basterfield L, Adamson AJ, Parkinson KN, Maute U, Li PX, Reilly JJ. Surveillance of

physical activity in the UK is flawed: validation of the Health Survey for England physical

activity questionnaire. Arch Dis Child 2008;93:1054-8.

21 Taber DR, Stevens J, Murray DM, Elder JP, Webber LS, Jobe JB, et al. The effect of a

physical activity intervention on bias in self-reported activity. Ann Epidemiol

2009;19:316-22.

22 Puyau MR, Adolph AL, Vohra FA, Butte NF. Validation and calibration of physical activity

monitors in children. Obes Res 2002;10:150-7.

23 Schmitz KH, Treuth M, Hannan P, McMurray R, Ring KB, Catellier D, et al. Predicting

energy expenditure from accelerometry counts in adolescent girls. Med Sci Sports Exerc

2005;37:155-61.

24 Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews

and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535.

25 Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Effect sizes based on means. In:

Introduction to meta-analysis. 1st ed. John Wiley and Sons, 2009:21-32.

26 Mattocks C, Leary S, Ness A, Deere K, Saunders J, Kirkby J, et al. Intraindividual variation

of objectively measured physical activity in children. Med Sci Sports Exerc 2007;39:622-9.

27 Kristensen PL, Moller NC, Korsholm L, Wedderkopp N, Andersen LB, Froberg K. Tracking

of objectively measured physical activity from childhood to adolescence: the European

youth heart study. Scand J Med Sci Sports 2008;18:171-8.

28 Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Multiple outcomes or time-points

within a study. In: Introduction to meta-analysis. 1st ed. John Wiley and Sons, 2009:225-38.

29 Cohen J. Statistical power analysis for the behavioral sciences. Lawrence Erlbaum

Associates, 1987.

30 Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Meta-regression. In: Introduction

to meta-analysis. 1st ed. John Wiley and Sons, 2009:187-203.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 6 of 11

RESEARCH

What is already known on this topic

Few children meet the recommendations for physical activity, and this is of concern as greater activity is associated with better health

outcomes and less obesity

Physical activity interventions nevertheless have little effect on the prevention of childhood obesity, and the reason for this has been

unclear

What this study adds

Physical activity interventions have little effect on the overall activity of children, which may explain, at least in part, why such strategies

have had limited success in preventing childhood obesity

31 Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Reporting the results of a

meta-analysis. In: Introduction to meta-analysis. 1st ed. John Wiley and Sons, 2009:365-70.

32 Kriemler S, Zahner L, Schindler C, Meyer U, Hartmann T, Hebestreit H, et al. Effect of

school based physical activity programme (KISS) on fitness and adiposity in primary

schoolchildren: cluster randomised controlled trial. BMJ 2010;340:c785.

33 Baranowski T, Baranowski J, Thompson D, Buday R, Jago R, Griffith MJ, et al. Video

game play, child diet, and physical activity behavior change a randomized clinical trial.

Am J Prev Med 2011;40:33-8.

34 Caballero B, Clay T, Davis SM, Ethelbah B, Rock BH, Lohman T, et al. Pathways: a

school-based, randomized controlled trial for the prevention of obesity in American Indian

schoolchildren. Am J Clin Nutr 2003;78:1030-8.

35 Cliff DP, Okely AD, Morgan PJ, Steele JR, Jones RA, Colyvas K, et al. Movement skills

and physical activity in obese children: randomized controlled trial. Med Sci Sports Exerc

2011;43:90-100.

36 Goldfield GS, Mallory R, Parker T, Cunningham T, Legg C, Lumb A, et al. Effects of

open-loop feedback on physical activity and television viewing in overweight and obese

children: a randomized, controlled trial. Pediatrics 2006;118:e157-66.

37 Goran MI, Reynolds K. Interactive multimedia for promoting physical activity (IMPACT)

in children. Obes Res 2005;13:762-71.

38 Gorely T, Nevill ME, Morris JG, Stensel DJ, Nevill A. Effect of a school-based intervention

to promote healthy lifestyles in 7-11 year old children. Int J Behav Nutr Phys Act 2009;6:5.

39 Haerens L, Deforche B, Maes L, Cardon G, Stevens V, De Bourdeaudhuij I. Evaluation

of a 2-year physical activity and healthy eating intervention in middle school children.

Health Educ Res 2006;21:911-21.

40 Harvey-Berino J, Rourke J. Obesity prevention in preschool native-american children: a

pilot study using home visiting. Obes Res 2003;11:606-11.

41 Hughes AR, Stewart L, Chapple J, McColl JH, Donaldson MD, Kelnar CJ, et al.

Randomized, controlled trial of a best-practice individualized behavioral program for

treatment of childhood overweight: Scottish Childhood Overweight Treatment Trial

(SCOTT). Pediatrics 2008;121:e539-46.

42 Jago R, Baranowski T, Baranowski JC, Thompson D, Cullen KW, Watson K, et al. Fit for

Life Boy Scout badge: outcome evaluation of a troop and internet intervention. Prev Med

2006;42:181-7.

43 Klesges RC, Obarzanek E, Kumanyika S, Murray DM, Klesges LM, Relyea GE, et al. The

Memphis Girls’ health Enrichment Multi-site Studies (GEMS): an evaluation of the efficacy

of a 2-year obesity prevention program in African American girls. Arch Pediatr Adolesc

Med 2010;164:1007-14.

44 Maddison R, Foley L, Ni MC, Jiang Y, Jull A, Prapavessis H, et al. Effects of active video

games on body composition: a randomized controlled trial. Am J Clin Nutr 2011;94:156-63.

45 Patrick K, Calfas KJ, Norman GJ, Zabinski MF, Sallis JF, Rupp J, et al. Randomized

controlled trial of a primary care and home-based intervention for physical activity and

nutrition behaviors: PACE+ for adolescents. Arch Pediatr Adolesc Med 2006;160:128-36.

46 Reilly JJ, Kelly L, Montgomery C, Williamson A, Fisher A, McColl JH, et al. Physical activity

to prevent obesity in young children: cluster randomised controlled trial. BMJ

2006;333:1041.

47 Roemmich JN, Gurgol CM, Epstein LH. Open-loop feedback increases physical activity

of youth. Med Sci Sports Exerc 2004;36:668-73.

48 Salmon J, Ball K, Hume C, Booth M, Crawford D. Outcomes of a group-randomized trial

to prevent excess weight gain, reduce screen behaviours and promote physical activity

in 10-year-old children: switch-play. Int J Obes (Lond) 2008;32:601-12.

49 Taylor RW, McAuley KA, Barbezat W, Strong A, Williams SM, Mann JI. APPLE Project:

2-y findings of a community-based obesity prevention program in primary school age

children. Am J Clin Nutr 2007;86:735-42.

50 Verstraete SJ, Cardon GM, De Clercq DL, De Bourdeaudhuij IM. A comprehensive physical

activity promotion programme at elementary school: the effects on physical activity,

physical fitness and psychosocial correlates of physical activity. Public Health Nutr

2007;10:477-84.

51 Wafa SW, Talib RA, Hamzaid NH, McColl JH, Rajikan R, Ng LO, et al. Randomized

controlled trial of a good practice approach to treatment of childhood obesity in Malaysia:

Malaysian Childhood Obesity Treatment Trial (MASCOT). Int J Pediatr Obes 2011;6:e62-9.

52 Weintraub DL, Tirumalai EC, Haydel KF, Fujimoto M, Fulton JE, Robinson TN. Team

sports for overweight children: the Stanford Sports to Prevent Obesity Randomized Trial

(SPORT). Arch Pediatr Adolesc Med 2008;162:232-7.

53 Wilson DK, Evans AE, Williams J, Mixon G, Sirard JR, Pate R. A preliminary test of a

student-centered intervention on increasing physical activity in underserved adolescents.

Ann Behav Med 2005;30:119-24.

54 Robinson TN, Matheson DM, Kraemer HC, Wilson DM, Obarzanek E, Thompson NS, et

al. A randomized controlled trial of culturally tailored dance and reducing screen time to

prevent weight gain in low-income African American girls: Stanford GEMS. Arch Pediatr

Adolesc Med 2010;164:995-1004.

55 Peralta LR, Jones RA, Okely AD. Promoting healthy lifestyles among adolescent boys:

the Fitness Improvement and Lifestyle Awareness Program RCT. Prev Med

2009;48:537-42.

56 Farpour-Lambert NJ, Aggoun Y, Marchand LM, Martin XE, Herrmann FR, Beghetti M.

Physical activity reduces systemic blood pressure and improves early markers of

atherosclerosis in pre-pubertal obese children. J Am Coll Cardiol 2009;54:2396-406.

57 Fitzgibbon ML, Stolley MR, Schiffer LA, Braunschweig CL, Gomez SL, Van HL, et al.

Hip-Hop to Health Jr. Obesity Prevention Effectiveness Trial: postintervention results.

Obesity (Silver Spring) 2011;19:994-1003.

58 Wilson DK, Van Horn ML, Kitzman-Ulrich H, Saunders R, Pate R, Lawman HG, et al.

Results of the “Active by Choice Today” (ACT) randomized trial for increasing physical

activity in low-income and minority adolescents. Health Psychol 2011;30:463-71.

59 Bäcklund C, Sundelin G, Larsson C. Effects of a 2-year lifestyle intervention on physical

activity in overweight and obese children. Adv Physiother 2011;13:97-109.

60 Puder JJ, Marques-Vidal P, Schindler C, Zahner L, Niederer I, Burgi F, et al. Effect of

multidimensional lifestyle intervention on fitness and adiposity in predominantly migrant

preschool children (Ballabeina): cluster randomised controlled trial. BMJ 2011;343:d6195.

61 Magnusson KT, Sigurgeirsson I, Sveinsson T, Johannsson E. Assessment of a two-year

school-based physical activity intervention among 7-9-year-old children. Int J Behav Nutr

Phys Act 2011;8:138.

62 Basterfield L, Adamson AJ, Pearce MS, Reilly JJ. Stability of habitual physical activity

and sedentary behavior monitoring by accelerometry in 6- to 8-year-olds. J Phys Act

Health 2011;8:543-7.

63 Cochrane Collaboration. Cochrane handbook for systematic reviews of interventions.

2011. www.cochrane-handbook.org.

64 Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in

meta-analyses. BMJ 2003;327:557-60.

65 Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Publication bias. In: Introduction to

meta-analysis. 1st ed. John Wiley and Sons, 2009: 277-91.

66 Marshall SJ, Biddle SJ, Gorely T, Cameron N, Murdey I. Relationships between media

use, body fatness and physical activity in children and youth: a meta-analysis. Int J Obes

Relat Metab Disord 2004;28:1238-46.

67 Webber LS, Catellier DJ, Lytle LA, Murray DM, Pratt CA, Young DR, et al. Promoting

physical activity in middle school girls: Trial of Activity for Adolescent Girls. Am J Prev

Med 2008;34:173-84.

68 Marcus C, Nyberg G, Nordenfelt A, Karpmyr M, Kowalski J, Ekelund U. A 4-year,

cluster-randomized, controlled childhood obesity prevention study: STOPP. Int J Obes

(Lond) 2009;33:408-17.

69 McCallum Z, Wake M, Gerner B, Baur LA, Gibbons K, Gold L, et al. Outcome data from

the LEAP (Live, Eat and Play) trial: a randomized controlled trial of a primary care

intervention for childhood overweight/mild obesity. Int J Obes (Lond) 2007;31:630-6.

70 Wake M, Baur LA, Gerner B, Gibbons K, Gold L, Gunn J, et al. Outcomes and costs of

primary care surveillance and intervention for overweight or obese children: the LEAP 2

randomised controlled trial. BMJ 2009;339:b3308.

71 Riddoch CJ, Bo AL, Wedderkopp N, Harro M, Klasson-Heggebo L, Sardinha LB, et al.

Physical activity levels and patterns of 9- and 15-yr-old European children. Med Sci Sports

Exerc 2004;36:86-92.

72 Frémeaux AE, Mallam KM, Metcalf BS, Hosking J, Voss LD, Wilkin TJ. The impact of

school-time activity on total physical activity: the activitystat hypothesis (EarlyBird 46). Int

J Obes (Lond) 2011;35:1277-83.

Accepted: 21 August 2012

Cite this as: BMJ 2012;345:e5888

This is an open-access article distributed under the terms of the Creative Commons

Attribution Non-commercial License, which permits use, distribution, and reproduction in

any medium, provided the original work is properly cited, the use is non commercial and

is otherwise in compliance with the license. See: http://creativecommons.org/licenses/bync/

2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 7 of 11

RESEARCH

Figures

Fig 1 Summary of study selection process

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 8 of 11

RESEARCH

Fig 2 Forest plot showing standardised mean difference in change in physical activity between intervention and control

groups for each included study. NA=not available

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 9 of 11

RESEARCH

Fig 3 Standardised mean differences in changes in physical activity between intervention and control groups by subgroups

of studies. BMI=body mass index; cpm=counts per minute; PA=physical activity

Fig 4 Intervention effects in total physical activity in relation to continuous study level covariates: age at baseline (top left),

body mass index (BMI) at baseline (top right), study size (bottom left), and study duration (bottom right)

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 10 of 11

RESEARCH

Fig 5 Assessment of publication bias: funnel plots of observed effects in total physical activity for each study (top) and

meta-regression residuals of total physical activity for each comparison with body mass index status as a study level

covariate (bottom)

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2012;345:e5888 doi: 10.1136/bmj.e5888 (Published 27 September 2012) Page 11 of 11

RESEARCH

Leave a Reply

Your email address will not be published. Required fields are marked *