delayed.oscillator

Climate, paleoclimate, huevos rancheros, and general asymmetry

Seeing Red (Noise): Galactic Cosmic Ray Fluxes and Tree Rings

with 9 comments

First off, thanks to delayed.oscillator for inviting me to participate.  I’m excited to be involved.  Secondly, please forgive some of the formatting – I’m still new to this whole blogging thing.  And away we go …

A recent paper (Dengel et al), published in the usually rigorous journal New Phytologist purports to show a statistically significant positive correlation between Galactic Cosmic Ray (GCR) fluxes and tree growth of sitka spruce (Picea sitka) from a plantation or managed forest in Scotland. Their hypothesis follows: High GCR fluxes stimulate aerosol nucleation in the atmosphere, leading to higher aerosol concentrations. The increased aerosol concentrations increase scattering of radiation, increasing the ratio of diffuse to direct radiation fluxes. The increased diffuse radiation penetrates more deeply into tree canopies, stimulating photosynthesis and growth.

Others have already shown that links between GCR fluxes and the Earth’s climate are quite dubious. Here we show that the conclusions drawn by the authors of this study are likely based on errors in their statistical analysis and faults in their study design.

The primary evidence in this paper for the GCR/diffuse radiation link to tree growth is tied to two Pearson correlations. One is a correlation between tree growth and March-August total diffuse radiation (n=45, r=+0.29, P=0.05). The second is a correlation between tree growth and GCR fluxes (n=45, r=+0.39, P=0.008). On the surface, both correlations appear to pass the traditionally accepted significance levels of P<=0.05, supporting the authors’ hypothesis.

The authors use a nominal sample size equal to 45, the total number of years in their dataset. This is fine, assuming that the underlying observations represent independent samples, e.g. a ‘white noise’ time series. To test this, we digitized the tree ring growth anomaly time series from the paper and estimated the lag-1 autocorrelation. If each observation were independent, the autocorrelation should be near 0. For this series, the autocorrelation is actually quite high (r=0.484) and significant (P=0.0008), indicating this time series has significant persistence (‘red’). The individual observations are not independent, and the significance of a Pearson correlation with n=45 will be overestimated.

This is easy enough to account for, however:

n'=n\frac{(1-r)} {(1+r)}

where n=original sample size, r is the autocorrelation of the underlying time series, and n’ is the new (effective) sample size. Plugging into this equation, we find that the sitka spruce tree growth time series has an effective sample size of only 16, instead of 45.

This change will not influence the actual correlation, but it will affect the test statistic used to determine statistical significance. For both correlations, we can recalculate the test statistic and significance level with the new effective sample size of 16:

GCR____        n            T Statistic        Significance
Original          45               2.7773               0.008
Revised           16               1.5647               0.139

Radiation        n            T Statistic        Significance
Original          45               1.9870               0.050
Revised           16               1.1195                0.281

In both cases, the correlations now do not meet the criteria (P<0.05) to be considered statistically significant.

It is also very clear, from reading the paper, that the researchers were considering many, many possible associations — looking for a statistical correlation between the tree rings and some variable, regardless of whether there was a strong a priori theoretical basis to think there should be.  Some of their correlations make sense – boreal summer temperatures concurrent with the growth year, for example, is reasonable since one would expect trees growing in Scotland could (possibly) be sensitive to temperatures during the growing season.  Others are more tenuous, as when the authors attempt to correlate tree growth to diffuse radiation from the previous year.

In cases where one is data mining (as in this paper), it is important to guard against significant correlations that may be due simply to the overwhelming number of statistical tests made.  Conceptually, it helps to think of the P value as your percent chance of a ‘false positive’, the chance that a statistically significant correlation is due to chance, rather than something meaningful. For most purposes, a P value of 0.05 (a 5% chance of a false positive) or less is considered acceptable. Each time you attempt another test, however, you increase the opportunity for this type of error. So, if two tests are conducted, the chance of a false positive, at a P=0.05 acceptable threshold, is actually 9.75% (assuming independent comparisons). In the case of total diffuse radiation, Dengel et al compared their tree growth time series against total diffuse radiation for each month of the current and previous growth years, a total of 24 tests (70.8% chance of a false positive).  This makes it very likely that some of the correlations will be significant due to chance alone.

To adjust for this, researchers may use some flavor of what is called a Bonferroni correction, modifying the original P value to account for multiple comparisons.  One simple way to do this is to divide your original acceptable threshold by the number of tests you conduct, increasing the burden of proof to accept a significant result.  In the diffuse radiation example (tree growth versus diffuse radiation), that means a P=0.05 should actually be P=0.05/24=0.002.  In other words, to have a P value confidence of 0.05, you actually need to meet a much stricter threshold, P=0.002. This burden is not met, either in the original analysis or with the adjusted sample size. The Bonferroni correction leads to less false positives (Type I errors), although it may increase the rate of false negatives (Type II errors).

Aside from not accounting for these features of their data and analysis, there are also some methodological oddities in the study design, where Dengel et al deviate from some standard practices in dendrochronology.  First, they do not report many of the standard statistics used to assess the quality of a tree ring chronology-e.g. the mean interseries correlation (the common signal amongst trees) nor the mean sensitivity (a measure of a variable vs. complacent the ring width series).  This makes it impossible for a reader to assess how well the trees correlate with each other, and whether the chronology really represents a common signal among the trees rather than simply an assemblage of noise.  Second, they appear to use essentially randomly sampled trees from a managed forest. Normally, to find a tree (or set of trees) with a strong climate response researchers target trees where climate is the most limiting factor for growth. This often means trees near their climatic limits, where they may be stressed by temperature, moisture, or even radiation. Plantations or managed forests are typically not limited by these factors, because the goal is to manage growth for some purpose (e.g., timber, wildlife, et).  Often this management can be quite intensive, involving fertilizer application, protection from pests, and thinning.  Even at a location where tree growth is limited by climate, trees are not simply randomly sampled to try to find a climate signal. In particular, trees that may be influenced by competitive interactions with other trees (shading, etc) are generally avoided, because the signal in the tree rings will not necessarily best represent a response to larger scale climate variability.  Randomly sampling trees through an even aged stand, as appears to be done in this study, will likely result in a large-scale signal that is equivocal or, worse, misleading. This is evidenced clearly by Figure 2 in the paper, where the authors attempted (largely unsuccessfully) to correlate tree growth against precipitation, temperature, and vapor pressure deficit.  Dengel et al also use very short time series (only 45 years of growth), truncate the trees such that the juvenile growth is partially removed, and appear to use a relatively stiff spline in their detrending (potentially problematic in trees that may have experienced growth changes related to stand dynamics).

The criticisms I have made are not particularly nuanced or obscure, and are largely standard practice in climatology and dendrochronology.  The lack of adherence to these practices likely led Dengel et al astray.  I largely suspect their results will not be reproducible in other studies with a more typical design and analysis, but time will tell.

About these ads

Written by transienteddy

October 28, 2009 at 2:22 am

9 Responses

Subscribe to comments with RSS.

  1. I hope you are going to submit a version of this comment to the journal. I wrote to the authors last week to recommend they consider autocorrelation as an alternative explanation (and implied they should quickly write a retraction), but no reply yet.

    The paper, in its rush to account for the correlation between rings and rays, forgets that the cosmic radiation is controlled by solar activity, and that this solar activity could more directly affect tree growth. I don’t subscribe to this, but it is more plausible that a direct effect of rays.

    Climate skeptics have begun to notice this paper. Always desperate to accept any work, however risible, that they believe supports their position, and any flaw, however minute (or imaginary) in the rest of the literature.

    richardjamest

    October 28, 2009 at 7:57 am

  2. Hi Richard,

    I considered submitting a comment to New Phytologist, but the cost/benefit makes it a little difficult to justify taking the time to do it. For example, Dengel et al got a fair bit of press for this. If i were to submit a comment, it would go back and forth between myself, reviewers, and the editorial board for several months. If it passes review, it will get stuck in the back of an issue maybe a year from now, long past the time that people are even talking about GCR and tree rings. It’s just not worth the effort (for me, anyway).

    In the end, I’ve found the literature to be pretty good at self correcting. Considering the difficulty that anyone will have reproducing their results with proper statistics and study design, I think this stuff will disappear off the radar pretty soon.

    transienteddy

    October 28, 2009 at 2:31 pm

  3. There was comment published alongside Dengel’s article by Kulmala et al. I cannot read it from here, but from what Dengel writes (I now have made contact) it is looking at another set of data rather than a direct critism. I’ll get access tomorrow.

    richardjamest

    October 28, 2009 at 6:49 pm

  4. You really should do a minimal-effort comment. The journal has stuffed up – when you get a very surprising statistical result they should have got a referee who equipped to check out the stats.

    litoria

    November 2, 2009 at 9:44 pm

  5. I don’t think the correction n(1-r)/(1+r) is correct. I tried some simulations, and it can be very conservative. gls seems to have much better type I error rates. Some R code below.
    I am also not sure you can critisise the authors for choosing a plantation. If you want to investigate the effects of sky conditions on forests, then choosing a site where climate affects are weak would make sense.

    library(nlme)
    rho<-c(none=.000001,low=.1,moderate=.3,moderatehigh=.6,high=.9)
    res<-lapply(rho,function(rx){##slow – time for coffee
    n=100
    x<-arima.sim(list(ar=rx),n=n)
    sapply(rho,function(ry){
    n2<-n*(1-ry)/(1+ry)#effective number of df
    res<-replicate(500,{
    y<-arima.sim(list(ar=ry),n=n)
    r<-cor(x,y)
    stat<-sqrt(n-2)*r/sqrt(1-r^2)
    stat2<-sqrt(n2-2)*r/sqrt(1-r^2)
    mod <- gls(y ~ x, correlation = corAR1(), method = "ML")
    c(pearson=pt(stat,n-2),correctedpearson=pt(stat2,n2-2), gls=anova(mod)$p[2])
    })
    rowMeans(res<.05)

    })

    })
    res

    richardjamest

    November 6, 2009 at 12:47 pm

    • Hi Richard,

      Thanks — n(1-r)/(1+r) is a common way to adjust for autocorrelation, although as one approaches larger r values it will indeed become very (perhaps too) conservative. I’ll be interested to take a look at your code. Another approach would be some sort of phase randomizing Monte Carlo method to estimate the p value for the correlation of the actual data. I’ll try this if I get some time. Perhaps Eddy can weigh in on this as well.

      On the issue of the plantation, while you are right that if you were looking for a sunlight signal in tree ring you would want to choose a site that was unlikely to be temperature or precipitation limited, my concern would be that a managed forest or plantation is going to have some potentially undesirable growth patterns due to competition (between trees) or management practices (i.e. thinning). Without the data, all I could do is speculate about the specific features of the Dengel chronology that could be related to stand dynamics (either intentional thinning or other management practices or release/suppression interactions between trees), and I don’t want to do that. But in general the concern is that non-climatic (including light) factors could be influencing tree growth in a planted, managed forest of young, non native trees.

      delayedoscillator

      November 9, 2009 at 12:44 pm

    • Hi Richard,

      I did a few more tests. First, I tried a random phase Monte Carlo. For my emulation of their correlation between ring width and GCR, I got r = 0.40, p = 0.12 for n = 10,000 phase randomized simulations.

      Alternatively, if I account for the autocorrelation in both ring width and in the GCR data, I get a critical r-value for a = 0.05 of r = 0.55 and an estimated p = 0.14.

      delayedoscillator

      November 10, 2009 at 1:50 pm

  6. I’m just a poor backwoods mathematician trying to make it by in a cold cruel world, but seems to me that in order to make any such lofty claims, one should at least consider trees from some other place (at the same latitude) on which to conduct the same experiments. I should wonder if there is any industry in Scotland upwind from these trees whose effluent might be affected by cosmic rays and that in turn affects this stand of trees; a distillery perhaps (I know that scotch and sunshine have an effect on me.)

    It’s been years since I have I mucked in statistics but correlation is still miles from causality.

    nancynaive

    December 28, 2009 at 4:37 pm

    • Hi Nancy,

      ‘I know that scotch and sunshine have an effect on me’ — Hah! Perfect :)

      In all seriousness — yes, we’d want to see this repeated (with statistical significance this time!), as well as have a mechanism.

      delayedoscillator

      December 29, 2009 at 3:22 am


Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: