Assignment 16
Statistics and Significance
Adapted from "Determining confidence: sex and statistics" by T. L. Derting et al., 2005

Goals: As you conducted your research projects and discussed various experiments in class, you may have wondered how one determines whether an experimental treatment truly has a biological effect, or whether differences among experimental groups are simply due to chance.  The following activity is designed to guide your exploration of statistics by:
  1. connecting data interpretation and confidence level with statistical testing and
  2. connecting statistical analyses of data with support or rejection of hypotheses.

Objectives: Students will be able to:

  • explain how sample size affects the confidence one can have in experimental results
  • state the meaning of a probability value and its role in statistical analysis
  • use probability values to determine rejection or support of scientific hypotheses

Sample Size and Confidence

In your groups, think about the following scenario and then answer the questions which follow :

Suppose you are the head of a drug-testing team.  You have obtained positive results from in-vitro experimental tests on human cells and are now ready to test the drug on a sample of humans.  You have a pool of 10,000 people on whom to test the drug. 
  • What sample size of people will you use - 10, 100, or 1000?  You may assume that one individual by chance alone responds unpredictably to the drug.
  • Explain your answer in terms of your confidence that the drug effects on all individuals measured are truly representative of the drug.
  • Why would you not use a smaller or larger sample size?

Probability Values and Confidence

Based on discussion of your responses to the questions above, it is expected that you realize that sample size is important in determining the impact of chance effects on experimental outcomes.  Confidence in an experimental outcome is usually much improved as you increase your sample size.  Consideration of sample size alone does not, however, tell us the probability that differences among experimental groups are real and not just due to chance.

Scientists use probability values (i.e., P-values) that are associated with specific statistical tests to decide whether differences in outcomes among experimental groups are real.  Here you will apply your understanding of P-values and statistical outcomes using data from a published scientific article. As you work on the problems below, use sections 8.1 and 8.2 in your Course Manual as a reference.

Problem

The ecological impact of endocrine disrupting chemicals (EDCs) is universally present, resulting from industrial chemicals, pesticides, and surfactants that infiltrate air, soil, food, and water. Although controversy exists regarding the linkage of endocrine disruptors to negative impacts on human health, it is evident that these compounds are present on a global scale with high levels occurring in the blood or body fats of humans and wildlife, even in remote areas of the arctic (Polischuk et al. 1995; Soto et al. 1995).

Exposure to EDCs has been listed as one of the causes of the decline in amphibians over the past decade.  Reeder et al. (2005) studied museum specimens of the cricket frog (Acris crepitans), looking for relationships between EDC use, population declines, and the occurrence of intersexuality (i.e., testes contain sperm and oocytes).  Specimens were examined from three regions of Illinois , differing in human population density and land use, for five time periods which differed in relation to use of EDCs.

Study Figure 4 below from Reeder et al.'s published article.  Consider the question:

  • Do EDCs have primarily a feminizing or masculinizing effect on cricket frogs?

How confident are you in your answer? Can you reliably predict the effect of EDCs during a new time period or in a new geographic region? How do scientists determine a level of confidence for a particular outcome or prediction?

Depending on whether you have been assigned to Group 1 or Group 2, focus on the data which are presented in Figure A or B and answer the set of questions assigned to your group.

figure a

Percentages of cricket frog specimens by (A) gonadal sex and time period and by (B) gonadal sex and geographic region. (Data reprinted with permission of author)

Group 1 (Figure A)

  1. Explain what the authors mean by: "The proportion of specimens in each gonadal sex class differed significantly among the time periods of collection (P < 0.001)".
  2. Based on the statement in question #1, would you accept or reject the null hypothesis that there were no differences in gonadal sex among the time periods of collection?  Justify your answer in terms of the probability that the results occurred by chance alone.
  3. What level of confidence do you have in your acceptance or rejection of the null hypothesis?
Group 2 (Figure B)
  1. Explain what the authors mean by "The proportion of specimens in each gonadal sex class differed significantly among  the geographical regions (P < 0.001)".
  2. Based on the statement in question #1, would you accept or reject the null hypothesis that there were no differences in gonadal sex among the geographical regions?  Justify your answer in terms of the probability that the results occurred by chance alone.
  3. What level of confidence do you have in your acceptance or rejection of the null hypothesis?
  4. Provide an ecological explanation for the statement, "In the 1990s, very few museum specimens of cricket frogs were available from regions that previously had the most elevated intersex rates."

After all groups have discussed the questions above, return to the question orginally asked: Do EDCs have primarily a feminizing or masculinizing effect on cricket frogs? Can you provide an answer in which you are confident?

References Cited
Polischuk SC, Letcher RJ, Norstrom RJ, Ramsay MA.  1995.  Preliminary result of fasting on the kinetics of organochlorines in polar bears (Ursus maritimus).  Sci Total Environ. 1995 Jan 15;160-161:465-72.

Soto AM, Sonnenschein C, Chung KL, Fernandez MF, Olea N, Serrano FO  1995.  The E-SCREEN assay as a tool to identify estrogens: an update on the estrogenic environmental pollutants.  Environ. Health Perspect. 103:113-122.

Reeder et al.  2005.  Intersexuality and the cricket frog decline: Historic and geographic trends.  Environ. Health Perspect. 113:261-265.