To compare means of two samples, we need to perform an independent samples t-test, more commonly called Student’s t-Test. To perform this test, we need to know some statistics about the samples: each sample’s mean (x̄), standard deviation (s), and sample size (n). Here is an example:
There is data contained in a table of two columns. Column 1 is called ‘category’ and Column 2 is called ‘length.’ The category column contains two options, a and b, while the length column contains lengths in R type num. We want to find out if the mean length of the a values are different than the mean length of the b values. The data looks something like this. The hypothesis statements look like this:
Ho (null hypothesis): x̄a = x̄b or x̄a - x̄b = 0
Ha (alternative hypothesis): x̄a ≠ x̄b or x̄a - x̄b ≠ 0
To assess the truth of the null hypothesis, we need to formulate a test statistic (t) and then using that test statistic, find the p-value. This is basic hypothesis testing. There are a few ways to determine the test statistic, depending on factors such as are the sample sizes of the two samples equal, as well as their variance. For this exercise, we will assume that the two sample sizes are not equal and neither are the variances. That means we will perform a Welch’s t-Test, a version of Student’s t-Test with a slight modification of how to calculate t.
In Welch’s t-Test, the t statistic is defined by the formula:
You can see different scenarios of how to calculate t in the Student’s t-Test wiki page. To make a long story short, you don’t need to know how to calculate t to run this test in R. R has a t.test function that takes in two vectors of samples and will tell you t and the p-value. In our example, we are performing a two-sided test (as opposed to one-sided) because we want to know if the means are different than each other – either less than or greater than. Since the variances of the two samples are not equal and we are performing a Welch’s t-Test, we need to tell the t.test function that as well. In our example, the data is called ‘sample_ab.’ The t-test function call will look like this:
t.test(sample_ab[sample_ab$category == 'a',"length"], sample_ab[sample_ab$category == 'b',"length"], alternative="two.sided", var.equal=FALSE)
And the output, depending on the data, will look like this:
data: sample_ab[sample_ab$category == "a", "length"] and sample_ab[sample_ab$category == "b", "length"]
t = -1.9411, df = 21.324, p-value = 0.06558
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
-915.90428 31.13761
sample estimates:
mean of x mean of y
66.41667 508.80000
This output shows us t, the p-value, and x̄a & x̄b (as mean of x and mean of y). The default confidence interval for the t.test function is 95%. Since the p-value of .066 > .05 (.05 is the alpha level when there is a 95% confidence level), then we cannot reject the null hypothesis and we cannot conclude at a 95% confidence level that the mean of a is different than the mean of b.
References:
http://eprints.uwe.ac.uk/27232/27/p030.pdf – paper I need to read about Welch’s t-Test
http://mathworld.wolfram.com/HypothesisTesting.html – WolfraMathWorld Hypothesis Testing
https://en.wikipedia.org/wiki/Student%27s_t-test, https://en.wikipedia.org/wiki/Welch%27s_t-test – Wiki
https://stat.ethz.ch/R-manual/R-devel/library/stats/html/t.test.html – R t-Test function