tag:blogger.com,1999:blog-3643556495084083743.post8125997053098481880..comments2022-12-09T08:52:54.666-08:00Comments on Skeptic Wonder: A stats question RE T-tests and U-testsPsi Wavefunctionhttp://www.blogger.com/profile/10829712736757471647noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-3643556495084083743.post-87598511922675306132010-02-11T01:58:11.927-08:002010-02-11T01:58:11.927-08:00Others have said it already, but just to put it an...Others have said it already, but just to put it another way: The thing of interest is <b>effect size</b> (e.g. difference in means), not p value. In your examples the effect size is very small, and that's what you should consider. The p value is perhaps useful (or perhaps not) if you have very small sample sizes - then you may want to check that the difference is unlikely to be due to random sampling effects.<br /><br />For large samples you practically always get a significant difference (shrug).<br /><br />And by the way, Past is <b>much</b> hotter than R.Oyvind Hammerhttp://folk.uio.no/ohammer/noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-81946731668443169362010-02-09T23:43:45.535-08:002010-02-09T23:43:45.535-08:00Thanks a lot for your input, guys, I really apprec...Thanks a lot for your input, guys, I really appreciate it!<br /><br />I'll ponder over this a bit later, when I'm less swamped with course-related stuff...(three exams this week o_O) I'd take data analysis issues over exams/classes any day of the week...Psi Wavefunctionhttps://www.blogger.com/profile/10829712736757471647noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-65377375397584667432010-02-07T23:07:12.357-08:002010-02-07T23:07:12.357-08:00If you have very large samples, even very small di...If you have very large samples, even very small differences can become statistically significant. But that doesn't mean that the differences are biologically significant.AYDIN ÖRSTANhttps://www.blogger.com/profile/09891160904748206385noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-54070430615285379472010-02-07T16:58:16.085-08:002010-02-07T16:58:16.085-08:00Hi Psi,
A couple of thoughts, but since I've...Hi Psi, <br /><br />A couple of thoughts, but since I've only learnt stats 'on the fly' as I've needed it you should take it with a couple of grains of salt. <br /><br />1. I don't know if there is any biological or methodological reason for your treatments/cells to differ but I do know if there is a difference then your large sample sizes will almost certainly find it. You might want to think in terms of 'effect size' as well as p-value - in the case the difference is greater than you'd expect by chance but the difference in mean size falls between 0.1 and 1mm - does this matter? How does this compare with the effect size between treated an control lines?<br /><br />2) If you are testing different combinations of 10 replicates of an experiment then you will expect to see some 'signficant' results just by chance (I think it was John Maynard Smith who said "Statistics was invented by biologists so they could do twenty experiments a year and publish one false result in Nature."). You might consider a correction such as Bonferroni's to deal with the problem of multiple testing.<br /><br />3) R is awesome. And it does have a GUI (R Commander) that has T-tests and ANOVA and all that good stuff just a couple of mouse clicks away. You should definitely consider it<br /><br />As I said, just a few thoughts, sorry I can't provide The Answer.David Winterhttps://www.blogger.com/profile/09704684760112027351noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-31322002972390845932010-02-07T05:30:39.348-08:002010-02-07T05:30:39.348-08:00Sorry for not seeing the sample sizes! I've be...Sorry for not seeing the sample sizes! I've been thinking for the past half hour about your problem, and since you mentioned that the variances appear to be different, according to your T-test, you maybe should try out your luck with adapted t-tests for samples with unequal variance or unequal sample sizes (or both!).<br />See for example: http://en.wikipedia.org/wiki/Student's_t-test#Unequal_sample_sizes.2C_unequal_variance or http://en.wikipedia.org/wiki/Welch's_t_test .<br /><br />By doing a test corrected for unequal variance, you would also make this guy happy ;): http://beheco.oxfordjournals.org/cgi/content/full/17/4/688<br /><br />I don't know if PAST or excel allow these tests. I'm a big fan of R, but I found that most biologists don't feel comfortable running scripts and prefer a visual interface.. It's really powerful though, and definitely allows you to perform the tests I described before (http://sekhon.berkeley.edu/stats/html/t.test.html).<br /><br />Statistics can be annoyingly complex.. Don't lose hope just yet though!Lucas Brouwershttps://www.blogger.com/profile/15192035237302508309noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-55958254137256962562010-02-07T02:29:10.019-08:002010-02-07T02:29:10.019-08:00I did specify sample sizes at all times. I thought...I did specify sample sizes at all times. I thought that larger sample sizes --> lower chance of false positives, just because any accidental differences would be corrected for more and more; ie larger sample sizes should represent the 'true' curve more closely, and in this case the 'true' curves should be the same!<br /><br />PAST also yields the same results! But thanks, it looks like a really nice program! And unlike Excel, an actual STATS program! =D<br /><br />I can't understand how the 95% CI could be so small!Psi Wavefunctionhttps://www.blogger.com/profile/10829712736757471647noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-66502586608397097022010-02-06T02:30:39.380-08:002010-02-06T02:30:39.380-08:00It is really important to state the sample sizes (...It is really important to state the sample sizes (as you say they are huge)! Tests like this have a tendency to quickly become incredibly significant if the numbers are high enough..Lucas Brouwershttps://www.blogger.com/profile/15192035237302508309noreply@blogger.comtag:blogger.com,1999:blog-3643556495084083743.post-7728056553289674782010-02-05T20:13:50.167-08:002010-02-05T20:13:50.167-08:00I'd repeat the t-test with another program. Th...I'd repeat the t-test with another program. There are t-test calculators on the web. Or else, use PAST; it's easy & free: http://folk.uio.no/ohammer/past/AYDIN ÖRSTANhttps://www.blogger.com/profile/09891160904748206385noreply@blogger.com