Statistics deals with uncertainty. When we make a statement; we also need to say how certain we are that this statement is correct. We would like this to be 100%, but realise that this is not possible. <\/p>\n\n\n\n
The decision to reject a null hypothesis<\/a> is made on the basis of a statistical test<\/strong>. What statistical test is being used, depends on the distribution of the outcome variable<\/strong> and can be parametric<\/a> or non parametric<\/a>. A parametric test (ie t-test) can only be used if the data are Normally distributed<\/a>. If the data are not Normally distributed, parametric statistics can’t be used and we will have to use a non parametric test (ie Wilcoxon test).<\/p>\n\n\n\n A statistical test will calculate a test statistic<\/strong>. The test statistic is different for different tests. For example, a parametric<\/a> t-test calculates the t-value and a non parametric<\/a> Wilcoxon test calculates the W value. The p value is the probability the<\/strong> test statistic takes a more extreme value <\/strong>and is used as a threshold to make a decision whether the null hypothesis is true or not.<\/p>\n\n\n\n P Value = Probability the test statistic takes a more extreme value<\/strong><\/p>\n\n\n\n It is generally accepted (in medical statistics) that something is \u2018proven\u2019, or statistically significant, if the probability the test statistic takes <\/strong>a more extreme value is less than 5%. On the basis of this, the null hypothesis is accepted or rejected.<\/p>\n\n\n\n In medical statistics we are usually satisfied something is statistically significant if p less than 5%. If p less than 5%, we feel this is unlikely to be due to chance and the null hypothesis is rejected in favour of the alternate hypothesis.<\/em><\/p>\n\n\n\n Statistically significant<\/em>: p value < 5%.<\/p>\n\n\n\n Please bear in mind that the p value indicates how incompatible data are with a statistical model. P values do NOT<\/strong> measure the probability that the studied hypothesis is true and do NOT<\/strong> measure the size of an effect (the alternative hypothesis is NOT<\/strong> ‘more true’ if the p value is lower)1<\/a><\/sup>. Please see also in p values their use and abuse.<\/a><\/p>\n\n\n\n It is customary to round the p value to three decimal figures. However, when p is less than one in a thousand p < 0.001 is used.<\/p>\n\n\n\n That something is statistically significant doesn\u2019t necessarily mean it is also clinically significant<\/em><\/strong>. It might well be that, although statistically there is a difference, it is of no clinical importance.<\/p>\n\n\n\n Also, if we were unable to demonstrate a statistically significant difference; this doesn\u2019t mean there is no difference. It might well be that with more patients in our study, we can demonstrate a significant difference (underpowered study<\/a>, type 2 error<\/a>).<\/p>\n\n\n\n If p = 5%, there is a probability of 1 in 20 that we drew the wrong conclusion (incorrectly reject the null hypothesis, type 1 error<\/a>). However, it is generally a reasonable trade off in clinical studies.<\/p>\n\n\n\n P value correction<\/strong><\/p>\n\n\n\n In a large data set, there may be many variables (columns when the data are in tidy format<\/a>) with an opportunity to do multiple tests until something turns up that is ‘significant’. However, when performing multiple tests, it is important to bear in mind that, when p = 5%, one in twenty tests will be significant by chance. To address this, p values should be corrected to the number of tests that have been performed. A number of methods have been described that include Bonferroni and Holme’s methods and are included in R. <\/p>\n\n\n\n For example, when comparing two groups of patients, multiple comparison tests were performed on many variables that included the patient’s height. The height variable had a p value of 0.001 and was considered ‘significant’. However, this should be evaluated in view of the number of test performed. To correct the p value using Bonferroni’s method in R:<\/p>\n\n\n\n n <\/strong>is the number of tests that have been performed<\/p>\n\n\n\n It can be seen that, if 50 or more tests were being performed, this p value can no longer be regarded as ‘significant’.<\/p>\n\n\n\n Fishing for p values<\/strong><\/p>\n\n\n\n It is important to consider that data should be evaluated in clinical perspective and should make clinical sense. There is no place for ‘fishing for p values’ until one is ‘significant’. On its own a p value has little importance2<\/a><\/sup>. A p value is only part of the data evaluation and reporting should be with full transparency and further evidence to justify the conclusions.<\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":" Statistics deals with uncertainty. When we make a statement; we also need to say how certain we are that this statement is correct. We would like this to be 100%, but realise that this is not possible. The decision to reject a null hypothesis is made on the basis of a statistical test. What statistical […]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"inline_featured_image":false,"footnotes":""},"class_list":["post-947","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/pages\/947","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/comments?post=947"}],"version-history":[{"count":21,"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/pages\/947\/revisions"}],"predecessor-version":[{"id":4831,"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/pages\/947\/revisions\/4831"}],"wp:attachment":[{"href":"https:\/\/pcool.dyndns.org\/index.php\/wp-json\/wp\/v2\/media?parent=947"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}p = 0.001<\/mark> <\/mark># set the p value<\/mark>\n<\/mark>p.adjust(p, method='bonferroni', n=20)<\/mark>\n[1] 0.02<\/mark><\/em>\np.adjust(p, method='bonferroni', n=50)<\/mark>\n[1] 0.05<\/mark>\n<\/mark>p.adjust(p, method='bonferroni', n=100)<\/mark>\n[1] 0.1<\/mark><\/em><\/code><\/pre>\n\n\n\n