The goal of scientific research is to increase our understanding of the world around us. To do this, researchers study different groups of people or populations. These populations can be as small as a few individuals from one workplace or as large as thousands of people representing a cross-section of Canadian society. The results of this research often provide insights into how work and health interact in those groups. But how do we know if a study's results can be applied to another group or population?
To answer this question, we first need to understand the concept of generalizability.
In its simplest form, generalizability can be described as making predictions based on past observations.
In other words, if something has often happened in the past, it will likely occur in the future. In studies, once researchers have collected enough data to support a hypothesis, they can develop a premise to predict the outcome in similar circumstances with a certain degree of accuracy.
Two aspects of generalizability
Generalizing to a population. Sometimes when scientists talk about generalizability, they are applying results from a study sample to the larger population from which the sample was selected. For instance, consider the question, “What percentage of the Canadian population supports the Liberal party?” In this case, it would be important for researchers to survey people who represent the population at large. Therefore they must ensure that the survey respondents include relevant groups from the larger population in the correct proportions. Examples of relevant groups could be based on race, gender or age group.
Generalizing to a theory. More broadly, the concept of generalizability deals with moving from observations to scientific theories or hypotheses. This type of generalization amounts to taking time- and place-specific observations to create a universal hypothesis or theory. For instance, in the 1940s and 1950s, British researchers Richard Doll and Bradford Hill found that 647 out of 649 lung cancer patients in London hospitals were smokers. This led to many more research studies, with increasing sample sizes, with differing groups of people, with differing amounts of smoking and so on. When the results were found to be consistent across person, time and place, the observations were generalized into a theory: “cigarette smoking causes lung cancer.”
Requirements for generalizability
For generalizability we require a study sample that represents some population of interest — but we also need to understand the contexts in which the studies are done and how those might influence the results.
Suppose you read an article about a Swedish study of a new exercise program for male workers with back pain. The study was performed on male workers from fitness centres. Researchers compared two approaches. Half of the participants got a pamphlet on exercise from their therapist, and half were put on an exercise program led by a former Olympic athlete. The study findings showed that workers in the exercise group returned to work more quickly than workers who received the pamphlet.
Assuming the study was well conducted, with a strong design and rigorous reporting, we can trust the results. But to what populations could you generalize these results?
Some factors that need to be considered include: How important is it to have an Olympian delivering the exercise program? Would the exercise program work if delivered by an unknown therapist? Would the program work if delivered by the same Olympian but in a country where he or she is not well-known? Would the results apply to employees of other workplaces that differ from fitness centres? Would women respond the same way to the exercise program?
To increase our confidence in the generalizability of the study, it would have to be repeated with the same exercise program but with different providers in different settings (either worksites or countries) and yield the same results.
Source: At Work, Issue 45, Summer 2006: Institute for Work & Health, Toronto
{"appState":{"pageLoadApiCallsStatus":true},"articleState":{"article":{"headers":{"creationTime":"2016-03-26T22:55:36+00:00","modifiedTime":"2016-03-26T22:55:36+00:00","timestamp":"2022-09-14T18:12:02+00:00"},"data":{"breadcrumbs":[{"name":"Academics & The Arts","_links":{"self":"//dummies-api.dummies.com/v2/categories/33662"},"slug":"academics-the-arts","categoryId":33662},{"name":"Math","_links":{"self":"//dummies-api.dummies.com/v2/categories/33720"},"slug":"math","categoryId":33720},{"name":"Statistics","_links":{"self":"//dummies-api.dummies.com/v2/categories/33728"},"slug":"statistics","categoryId":33728}],"title":"Generalizing Statistical Results to the Entire Population","strippedTitle":"generalizing statistical results to the entire population","slug":"generalizing-statistical-results-to-the-entire-population","canonicalUrl":"","seo":{"metaDescription":"Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is cal","noIndex":0,"noFollow":0},"content":"<p>Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called <i>generalization,</i> and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. </p>\n<p>For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, \"Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!\"</p>\n<p>Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about <i>all</i> Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America.</p>\n<p>If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else.</p>\n<p class=\"Tip\">To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.</p>","description":"<p>Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called <i>generalization,</i> and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. </p>\n<p>For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, \"Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!\"</p>\n<p>Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about <i>all</i> Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America.</p>\n<p>If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else.</p>\n<p class=\"Tip\">To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.</p>","blurb":"","authors":[],"primaryCategoryTaxonomy":{"categoryId":33728,"title":"Statistics","slug":"statistics","_links":{"self":"//dummies-api.dummies.com/v2/categories/33728"}},"secondaryCategoryTaxonomy":{"categoryId":0,"title":null,"slug":null,"_links":null},"tertiaryCategoryTaxonomy":{"categoryId":0,"title":null,"slug":null,"_links":null},"trendingArticles":null,"inThisArticle":[],"relatedArticles":{"fromBook":[],"fromCategory":[{"articleId":263501,"title":"10 Steps to a Better Math Grade with Statistics","slug":"10-steps-to-a-better-math-grade-with-statistics","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263501"}},{"articleId":263495,"title":"Statistics and Histograms","slug":"statistics-and-histograms","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263495"}},{"articleId":263492,"title":"What is Categorical Data and How is It Summarized?","slug":"what-is-categorical-data-and-how-is-it-summarized","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/263492"}},{"articleId":209320,"title":"Statistics II For Dummies Cheat Sheet","slug":"statistics-ii-for-dummies-cheat-sheet","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/209320"}},{"articleId":209293,"title":"SPSS For Dummies Cheat Sheet","slug":"spss-for-dummies-cheat-sheet","categoryList":["academics-the-arts","math","statistics"],"_links":{"self":"//dummies-api.dummies.com/v2/articles/209293"}}]},"hasRelatedBookFromSearch":true,"relatedBook":{"bookId":282603,"slug":"statistics-for-dummies-2nd-edition","isbn":"9781119293521","categoryList":["academics-the-arts","math","statistics"],"amazon":{"default":"//www.amazon.com/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","ca":"//www.amazon.ca/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","indigo_ca":"//www.tkqlhce.com/click-9208661-13710633?url=//www.chapters.indigo.ca/en-ca/books/product/1119293529-item.html&cjsku=978111945484","gb":"//www.amazon.co.uk/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20","de":"//www.amazon.de/gp/product/1119293529/ref=as_li_tl?ie=UTF8&tag=wiley01-20"},"image":{"src":"//catalogimages.wiley.com/images/db/jimages/9781119293521.jpg","width":250,"height":350},"title":"Statistics For Dummies","testBankPinActivationLink":"","bookOutOfPrint":true,"authorsInfo":"\n <p><p><b>Deborah Rumsey, PhD, </b>is a Professor of Statistics and Statistics Education Specialist at The Ohio State University. She is the author of <i>Statistics For Dummies, Statistics Workbook For Dummies, Statistics II For Dummies, </i>and<i> Probability For Dummies</i>.</p>","authors":[{"authorId":9121,"name":"Deborah J. Rumsey","slug":"deborah-j-rumsey","description":" <p><b>Deborah Rumsey, PhD, </b>is a Professor of Statistics and Statistics Education Specialist at The Ohio State University. She is the author of <i>Statistics For Dummies, Statistics Workbook For Dummies, Statistics II For Dummies, </i>and<i> Probability For Dummies</i>. ","hasArticle":false,"_links":{"self":"//dummies-api.dummies.com/v2/authors/9121"}}],"_links":{"self":"//dummies-api.dummies.com/v2/books/282603"}},"collections":[],"articleAds":{"footerAd":"<div class=\"du-ad-region row\" id=\"article_page_adhesion_ad\"><div class=\"du-ad-unit col-md-12\" data-slot-id=\"article_page_adhesion_ad\" data-refreshed=\"false\" \r\n data-target = \"[{"key":"cat","values":["academics-the-arts","math","statistics"]},{"key":"isbn","values":[null]}]\" id=\"du-slot-6322197242359\"></div></div>","rightAd":"<div class=\"du-ad-region row\" id=\"article_page_right_ad\"><div class=\"du-ad-unit col-md-12\" data-slot-id=\"article_page_right_ad\" data-refreshed=\"false\" \r\n data-target = \"[{"key":"cat","values":["academics-the-arts","math","statistics"]},{"key":"isbn","values":[null]}]\" id=\"du-slot-6322197242c15\"></div></div>"},"articleType":{"articleType":"Articles","articleList":null,"content":null,"videoInfo":{"videoId":null,"name":null,"accountId":null,"playerId":null,"thumbnailUrl":null,"description":null,"uploadDate":null}},"sponsorship":{"sponsorshipPage":false,"backgroundImage":{"src":null,"width":0,"height":0},"brandingLine":"","brandingLink":"","brandingLogo":{"src":null,"width":0,"height":0},"sponsorAd":"","sponsorEbookTitle":"","sponsorEbookLink":"","sponsorEbookImage":{"src":null,"width":0,"height":0}},"primaryLearningPath":"Advance","lifeExpectancy":null,"lifeExpectancySetFrom":null,"dummiesForKids":"no","sponsoredContent":"no","adInfo":"","adPairKey":[]},"status":"publish","visibility":"public","articleId":201267},"articleLoadedStatus":"success"},"listState":{"list":{},"objectTitle":"","status":"initial","pageType":null,"objectId":null,"page":1,"sortField":"time","sortOrder":1,"categoriesIds":[],"articleTypes":[],"filterData":{},"filterDataLoadedStatus":"initial","pageSize":10},"adsState":{"pageScripts":{"headers":{"timestamp":"2022-11-03T10:50:01+00:00"},"adsId":0,"data":{"scripts":[{"pages":["all"],"location":"header","script":"<!--Optimizely Script-->\r\n<script src=\"//cdn.optimizely.com/js/10563184655.js\"></script>","enabled":false},{"pages":["all"],"location":"header","script":"<!-- comScore Tag -->\r\n<script>var _comscore = _comscore || [];_comscore.push({ c1: \"2\", c2: \"15097263\" });(function() {var s = document.createElement(\"script\"), el = document.getElementsByTagName(\"script\")[0]; s.async = true;s.src = (document.location.protocol == \"\" ? \"//sb\" : \"//b\") + \".scorecardresearch.com/beacon.js\";el.parentNode.insertBefore(s, el);})();</script><noscript><img src=\"//sb.scorecardresearch.com/p?c1=2&c2=15097263&cv=2.0&cj=1\" /></noscript>\r\n<!-- / comScore Tag -->","enabled":true},{"pages":["all"],"location":"footer","script":"<!--BEGIN QUALTRICS WEBSITE FEEDBACK SNIPPET-->\r\n<script type='text/javascript'>\r\n(function(){var g=function(e,h,f,g){\r\nthis.get=function(a){for(var a=a+\"=\",c=document.cookie.split(\";\"),b=0,e=c.length;b<e;b++){for(var d=c[b];\" \"==d.charAt(0);)d=d.substring(1,d.length);if(0==d.indexOf(a))return d.substring(a.length,d.length)}return null};\r\nthis.set=function(a,c){var b=\"\",b=new Date;b.setTime(b.getTime()+6048E5);b=\"; expires=\"+b.toGMTString();document.cookie=a+\"=\"+c+b+\"; path=/; \"};\r\nthis.check=function(){var a=this.get(f);if(a)a=a.split(\":\");else if(100!=e)\"v\"==h&&(e=Math.random()>=e/100?0:100),a=[h,e,0],this.set(f,a.join(\":\"));else return!0;var c=a[1];if(100==c)return!0;switch(a[0]){case \"v\":return!1;case \"r\":return c=a[2]%Math.floor(100/c),a[2]++,this.set(f,a.join(\":\")),!c}return!0};\r\nthis.go=function(){if(this.check()){var a=document.createElement(\"script\");a.type=\"text/javascript\";a.src=g;document.body&&document.body.appendChild(a)}};\r\nthis.start=function(){var t=this;\"complete\"!==document.readyState?window.addEventListener?window.addEventListener(\"load\",function(){t.go()},!1):window.attachEvent&&window.attachEvent(\"onload\",function(){t.go()}):t.go()};};\r\ntry{(new g(100,\"r\",\"QSI_S_ZN_5o5yqpvMVjgDOuN\",\"//zn5o5yqpvmvjgdoun-wiley.siteintercept.qualtrics.com/SIE/?Q_ZID=ZN_5o5yqpvMVjgDOuN\")).start()}catch(i){}})();\r\n</script><div id='ZN_5o5yqpvMVjgDOuN'><!--DO NOT REMOVE-CONTENTS PLACED HERE--></div>\r\n<!--END WEBSITE FEEDBACK SNIPPET-->","enabled":false},{"pages":["all"],"location":"header","script":"<!-- Hotjar Tracking Code for //www.dummies.com -->\r\n<script>\r\n (function(h,o,t,j,a,r){\r\n h.hj=h.hj||function(){(h.hj.q=h.hj.q||[]).push(arguments)};\r\n h._hjSettings={hjid:257151,hjsv:6};\r\n a=o.getElementsByTagName('head')[0];\r\n r=o.createElement('script');r.async=1;\r\n r.src=t+h._hjSettings.hjid+j+h._hjSettings.hjsv;\r\n a.appendChild(r);\r\n })(window,document,'//static.hotjar.com/c/hotjar-','.js?sv=');\r\n</script>","enabled":false},{"pages":["article"],"location":"header","script":"<!-- //Connect Container: dummies --> <script src=\"//get.s-onetag.com/bffe21a1-6bb8-4928-9449-7beadb468dae/tag.min.js\" async defer></script>","enabled":true},{"pages":["homepage"],"location":"header","script":"<meta name=\"facebook-domain-verification\" content=\"irk8y0irxf718trg3uwwuexg6xpva0\" />","enabled":true},{"pages":["homepage","article","category","search"],"location":"footer","script":"<!-- Facebook Pixel Code -->\r\n<noscript>\r\n<img height=\"1\" width=\"1\" src=\"//www.facebook.com/tr?id=256338321977984&ev=PageView&noscript=1\"/>\r\n</noscript>\r\n<!-- End Facebook Pixel Code -->","enabled":true}]}},"pageScriptsLoadedStatus":"success"},"navigationState":{"navigationCollections":[{"collectionId":287568,"title":"BYOB (Be Your Own Boss)","hasSubCategories":false,"url":"/collection/for-the-entry-level-entrepreneur-287568"},{"collectionId":293237,"title":"Be a Rad Dad","hasSubCategories":false,"url":"/collection/be-the-best-dad-293237"},{"collectionId":294090,"title":"Contemplating the Cosmos","hasSubCategories":false,"url":"/collection/theres-something-about-space-294090"},{"collectionId":287563,"title":"For Those Seeking Peace of Mind","hasSubCategories":false,"url":"/collection/for-those-seeking-peace-of-mind-287563"},{"collectionId":287570,"title":"For the Aspiring Aficionado","hasSubCategories":false,"url":"/collection/for-the-bougielicious-287570"},{"collectionId":291903,"title":"For the Budding Cannabis Enthusiast","hasSubCategories":false,"url":"/collection/for-the-budding-cannabis-enthusiast-291903"},{"collectionId":291934,"title":"For the Exam-Season Crammer","hasSubCategories":false,"url":"/collection/for-the-exam-season-crammer-291934"},{"collectionId":287569,"title":"For the Hopeless Romantic","hasSubCategories":false,"url":"/collection/for-the-hopeless-romantic-287569"},{"collectionId":287567,"title":"For the Unabashed Hippie","hasSubCategories":false,"url":"/collection/for-the-unabashed-hippie-287567"},{"collectionId":295430,"title":"Have a Beautiful (and Tasty) Thanksgiving","hasSubCategories":false,"url":"/collection/have-a-wonderful-thanksgiving-295430"}],"navigationCollectionsLoadedStatus":"success","navigationCategories":{"books":{"0":{"data":[{"categoryId":33512,"title":"Technology","hasSubCategories":true,"url":"/category/books/technology-33512"},{"categoryId":33662,"title":"Academics & The Arts","hasSubCategories":true,"url":"/category/books/academics-the-arts-33662"},{"categoryId":33809,"title":"Home, Auto, & Hobbies","hasSubCategories":true,"url":"/category/books/home-auto-hobbies-33809"},{"categoryId":34038,"title":"Body, Mind, & Spirit","hasSubCategories":true,"url":"/category/books/body-mind-spirit-34038"},{"categoryId":34224,"title":"Business, Careers, & Money","hasSubCategories":true,"url":"/category/books/business-careers-money-34224"}],"breadcrumbs":[],"categoryTitle":"Level 0 Category","mainCategoryUrl":"/category/books/level-0-category-0"}},"articles":{"0":{"data":[{"categoryId":33512,"title":"Technology","hasSubCategories":true,"url":"/category/articles/technology-33512"},{"categoryId":33662,"title":"Academics & The Arts","hasSubCategories":true,"url":"/category/articles/academics-the-arts-33662"},{"categoryId":33809,"title":"Home, Auto, & Hobbies","hasSubCategories":true,"url":"/category/articles/home-auto-hobbies-33809"},{"categoryId":34038,"title":"Body, Mind, & Spirit","hasSubCategories":true,"url":"/category/articles/body-mind-spirit-34038"},{"categoryId":34224,"title":"Business, Careers, & Money","hasSubCategories":true,"url":"/category/articles/business-careers-money-34224"}],"breadcrumbs":[],"categoryTitle":"Level 0 Category","mainCategoryUrl":"/category/articles/level-0-category-0"}}},"navigationCategoriesLoadedStatus":"success"},"searchState":{"searchList":[],"searchStatus":"initial","relatedArticlesList":[],"relatedArticlesStatus":"initial"},"routeState":{"name":"Article3","path":"/article/academics-the-arts/math/statistics/generalizing-statistical-results-to-the-entire-population-201267/","hash":"","query":{},"params":{"category1":"academics-the-arts","category2":"math","category3":"statistics","article":"generalizing-statistical-results-to-the-entire-population-201267"},"fullPath":"/article/academics-the-arts/math/statistics/generalizing-statistical-results-to-the-entire-population-201267/","meta":{"routeType":"article","breadcrumbInfo":{"suffix":"Articles","baseRoute":"/category/articles"},"prerenderWithAsyncData":true},"from":{"name":null,"path":"/","hash":"","query":{},"params":{},"fullPath":"/","meta":{}}},"dropsState":{"submitEmailResponse":false,"status":"initial"},"sfmcState":{"status":"initial"},"profileState":{"auth":{},"userOptions":{},"status":"success"}}
Making conclusions about a much broader population than your sample actually represents is one of the biggest no-no's in statistics. This kind of problem is called generalization, and it occurs more often than you might think. People want their results instantly; they don't want to wait for them, so well-planned surveys and experiments take a back seat to instant Web surveys and convenience samples. For example, a researcher wants to know how cable news channels have influenced the way Americans get their news. He also happens to be a statistics professor at a large research institution and has 1,000 students in his classes. He decides that instead of taking a random sample of Americans, which would be difficult, time-consuming, and expensive, he will just put a question on his final exam to get his students' answers. His data analysis shows that only 5 percent of his students read the newspaper and/or watch network news programs anymore; the rest watch cable news. For his class, the ratio of students who exclusively watch cable news compared to those students who don't is 20 to 1. The professor reports this and sends out a press release about it. The cable news channels pick up on it and the next day are reporting, "Americans choose cable news channels over newspapers and network news by a 20-to-1 margin!" Do you see what's wrong with this picture? The problem is that the professor's conclusions go way beyond his study, which is wrong. He used the students in his statistics class to obtain the data that serves as the basis for his entire report and the resulting headline. Yet the professor reports the results about all Americans. It's safe to say that a sample of 1,000 college students taking a statistics class at the same time at the same college doesn't represent a cross section of America. If the professor wants to make conclusions in the end about America, he has to select a random sample of Americans to take his survey. If he uses 1,000 students from his class, then his conclusions can be made only about that class and no one else. To avoid or detect generalization, identify the population that you're intending to make conclusions about and make sure the selected sample represents that population. If the sample represents a smaller group within that population, then the conclusions have to be downsized in scope also.