Introduction

Overview

The International Adult Literacy Survey was the first comparative assessment of adult literacy skills ever undertaken internationally. Over 75,000 adults from 22 countries were interviewed and tested in their homes in 15 languages between 1994 and 1998. The purpose of the study was to improve understanding of the nature and magnitude of the literacy issues faced by nations and to investigate the factors that influence the development of adult literacy skills in various settings — at home, at work and across countries.

In 1994, nine countries — Canada (English and French-speaking populations), France,¹ Germany, Ireland, the Netherlands, Poland, Sweden, Switzerland (German and French-speaking regions) and the United States — fielded the world's first large-scale, comparative assessment of adult literacy. Data for seven of these countries were published in December 1995. Five additional countries or territories — Australia, the Flemish community in Belgium, Great Britain, New Zealand and Northern Ireland — administered the IALS instruments in 1996 and published results in November 1997. Finally, nine other countries or regions — Chile, the Czech Republic, Denmark, Finland, Hungary, Italy, Norway, Portugal, Slovenia and the Italian-speaking region of Switzerland — participated in a new wave of collection in 1998. Results for these latter countries became first available in June 2000; findings for most of them are included in this monograph.

Definition of Literacy

In IALS proficiency levels along a continuum denote how well adults use information to function in society. Thus, literacy is defined as the ability to understand and employ printed information in daily activities, at home, at work and in the community — to achieve one's goals, and to develop one's knowledge and potential. In denoting a broad set of information — processing competencies, this conceptual approach points to the multiplicity of skills that constitute literacy in advanced industrialized countries.

The conceptual framework and the definitions of the literacy domains used for the assessment built on the seminal work of Irwin Kirsch and Peter Mosenthal (see Annex C). In particular, the IALS assessment was based on the theoretical and methodological insights offered by four large-scale North-American surveys: The Functional Reading Study conducted in the U.S. by the Educational Testing Service (ETS) in the early 1970s; the Young Adult Literacy Survey fielded in the U.S. by ETS in 1985; the Survey of Literacy Skills Used in Daily Activities undertaken by Statistics Canada in 1989; and the National Adult Literacy Survey conducted in the United States by ETS between 1989 and 1992². Literacy is measured operationally in terms of three domains, each encompassing a common set of skills relevant for diverse tasks:

• Prose literacy — the knowledge and skills needed to understand and use information from texts including editorials, news stories, poems and fiction.
• Document literacy — the knowledge and skills required to locate and use information contained in various formats, including job applications, payroll forms, transportation schedules, maps, tables and charts.
• Quantitative literacy — the knowledge and skills required to apply arithmetic operations, either alone or sequentially, to numbers embedded in printed materials, such as balancing an account, figuring out a tip, completing an order form or determining the amount of interest on a loan from an advertisement.

The IALS employed a sophisticated methodology developed and applied by the Educational Testing Service to measure literacy proficiency for each domain on a scale ranging from 0-500 points. Literacy ability in each domain is expressed by a score, defined as the point at which a person has an 80 percent chance of successful performance from among the set of tasks of varying difficulty included in the assessment³. Five levels of literacy correspond to measured ranges of scores achieved.Level1 indicates persons with very poor skills, where the individual may, for example, be unable to determine the correct amount of medicine to give a child from information printed on the package. Level 2 respondents can deal only with material that is simple, clearly laid out, and in which the tasks involved are not too complex. It denotes a weak level of skill, but more hidden than Level 1. It identifies people who can read, but test poorly. They may have developed coping skills to manage everyday literacy demands, but their low level of proficiency makes it difficult for them to face novel demands, such as learning new job skills. Performance at Levels 3, 4 and 5 requires the ability to integrate several sources of information and solve more complex problems.

Study Design

The data presented in this report were collected by the countries participating in successive cycles of data collection between 1994 and 1998, using nationally representative samples of the adult population aged 16-65. The fact that some countries collected data a few years earlier or later than others is thought not to affect the international comparability of the survey data because the literacy profiles of nations are quite stable and are normally expected to change only slowly with the passing of time⁴.

The survey was conducted in people's homes by experienced interviewers. The design used for IALS combined educational assessment techniques with methods of household survey research. Multiple quality control measures were implemented throughout the course of the study in order to ensure that high-quality data would be obtained. Annex B describes the measures taken to improve data quality and comparability and addresses specific issues concerning validity, reliability and comparability.

In brief, respondents were first asked a series of questions to obtain background and demographic information. Once this background questionnaire was completed, the interviewer presented a booklet containing six simple tasks. If a respondent failed to complete at least two of these correctly, the interview was adjourned. Respondents who completed two or more tasks correctly were then given a much larger variety of tasks, printed in a separate booklet. The assessment was not timed, and respondents were urged to try each exercise. Respondents were thus given maximum opportunity to demonstrate their skills.

Survey and Research Team

IALS was a large-scale co-operative effort by governments, national statistical agencies, research institutions and the Organisation for Economic Co-operation and Development (OECD). Overall responsibility for the study was shared between Mr. T. Scott Murray and Ms. Nancy Darcovich of Statistics Canada and Mr. Albert Tuijnman, formerly of the OECD. The development and implementation of the survey were co-ordinated by Statistics Canada and the Educational Testing Service (ETS) of Princeton, New Jersey. Mr. Irwin Kirsch and Mr. Kentaro Yamamoto were ETS Project co-Directors. Ms. Marilyn Binkley of the National Center for Education Statistics (NCES) was the National Study Director for the U.S. component of the study. Mr. Stan Jones, consultant to Statistics Canada, acted as International Project Advisor.

Data collection constituted the largest cost to the countries that participated in the IALS program of work. Most paid the full cost of data collection⁵ and adhered to the international data collection guidelines specified by Statistics Canada and ETS. The costs of the international co-ordination, data analysis and reporting for the first survey cycle were covered principally by the Canadian Government and NCES. In further cycles the participating countries were required to assist in offsetting some of the international overhead costs. Limited funding was also obtained from the European Union and the OECD. NCES and the Division of Adult Education and Literacy, Office of Vocational and Adult Education of the U.S. Department of Education funded the national study in the United States.

Organization of this Monograph

This monograph presents summary findings for 21 of the 22 nations that took part in the assessment. The results presented in the next chapter were computed at ETS and Statistics Canada by analysts using sophisticated but recognized procedures for scaling and the calculation of plausible values, population mean scores and standard errors. The results presented are consistent with those published previously in the IALS final report⁶.

Chapter 1 presents 10 benchmarks for assessing adult literacy in North America against the backdrop of results achieved by other nations, the majority of them economically advanced Member countries of the OECD. Chapter 2 proposes ten targets and tools that might be employed to improve literacy in America. Successful policies will require a broad and encompassing approach, targeting different audiences and addressing a range of policy domains related to life-long learning. Chapter 3, finally, presents some overall conclusions for policy.

---

¹ France withdrew its data in November 1995, after the comparative results had become available, citing concerns about comparability. The French results are therefore not included in this monograph. A new data collection was undertaken in France in 1998 as part of a European Union financed research study that applied the same methods and the same test instruments as were used in the original IALS. The results of this study are reported in Carey (2000), see Annex C.

² See Kirsch, I.S., Jungeblut, A., and Mosenthal, P.B. (1998), "The measurement of adult literacy", pp. 105-134 in Murray, T.S., Kirsch, I.S., and Jenkins, L.B. (Eds.), Adult literacy in OECD countries : Technical report on the first international adult literacy survey, United States Department of Education, National Center for Education Statistics, Washington, DC.

³ The RP-80 criterion is explained in Yamamoto, K. (1998), "Scaling and scale linking", pp. 161-178, in Murray, T.S., Kirsch, I.S., and Jenkins, L.B. (Eds.), op. cit.

⁴ The possibility that a nation succeeds in significantly altering its literacy profiles within the course of 4-5 years cannot be ruled out. Major educational reforms, for example, can influence the literacy profiles of specific sub-populations.

⁵ Chile and Poland received limited financial support from UNESCO and Slovenia did the same from the World Bank.

⁶ OECD and Statistics Canada (2000). Literacy in the Information Age: Final Report of the International Adult Literacy Survey, Paris and Ottawa.