«The cost of not assessing learning outcomes UNESCO The constitution of the United Nations Educational, Scientific and Cultural Organization (UNESCO) ...»
INFORMATION PAPER NO. 26 JANUARY 2016
The cost of not assessing learning outcomes
The constitution of the United Nations Educational, Scientific and Cultural Organization (UNESCO) was adopted by
20 countries at the London Conference in November 1945 and entered into effect on 4 November 1946. The
Organization currently has 195 Member States and 10 Associate Members.
The main objective of UNESCO is to contribute to peace and security in the world by promoting collaboration among nations through education, science, culture and communication in order to foster universal respect for justice, the rule of law, and the human rights and fundamental freedoms that are affirmed for the peoples of the world, without distinction of race, sex, language or religion, by the Charter of the United Nations.
To fulfil its mandate, UNESCO performs five principal functions: 1) prospective studies on education, science, culture and communication for tomorrow's world; 2) the advancement, transfer and sharing of knowledge through research, training and teaching activities; 3) standard-setting actions for the preparation and adoption of internal instruments and statutory recommendations; 4) expertise through technical co-operation to Member States for their development policies and projects; and 5) the exchange of specialized information.
UNESCO is headquartered in Paris, France.
UNESCO Institute for Statistics The UNESCO Institute for Statistics (UIS) is the statistical office of UNESCO and is the UN depository for global statistics in the fields of education, science and technology, culture and communication.
The UIS was established in 1999. It was created to improve UNESCO's statistical programme and to develop and deliver the timely, accurate and policy-relevant statistics needed in today’s increasingly complex and rapidly changing social, political and economic environments.
The UIS is based in Montreal, Canada.
Published in 2016 by:
The designations employed and the presentation of material throughout this publication do not imply the expression of any opinion whatsoever on the part of UNESCO concerning the legal status of any country, territory, city or area or of its authorities or concerning the delimitatio
The global assessment landscape
How much might testing cost?
Cost per student and per graduate
How much does NOT testing cost?
List of figures Figure 1. Types of assessments
Figure 2. Intensity of assessment activity in the world: Findings from the Learning Assessment Capacity Index (LACI)
Figure 3. Distribution of country costs for a major international assessment
Figure 4. Cost assessments per year per student (ISCED 2-3) in terms of actual government expenditure per graduat (ISCED 1, 2, 3) (in US$PPP)
Figure 5. Achievement gains (percentage of GDP)
List of tables Table 1. International assessment costs (in US$)
Table 2a. Annual cost of expanding selected assessments (in US$)
Table 2b. Annual cost of expanding selected assessments (per US$100,000)
Table 3. Annual government expenditure per secondary student and cost assessment per student
Table 4. Expenditure per graduate (ISCED 1, 2 and 3) (in US$PPP), 2013 or latest year available
Table 5. Gains arising from assessment data (in millions US$PPP)
-3Introduction Sustainable Development Goal 1 (SDG) 4 calls for the international community to “ensure inclusive and equitable quality education and promote lifelong learning opportunities for all”.
Education is a fundamental human right. As a catalyst for development, it is a key contributor to reducing inequality and scaling down poverty. Full access to quality education at all levels is an essential condition for accelerating progress towards the achievement of other SDGs.
Recently, the international community has gone a step further in monitoring education by attempting to measure learning. Currently there is no single approach or best way to monitor learning internationally. However, the rationale to come up with a viable approach to assess learning on a universal basis is growing stronger as the Education for All (EFA) and Millennium Development Goals (MDGs) come to a close and the international community advances towards the SDGs.
The focus on the quality of education has led to an emphasis on the measurement of learning outcomes at all levels. Input data, such as knowing how many children are enrolled in school or how many teachers are hired, are not enough. There is a need to know if children are learning in schools and to measure learning outcomes on a global scale to monitor progress. Five of the seven education targets in SDG 4 focus on learning outcomes (i.e. the effect of education on individual children, young people and adults).
This is a shift from previous global education targets, such as those in the MDGs, which solely focused on ensuring access to, participation and completion in formal primary education and on gender parity in primary, secondary and tertiary education. The Education 2030 targets highlight that enrolment and participation (e.g. in early childhood development programmes, formal schooling or adult education programmes) are the means to attain results and learning outcomes at every stage.
This includes a range of topics, such as school readiness for young children; academic competencies for children in primary and secondary education; functional literacy and numeracy skills; and skills for work, global citizenship as well as sustainable development for youth and adults. Indicators for global monitoring must emphasise this renewed focus on outcome measures. While the Inter-Agency and Expert Group on Sustainable Development Goal 4 (IAEG-SDGs)2 is in the process of finalising the list of global monitoring indicators, it is clear today that a cross-national measure of reading and mathematics will be needed.
With the adoption of the SDGs and the Framework for Action for Education 2030, attention has turned to defining a plan and framework for monitoring progress towards the targets. This is the second part of the process to be tackled by the IAEG-SDGs, although UNESCO and partners have already started developing an approach to implement the agenda.
https://sustainabledevelopment.un.org/content/documents/7891TRANSFORMING%20OUR%20WORLD.pdf The IAEG-SDGs was established by the UN Statistical Commission to develop an indicator framework for all SDGs. More information on the November meeting is available at http://unstats.un.org/sdgs/.
-4No single organization can produce all of the data needed to monitor SDG 4 – which covers a wide range of issues from learning outcomes to global citizenship. Therefore, the first priority will be to develop the measurement framework for SDG 4, building on the progress made thus far to define indicators, assess data availability, coverage and evaluating the existing methodologies.
In addition and as agreed by the IAEG-SDG, reliable measures are needed at each level to generate data that are comparable across time and disaggregated by age, sex, disability, socioeconomic status, geographical location (urban/rural areas) and other relevant factors.3 Since five of the seven education targets in SDG 4 focus on learning outcomes of children, young people and adults, this clearly is an area of priority for the Global Alliance for Learning (GAL) which is faced with the challenge of putting in place a system to measure and monitor progress in achieving these targets. To date, the IAEG-SDGs has identified 11 global indicators.
One of the first challenges will be to address the development of metrics that could serve indicators related to learning outcomes, such as SDG Target 4.1: “Percentage of children/young people i) in Grade 2/3; ii) at the end of primary education; and iii) at the end of lower secondary education achieving at least a minimum proficiency level in (a) reading and (b) mathematics”.
The global assessment landscape
Assessing learning outcomes has never been as dynamic as today. A profusion of assessments exist at international, regional and national levels (see Figure 1), with research articles flourishing and media attention riveted on any new results from an international survey. League tables stir debate in every country, and opposition to these exercises is fierce in many of them.
In general, large-scale assessments can be divided into two categories: school-based or
household surveys. School-based assessments include three types:
a. National assessments designed to measure specific learning outcomes at a particular age or grade that are considered relevant for national policymakers.
b. Cross-national initiatives (either regional or international) administered in a number of countries, based on an agreed upon framework, following similar procedures to yield comparable data on learning outcomes.
c. Public examinations intended to certify specific learning outcomes linked to curricula and often used to select students for continuing education programmes or attainment of a certain cycle.
In contrast, household-based learning assessments can be used to target populations that may or may not be enrolled in or attend school. They include citizen-led assessments and any household surveys with an assessment component in the data collection.
Source: UNESCO Institute for Statistics (UIS) The number of countries participating in national and cross-national assessments varies by region. Assessment activity is high in South Asia, North America and Europe (see Figure 2).
However, in sub-Saharan Africa, few countries participate in international assessments or conduct national assessments. Therefore, it will be essential to find ways to link existing national and cross-national assessments to produce a base for initial monitoring.
Figure 2. Intensity of assessment activity in the world: Findings from the Learning Assessment Capacity Index (LACI) Note: The index is composed of five criteria based on the following questions: i) Does the country conduct national assessments in primary education between 2010 and 2015? ii) Does the country conduct national assessments in secondary education between 2010 and 2015? iii) Did the country have a national assessment for the two points of measurement in primary and secondary education between 2010 and 2015? iv) Does the country participate in a regional assessment between 2010 and 2015? iv) Does the country participate in an international assessment between 2010 and 2015? A negative response amounts to 0 and a positive response equals 1, with the index ranging from 0 to 5.
Source: UNESCO Institute for Statistics (UIS)
-6To date, there is no way of comparing national assessments in terms of their metrics. Despite some statistical attempts, cross-national comparability does not exist for several reasons: there is no single measure at any educational level (i.e. the grade at the end of primary education and at the end of lower secondary education varies from country to country); heterogeneity in the quality of national assessments; and each assessment body has its own framework of methodologies that are not necessarily comparable.