This year, we were presented with the most rigorous, in-depth, comprehensive and international survey of worldwide financial skills ever seen: The Programme for International Student Assessment (PISA) report into Financial Literacy Skills for the 21st C, Students and Money.
Back in 2012 PISA tested 29,000 15 and 16 year olds in 18 countries who sat a two hour test comprising four papers. PISA ranked students in five levels of proficiency from basic to advanced. Scores for each country’s students were aggregated with a mean of 500. Top-score was Shanghai (603) and lowest Colombia (377).
Despite the UK not taking part, we are enthusiastic and supportive of the objectives of the PISA tests and the need for internationally comparable assessments to support policy choices, implementation and evaluation. However, the financial education test itself and its parameters are not above reproach.
We interpret PISA tests as not testing as stated, the likely ‘readiness of 15-year-old students for life beyond compulsory education and their capacity to use knowledge and skills’. PISA defines financial literacy as:
“A combination of awareness, knowledge, skill, attitude and behaviour necessary to make sound financial decisions and ultimately achieve individual financial well-being’
But the sample questions provided focuses on ‘skills’ and to a lesser extent ‘knowledge’. It does not sufficiently address ‘awareness, attitude and behaviour’. It is not surprising that mathematics are the most significant drivers of performance in the PISA tests.
For example consider one of the sample questions:
To receive a Full Credit (level 5 advanced capability) requires an answer that simply states Mrs Jones will have extra money and she will pay a lower interest rate. In the explanatory notes it is stated the student has all the information needed to assess the options. Many would disagree with this premise. Does the student possess all the relevant information to demonstrate financial literacy as defined above? What they do have is the information necessary to make a mathematical calculation and require the knowledge to understand what the words and symbols for interest, loan and monthly payment mean.
For PISA’s test to hold water, a full credit answer should include more information. What are the terms of the loans? Is the interest simple or compounded and how often is the principal adjusted? Are there set-up charges; early repayment options or penalties; does the loan require security or a guarantor?’ Financial literacy, may be better expressed as capability, includes the knowledge and confidence to ask these kinds of questions and the skills to interpret the answers. It is not just the maths and even that is incomplete without further information.*
A secondary concern is how results have been reported and how each community may use and interpret them. Headlines like: ‘Top of the Class Shanghai’ and ‘bottom of the class Colombia’ is not entirely useful. There is not a lot of difference between countries – 16 of the top 18 score in a range 466 to 541 or less than one PISA level (75 points) difference from the mean.
Recently the Financial Times featured a chart derived from PISA comparing hours of financial education versus scores by country. Many of us in the financial education sector strongly question any conclusion that financial education is ineffective based on this relationship.
Relevance and quality must be considered and indeed PISA shows the direct use of financial products is mildly correlated with performance. Like PISA we are concerned that many who deliver financial education are not skilled in doing so. MyBnk ensures all instructors are trained and tested in the relevant financial knowledge of the cohort they are working with. Furthermore, trainers are selected because they can actively engage with young people and are trained to bring the subject matter to life.
The need for financial capability has been universally accepted by civil society and policy makers. In England this September 2014, financial education becomes compulsory on the National Curriculum and the Money Advice Service (MAS) is currently preparing a UK Financial Capability Strategy. How financial capability is best achieved and measured are today’s questions. PISA is a substantial report which provides test-based, comparable scores which complement the vast majority of ‘self-assessed’ evaluations currently available. Like others, we will embrace some of the methodology particularly in maths and consider the conclusions of other points. We expect providers and policy makers to do likewise.
The UK’s provisional agreement to participate in PISA’s 2015 tests presents an opportunity to further our understanding on the effectiveness of financial education – however, the current model will not offer a full and true picture of the financial literacy landscape and its impact on the lives of young people.
Our objective of a financially literate and enterprise-driven generation includes the need to develop ‘attitudes and behaviours’ as well as utilise maths and literacy. Engaging young people with all facets of quality financial education is key to achieving that.
*In relation to PISA question: We can calculate the loan principal is (at best) adjusted annually Mrs Joans still owes 7400 zeds at year end. A more real-life example would adjust the principal monthly and 15% simple interest would equate to 14.5% of the initial loan. This matters as the loan would be paid off in seven years five months rather than eight years seven months. The worst case scenario is the principal is not adjusted through the loan period in which case it would take more than twelve years to pay off.
