Improving the Measurement of Critical Thinking

Our Roadmap to Improving the Measurement of Critical Thinking

Our previous blog article discussed some methods of measuring critical thinking skills, as well as their limitations. Read on to find out our recommendations for improving the measurement of critical thinking.

Figure 1

1. Reducing Verbal Loading

Critical thinking assessments should measure critical thinking skills. Essay-based assessments have the unfortunate side-effect of measuring reading and writing skills. This negatively impacts exam takers who struggle with reading or writing. To make critical thinking measurement more accurate and to avoid disadvantaging some test takers, assessments should use limited words and simplified vocabulary.

At Endoxa Learning, we achieve this by dual coding arguments, displaying them as argument graphs (Figure 1). By mapping arguments visually and replacing complex sentences with arrows and boxes, we make it easier to understand the structure of an argument. Testing includes hiding statements from the graph and asking the student to choose the correct answer, as well as quizzing them on their understanding of the overall argument. Unlike in essay-based assessments, students can work out an argument’s strengths and weaknesses using its visualised structure, without wading through complicated prose.

Figure 2

2. Testing on Arguments of a Range of Complexities

While short form tests reduce the necessity for strong reading and writing skills, they do so at the expense of argument complexity. To measure critical thinking effectively, tests must measure participants’ understanding of longer arguments that contain more controversy or have more than one branch.

At Endoxa Learning, we test understanding of a range of arguments, with varying complexity. These can be a few short steps, or a much larger argument that includes different theories and case studies (Figure 2).

Figure 3

This sets argument graphs apart from short form assessments, which cannot include complex arguments like the existence of God, climate change or other important debates. We automate testing of students’ understanding of longer arguments, which means that checking an argument graph is much quicker than marking a traditional essay. As a result, the marking workload for teachers is reduced significantly. Additionally, Endoxa Learning can still test on shorter sections of an argument since each argument step is introduced individually (Figure 3).

3. Testing on Relevant Content

Whether essay-based or multiple-choice, critical thinking tests often share the flaw of relying on irrelevant scenarios, with which test takers may be unfamiliar. In the most extreme case, that of critical thinking as an A Level, students are required to process huge amounts of knowledge, only for students to be tested on skill rather than knowledge. Critical thinking is a transferable skill, because it is useful anywhere there is an argument, debate or problem to understand. So, critical thinking can be measured in contexts that students and professionals are already familiar with, in order to ensure they are being tested on critical thinking as opposed to knowledge about an irrelevant scenario. Furthermore, combining critical thinking with relevant subjects can reduce the workload of those setting and marking the test, which is especially useful in an educational setting. This has already been attempted in some areas of education, but only at a limited level and mostly in highly specialised contexts such as nursing.

At Endoxa Learning, critical thinking assessment is integrated within content that is relevant to students. We produce lessons on debates that students already need to learn as part of the curriculum, rather than producing generic scenarios as the basis of the assessment. For example, Figure 3 is based on the theory of plate tectonics, which students learn in KS3 geography. Students are tested on critical thinking skills without having to learn unhelpful information. Moreover, they can also learn crucial subject knowledge, using a system that stays the same whether they are learning about coastal management, economics or the existence of God.

Summary

Endoxa Learning uses three key methods to solve the limitations of assessing critical thinking in education:

1. Information is dual coded to reduce verbal loading.

2. Students can be tested on arguments of a range of complexities.

3. We test critical thinking using content that students already need to learn.

Using this approach, we take the positive aspects of in depth, essay-based assessments and combine them with the lower verbal loading of short form tests.

Given the value of critical thinking tests for employers, we also plan to adapt Endoxa Learning to provide employment aptitude tests. These will share the benefits of the tests discussed in previous sections, while making them even more useful for employers who need a clear measure of critical thinking skills among applicants.

To learn more about measuring critical thinking and to find the research supporting this blog article, click here to read our longer article.