prior probability

That is the title of this new paper by a team of researchers affiliated with the MIT Media Lab (Kosmyna, et al., 2025). Although the sample size of their study is small (n = 54), this work is one of the few in the critical thinking literature to conduct an actual experiment: the participants in their study were randomly assigned into one of three groups — the ChatGPT group, the Google Search group, and the “brain-only” group — and the members of each group then had to complete a critical-thinking essay-writing task while they were hooked up to an electroencephalogram (EEG), a machine that measures electrical activity in the brain. Although it looks like the the MIT Media Lab team may have cherry-picked or p-hacked their results (see here), they claim that ChatGPT decimates critical thinking! (Here is a plain-English summary of their findings.) For further reference, below is a compilation of my previous posts on the subject of “Critical thinking in the age of A.I.”:

1. The impact of ChatGPT on critical thinking: prologue (11 June)
2. What is *critical thinking*? A Humean-Bayesian approach (16 June)
3. Critiques of *critical thinking* theory, pedagogy, and practice: an annotated bibliography (17 June)
4. ChatGPT’s pincer attack on critical thinking (18 June)
5. Critical thinking as a communal activity (19 June)
6. John List on critical thinking in the age of A.I. (20 June)

I forgot to wish everyone a “Happy Juneteenth” on 19 June. My bad! Also, shout out to my family and friends in the beautiful island of Jamaica; hope to see you soon!

Translation: Have a great summer! Below are two melodious classics to officially kick off the summer season:

I want to conclude (for now) my series on “Critical thinking in the age of A.I.” with an insight from my colleague and friend John A. List (a/k/a @Econ_4_Everyone), an experimental economist at the University of Chicago. When asked, What skills will A.I. models like ChatGPT make more important? Without skipping a beat, his response was “critical thinking skills”! (See his tweet from 21 May, which I have reposted below. Hat tip: information monster Tyler Cowen.)

But for me, even more important than this response was his economic reasoning: critical thinking is more essential than ever because A.I. models have reduced the cost of creating information to almost zero. In Professor List’s own words, “in the past there was value in creating large quantities of information. That is now costless. The new currency is how to generate, assimilate, interpret, and make that large amount of information actionable”. But this observation begs the $64 question: how can we teach critical thinking or improve our own critical thinking skills?

As it happens, Professor List has written up a new paper titled “Enhancing Critical Thinking Skill Formation: Getting Fast Thinkers to Slow Down”, which is available here, and now all I can say is that I wish I had discovered this paper sooner! Professor List has not only conducted some of the most ingenious field experiments of all time; he is also my favorite living economist. I have therefore decided to hit pause on my series on “Critical thinking in the age of A.I.” in order to go back to the drawing board, so to speak. I will study List’s new paper as well as this devastating critique of education research by my colleagues James Rebele and E. Kent St Pierre and will report back soon …

Works cited:

John List, “Enhancing Critical Thinking Skill Formation: Getting Fast Thinkers to Slow Down“, Artefactual Field Experiments 00726, The Field Experiments Website (2021).

James E. Rebele and E. Kent St Pierre, “Stagnation in accounting education research”, Journal of Accounting Education, Vol. 33, no. 2 (2015), pp. 128-137.

N.B.: This is Part 5 of my series on “Critical thinking in the age of A.I.”

Yesterday (see here), I responded to John McPeck’s critique that there are no general thinking skills; today, I will respond to the other objections to the theory, pedagogy, and practice of critical thinking that I had surveyed in a previous post, including those by Alston (2001), Martin (2002), Paul (1981), and Thayer-Bacon (2000). Broadly speaking, the common thread tying together these sundry criticisms is “dissatisfaction with focusing [exclusively] on the logical analysis and evaluation of reasoning and arguments” (Hitchcock 2024, 12.2). Instead, these critics see critical thinking as “a social, interactive, personally engaged activity like that of a quilting bee … rather than as an individual, solitary, distanced activity symbolized by Rodin’s The Thinker” (ibid.).

For me, one of the most memorable and moving examples of this more social or communal approach to critical thinking is bell hook’s innovative “engaged pedagogy” method of teaching: in her introductory course on black women writers, students are required to write an autobiographical paragraph about an early racial memory and then to read their work aloud to the class as whole. (hooks 1994: 84) The goal of hook’s communal approach to critical thinking is thus to affirm “the uniqueness and value of each voice” in the class by “creating a communal awareness of the diversity of the group’s experiences” (Hitchcock 2024, 12.2).

On this note, one of the advantages of my own Humean/Bayesian approach to critical thinking (see here) is that this approach can be practiced by groups as well as by individuals. In fact, the original inspiration for my Humean/Bayesian approach is the common law jury of lore, where a group of six or 12 ordinary people are chosen at random and asked to evaluate the evidence presented by the parties in the case. Although a single judge acts as a gatekeeper both before and during the trial (overseeing the jury-selection process and deciding which pieces of evidence are relevant and which are “privileged”, “prejudicial”, or otherwise inadmissible), it is the jury as a whole, meeting outside the presence of the judge, who decides how much weight to assign to the evidence.

Both my common law jury model and bell hooks’ communal approach to critical thinking highlights another limitation of large language models like ChatGPT: although these models share a communal aspect, since they are trained on massive datasets created by a large number of people, at the same time they are designed to be used by a single user only. But is ChatGPT an inherently individualized and atomistic tool, isolating (and perhaps alienating) its users from each other and the world at large, or is there any way to make ChatGPT usage more of a communal, social, or shared experience (like, say, Wikipedia)? I will conclude my series on “ChatGPT and critical thinking” in my next post.

Work cited: bell hooks, Teaching to Transgress: Education as the Practice of Freedom, New York & London: Routledge (1994).

N.B.: This is Part 4 of my series on “Critical thinking in the age of A.I.”

In my previous post, I turned to David Hitchcock’s survey article in the SEP to revisit several salient critiques of the theory, pedagogy, and practice of critical thinking, and I concluded my review of these criticisms with two questions: “Which of these objections … is the most damaging one to my own Humean/Bayesian approach to critical thinking, and which … are also relevant to large language models like ChatGPT?” Let’s begin with John McPeck (1981)’s critique that there are no general thinking skills, since thinking is always thinking about some subject-matter within a specific field, the so-called “strong subject-specificity thesis”. Although McPeck’s claim is ultimately unpersuasive, it does help us see why ChatGPT represents such a grave danger to higher ed.

To begin with, there are two problems with McPeck’s critique. One is purely academic or definitional: the concept of a “field” or a “subject” is a vague one, since all knowledge is interconnected. The other problem, however, is fatal: even if we could agree on such definitions, McPeck’s strong subject-specificity thesis is subject to “obvious counter-examples”, including the existence of general inter-field principles (e.g. parsimony, originality, usefulness, etc.), the general hypothetico-deductive model of reasoning (i.e. making predictions and comparing them with observed data), as well as general or overarching logical principles, like the ability to recognize confusion of necessary and sufficient conditions. (For my part, I would add my Humean-Bayesian approach to critical thinking as another counter-example, for however narrowly or broadly the concept of a field or subject is defined, one has to be able to evaluate evidence and update one’s beliefs when new evidence becomes available.)

Nevertheless, as Hitchcock concedes (2024, 12.1), a prerequisite for critical thinking is “background knowledge”, the basic facts of the field (however defined) one is operating in: “It is common ground in debates about the generality or subject-specificity of critical thinking dispositions and abilities that critical thinking about any topic requires background knowledge about the topic.” For example, even the most sophisticated understanding of my Humean-Bayesian approach to critical thinking is of no help unless accompanied by some knowledge of what counts as relevant evidence and how much weight to assign to such evidence.

And it is on this particular point (i.e. the all-important question of what counts as relevant evidence and how much weight to assign to the evidence), we can now begin to understand how ChatGPT poses a powerful pincer attack on critical thinking. (A pincer movement, or “double envelopment” or “hammer and anvil” tactic, is a military maneuver in which one’s forces simultaneously attack both flanks or sides of one’s adversary with the aim of encircling and trapping the enemy formation.) To the point, ChatGPT attacks critical thinking in two ways. First off, it dispenses with the need to learn any basic facts or develop any background knowledge at all about the topic being explored by the user, since most AI models like ChatGPT are trained on massive datasets primarily consisting of publicly-available Internet content, including presumably Google Scholar, the Stanford Encyclopedia of Philosophy, and Wikipedia, and thus already has all the relevant background knowledge at its disposal.

Secondly, ChatGPT not only does the thinking for us, so to speak, by providing plausible answers to any particular problem or question one may have; it also provides well-reasoned answers to our problems and questions, thus dispensing with the need for any thinking altogether, let alone critical thinking! (See, for example, this intriguing new paper by Damien Charlotin (HEC Paris) and Niccolò Ridi (King’s College London) in which two popular LLM models, Google’s Gemini 2.0 and OpenAI’s GPT4o, competed in the Jessup International Law Moot Court Competition and garnered higher scores than many humans in legal reasoning.)

To recap the potential double-edged threat that ChatGPT poses to critical thinking, why would anyone take the time to learn basic facts or develop background knowledge (or write a legal brief for a moot court competition) if ChatGPT already has those facts and background knowledge at its disposal? Likewise, why waste any time with my Humean/Bayesian approach to critical thinking if ChatGPT can not only evaluate and weigh the evidence for us, but can do so faster and maybe even more accurately? In short, why take the trouble to think if ChatGPT can think for us? Is there any effective way to escape or counteract this powerful pincer attack? And what about the many other critiques of the theory, pedagogy, and practice of critical thinking that I surveyed in my previous post? I will turn to those critiques next …

N.B.: This is Part 3 of my series on “Critical thinking in the age of A.I.”

As I mentioned in my previous post, critical thinking is supposed to help us overcome our biases, but is my own approach to critical thinking itself biased, incomplete, or otherwise off-base? (Recall how I defined critical thinking in Humean and Bayesian terms: careful evaluation and scrutiny of the available evidence, followed by periodic “updating” as new evidence becomes available.) David Hitchcock identifies several salient criticisms of the theory, pedagogy, and practice of critical thinking in his survey article for the Stanford Encyclopedia of Philosophy:

• Kal Alston (2001), “Re/Thinking Critical Thinking: The Seductions of Everyday Life”, Studies in Philosophy and Education, 20(1): 27–40. doi:10.1023/A:1005247128053

Among other things, Alston argues that the theory, pedagogy, and practice of critical thinking are biased because they favor written and spoken assignments over other forms of expression and focus attention to written and spoken communications over attention to human problems. For Alston, “critical thinking that reads arguments, texts, or practices merely on the surface without connections to feeling/desiring/doing or action lacks an ethical depth that should infuse the difference between mere cognitive activity and something we want to call critical thinking” (Alston 2001: 34, quoted in Hitchcock 2024, 12.2).

By way of example, she reports that the students in her women’s studies class were able to see the flaws in the Cinderella myth that pervades much romantic fiction but in their own romantic relationships still acted as if all failures were the woman’s fault and still accepted the notions of love at first sight and living happily ever after. Students, she writes, should “be able to connect their intellectual critique to a more affective, somatic, and ethical account of making risky choices that have sexist, racist, classist, familial, sexual, or other consequences for themselves and those both near and far” (Ibid.).

• David Hitchcock, “Critical Thinking“, in Edward N. Zalta & Uri Nodelman (eds.), Stanford Encyclopedia of Philosophy (Summer 2024 Edition).

What is the relationship of critical thinking to problem solving, decision-making, higher-order thinking, creative thinking, and other recognized types of thinking? As Hitchcock points out, the nature of this relationship is unclear. If critical thinking is conceived broadly to cover any careful thinking about any topic for any purpose, then problem solving and decision making will be kinds of critical thinking, if they are done carefully. If, however, critical thinking is conceived more narrowly as consisting solely of appraisal of intellectual products, then it will be disjoint with such constructive activities as problem solving and decision making. (See Hitchcock 2024, 12.3.)

• Jane Roland Martin, “Critical Thinking for a Humane World”, in Stephen P. Norris (ed.), The Generalizability of Critical Thinking, New York: Teachers College Press (1992), pp. 163–180.

Jane Roland Martin also highlights the problem of bias. For Martin, the theory, pedagogy, and practice of critical thinking are biased because of their supposed indifference to the situation of others over care for them (the indifference problem), their distancing from the object of inquiry over closeness to it (the distancing problem), and their orientation to thought over orientation to action (the thought over action problem). (See Hitchcock 2024, 12.2.)

• John E. McPeck, Critical Thinking and Education, New York: St. Martin’s Press (1981).

For McPeck, it is a conceptual truth that all critical thinking abilities are specific to a subject (the “strong subject-specificity thesis”). He therefore argues that there are no general thinking skills, since thinking is always thinking about some subject-matter. (See Hitchcock 2024, 12.1.)

• Richard W. Paul (1981), “Teaching Critical Thinking in the ‘Strong’ Sense: A Focus on Self-Deception, World Views, and a Dialectical Mode of Analysis”, Informal Logic, 4(2): 2–7.

Paul “bemoans the tendency of atomistic teaching of methods of analyzing and evaluating arguments to turn students into more able sophists, adept at finding fault with positions and arguments with which they disagree but even more entrenched in the egocentric and sociocentric biases with which they began” (Hitchcock 2024, 12.2).

• Barbara J. Thayer-Bacon, Transforming Critical Thinking: Thinking Constructively, New York: Teachers College Press (2000).

For her part, Thayer-Bacon argues that the theory, pedagogy, and practice of critical thinking are biased because they privilege reason over emotion, imagination, and intuition and solitary thinking over collaborative thinking. She contrasts the embodied and socially embedded learning of her elementary school students in a Montessori school, who used their imagination, intuition, and emotions as well as their reason, with conceptions of critical thinking as “thinking that is used to critique arguments, offer justifications, and make judgments about what are the good reasons, or the right answers” (Thayer-Bacon 2000, pp. 127-128, quoted in Hitchcock 2024, 12.2).

So, which of these objections (if any) is the most damaging one to my own Humean/Bayesian approach to critical thinking, and which of the critiques above are also relevant to large language models like ChatGPT? I will address these key questions in my next post …

N.B.: This is Part 2 of my series on “Critical thinking in the age of A.I.”

Previously (see here), I posed a question that has been troubling me since ChatGPT was unleashed on the world on 30 November 2022: What impact will ChatGPT and other large language models have on higher education? That is, if the primary mission of higher ed is to promote critical thinking (paging John Dewey!), what impact will these models have on students’ ability to think and reason for themselves? Today, however, I want to take a step back and ask an even more fundamental question. To the point, what is critical thinking? Simply put, what do we mean when we say “the primary mission of higher ed is to promote critical thinking”?

Alas, there is no one standard or universal or commonly-accepted definition of “critical thinking”? Worse yet, some definitions are totally circular. (See, for example, this entry in the Stanford Encyclopedia of Philosophy, which defines critical thinking as “careful thinking directed to a goal”.) Some scholars simply assume that critical thinking is the ability to “think for yourself”. (See, for example, this article in the Harvard Business Review.) The problem with this standard definition, however, is that it is incomplete, since we all have deep cognitive biases that distort our thinking. At a minimum, then, critical thinking has to consist of some reliable method for overcoming our biases.

That is why I propose we take a general Humean/Bayesian approach to critical thinking, for it was the great David Hume who said (in his essay on miracles), “A wise man … proportions his belief to the evidence wise man”, and it was Hume’s contemporary, the Reverend Thomas Bayes (in his posthomous essay on the “doctrine of chances”), who developed an ingenious method for updating our beliefs when new evidence becomes available. For me, then, a critical thinker is someone who is able to overcome his cognitive biases by emulating Hume and Bayes — specifically, by taking the following two intellectual steps: careful evaluation and scrutiny of the available evidence (Hume), followed by periodic “updating” as new evidence becomes available (Bayes).

With this definition in mind, we can now begin to address my original question: What impact will ChatGPT have on critical thinking? (To be continued …)

As I remember my late father, Don Francisco Guerra (1943-2024), on this Father’s Day, I say “hasta pronto y hasta luego” to our beloved México with our favorite mariachi ballad, for it was my Cuban father who introduced me to Mexican culture and mariachi music when I was a boy. I will never forget our family vacations in the little beach towns of Rosarito and San Felipe …