AI and the Demise of Programmed Instruction

#6 in the AI Diaries series. Previous Posts: 1 2 3 4 5

It’s a new academic year, and for behavior analysts engaged in instruction that means creating conditions to precisely shape student behavior.

Unfortunately, behavior can be shaped only when you have unambiguous access to it, and with artificial intelligence in the mix it’s hard to know whose behavior is driving an instructional program. Can behavioral instruction survive?

---

Behavioral Instruction: Good for What Ails a “Nation at Risk”

For my money the most exciting thing B.F. Skinner did, once he wandered out of the lab and into the arena of messy everyday affairs, was demonstrate how to make instruction precise, effective, individualized, and self-managed. I’m talking about the teaching machine, or really the general concept of programmed instruction.

Although Skinner was not the first to try to improve on traditional educational practices, he was exceptionally clear about the behavioral technology that makes programmed instruction effective. This post is about the technology behind the behavioral technology, but before getting to that let’s acknowledge how prescient Skinner was in placing education high on his practical to-do list. Skinner’s seminal article, “Teaching machines,” came out in Science in 1958, and the book Technology of Teaching came out in 1968 (see Note 0).

Note 0: For Students

Students, if you’re not up on this bit of behavioral history, here’s a great short video of Skinner describing how his stuff worked. If you’ve not read Technology of Teaching, you’re missing out. This is some of Skinner’s best writing, and it’s about the very engine of a successful modern society. Good news: The B.F. Skinner Foundation offers a PDF copy for only $10. What are you waiting for?

From the outset, Skinner understood that the U.S. educational system was pretty ineffective, and soon politicians and the general public had objective reason to agree. Standardized testing of educational achievement had begun to infiltrate school systems, and it repeatedly showed U.S. students lagging behind their international peers, with scores declining annually across a period of two decades. In 1983, a breathless report called A Nation At Risk (download executive summary here) described the trend as a threat to the nation’s very survival:

If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war.

This perspective echoed the views of the nation’s founders about the need for an educated populace. Thomas Jefferson had written, “I know no safe depository of the ultimate powers of the society but the people themselves” and thought it essential that they be, “enlightened enough to exercise their control with a wholesome discretion” (quoted in A Nation at Risk). Current events seem to suggest that Jefferson was right and A Nation at Risk was not excessively histrionic.

But back to technology. It would be lovely to say that, when confronted the existential crisis described in A Nation At Risk, society followed Skinner’s lead and embraced effective instructional practices. But of course that never happened, making the teaching machine an early object lesson in the still-current uphill battle of getting useful behavioral interventions and programs adopted at scale.

Difficult to Love

One reason teaching machines didn’t catch on in a big way is that physical technology was not really up to the demands of the behavioral technology.

Skinner’s teaching machine (above) was a fairly crude analog device and laborious programming was required to create successful lessons. As Ludy Benjamin (1988) detailed in a nice retrospective on teaching machines, although there were discussions in the 1950s and 1960s about installing the machines in public classrooms, this never happened at scale because the machines were expensive and unreliable (see Note 1), and they required dedicated classroom space that not every school had to spare.

Note 1: Unreliable

“Skinner had chronic difficulties in persuading his manufacturing partners to produce machines to a quality benchmark that satisfied him“

Moreover, few suitable commercial programs were available, and teachers neither knew how to create programs nor had the time for creating them (see Note 2).

Note 2: And This

And there’s more. As Benjamin (1988) reports, the teaching-machine movement had the bad fortune of coming along just as society was experiencing Orwellian jitters about humanity being ruled by machines, and teachers tended to see the machines as an existential threat to their profession. To some, the machines felt like competition, not supportive technology.

A most important, and frequently repeated, concern was the impact of the machines on teachers, specifically that teachers would lose their jobs because of the ma- chines. Some educators saw the machines as an excuse for administrators to increase the student-teacher ratio…. Several magazines noted that the teacherless school seemed a reasonable possibility. In at least two instances, pressure from teachers led school administrators to seek legislation that would ban teaching machines from the classroom. (Benjamin, 1988, p. 710; reference citations omitted)

Then there were the inevitable claims that such highly regimented instruction would transform students into mindless robots who didn’t know how to think for themselves. In such discussions it mattered little whether the machines actually worked in the sense of empirical outcome data.

Unsurprisingly, after the initial splash of Technology of Teaching and the Science article, interest in teaching machines quickly evaporated. According to Benjamin, between the 1960s and 1970s there was an 85% drop in “teaching machine” citations in the scholarly literature. The sad, simple take-home:

The story of Skinner’s machines covers a surprisingly short period from about 1949 to about 1969, with antecedents in the 1930s. However, by the end of this period there was limited tangible effect on education. (Longman, 2021).

Digitalia

In hindsight, it seems obvious that programmed instruction was ahead of its time. Skinner had imagined something that was better suited to the flexible computer technology of today than to the stodgy mechanical devices of his era (see Note 3). The individualized, branching logic that Skinner’s behavioral technology required is far easier to create with digital programming than analog machines. But by the time microcomputers made computing power widely available, Skinner’s work in education was mostly forgotten.

Note 3: On the Magic of Technology

Digital devices are not magical, although everyday people have a prodigious capacity to blindly assume that they are. When my kids were in school, you could walk into classrooms and find piles of microcomputers and iPads that schools had purchased but educators had absolutely no idea how to use to benefit their students. The “educational” software available for those devices was anything but. The core problem, of course, is that human beings rarely understand the distinction between behavioral and physical technologies. A gadget by itself will not help people learn unless it intersects productively with principles of behavior. It might be said that Skinner’s major accomplishment was to show that the device could be quite primitive as long as the associated behavioral technology was sound. Human designers, however, tend to produce the opposite.

To be sure, even with digital assistance, creating and maintaining programmed instruction is labor- and resource-intensive. For instance, when I entered graduate school, a professor at West Virginia University was using a mainframe computer to teach Introductory Psychology. His system required a ton of pricey hardware and a massive team of content creators and programmers. More recently, the brilliant online learn-to-read system called Headsprout sprung from a laborious development process requiring millions of dollars and countless person-hours. These two examples show that you don’t just throw together programmed instruction on a whim like you might a lecture or a classroom activity (and see Note 4).

Note 4: Also, Being Good is Not Enough to Make You Popular

It’s also informative that, due in part to their sizeable infrastructure demands, neither of the systems just mentioned lasted. When the professor retired, the university promptly trashed his setup and returned to traditional lectures. Headsprout is no longer marketed — alas, poor Headsprout, we barely knew ye! Headsprout’s developers (primarily behavior analysts) did pretty much everything right in terms of prompting and shaping active responding, creating a logical sequence of tasks, and making progress contingent on student mastery. It worked for a wide variety of children (see here and here and here and here and here). Alas, efficacy is rarely the only determiner of adoption, and apparently not enough customers clamored for the product to justify it in business terms. Though I have no hard evidence of this, I’m guessing that, just as with old-fashioned teaching machines, teachers objected (without evidence) that Headsprout taught only rote performance, not REAL understanding (insert eye roll here). In the end, Headsprout was never a commercial success and was discontinued in 2013.

Still, because a computer can deliver programmed instruction more deftly than an analog machine, from our contemporary digital perches we might still hold out hope of sound instructional technology being widely accepted and implemented — finally! — to save our poor beleaguered students from a failing education system. Optimists might even see windows of opportunity in the growth in online instruction that accelerated considerably in the wake of the COVID-19 pandemic. The physical technology that programmed instruction required now seems to be in place on a broad scale.

So maybe, just maybe, the educational establishment will eventually come around to embracing better ways of teaching. But even if it does (which I doubt, because historically humans have shown little interest in better mousetraps), there’s a rub.

Pop Quiz!

Programmed instruction rests on two cornerstones. The first is that the pace of instruction is tailored to each student’s progress (we’ll return to that point briefly at the end of this post). The second is that students respond frequently, with their responses shaped by contingent, individualized feedback. About that, Skinner (1961) wrote:

The effect on each student is surprisingly like that of a private tutor…. There is a constant exchange between program and student. Unlike lectures, textbooks and the usual audio-visual aids, the machine does not simply present something to be learned; it induces sustained activity. The student is always alert and busy…. The machine, like the private tutor, reinforces the student for every correct response, using this immediate feedback not only to shape [student] behavior most efficiently but also to maintain it in strength in a manner that the layman would describe as “holding the student’s interest.” (p. 97)

Thus, everything in programmed instruction hinges on the seemingly simple prerequisite of active student engagement. But engagement requires responding, and these days it’s getting hard to know whether responses entered into an instructional program really belong to a given student.

To help me illustrate a point, please take this one-question pop quiz.

The problem of confusing correlation with causation is also called…
(a) the file drawer problem
(b) the circus of circumstances
(c) spreading activation
(d) confirmation bias
(e) overcorrelation
(f) inference overreach

Apologies: This is a bogus question. Sorry about the subterfuge. But you probably don’t need to be a Nobel Prize recipient to know that the correct answer should be “none of the above.” All of the response options were pulled randomly from textbooks, or simply made up with jargony-sounding words. Although in the present context these answers all are nonsense to varying degrees, there is one thing that distinguishes among them: when presented with this question, ChatGPT loved answer (f). It confidently told me:

The problem of confusing correlation with causation is most often called inference overreach (f). That term refers to drawing conclusions that go beyond what the evidence supports—like assuming that because two things co-occur, one must cause the other.

Okay, fine; everyone knows that for all of its impressive capabilities AI can still stumble on occasion. What’s really interesting here is how live human undergraduates respond to the same question. I recently inserted it into an exam taken remotely, via course management software (see Note 5), by sophomore-level research methods students

Note 5: Regarding Online Course Delivery and Assessment

Just to be extra clear, I’m not claiming that my class is a model example of effective behavioral instruction or that multiple choice tests are a good practice or that it’s ideal to conduct remote assessments under conditions in which students can access outside information. My class is what the (decidedly un-behavioral) institutional context allows it to be. But the example holds: With digital devices everywhere and access to AI all but ubiquitious, it’s getting touch to know where a student answer comes from.

About preventing unauthorized student consultation of sources during assessments: Yes, I’m aware that there’s technology that can block a student’s device from accessing sites and programs other than the course management system, but students can easily defeat this with a second device. There’s also video technology that scrutinizes a student’s every move during testing, flagging possible instances of gazing toward an outside source, but the technology is expensive (my university doesn’t license it), not foolproof, and reliant on a human inspecting the video recordings to verify transgressions. Anyway, history says that as long as “free” answers can be accessed, students are going to find a way to access them.

About the tests in my class mentioned above: Students take the test concurrently under a degree of time pressure. Each student’s test contains a random selection of items from extensive test banks, with the questions presented in random order. Only one item is shown at a time and students aren’t shown the correct answers unless they come to my office. By the way, I’m completely comfortable with students having access to course materials while testing, because looking something up in context requires a bit of brainpower. This is very different than simply being given the answer, because you need to know roughly where to look, and you have to think to recognize when you’ve found the right answer. In fact, I challenge you to show hard evidence that testing in the traditional way produces better learning outcomes than this. And I know of at least one study showing that this particular form of “cheating” in programmed instruction — looking up the answers in course-related materials — has no adverse effects on learning outcomes compared to a no-looking-up control.

If the students chose answers randomly, each response option would be selected about by about 17% of the time. If they gravitated toward terms mentioned recently in the course (even if irrelevant to the question itself), they might prefer something like confirmation bias. Instead, nearly two-thirds of the students (61%) chose the never-employed-in-my-class term inference overreach (according to chi square, that’s significantly different from chance responding, p < .0001).

It seems pretty likely that the majority of my students responded as any self-respecting individual of their generation would, by providing the answer an autonomous artificial intelligence (AI) tool told them to, rather than answering the question themselves (see Note 6).

Note 6: For Verbal Behavior Nerds

In the language of Skinner’s Verbal Behavior, the problem is that transcription occurs when the instructional program expects an intraverbal.

This example illustrates how hard it has become to know for sure if a response is truly a student’s own. AI has advanced to the point where you can consult it about almost anything and receive a credible answer almost instantly. It takes maybe 5 seconds to paste a test question from my university’s course management software into an AI tool and get back an answer that’s usually correct (fyi, I had to work pretty hard to create bogus questions that AI didn’t recognize as bogus).

Signs of Corruption

One popular AI agent, ChatGPT, was launched online in 2022, and during the span 2023-2025 its use by the general public doubled (with by far the most use in the college student age demographic). Especially in the last year or so widely-available AI tools became really impressive in their mastery of academic subjects. We don’t know for sure exactly how many students currently use AI to complete college assignments, but a sizable majority is a safe bet.

And with that in mind it’s rather suspicious that, although students at my university are definitely not getting smarter (according to their admissions credentials at least), and I’m probably not getting better at teaching (according to the general consensus of friends and colleagues), the students certainly are doing systematically better on online tests.

A bit of context here. Across many years of in-person instruction the mean scores on my tests did not budge in any meaningful way. When the course moved to online, scores also remained constant… until AI started to become ubiquitous.

Here are illustrative data, for one exam from three recent semesters. The Fall ’24 data are pretty similar to several previous semesters of online testing, and to several years before that of in-person testing, during which test mean hovered around 80% correct. You can see that after years of holding steady scores are suddenly increasing. Additionally, the amount of time students need to answer questions is decreasing. Both outcomes strongly suggest an infusion of digital assistance.

Semester	M TEST score	M time (sec) to complete an item	% students averaging <30 Sec/item
Fall 2024	80%	41	8
Spring 2025	83%	36	23
Fall 2025	87%	34	35

It doesn’t seem coincidental that 2024 is the pivot point. That’s right around when tools like ChatGPT began to wrap their digital claws tightly around the throat of mainstream culture.

Whose Behavior?

Obviously the problems created by AI are bigger than programmed instruction, but for present purposes what I’m describing matters because of the nature of programmed instruction. It’s always an exercise in shaping. Shaping works only if there’s a contingency. And a meaningful contingency requires a response that definitively belongs to the learner.

In the Nothing New Under the Sun Department, it turns out we didn’t even need AI to have grounds for worrying about whose answers are being recorded in programmed instruction. Following the creation of effective teaching machines, and the realization that those machines came with financial and logistical challenges, there was a period of experimentation with instructional programming in good old fashioned print books. As an example, readers may be familiar with Holland and Skinner’s (1961) The Analysis of Behavior: A Program for Self-Instruction, which used the principles of behavior to teach, well, the principles of behavior (btw, the B.F. Skinner Foundation is working on an online version of the book).

In the 1960s and early 1970s a number of programmed textbooks were used in the public schools. These books putatively shaped similarly to teaching machines, but there was a loophole. The concept underpinning teaching machines was contingency: The correct answer would be revealed only after students had recorded their own answers. But with programmed books, this contingency became muddled. The reason: “Questions” were printed on one page and the “answers” on another. A student was supposed to record an answer, then flip to the next page to check the answer as feedback. Unsurprisingly, human nature being what it is, during the brief heyday of programmed textbooks books there was concern about the resulting potential for cheating (e.g., here and here and here and here).

Holland and Skinner themselves acknowledged this problem:

The correct response to each item appears on the following page, along with the next item in the sequence. Read each item, write your response…, and then turn the page to see whether your answer is correct….Writing out the answer is essential. It is also essential to write it before looking at the correct answer. When the student… glances ahead without first putting down an answer of his own, he commits himself to only a vague and poorly formulated guess. This is not effective and in the long run makes the total task more difficult. (Holland & Skinner, 1961, pp. vii-viii, emphasis added).

A lot of “glancing ahead” did take place. In one illustrative case with which I’m familiar, three 6th graders (Note 7) were placed into an accelerated math track. Because no teacher was available to work separately with them, they were sequestered at an unsupervised table at the back of their classroom and given a programmed pre-algebra textbook. And there they sat, day after day, book in one hand and pencil in the other, transcribing answers as fast as their little fingers could scribble. Rarely, in fact, did they do anything other than look at a question on one page and flip to another page for the answer before flipping back to record it.

Had you asked those students why they did this, they’d have stared in disbelief at the magnitude of your hoopleheadedness (Note 7) and replied, Why the heck NOT?

Note 7: Students and Hoopleheads

The perpetrators were me and my classmates Greg and Suzanne. “Hooplehead” is Appalachian slang for fool or idiot.

And this helps to define a rarely-discussed, but potentially fatal, problem that arises when programmed instruction is part of a larger curriculum.

Mixed Contingencies

Skinner imagined that programmed instruction would be so reinforcing that people would happily “learn for the sake of learning.” More specifically:

We might say that the human organism is reinforced by any simple gain in competence. When we guarantee a consistent gain by breaking the material to be learned into small steps, we raise the frequency of reinforcement to a maximum and reduce aversive consequences to a minimum. (Skinner, 1961, p. 97)

But instruction normally is embedded in a curriculum, completion of which is tied to long-term outcomes like course grades, diplomas, and eventual employment. This inevitably means student responses are governed by multiple layers of contingencies. Sure, well-designed instruction can make it fun to learn and answer correctly, but educational systems also create escape contingencies in which making work go away, and getting closer to those long-term outcomes, is a sizable conditioned reinforcer.

Readily available, “free” correct answers eliminate work and achieve “progress” almost effortlessly, and thus are a hard temptation for students to resist (see Note 8).

Note 8: Very Meta

The lure of cheating to avoid effort and attain a goal quickly is so great that, in a phenomenon computer science folks call “reward hacking,” even autonomous AI agents will do it.

But at the same time, of course, those “free” correct answers undermine the core requirement of behavioral instruction (actually, of ALL instruction, one would hope), that a student actually know the answer before recording a correct response (see Note 9).

Note 9: Behaviorspeak

In other words, this is a problem of competing repertoires: Any time and effort invested in trafficking in “free” answers is time and effort siphoned away from the behaviors that instruction was designed to shape and reinforce.

If you work in education, you’re already aware that there’s chaos afoot when it comes to figuring out how to assure that students with access to AI’s prodigious capabilities are actually learning. How much is at stake when education is undermined in this way depends on the effectiveness of the underlying instructional technology. Honestly, with a lot of classes taught at my university, I’m not sure it matters whether students or ChatGPT complete the work. Either way, check in with the students a month after the semester ends and they know absolutely nothing. Basement effects mean that AI is no threat to failed instruction (see Note 10).

Note 10: Baseline-Dependent Effects

And who knows? AI might actually help (a little) when students use it in the context of lousy instruction. After all, in transcribing what an AI bot says a student is at least engaging in some topically-appropriate behavior… which could be more than occurs some college classes I’ve seen. Transcription isn’t exactly what people like to call “deep processing,” but it’s potentially more meaningful than, say, doodling while an off-topic lecturer rambles interminably about that time his car got stuck in the mud at Death Valley National Park.

But with data-driven, best-practices behavioral instruction, it’s different. When the conditions are right, just about any student can succeed. Undermine that and a great deal of value is lost.

As you read this, AI is undermining instruction as fast as students’ little fingers can put it to work. Which means that even if we could wave a magic wand and get what behavior analysts have always wanted — the adoption of effective instructional practices at a societal scale — there’s no guarantee this would result in substantive educational benefits.

Solutions?

Programmed instruction’s pioneers could not have anticipated AI or the magnitude of the threat it currently poses to education. But now that this threat is upon us, what are we to do about it? To prevent cheating with digital technology may require direct human supervision of student work — which is kind of at odds with programmed instruction’s emphasis on individualized pacing that frees students from oppressive group-based routines. Alternatively, maybe technology that prevents students from accessing outside help while engaged with instructional programs can be improved (it’s clunky and fallible now), although every instructor knows that as long as contingencies reward the completing of lessons quickly and effortlessly, prevention is an arms race in which students tend to lead.

Lots of additional considerations apply, but you get the idea. If we hold out hope of effective programmed instruction ever being widely adopted, to beneficial effect, we are going to have to figure out how be sure that students are doing their own work (see Note 11).

Note 11: Irony of the Century, or: Hello, Teacher…(?)

Consider: Probably the greatest potential for developing individually-sensitive programmed instruction lies in the near-infinite computing capacity of autonomous AI. With the right guidance, AI could create intricately branching, student-specific, topically-sophisticated and pedagogically-sound instructional programs essentially on the fly, as needed, all but instantly. Brilliant! But all of that computing power and design expertise is worthless if students, on their end, are also using AI to generate their responses.

Otherwise, programmed instruction will produce students like those coming out of the educational mainstream, who spend plenty of time in lessons but emerge knowing next to nothing. As demonstrated by those recalcitrant 6th grade math students you met above, that is as bad as having no instructional technology at all.

I’m interested in your thoughts on the AI problem in education and possible solutions, especially regarding how to create valid applications of programmed instruction when it’s unclear who’s doing the responding. Please use the Comments section below to weigh in.

---

THEME ART