Wednesday, February 26, 2014

Common Ground In Teaching And Tech

Teaching and tech have much more in common than a few letters. Two decades ago American software engineering was perceived as falling behind the overseas competition, just as education is today. The software industry responded with innovation in practice, governance, management, and culture. The software industry was not the first to confront this, as the manufacturing sector had faced, and continues to face, a similar challenge. Indeed, the software industry was able to leverage significant ideas and concepts from manufacturing and repurpose them to improving their own practices. Education now has the opportunity to leverage the lessons from both technology and manufacturing.

Widespread Misunderstandings

When I refer to software engineering I do not mean simply writing code, but the entire process built around understanding users, their mental models, their needs, their expectations, and translating these into high quality products of value to the customer.

Teaching as well is frequently misunderstood. Effective teaching is far more subtle and nuanced than simply presenting information to students. The best teachers build relationships with their students, find effective ways to make the content relevant and interesting, and to the extent possible adjust the content to the specific needs of the students, some even engage the parents. And that's on a good day.

The understanding from outside either profession is significantly different from that of actual practitioners. The public perception of these professions is deeply flawed. Even more damaging is that the misunderstanding sometimes extends into the management and policy making circles. Importantly for the reader, many technologists and educators do not truly understand one another. My intent is to foster communication and cooperation by pointing out strong parallels between these two schools of thought.

Impetus for Change

Both professions have confronted, and are confronting, a crisis of confidence. For both teaching and tech, one of the significant concerns is that America is falling behind. For teaching we see this almost daily in the media. Software engineering has a head start confronting this issue. In 1993 Edward Yourdon portrayed the Decline and Fall of the American Programmer relative to their international competition.

In both cases this precipitated collections of procedural and technological solutions. For software this included the Rational Unified Process and several Computer Aided Software Engineering (CASE) tools, which turned out not to be a panacea. Indeed, the misapplication, the overly rigid application, of the Rational Process was a factor in the raise of agile methods.

The important aspect is that the recognition of an issue lead to a period of experimentation and trial, and only after this period did more fruitful approaches emerge. Agile methodologies themselves underwent a Cambrian explosion of sorts, followed by a Darwinian evolution leaving behind more fit solutions, and more openness to change.

Instruction faces a similar situation to software engineering after Yourdon published his 1993 book. There is a recognition that there is a problem, and many solutions are materializing. With MOOCS, apps, web apps, videos, CCSS, and standardized testing all making a showing, it is unclear which approaches will provide the greatest advantage in the future. Each approach has its champions, but clearly none of them stand alone as a complete solution. This plethora of solutions reflects varying perceptions of exactly what the issues are with education. Perhaps, though, something can be borrowed from the software engineering venue and teaching can embrace change and data driven adaptation. Moreover, in software engineering this shift to agile was a grass roots movement.

Location Independence

Both education and software are rooted in the transfer of information. In most cases there is no need for a physical presence or for the transfer of material goods. This reduces the importance of location and increases the importance of interactions. Indeed, the realization is coming to fruition that face to face time is better utilized by focusing on personal interactions during office hours or in class time.

A software engineer can collaborate with, and market to anywhere on the planet without leaving the neighborhood coffee shop. Similarly, many universities now have a global reach. This globalization directly effects the consumer as well. Online web services and online instruction on almost any topic can be accessed from almost anywhere. Where previously I might visit an accountant to process my taxes, now I can easily prepare my taxes online with a well known and trusted service. Generally though, determining which products and resources represent a true value is difficult. While some resources are undoubtedly first class, others fall far short.

Global access to information and services reduces the value of memorization, and significantly increases the value of critical thinking to ascertain which are truly useful, and sometimes whether online information is correct.

Assessment and tests

Assessment easily warrants its own discussion in either field. We limit ourselves to outlining a few relationships between assessment in software and in education. Continuous testing is a generally accepted part of modern software development. There is a small school that feels that the testing should drive the development process, and some people are simply confused about the difference between the two.

Predominantly, testing is viewed as an important step in the production of quality software; as a means to an end, but not a primary goal. Considerable effort has gone into creating software testing tools and techniques. A common goal is to maximize the value of the tests while minimizing their impact on the development process. For example unit tests are designed to run quickly and in an automated fashion. Lightweight testing, or better yet adaptive learning, can play a similar role in education.

At the end of the day, software is valued by success in the marketplace, by delivering value to the customer. This is a fundamental difference between software and education, choice. The market can choose any alternative, of if the software does not address any need, the market may choose to forgo it entirely. This provides a powerful incentive to correctly assess the value and correctness of software.

Education, on the other hand, is compulsory. This, with a few exceptions, places the burden of assessment on the educational process itself, and shifts it from a question of market acceptance to one of accountability.

The public perception of testing in education is dominated by high stakes, high stress tests like final exams, midterms, SATs and thanks to recent media coverage, PISA tests. These tests capture the student's learning at a specific point in time. Tests like this are referred to as summative tests.

The true picture is much richer. Another type of assessment, formative assessment, has an entirely different emphasis. These are ongoing assessments focused on providing feedback on how effectively the student is learning. This assessment is usually a lower stakes and lower stress - perhaps not even graded - mechanism. Formative assessment is also intended to provide feedback on the effectiveness of the teaching process. At a conceptual level, this might be expected to provide a strong incentive to innovate in education. Also, well designed feedback would mitigate any risk associated with innovation by allowing a quick course correction onto a productive path.

The technology industry struggles with assessment as well. Particularly telling is this interview with Laszlo Bock, a senior vice president at Google.

We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship.

This raises fundamental questions about the effectiveness of the interview, aka assessment, process in the software industry as a whole.

One of the things we've seen from all our data crunching is that G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there's a slight correlation.

He goes on to explain that the academic environment is highly specialized, and that success within that context does not imply that someone will be successful within another environment or context. Both teaching and tech have a critical need to access ability to put knowledge to work in real world contexts.

Iterative Improvement

The application of iterations is perhaps the most subtle of the points raised here. The software engineering world uses iterations in the development process, and also iteratively improves the process itself. Both the product and process iterations are driven by assessments, participant feedback, and comparisons against organizational goals.

The current state of education is reminiscent of software engineering before the acceptance of agile iterative methods. At that point software development was dominated by inflexible methods characterized by large up front planning.

The famous quote from German strategist von Moltke "No Battle Plan Survives Contact With the Enemy" dramatically captures the need to adjust any plan in the field to adapt to real world conditions. Software engineering has embraced this need for continual adaption and seen significant increases in quality and productivity.

These adaptations to the situation on the ground are driven by the people on the ground. This puts a greater freedom and greater responsibility into the hands of the practitioner with boots on the ground.

Education would profit as well from this shift in emphasis. From up front planning and set curricula to general goals and continual adaption to the needs of a particular situation driven by the teacher's own judgment.

Personalizing the Experience

Education, by its very nature, focuses on presenting new ideas and integrating them into the audience's mental model. The most successful approach caters the educational experience to the individual student. The teacher builds a relationship with the student and adjusts the content to address their individual interests and well as to expand on their strengths and address their weaknesses. This path is proven to be effective and well received by students and teachers alike.

The application of personalized learning is limited by its labor intensive nature. However, computers can ease the labor requirement in certain cases. The most direct application is to provide asynchronous communications through email and social media. The interaction and feedback is still timely, but not always immediate and face to face.

Well designed software provides multiple modes and multiple paths for the learner to explore different scenarios. This encourages the learner to freely choose their path through content exercising their interests, and filling in gaps in their understanding.

Several companies carry the concept in another direction with adaptive learning. Adaptive learning assesses the state of the students knowledge and further instruction is focused on their weakest areas. Extensive effort has been put into allowing automatic identification of the learners problem areas. Indicators include scores on exams, the time taken to answer questions, or to read through a section, even eyetracking and body language are sometimes used.

The edtech entrepreneur is concerned with personalization from their earliest days. Almost from the outset, she will be concerned with product market fit, or how the product addresses the specific needs of the educational community, or that part of the community they are addressing. This requires building relationships with educators, administrators, and frequently with students. The entrepreneur must understand the market's interests and needs, and modify the product to fulfill those interests and needs. Sound familiar?

The parallel continues through the lifetime of the company. In addition to expanding the relationships with the educational community, the company must cater its advertising to that market. What is now called targeted and retargeted marketing is essential to fitting the advertising message to the recipient. Retargeted marketing expresses some of the same principles as adaptive learning. Programs track the actions of the user and collect metrics which are considered effective measures of their interests. Specific marketing messages are then delivered as guided by the collected information. Retargeting also makes use of spaced repetition to expose the potential customer to brand information multiple times. The success rate of retargeted marketing can be taken as a confirmation of the approach. Perhaps this success can even cast a favorable light on similar approaches for education.

Rigidity and Resistance

Neither software engineering nor instruction are physical products. Both are inherently flexible in their application and delivery. However, both are surrounded by processes, procedures, and bureaucracies that generate rigidity.

Software engineering practice is likely more flexible because it is newer, and until recently was not a major focus of public concern. Education is more foundational to society and has been, appropriately, a focus of public policy for centuries. This entanglement with public policy sets several, sometimes inconsistent, priorities and makes it difficult to obtain funding for best practices while political and marketing forces push in other directions. Education, though, has the advantage that by its very nature it focuses on presenting new ideas and integrating them into the audience's mental model.

Even the best ideas in the best environment may be slow to gain traction. This is compounded in the fields we consider by tremendous ranges in the effort required and potential profits for different approaches. For example, perhaps the most widespread edTech application is online testing. A testing framework is applicable across multiple grades and subjects, and quite frankly does not require any great innovation to develop. With limited effort, the returns are significant.

This contrasts sharply with content creation. Content creation is significantly more difficult, especially for more advanced material. To be effective, it must be targeted to a specific level of a specific topic. The effort to create it is greater and the addressable market, hence the possible profitability, is limited.

The MOOC is perhaps a middle ground, but even it makes only limited use of technology, with most of the online content in the form of video clips with limited interactive content.

Changing thought patterns and work styles to make the best use of advancing technology is difficult. Bret Victor gave an excellent talk on innovations that didn't catch on in computer science, despite their promise. I close this section with a quote from that talk that applies equally to teaching and technology.

It's easy to think that technology is always getting better because of Moor's Law, because computers are always getting more capable but ideas that require people to unlearn what they have learned and think in new ways there is often an enormous amount of resistance.

Silos are common

The concept of addressing material from multiple viewpoints, across multiple courses, is gaining traction. It has even found its way into the CCSS, but there is a long way to go. For example, history and science curricula are rarely developed in concert. However, in the real world these topics do not play out in isolation. Each strongly interacts with and depends on the other as they evolve. Curricula that acknowledge this and that are developed to reflect this interdependence present a truer picture, and provide a more complete understanding to the learner.

Historically, this has not been the case, and it remains rare. Human instinct is to scope out, mark, and protect territory1. This is particularly true for an interloper from another department, say science, getting involved in the definition of history curricula.

An often overlooked aspect of the software engineering evolution is that it owes a great deal to W. Edwards Deming. It is worth it to gain at least a passing familiarity with his work, especially his 14 points. Point 9, break down barriers, directly refers to the need for disparate groups to work together for the benefit of the system as a whole. To break down silos and gain advantage through cooperation.

This strengthens the parallel that I draw between teaching and technology. The technology sector was able to extract relevant core elements from Demming's work, adapt them to a new discipline, and apply them to their own work. I suggest that education follow a similar path by adapting some of these core concepts from both manufacturing and software and applying them to the education culture and process.

One critical, but often neglected, silo is that of the profession itself. Technology development is significantly strengthened by involving both the user and the decision maker in product design and development. However, this is frequently not the case - indeed, I have seen the technology side's disdain for the end user harm several products.

Similarly I have seen less than welcoming reactions to parents and other outsiders to the teaching profession getting involved in the teaching practice itself. The teaching profession is an important public trust, so the public must be seen as automatically having standing in a discussion of teaching standards and practices. This places a greater responsibility on the teaching profession for public engagement and education. The educator can educate, not only their students, but their constituents as well.

A monoculture is a close cousin to the silo, and reflects many of the same behavior patterns. Both are harmful. We can counter these tendencies by including and encouraging visionaries, innovators, and communicators within our group. The visionary seeking a path to a better future, the innovator always asking is there a better way, and the communicator drawing in participation from other groups and casting the language of each group into a form the other can understand. Or better yet, cultivate some of these skills in ourselves.

Patterns and Antipatterns

An important set of concepts and techniques in software engineering are the design patterns. Patterns can be thought of as best practices to achieve a specific goal in a specific context.

Design patterns are a powerful idea that was slow to gain traction. While the concept was introduced to architecture in 1977, and to software in 1987, it did not gain significant traction in software for almost another decade. The design patterns work did not introduce the design patterns themselves, but cataloged and established common names for wide spread best practices and grouped them into categories. This common vocabulary and taxonomy allows easier communication and, interestingly, for more effective education.

There has been some movement toward a design pattern catalog for instruction. A truly organically derived education pattern catalog and its attendant vocabulary and taxonomy of concepts would be a great value to the profession.

A closely related concept is the antipattern. These are common practices that do not represent effective solutions. Antipatterns frequently appear reasonable, but experience has proven otherwise. Interestingly, the antipattern concept has been extended to management, and includes the silo as an organizational antipattern.

The antipattern concept may be catching on in education in the form of recognizing popular myths. But it can be carried much further.

What's Really Important

In the 1990's software engineering was flooded with potential solutions to developer innovation and productivity. So many that it was necessary to take a step back and summarize what was really important and what was not. The Agile Manifesto and its accompanying 12 principles, is a collection of fundamental values for the process of creating high quality software.

The Agile Manifesto does not specify any specific development methodology. It provides a set of values and principles that any development methodology should follow. Indeed, it explicitly promotes adoption of methods and processes that work best with a specific team in a specific domain.

This is reflected in education with the need to adjust instructional techniques to meet the needs of a particular student or class in learning a specific topic. I recently encountered The Paideia Principles, which take a similar approach to teaching, providing a set of principles rather specifying a fixed methodology. A generally applicable set of principles would probably be generated and vetted by a larger group.

Such a set of principles also provides a powerful tool for communications, focuses public policy debate, and encourages freedom for individual schools and teachers to manifest the principles as best suits their environment.

1) Stack, Robert David, Human Territoriality: Its Theory and History, PP 169-215, Cambridge University Press, 1986