Wednesday, October 15, 2014

Why We Stick With Slides and Other Archaic Practices

Despite a vast sea of problems with slide presentations (aka PowerPoint), we continue their use.

Rather than moving away from slides, this has spawned a whole industry on improving slide presentations. The N.Y. Times article raises the point that PowerPoint is deeply embedded in the military culture. This clearly extends beyond military culture, PowerPoint is deeply embedded in the educational culture as well. How did we get here?

In certain contexts slides are highly effective. Most of us have had positive experiences with them used in an educational context, building an association between slides and effective presentation. Further, the presenters included faculty in positions of authority, building an association between slides and success, which we want to emulate. This effect is magnified by the Microsoft marketing machine, which has placed into the field, according to Tufte, several hundred million copies of Microsoft PowerPoint turning out trillions of slides each year.

So we have something that is proven to be successful, albeit in a limited context, and it is backed by a large, over-hyping, marketing machine. It is no surprise that it has obtained a massive market presence, and that an entire ecosystem has grown up around it. This ecosystem includes significant business interests and self proclaimed experts who are motivated to expand their reach as far as possible.

This brings us to an important point. Many of the people who produce this static content think they are doing a good job and some actively oppose more advanced techniques.

In reality, the effective contexts for slides are generally associated with situations where the presenter is the center of attention. The slide is limited to the role of a visual aid and the dynamism and interactivity is provided by the speaker.

Modern digital content provides an entirely new context that allows, even demands, an entirely new approach. In this new context the student interacts directly with the content with no intermediary. In this new context the interactivity and the adaptation to the student is provided by the content itself.

Wednesday, February 26, 2014

Common Ground In Teaching And Tech

Teaching and tech have much more in common than a few letters. Two decades ago American software engineering was perceived as falling behind the overseas competition, just as education is today. The software industry responded with innovation in practice, governance, management, and culture. The software industry was not the first to confront this, as the manufacturing sector had faced, and continues to face, a similar challenge. Indeed, the software industry was able to leverage significant ideas and concepts from manufacturing and repurpose them to improving their own practices. Education now has the opportunity to leverage the lessons from both technology and manufacturing.

Widespread Misunderstandings

When I refer to software engineering I do not mean simply writing code, but the entire process built around understanding users, their mental models, their needs, their expectations, and translating these into high quality products of value to the customer.

Teaching as well is frequently misunderstood. Effective teaching is far more subtle and nuanced than simply presenting information to students. The best teachers build relationships with their students, find effective ways to make the content relevant and interesting, and to the extent possible adjust the content to the specific needs of the students, some even engage the parents. And that's on a good day.

The understanding from outside either profession is significantly different from that of actual practitioners. The public perception of these professions is deeply flawed. Even more damaging is that the misunderstanding sometimes extends into the management and policy making circles. Importantly for the reader, many technologists and educators do not truly understand one another. My intent is to foster communication and cooperation by pointing out strong parallels between these two schools of thought.

Impetus for Change

Both professions have confronted, and are confronting, a crisis of confidence. For both teaching and tech, one of the significant concerns is that America is falling behind. For teaching we see this almost daily in the media. Software engineering has a head start confronting this issue. In 1993 Edward Yourdon portrayed the Decline and Fall of the American Programmer relative to their international competition.

In both cases this precipitated collections of procedural and technological solutions. For software this included the Rational Unified Process and several Computer Aided Software Engineering (CASE) tools, which turned out not to be a panacea. Indeed, the misapplication, the overly rigid application, of the Rational Process was a factor in the raise of agile methods.

The important aspect is that the recognition of an issue lead to a period of experimentation and trial, and only after this period did more fruitful approaches emerge. Agile methodologies themselves underwent a Cambrian explosion of sorts, followed by a Darwinian evolution leaving behind more fit solutions, and more openness to change.

Instruction faces a similar situation to software engineering after Yourdon published his 1993 book. There is a recognition that there is a problem, and many solutions are materializing. With MOOCS, apps, web apps, videos, CCSS, and standardized testing all making a showing, it is unclear which approaches will provide the greatest advantage in the future. Each approach has its champions, but clearly none of them stand alone as a complete solution. This plethora of solutions reflects varying perceptions of exactly what the issues are with education. Perhaps, though, something can be borrowed from the software engineering venue and teaching can embrace change and data driven adaptation. Moreover, in software engineering this shift to agile was a grass roots movement.

Location Independence

Both education and software are rooted in the transfer of information. In most cases there is no need for a physical presence or for the transfer of material goods. This reduces the importance of location and increases the importance of interactions. Indeed, the realization is coming to fruition that face to face time is better utilized by focusing on personal interactions during office hours or in class time.

A software engineer can collaborate with, and market to anywhere on the planet without leaving the neighborhood coffee shop. Similarly, many universities now have a global reach. This globalization directly effects the consumer as well. Online web services and online instruction on almost any topic can be accessed from almost anywhere. Where previously I might visit an accountant to process my taxes, now I can easily prepare my taxes online with a well known and trusted service. Generally though, determining which products and resources represent a true value is difficult. While some resources are undoubtedly first class, others fall far short.

Global access to information and services reduces the value of memorization, and significantly increases the value of critical thinking to ascertain which are truly useful, and sometimes whether online information is correct.

Assessment and tests

Assessment easily warrants its own discussion in either field. We limit ourselves to outlining a few relationships between assessment in software and in education. Continuous testing is a generally accepted part of modern software development. There is a small school that feels that the testing should drive the development process, and some people are simply confused about the difference between the two.

Predominantly, testing is viewed as an important step in the production of quality software; as a means to an end, but not a primary goal. Considerable effort has gone into creating software testing tools and techniques. A common goal is to maximize the value of the tests while minimizing their impact on the development process. For example unit tests are designed to run quickly and in an automated fashion. Lightweight testing, or better yet adaptive learning, can play a similar role in education.

At the end of the day, software is valued by success in the marketplace, by delivering value to the customer. This is a fundamental difference between software and education, choice. The market can choose any alternative, of if the software does not address any need, the market may choose to forgo it entirely. This provides a powerful incentive to correctly assess the value and correctness of software.

Education, on the other hand, is compulsory. This, with a few exceptions, places the burden of assessment on the educational process itself, and shifts it from a question of market acceptance to one of accountability.

The public perception of testing in education is dominated by high stakes, high stress tests like final exams, midterms, SATs and thanks to recent media coverage, PISA tests. These tests capture the student's learning at a specific point in time. Tests like this are referred to as summative tests.

The true picture is much richer. Another type of assessment, formative assessment, has an entirely different emphasis. These are ongoing assessments focused on providing feedback on how effectively the student is learning. This assessment is usually a lower stakes and lower stress - perhaps not even graded - mechanism. Formative assessment is also intended to provide feedback on the effectiveness of the teaching process. At a conceptual level, this might be expected to provide a strong incentive to innovate in education. Also, well designed feedback would mitigate any risk associated with innovation by allowing a quick course correction onto a productive path.

The technology industry struggles with assessment as well. Particularly telling is this interview with Laszlo Bock, a senior vice president at Google.

We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship.

This raises fundamental questions about the effectiveness of the interview, aka assessment, process in the software industry as a whole.

One of the things we've seen from all our data crunching is that G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there's a slight correlation.

He goes on to explain that the academic environment is highly specialized, and that success within that context does not imply that someone will be successful within another environment or context. Both teaching and tech have a critical need to access ability to put knowledge to work in real world contexts.

Iterative Improvement

The application of iterations is perhaps the most subtle of the points raised here. The software engineering world uses iterations in the development process, and also iteratively improves the process itself. Both the product and process iterations are driven by assessments, participant feedback, and comparisons against organizational goals.

The current state of education is reminiscent of software engineering before the acceptance of agile iterative methods. At that point software development was dominated by inflexible methods characterized by large up front planning.

The famous quote from German strategist von Moltke "No Battle Plan Survives Contact With the Enemy" dramatically captures the need to adjust any plan in the field to adapt to real world conditions. Software engineering has embraced this need for continual adaption and seen significant increases in quality and productivity.

These adaptations to the situation on the ground are driven by the people on the ground. This puts a greater freedom and greater responsibility into the hands of the practitioner with boots on the ground.

Education would profit as well from this shift in emphasis. From up front planning and set curricula to general goals and continual adaption to the needs of a particular situation driven by the teacher's own judgment.

Personalizing the Experience

Education, by its very nature, focuses on presenting new ideas and integrating them into the audience's mental model. The most successful approach caters the educational experience to the individual student. The teacher builds a relationship with the student and adjusts the content to address their individual interests and well as to expand on their strengths and address their weaknesses. This path is proven to be effective and well received by students and teachers alike.

The application of personalized learning is limited by its labor intensive nature. However, computers can ease the labor requirement in certain cases. The most direct application is to provide asynchronous communications through email and social media. The interaction and feedback is still timely, but not always immediate and face to face.

Well designed software provides multiple modes and multiple paths for the learner to explore different scenarios. This encourages the learner to freely choose their path through content exercising their interests, and filling in gaps in their understanding.

Several companies carry the concept in another direction with adaptive learning. Adaptive learning assesses the state of the students knowledge and further instruction is focused on their weakest areas. Extensive effort has been put into allowing automatic identification of the learners problem areas. Indicators include scores on exams, the time taken to answer questions, or to read through a section, even eyetracking and body language are sometimes used.

The edtech entrepreneur is concerned with personalization from their earliest days. Almost from the outset, she will be concerned with product market fit, or how the product addresses the specific needs of the educational community, or that part of the community they are addressing. This requires building relationships with educators, administrators, and frequently with students. The entrepreneur must understand the market's interests and needs, and modify the product to fulfill those interests and needs. Sound familiar?

The parallel continues through the lifetime of the company. In addition to expanding the relationships with the educational community, the company must cater its advertising to that market. What is now called targeted and retargeted marketing is essential to fitting the advertising message to the recipient. Retargeted marketing expresses some of the same principles as adaptive learning. Programs track the actions of the user and collect metrics which are considered effective measures of their interests. Specific marketing messages are then delivered as guided by the collected information. Retargeting also makes use of spaced repetition to expose the potential customer to brand information multiple times. The success rate of retargeted marketing can be taken as a confirmation of the approach. Perhaps this success can even cast a favorable light on similar approaches for education.

Rigidity and Resistance

Neither software engineering nor instruction are physical products. Both are inherently flexible in their application and delivery. However, both are surrounded by processes, procedures, and bureaucracies that generate rigidity.

Software engineering practice is likely more flexible because it is newer, and until recently was not a major focus of public concern. Education is more foundational to society and has been, appropriately, a focus of public policy for centuries. This entanglement with public policy sets several, sometimes inconsistent, priorities and makes it difficult to obtain funding for best practices while political and marketing forces push in other directions. Education, though, has the advantage that by its very nature it focuses on presenting new ideas and integrating them into the audience's mental model.

Even the best ideas in the best environment may be slow to gain traction. This is compounded in the fields we consider by tremendous ranges in the effort required and potential profits for different approaches. For example, perhaps the most widespread edTech application is online testing. A testing framework is applicable across multiple grades and subjects, and quite frankly does not require any great innovation to develop. With limited effort, the returns are significant.

This contrasts sharply with content creation. Content creation is significantly more difficult, especially for more advanced material. To be effective, it must be targeted to a specific level of a specific topic. The effort to create it is greater and the addressable market, hence the possible profitability, is limited.

The MOOC is perhaps a middle ground, but even it makes only limited use of technology, with most of the online content in the form of video clips with limited interactive content.

Changing thought patterns and work styles to make the best use of advancing technology is difficult. Bret Victor gave an excellent talk on innovations that didn't catch on in computer science, despite their promise. I close this section with a quote from that talk that applies equally to teaching and technology.

It's easy to think that technology is always getting better because of Moor's Law, because computers are always getting more capable but ideas that require people to unlearn what they have learned and think in new ways there is often an enormous amount of resistance.

Silos are common

The concept of addressing material from multiple viewpoints, across multiple courses, is gaining traction. It has even found its way into the CCSS, but there is a long way to go. For example, history and science curricula are rarely developed in concert. However, in the real world these topics do not play out in isolation. Each strongly interacts with and depends on the other as they evolve. Curricula that acknowledge this and that are developed to reflect this interdependence present a truer picture, and provide a more complete understanding to the learner.

Historically, this has not been the case, and it remains rare. Human instinct is to scope out, mark, and protect territory1. This is particularly true for an interloper from another department, say science, getting involved in the definition of history curricula.

An often overlooked aspect of the software engineering evolution is that it owes a great deal to W. Edwards Deming. It is worth it to gain at least a passing familiarity with his work, especially his 14 points. Point 9, break down barriers, directly refers to the need for disparate groups to work together for the benefit of the system as a whole. To break down silos and gain advantage through cooperation.

This strengthens the parallel that I draw between teaching and technology. The technology sector was able to extract relevant core elements from Demming's work, adapt them to a new discipline, and apply them to their own work. I suggest that education follow a similar path by adapting some of these core concepts from both manufacturing and software and applying them to the education culture and process.

One critical, but often neglected, silo is that of the profession itself. Technology development is significantly strengthened by involving both the user and the decision maker in product design and development. However, this is frequently not the case - indeed, I have seen the technology side's disdain for the end user harm several products.

Similarly I have seen less than welcoming reactions to parents and other outsiders to the teaching profession getting involved in the teaching practice itself. The teaching profession is an important public trust, so the public must be seen as automatically having standing in a discussion of teaching standards and practices. This places a greater responsibility on the teaching profession for public engagement and education. The educator can educate, not only their students, but their constituents as well.

A monoculture is a close cousin to the silo, and reflects many of the same behavior patterns. Both are harmful. We can counter these tendencies by including and encouraging visionaries, innovators, and communicators within our group. The visionary seeking a path to a better future, the innovator always asking is there a better way, and the communicator drawing in participation from other groups and casting the language of each group into a form the other can understand. Or better yet, cultivate some of these skills in ourselves.

Patterns and Antipatterns

An important set of concepts and techniques in software engineering are the design patterns. Patterns can be thought of as best practices to achieve a specific goal in a specific context.

Design patterns are a powerful idea that was slow to gain traction. While the concept was introduced to architecture in 1977, and to software in 1987, it did not gain significant traction in software for almost another decade. The design patterns work did not introduce the design patterns themselves, but cataloged and established common names for wide spread best practices and grouped them into categories. This common vocabulary and taxonomy allows easier communication and, interestingly, for more effective education.

There has been some movement toward a design pattern catalog for instruction. A truly organically derived education pattern catalog and its attendant vocabulary and taxonomy of concepts would be a great value to the profession.

A closely related concept is the antipattern. These are common practices that do not represent effective solutions. Antipatterns frequently appear reasonable, but experience has proven otherwise. Interestingly, the antipattern concept has been extended to management, and includes the silo as an organizational antipattern.

The antipattern concept may be catching on in education in the form of recognizing popular myths. But it can be carried much further.

What's Really Important

In the 1990's software engineering was flooded with potential solutions to developer innovation and productivity. So many that it was necessary to take a step back and summarize what was really important and what was not. The Agile Manifesto and its accompanying 12 principles, is a collection of fundamental values for the process of creating high quality software.

The Agile Manifesto does not specify any specific development methodology. It provides a set of values and principles that any development methodology should follow. Indeed, it explicitly promotes adoption of methods and processes that work best with a specific team in a specific domain.

This is reflected in education with the need to adjust instructional techniques to meet the needs of a particular student or class in learning a specific topic. I recently encountered The Paideia Principles, which take a similar approach to teaching, providing a set of principles rather specifying a fixed methodology. A generally applicable set of principles would probably be generated and vetted by a larger group.

Such a set of principles also provides a powerful tool for communications, focuses public policy debate, and encourages freedom for individual schools and teachers to manifest the principles as best suits their environment.

1) Stack, Robert David, Human Territoriality: Its Theory and History, PP 169-215, Cambridge University Press, 1986

Thursday, January 2, 2014

Hands On Experimentation as a Teaching Tool

The real world is hardly cut and dried, it is full of uncertainties, probabilities, and hypotheses to be verified. Introducing this into a classroom though, is hardly straightforward. Even adults are sometimes unprepared to deal with uncertainty. And young children are even less well equipped to handle uncertainty.

The Common Core State Standards introduce Statistics & Probability in grade 6, and the Next Generation Science Standards introduce the concept of measurement error and multiple measurements at about the same time.

Laboratory experiments provide a strong example of measurement error and probability and their interaction with other subjects. Laboratory experiments can be introduced just after probability and measurement error.

Hands on work, laboratory work, practical work, it has many names. It has even more benefits. Perhaps so many benefits that it is sometimes difficult to prioritize them for a given situation1,2. Further many of these benefits are distinct from standard classroom instruction and as a result at times "outcome measures consisted almost exclusively of paper and pencil achievement tests that were often poorly linked to the laboratory activities."

Not only do we need to be careful in constructing the laboratory component of our instructional content, we have to be careful how we access its effectiveness. Designing each of these requires a familiarity with scientific experimentation and the scientific method.

We have talked about the contextual sensitivity of knowledge, and the value of presenting the same material from multiple viewpoints over time. The laboratory experiment exercises both of these principles.

The first hand exposure to scientific principles provides a new path to insight on classroom material. At least as importantly, it necessitates dealing with real world issues such as what it means to validate or invalidate a hypothesis, dealing with experimental data, presenting experimental results, and drawing conclusions from experimental results. Entire books have been written on these topics 5,6.

Perhaps the most important of these, and one of the most difficult to grasp, is the need for experimental verification of reality. This concept lies at the heart of the scientific method, a deep understanding of which generates much more confidence in science and helps in the ability to differentiate legitimate from illegitimate claims both inside and outside of science.

An effective lab will be designed to illustrate and explore concepts from the other components of the coursework, better still if they also relate to the broader curriculum.

How do we introduce lab work to students? Start early. This example introduces laboratory work to 7th graders. While it is highly, and perhaps overly, procedural, it does introduce lab work. For example, I would not check the student data as soon as it was gathered, I would allow the students to continue on to the analysis and conclusion, and stress the questions of whether or not the conclusion is what they expected, and is the conclusion consistent with established science. Then follow up with the question of why, or why didn't, they expect the result. While this first example may serve as an introduction to more advanced lab work, it should be clearly understood as such an introduction by both the students and teachers. Can you see additional ways that this lab could be improved?

To many students, a laboratory activity has meant manipulating equipment but not manipulating ideas. Multiple studies confirm that the frequently observed ritualistic, even 'mindless' student behaviors observed in many laboratory activities stifle students' personal engagement in decision-making in the laboratory. These kinds of activities rarely uncover students' underlying beliefs; they do not encourage students to wrestle with their prior knowledge in making sense of their experiences, and they do not encourage them to reflect on their own thinking.3

Contrast the first example with another lab where once again middle school students tackle experimental work, but this time in an almost completely unstructured format. Here the students are confronted with a real world situation, their fish are dying due to a high pH, which would spike after a few days. When the teachers who had setup they system failed to find a cause, they involved their students in investigating the mystery. The students now know this is a real world investigation. According to the article, the students reached out to experts from UW-Milwaukee, who visited the school and worked with the students. This added more realism and helped the students follow accepted scientific methodology.

'With science, it's got to be hands-on, it's got to be real world,' said Stewart [one of the teachers]. 'Students did their own research for this, there's a sense of ownership for them.'

The real world nature of this lab will be hard to duplicate, but the successful involvement of eighth graders is promising and inspiring. Involving outside experts lends additional realism. Interestingly, the local cable company, Time Warner, has a program to connect practitioners with educational programs. Perhaps because it is new, there do not seem to be many programs visible in my geographic area.

Most students are best served by a path that touches on elements from both of these examples. For example provide a clear stage and goals for the lab as in the first example, while drawing the real world relevance and involvement of the students in designing the actual actions and analysis from the second.

The importance of hands on experience and dealing with errors and uncertainty in raw data speaks loudly to the superiority of actual experimentation over simulations of experimentations. Simulations can provide reinforcement of classroom material, and have a place in our instructional repertoire. However, they can not provide the confidence in science and the scientific method that flows from hands on experiments and direct observation. Fundamentally, simulations behave the way they do because that's how we built them. They are at least a layer or two of abstraction removed from actual physical reality.

  • Learning in and from Science Laboratories: Enhancing Students' Meta-Cognition and Argumentation Skills. Avi Hofstein, Mira Kipnis, Per Kind, in Science Education Issues and Developments. 2008, Nova Science Publishers, Inc.
  • The Role of Laboratory Work in School Science: Educators' and Students' Perspectives. Dr. Ali Khalfan Al-Naqbi, Dr. Hassan H. Tairab, Journal of Faculty of Education UAEU. Year 18, Issue No. 22, 2005.
  • Learning and Teaching in the School Science Laboratory: An Analysis of Research, Theory, and Practice. Vincent N. Lunetta, Avi Hofstein, Michael P. Clough, Handbook of Research on Science Education, 2007, Lawrence Erlbaum Associates, Inc.
  • The Role of the Laboratory in Science Teaching: Neglected Aspects of Research, Avi Hofstein, Vincent N. Lunetta, Review of Educational Research, Summer, 1982, Vol. 52, No. 2, Pp 201-217.
  • Statistical Treatment of Experimental Data, Hugh D. Young, 1962, McGraw-Hill Book Company, Inc. Company.
  • Beautiful Evidence, Edward R. Tufte, 2006, Graphics Pr.