The Unique Value of Open Educational Resources

Open and Education are concepts that fit together and compliment each other incredibly well.

Open and Education are concepts that fit together and compliment each other incredibly well. We will look at one aspect, the overlap of open source and education that is open educational resources (OER). For something that is such a natural fit, OER is comparatively unknown and underutilized.

@katyamuses what is #OER?

— Tabitha Devore (@TeacherTabitha) November 13, 2014

@TeacherTabitha @katyamuses #OER r open educ resources, materials that r openly lic so that they can b freely reused + redist #EdTechBridge

— K12 Open Ed (@k12opened) November 13, 2014

@k12opened @TeacherTabitha @katyamuses #OER is much more than free content. #EdTechBridge

— Alex Kluge (@AlexVKluge) November 13, 2014

There is a well established, oft cited definition for OER.

teaching, learning and research materials in any medium, digital or otherwise, that reside in the public domain or have been released under an open license that permits no-cost access, use, adaptation and redistribution by others with no or limited restrictions.

World Open Educational Resources (OER) Congress

This permissive licensing is the minimum requirement for OER. What is the "much more" that I referred to in the tweet? What more can we do to increase the value of OER, both for the educational community and for the producers of the content? Can we build something akin to a rubric for accessing the value of an open educational project and how well we put that project to use?

The value and perception of OER strongly depends on perspective. OER presents different value propositions to educational institutions that consume them, students that consume them, educational institutions that produce them, and companies that produce them. And perhaps the most important value proposition, and definitely the least appreciated, the value to the student who produces content.

Licensing

We have established the minimum requirement of a permissive license. These licenses take many forms, ranging to completely unfettered, public domain, content to the GNU AGPL V3 and the Creative Commons noncommercial noderivatives license. Arguably, this last example is so restrictive that it is not an open license.

As soon as a work is created, it is protected by copyright. As soon as a story is written, a painting made, a photograph taken, or a film shot, it is copyrighted. This means that the creator, or their employer, owns the intellectual property rights for the original work. Like other forms of property, the owner can sell those rights, trade them, or simply give them away.

This is why there are so many different open source licenses, and so many uses of the phrase open educational resources. The creator of the original intellectual property can follow almost any path they want to share, trade, or sell it. The license is the terms that the creator attaches to the use of their creation.

Luckily, one of the most popular open license sets, Creative Commons, has a very good explanation of their licenses. One of the reasons for their popularity is the simplicity and straight forward nature of their licenses, and their focus on content, not software. If you are looking for an open source license for your content, this is a great place to start. GitHub, a large open source repository, also provides a guide for choosing an open license, this one targeted more to the software development community.

An interesting side effect of attaching a permissive license to your work is that you are immediately conscious of the need to design the project for uses and interests beyond your own. It's still early and we have already touched on constructionism.

Cost

The cost of textbooks, and education in general, is increasing dramatically. This is placing an increasing burden on students and school districts.

Textbook prices are increasing dramatically.

OERs promise to displace much of this cost. As tempting as it may be, it is important that schools and school districts not simply fixate on the cost savings. The most promising path includes a feedback loop where the the consumers of OER also invest in its improvement. Redirect some of the cost savings to resources for developing and obtaining expertise around the use of OER, and over a slightly longer term develop the ability to contribute to the creative process. Repeating this feedback loop over even a fraction of the 3 million teachers in the US will create a powerful force behind the improvement and adoption of open educational resources.

Value

A4: There are endless #OERs out there! As a teacher you need to be willing to investigate & find what works for you! #EdTechBridge #uneedu

— Emilie (@EmilieNewbern) November 20, 2014

Just because it is free and open does not make it a good fit for education, moreover, it does not make it a good fit for your classroom. Conversely, even if the content is not targeted to education, you may find it a great fit to your class. So what are some of the things we need to look for the evaluate the usefulness and fit of a particular open educational resource?

The 5Rs

Ideally, OER provides all 5 of these values, commonly refreeed to as the 5Rs.

• Retain: Users have the right to make, archive, and "own" copies of the content;
• Reuse: Content can be reused in its unaltered form;
• Revise: Content can be adapted, adjusted, modified or altered;
• Remix: The original or revised content can be combined with other content to create something new;
• Redistribute: Copies of the content can be shared with others in its original, revised or remixed form.

Retention can have a surprisingly strong effect, especially for students who normally can not markup and retain their text from year to year.

Correctness

A common and basic criteria for any resource, educational or not, open or not, is correctness. Instructional material, no matter how well written, well designed, or well marketed, is worse than useless if it is not correct. A messaging utility is a poor investment if it loses data or misdirects messages, no matter the cost.

Open source materials have a clear advantage here. The inclusive, community driven, nature of open source makes it more likely that the community will provide constructive feedback. A well organized open source project will invite the community to contribute updates and corrections directly. Such constructive community involvement is important for the health of open source projects.

Pedagogy

You can not talk about educational resources without talking about pedagogy. However, open resources can be used directly to teach, they can be used as resources in a larger project, or they can themselves be the target of an educational project. The evaluation of any resource will depend on how it is integrated into your class.

For direct teaching, the open resources often present the content as a whole, as with MIT's OpenCourseWare, Boundless or OpenStax College. For such complete content, we can ask questions such as:

• Are there clear goals and expected results?
• Does the product reflect an understanding of how people learn?
• Is the level of the content appropriate to my class?

Many open resources encourage you to adapt and mix them with your own content and sometimes other content. When open content is mixed into existing content, evaluation is based on fundamentally different questions. Now we must look first at how the new content compliments the existing content.

• Does it fill in a gap?
• Does it present the same material through a different lens?
• And most importantly, does it integrate with your content and with your presentation style?

Examples of content that can be incorporated into a larger whole include the Smithsonian 3D collection, Web Gallery of Art, and the author's own mathematics and physics visualizations.

Another interesting and valuable set of open resources for education is open data. Google has an excellent collection of public datasets, including extensive tools to visualize and analyze the data. CERN has recently made available extensive particle physics data along with software for its analysis. US census data is available including some wonderful visualizations, one of which details college majors and eventual careers. Data such as this can be used as an investigative tool by students, and in the case of the census data even to inform policy for the educator.

It is important to select data from high quality sources. It is easy to find data that is reflects bias on the part of the researcher or collection agency.

A far richer set of options is available for #OER than for a packaged end product because the open product is more open to mixing into the complete solution.

We will see a little bit later how a well organized open source project is more likely to get this, and many other factors, right than other approaches.

Myths

There are also a number of myths surrounding open source material. This results in additional effort to rework or reinvent already existing solutions. Experience has shown that where it is reasonable, it is better to utilize an existing solution and implement customizations and additions where necessary.

Frequently encountered arguments and fears against OER from the myths and obstacles paper cited above.

OER Myths	Prevalence
Not professional	61%
Difficult to find/use	56%
Time-consuming for teachers	50%
Lacking support	39%
Lack of business models	39%
Unreliable	33%
Unsustainable	31%
Other	14%
Can lead to centralization	3%
Unfair practice	3%
Disruptive	3%

We've seen some myths, how about some facts from actual studies?

Some actual facts about open educational resources.

OER Facts	Prevalence
increased interest	60.1%
Increased enthusiasm	59%
interested in a wider range of subjects	50%
Increased independence and self-reliance	48%
teachers share resources online	43.3%
teachers publish resources with a Creative Commons license	12.4%
OpenStax saved students over $3.7M in 2013

To be honest, using OER does require more effort, deeper thought, and a greater level of expertise than more traditional approaches. But we are all in the 90th percentile, right? Further, if we distribute this effort among us all, and form a strong ecosystem, the effort will be reduced dramatically. This is one of the main goals for this writeup. To get you involved not only in using, but in contributing to the OER ecosystem.

Usability

It is almost a tradition that open source products have poor usability. The target audience has often made the tradeoff of cost vs usability and polish. For educational content though, this is largely an unacceptable tradeoff. Wrestling with the design or interface will quickly distract and alienate the learner.

Much progress has been made on this front over the past several years, with open source products matching and sometime eclipsing their shrink wrapped rivals.

Accessibility

Accessibility is a close cousin to usability. Instruction, no matter how amazing it might be, is of little value if the learner can not access it.

So, what are the big barriers of accessibility. The traditional set of barriers are physical limitations such as blindness or hearing loss. Addressing these requires both intent and specialized knowledge.

Once again, feedback is important. A development community will almost never have complete coverage of the spectrum of hardware and software in use. Systems and combinations where things misbehave are important to know. Further, accessibility can be a strong differentiator in online reviews of content.

There are some great resources for implementing and evaluating accessibility.

New with the advent of technology is the so called digital divide. We can quickly recognize the opportunity for both lower cost and higher quality resources. However, a closer look quickly reveals that to take advantage of these opportunities, such as replacing printed commercial textbooks with open digital content, required widespread digital devices, connectivity and expertise. These things are not equally distributed. So, perversely, free resources can actually increase the digital divide.

Technology is such an important aspect of modern education that addressing the digital divide is a priority for local, state and federal governments.

The BYOD trend is another two edged sword. Students using their own devices to access and work with open educational resources is a powerful concept. It moves learning beyond specific devices and content provided by the school into a more real world context. This breaks the association of learning is something that happens in the classroom and transform it into a lifelong process that happens everywhere. However, it brings the unequal access to technology into sharp focus.

Even when students can keep OER, with their own notes and additions, after the course is over, it is of limited use if they don't have their own device. This is another example of how the educational, and OER, ecosystem is complex and layered. But that just means that there are many avenues to contribute.

Availability & Findability

Even the best content has limited value if you can't find it. This means that you have to know to look for it, and there has to be an easy path to find it. This is the first time we will see a significant business value behind OER as the open content becomes part of the sales funnel. OER can be a great way to quickly gain market awareness, and, as we will see, to build a strong user community.

Between two-thirds and three-quarters of all faculty classify themselves as unaware on OER. This shows clearly that one of the most important thresholds we need to cross is one of simple awareness.

Google Trends gives us an interesting picture of interest in a few OER topics, and how it has changed over time.

PhET stands out with its strong increase in search queries and brand identity.

Of these terms, the most striking thing is that only PhET shows a strong upward trend. Interest in Creative Commons has a gentle raise until late 2011, then shows a slight decline over time. Even Khan Academy enjoyed a spike from a March 2012 60 Minutes story but was flat afterwards.

It's also interesting to look at the web traffic rankings for some of the largest OER collections. We throw in Pearson, a high profile, but not open, education company for comparison. The open education sites have respectable numbers with the well connected Khan Academy leading the traffic race. Apparently marketing is effective for OER. No one organization has a commanding presence in the OER marketplace leaving considerable room for growth and evolution.

Kahn Academy and PhET show how to build an audience with OER.

Site	Global Traffic Rank	US Traffic Rank
Boundless Learning	5,219	2,727
Connexions	50,515	18,301
Creative Commons	5,492	4,282
Khan Academy	632	167
OER Commons	114,492	29,952
OpenStax	75,181	18,985
Pearson	5,459	1,187
PhET	3,391	1,378

The table shows a snapshot of traffic for select OER sites. We clearly see these sites have a slightly higher proportion of traffic in the US than the rest of the world. Also interesting is how that traffic is changing over time. They are showing significant growth over time, with OpenStax as an extreme case.

The traffic for OpenStax shows a massive increase over the past year, likely due to expanded use of their textbooks.

Comparing traffic statistics with search trends leads to some interesting conclusions. While clearly dominaning traffic, Khan Academy is far from the most active search term. PhET, which dominates search, is closer to the mean for traffic.

Clearly the principle driver for traffic to many educational sites is not brand search. Indeed, for the author, virtually no traffic comes from beand driven search. Almost all traffic comes from content driven subject matter search. Reputation and referrals from faculty and students also has a significant impact on overall traffic.

More to come soon

Why We Stick With Slides and Other Archaic Practices

Despite a vast sea of problems with slide presentations (aka PowerPoint), we continue their use.

• Death By PowerPoint
• We Have Met the Enemy and He Is PowerPoint
• The Cognitive Style Of PowerPoint: Pitching Out Corrupts Within

Rather than moving away from slides, this has spawned a whole industry on improving slide presentations. The N.Y. Times article raises the point that PowerPoint is deeply embedded in the military culture. This clearly extends beyond military culture, PowerPoint is deeply embedded in the educational culture as well. How did we get here?

In certain contexts slides are highly effective. Most of us have had positive experiences with them used in an educational context, building an association between slides and effective presentation. Further, the presenters included faculty in positions of authority, building an association between slides and success, which we want to emulate. This effect is magnified by the Microsoft marketing machine, which has placed into the field, according to Tufte, several hundred million copies of Microsoft PowerPoint turning out trillions of slides each year.

So we have something that is proven to be successful, albeit in a limited context, and it is backed by a large, over-hyping, marketing machine. It is no surprise that it has obtained a massive market presence, and that an entire ecosystem has grown up around it. This ecosystem includes significant business interests and self proclaimed experts who are motivated to expand their reach as far as possible.

This brings us to an important point. Many of the people who produce this static content think they are doing a good job and some actively oppose more advanced techniques.

In reality, the effective contexts for slides are generally associated with situations where the presenter is the center of attention. The slide is limited to the role of a visual aid and the dynamism and interactivity is provided by the speaker.

Modern digital content provides an entirely new context that allows, even demands, an entirely new approach. In this new context the student interacts directly with the content with no intermediary. In this new context the interactivity and the adaptation to the student is provided by the content itself.

Common Ground In Teaching And Tech

Teaching and Tech have much more in common than a few letters.

Teaching and tech have much more in common than a few letters. Two decades ago American software engineering was perceived as falling behind the overseas competition, just as education is today. The software industry responded with innovation in practice, governance, management, and culture. The software industry was not the first to confront this, as the manufacturing sector had faced, and continues to face, a similar challenge. Indeed, the software industry was able to leverage significant ideas and concepts from manufacturing and repurpose them to improving their own practices. Education now has the opportunity to leverage the lessons from both technology and manufacturing.

Widespread Misunderstandings

The public perception of these professions is deeply flawed.

When I refer to software engineering I do not mean simply writing code, but the entire process built around understanding users, their mental models, their needs, their expectations, and translating these into high quality products of value to the customer.

Teaching as well is frequently misunderstood. Effective teaching is far more subtle and nuanced than simply presenting information to students. The best teachers build relationships with their students, find effective ways to make the content relevant and interesting, and to the extent possible adjust the content to the specific needs of the students, some even engage the parents. And that's on a good day.

The understanding from outside either profession is significantly different from that of actual practitioners. The public perception of these professions is deeply flawed. Even more damaging is that the misunderstanding sometimes extends into the management and policy making circles. Importantly for the reader, many technologists and educators do not truly understand one another. My intent is to foster communication and cooperation by pointing out strong parallels between these two schools of thought.

Impetus for Change

One of the significant concerns is that America is falling behind.

Both professions have confronted, and are confronting, a crisis of confidence. For both teaching and tech, one of the significant concerns is that America is falling behind. For teaching we see this almost daily in the media. Software engineering has a head start confronting this issue. In 1993 Edward Yourdon portrayed the Decline and Fall of the American Programmer relative to their international competition.

In both cases this precipitated collections of procedural and technological solutions. For software this included the Rational Unified Process and several Computer Aided Software Engineering (CASE) tools, which turned out not to be a panacea. Indeed, the misapplication, the overly rigid application, of the Rational Process was a factor in the raise of agile methods.

The important aspect is that the recognition of an issue lead to a period of experimentation and trial, and only after this period did more fruitful approaches emerge. Agile methodologies themselves underwent a Cambrian explosion of sorts, followed by a Darwinian evolution leaving behind more fit solutions, and more openness to change.

Instruction faces a similar situation to software engineering after Yourdon published his 1993 book. There is a recognition that there is a problem, and many solutions are materializing. With MOOCS, apps, web apps, videos, CCSS, and standardized testing all making a showing, it is unclear which approaches will provide the greatest advantage in the future. Each approach has its champions, but clearly none of them stand alone as a complete solution. This plethora of solutions reflects varying perceptions of exactly what the issues are with education. Perhaps, though, something can be borrowed from the software engineering venue and teaching can embrace change and data driven adaptation. Moreover, in software engineering this shift to agile was a grass roots movement.

Location Independence

In most cases there is no need for a physical presence.

Both education and software are rooted in the transfer of information. In most cases there is no need for a physical presence or for the transfer of material goods. This reduces the importance of location and increases the importance of interactions. Indeed, the realization is coming to fruition that face to face time is better utilized by focusing on personal interactions during office hours or in class time.

A software engineer can collaborate with, and market to anywhere on the planet without leaving the neighborhood coffee shop. Similarly, many universities now have a global reach. This globalization directly effects the consumer as well. Online web services and online instruction on almost any topic can be accessed from almost anywhere. Where previously I might visit an accountant to process my taxes, now I can easily prepare my taxes online with a well known and trusted service. Generally though, determining which products and resources represent a true value is difficult. While some resources are undoubtedly first class, others fall far short.

Global access to information and services reduces the value of memorization, and significantly increases the value of critical thinking to ascertain which are truly useful, and sometimes whether online information is correct.

Assessment and tests

Testing is a means to an end, not a primary goal.

Assessment easily warrants its own discussion in either field. We limit ourselves to outlining a few relationships between assessment in software and in education. Continuous testing is a generally accepted part of modern software development. There is a small school that feels that the testing should drive the development process, and some people are simply confused about the difference between the two.

Predominantly, testing is viewed as an important step in the production of quality software; as a means to an end, but not a primary goal. Considerable effort has gone into creating software testing tools and techniques. A common goal is to maximize the value of the tests while minimizing their impact on the development process. For example unit tests are designed to run quickly and in an automated fashion. Lightweight testing, or better yet adaptive learning, can play a similar role in education.

At the end of the day, software is valued by success in the marketplace, by delivering value to the customer. This is a fundamental difference between software and education, choice. The market can choose any alternative, of if the software does not address any need, the market may choose to forgo it entirely. This provides a powerful incentive to correctly assess the value and correctness of software.

Education, on the other hand, is compulsory. This, with a few exceptions, places the burden of assessment on the educational process itself, and shifts it from a question of market acceptance to one of accountability.

The public perception of testing in education is dominated by high stakes, high stress tests like final exams, midterms, SATs and thanks to recent media coverage, PISA tests. These tests capture the student's learning at a specific point in time. Tests like this are referred to as summative tests.

The true picture is much richer. Another type of assessment, formative assessment, has an entirely different emphasis. These are ongoing assessments focused on providing feedback on how effectively the student is learning. This assessment is usually a lower stakes and lower stress - perhaps not even graded - mechanism. Formative assessment is also intended to provide feedback on the effectiveness of the teaching process. At a conceptual level, this might be expected to provide a strong incentive to innovate in education. Also, well designed feedback would mitigate any risk associated with innovation by allowing a quick course correction onto a productive path.

The technology industry struggles with assessment as well. Particularly telling is this interview with Laszlo Bock, a senior vice president at Google.

We looked at tens of thousands of interviews, and everyone who had done the interviews and what they scored the candidate, and how that person ultimately performed in their job. We found zero relationship.

This raises fundamental questions about the effectiveness of the interview, aka assessment, process in the software industry as a whole.

One of the things we've seen from all our data crunching is that G.P.A.'s are worthless as a criteria for hiring, and test scores are worthless — no correlation at all except for brand-new college grads, where there's a slight correlation.

He goes on to explain that the academic environment is highly specialized, and that success within that context does not imply that someone will be successful within another environment or context. Both teaching and tech have a critical need to access ability to put knowledge to work in real world contexts.

Iterative Improvement

The application of iterations is perhaps the most subtle of the points raised here. The software engineering world uses iterations in the development process, and also iteratively improves the process itself. Both the product and process iterations are driven by assessments, participant feedback, and comparisons against organizational goals.

The current state of education is reminiscent of software engineering before the acceptance of agile iterative methods. At that point software development was dominated by inflexible methods characterized by large up front planning.

The famous quote from German strategist von Moltke "No Battle Plan Survives Contact With the Enemy" dramatically captures the need to adjust any plan in the field to adapt to real world conditions. Software engineering has embraced this need for continual adaption and seen significant increases in quality and productivity.

These adaptations to the situation on the ground are driven by the people on the ground. This puts a greater freedom and greater responsibility into the hands of the practitioner with boots on the ground.

Education would profit as well from this shift in emphasis. From up front planning and set curricula to general goals and continual adaption to the needs of a particular situation driven by the teacher's own judgment.

Personalizing the Experience

Education, by its very nature, focuses on presenting new ideas and integrating them into the audience's mental model. The most successful approach caters the educational experience to the individual student. The teacher builds a relationship with the student and adjusts the content to address their individual interests and well as to expand on their strengths and address their weaknesses. This path is proven to be effective and well received by students and teachers alike.

The application of personalized learning is limited by its labor intensive nature. However, computers can ease the labor requirement in certain cases. The most direct application is to provide asynchronous communications through email and social media. The interaction and feedback is still timely, but not always immediate and face to face.

Well designed software provides multiple modes and multiple paths for the learner to explore different scenarios. This encourages the learner to freely choose their path through content exercising their interests, and filling in gaps in their understanding.

Several companies carry the concept in another direction with adaptive learning. Adaptive learning assesses the state of the students knowledge and further instruction is focused on their weakest areas. Extensive effort has been put into allowing automatic identification of the learners problem areas. Indicators include scores on exams, the time taken to answer questions, or to read through a section, even eyetracking and body language are sometimes used.

The edtech entrepreneur is concerned with personalization from their earliest days. Almost from the outset, she will be concerned with product market fit, or how the product addresses the specific needs of the educational community, or that part of the community they are addressing. This requires building relationships with educators, administrators, and frequently with students. The entrepreneur must understand the market's interests and needs, and modify the product to fulfill those interests and needs. Sound familiar?

The parallel continues through the lifetime of the company. In addition to expanding the relationships with the educational community, the company must cater its advertising to that market. What is now called targeted and retargeted marketing is essential to fitting the advertising message to the recipient. Retargeted marketing expresses some of the same principles as adaptive learning. Programs track the actions of the user and collect metrics which are considered effective measures of their interests. Specific marketing messages are then delivered as guided by the collected information. Retargeting also makes use of spaced repetition to expose the potential customer to brand information multiple times. The success rate of retargeted marketing can be taken as a confirmation of the approach. Perhaps this success can even cast a favorable light on similar approaches for education.

Rigidity and Resistance

Teaching and tech are inherently flexible, but are surrounded by processes, procedures and bureaucracies which generate rigidity.

Neither software engineering nor instruction are physical products. Both are inherently flexible in their application and delivery. However, both are surrounded by processes, procedures, and bureaucracies that generate rigidity.

Software engineering practice is likely more flexible because it is newer, and until recently was not a major focus of public concern. Education is more foundational to society and has been, appropriately, a focus of public policy for centuries. This entanglement with public policy sets several, sometimes inconsistent, priorities and makes it difficult to obtain funding for best practices while political and marketing forces push in other directions. Education, though, has the advantage that by its very nature it focuses on presenting new ideas and integrating them into the audience's mental model.

Even the best ideas in the best environment may be slow to gain traction. This is compounded in the fields we consider by tremendous ranges in the effort required and potential profits for different approaches. For example, perhaps the most widespread edTech application is online testing. A testing framework is applicable across multiple grades and subjects, and quite frankly does not require any great innovation to develop. With limited effort, the returns are significant.

This contrasts sharply with content creation. Content creation is significantly more difficult, especially for more advanced material. To be effective, it must be targeted to a specific level of a specific topic. The effort to create it is greater and the addressable market, hence the possible profitability, is limited.

The MOOC is perhaps a middle ground, but even it makes only limited use of technology, with most of the online content in the form of video clips with limited interactive content.

Changing thought patterns and work styles to make the best use of advancing technology is difficult. Bret Victor gave an excellent talk on innovations that didn't catch on in computer science, despite their promise. I close this section with a quote from that talk that applies equally to teaching and technology.

It's easy to think that technology is always getting better because of Moor's Law, because computers are always getting more capable but ideas that require people to unlearn what they have learned and think in new ways there is often an enormous amount of resistance.

Bret Victor - The Future of Programming

Silos are common

Teaching is an important public trust so the public automatically has standing in a discussion of teaching standards and practices.

The concept of addressing material from multiple viewpoints, across multiple courses, is gaining traction. It has even found its way into the CCSS, but there is a long way to go. For example, history and science curricula are rarely developed in concert. However, in the real world these topics do not play out in isolation. Each strongly interacts with and depends on the other as they evolve. Curricula that acknowledge this and that are developed to reflect this interdependence present a truer picture, and provide a more complete understanding to the learner.

Historically, this has not been the case, and it remains rare. Human instinct is to scope out, mark, and protect territory¹. This is particularly true for an interloper from another department, say science, getting involved in the definition of history curricula.

An often overlooked aspect of the software engineering evolution is that it owes a great deal to W. Edwards Deming. It is worth it to gain at least a passing familiarity with his work, especially his 14 points. Point 9, break down barriers, directly refers to the need for disparate groups to work together for the benefit of the system as a whole. To break down silos and gain advantage through cooperation.

This strengthens the parallel that I draw between teaching and technology. The technology sector was able to extract relevant core elements from Demming's work, adapt them to a new discipline, and apply them to their own work. I suggest that education follow a similar path by adapting some of these core concepts from both manufacturing and software and applying them to the education culture and process.

One critical, but often neglected, silo is that of the profession itself. Technology development is significantly strengthened by involving both the user and the decision maker in product design and development. However, this is frequently not the case - indeed, I have seen the technology side's disdain for the end user harm several products.

Similarly I have seen less than welcoming reactions to parents and other outsiders to the teaching profession getting involved in the teaching practice itself. The teaching profession is an important public trust, so the public must be seen as automatically having standing in a discussion of teaching standards and practices. This places a greater responsibility on the teaching profession for public engagement and education. The educator can educate, not only their students, but their constituents as well.

A monoculture is a close cousin to the silo, and reflects many of the same behavior patterns. Both are harmful. We can counter these tendencies by including and encouraging visionaries, innovators, and communicators within our group. The visionary seeking a path to a better future, the innovator always asking is there a better way, and the communicator drawing in participation from other groups and casting the language of each group into a form the other can understand. Or better yet, cultivate some of these skills in ourselves.

Patterns and Antipatterns

An important set of concepts and techniques in software engineering are the design patterns. Patterns can be thought of as best practices to achieve a specific goal in a specific context.

Design patterns are a powerful idea that was slow to gain traction. While the concept was introduced to architecture in 1977, and to software in 1987, it did not gain significant traction in software for almost another decade. The design patterns work did not introduce the design patterns themselves, but cataloged and established common names for wide spread best practices and grouped them into categories. This common vocabulary and taxonomy allows easier communication and, interestingly, for more effective education.

There has been some movement toward a design pattern catalog for instruction. A truly organically derived education pattern catalog and its attendant vocabulary and taxonomy of concepts would be a great value to the profession.

A closely related concept is the antipattern. These are common practices that do not represent effective solutions. Antipatterns frequently appear reasonable, but experience has proven otherwise. Interestingly, the antipattern concept has been extended to management, and includes the silo as an organizational antipattern.

The antipattern concept may be catching on in education in the form of recognizing popular myths. But it can be carried much further.

What's Really Important

In the 1990's software engineering was flooded with potential solutions to developer innovation and productivity. So many that it was necessary to take a step back and summarize what was really important and what was not. The Agile Manifesto and its accompanying 12 principles, is a collection of fundamental values for the process of creating high quality software.

The Agile Manifesto does not specify any specific development methodology. It provides a set of values and principles that any development methodology should follow. Indeed, it explicitly promotes adoption of methods and processes that work best with a specific team in a specific domain.

This is reflected in education with the need to adjust instructional techniques to meet the needs of a particular student or class in learning a specific topic. I recently encountered The Paideia Principles, which take a similar approach to teaching, providing a set of principles rather specifying a fixed methodology. A generally applicable set of principles would probably be generated and vetted by a larger group.

Such a set of principles also provides a powerful tool for communications, focuses public policy debate, and encourages freedom for individual schools and teachers to manifest the principles as best suits their environment.

1) Stack, Robert David, Human Territoriality: Its Theory and History, PP 169-215, Cambridge University Press, 1986

Hands On Experimentation as a Teaching Tool

The real world is hardly cut and dried, it is full of uncertainties, probabilities, and hypotheses to be verified. Introducing this into a classroom though, is hardly straightforward. Even adults are sometimes unprepared to deal with uncertainty. And young children are even less well equipped to handle uncertainty.

The Common Core State Standards introduce Statistics & Probability in grade 6, and the Next Generation Science Standards introduce the concept of measurement error and multiple measurements at about the same time.

Laboratory experiments provide a strong example of measurement error and probability and their interaction with other subjects. Laboratory experiments can be introduced just after probability and measurement error.

Hands on work, laboratory work, practical work, it has many names. It has even more benefits. Perhaps so many benefits that it is sometimes difficult to prioritize them for a given situation^1,2. Further many of these benefits are distinct from standard classroom instruction and as a result at times "outcome measures consisted almost exclusively of paper and pencil achievement tests that were often poorly linked to the laboratory activities."

Not only do we need to be careful in constructing the laboratory component of our instructional content, we have to be careful how we access its effectiveness. Designing each of these requires a familiarity with scientific experimentation and the scientific method.

We have talked about the contextual sensitivity of knowledge, and the value of presenting the same material from multiple viewpoints over time. The laboratory experiment exercises both of these principles.

The first hand exposure to scientific principles provides a new path to insight on classroom material. At least as importantly, it necessitates dealing with real world issues such as what it means to validate or invalidate a hypothesis, dealing with experimental data, presenting experimental results, and drawing conclusions from experimental results. Entire books have been written on these topics ^5,6.

Perhaps the most important of these, and one of the most difficult to grasp, is the need for experimental verification of reality. This concept lies at the heart of the scientific method, a deep understanding of which generates much more confidence in science and helps in the ability to differentiate legitimate from illegitimate claims both inside and outside of science.

An effective lab will be designed to illustrate and explore concepts from the other components of the coursework, better still if they also relate to the broader curriculum.

How do we introduce lab work to students? Start early. This example introduces laboratory work to 7th graders. While it is highly, and perhaps overly, procedural, it does introduce lab work. For example, I would not check the student data as soon as it was gathered, I would allow the students to continue on to the analysis and conclusion, and stress the questions of whether or not the conclusion is what they expected, and is the conclusion consistent with established science. Then follow up with the question of why, or why didn't, they expect the result. While this first example may serve as an introduction to more advanced lab work, it should be clearly understood as such an introduction by both the students and teachers. Can you see additional ways that this lab could be improved?

To many students, a laboratory activity has meant manipulating equipment but not manipulating ideas. Multiple studies confirm that the frequently observed ritualistic, even 'mindless' student behaviors observed in many laboratory activities stifle students' personal engagement in decision-making in the laboratory. These kinds of activities rarely uncover students' underlying beliefs; they do not encourage students to wrestle with their prior knowledge in making sense of their experiences, and they do not encourage them to reflect on their own thinking.³

Contrast the first example with another lab where once again middle school students tackle experimental work, but this time in an almost completely unstructured format. Here the students are confronted with a real world situation, their fish are dying due to a high pH, which would spike after a few days. When the teachers who had setup they system failed to find a cause, they involved their students in investigating the mystery. The students now know this is a real world investigation. According to the article, the students reached out to experts from UW-Milwaukee, who visited the school and worked with the students. This added more realism and helped the students follow accepted scientific methodology.

'With science, it's got to be hands-on, it's got to be real world,' said Stewart [one of the teachers]. 'Students did their own research for this, there's a sense of ownership for them.'

The real world nature of this lab will be hard to duplicate, but the successful involvement of eighth graders is promising and inspiring. Involving outside experts lends additional realism. Interestingly, the local cable company, Time Warner, has a program to connect practitioners with educational programs. Perhaps because it is new, there do not seem to be many programs visible in my geographic area.

Most students are best served by a path that touches on elements from both of these examples. For example provide a clear stage and goals for the lab as in the first example, while drawing the real world relevance and involvement of the students in designing the actual actions and analysis from the second.

The importance of hands on experience and dealing with errors and uncertainty in raw data speaks loudly to the superiority of actual experimentation over simulations of experimentations. Simulations can provide reinforcement of classroom material, and have a place in our instructional repertoire. However, they can not provide the confidence in science and the scientific method that flows from hands on experiments and direct observation. Fundamentally, simulations behave the way they do because that's how we built them. They are at least a layer or two of abstraction removed from actual physical reality.

• Learning in and from Science Laboratories: Enhancing Students' Meta-Cognition and Argumentation Skills. Avi Hofstein, Mira Kipnis, Per Kind, in Science Education Issues and Developments. 2008, Nova Science Publishers, Inc.
• The Role of Laboratory Work in School Science: Educators' and Students' Perspectives. Dr. Ali Khalfan Al-Naqbi, Dr. Hassan H. Tairab, Journal of Faculty of Education UAEU. Year 18, Issue No. 22, 2005.
• Learning and Teaching in the School Science Laboratory: An Analysis of Research, Theory, and Practice. Vincent N. Lunetta, Avi Hofstein, Michael P. Clough, Handbook of Research on Science Education, 2007, Lawrence Erlbaum Associates, Inc.
• The Role of the Laboratory in Science Teaching: Neglected Aspects of Research, Avi Hofstein, Vincent N. Lunetta, Review of Educational Research, Summer, 1982, Vol. 52, No. 2, Pp 201-217.
• Statistical Treatment of Experimental Data, Hugh D. Young, 1962, McGraw-Hill Book Company, Inc. Company.
• Beautiful Evidence, Edward R. Tufte, 2006, Graphics Pr.