Saturday, October 29, 2011

Easy feedback

It's true for students and it's true for organizations: the better the feedback, the better the learning. Feedback is becoming increasingly effortless. For example, I use software called RescueTime that tracks what I do on my computer and shows how I've spent my time each day. I also carry a device called Fitbit that tracks physical movement and wirelessly uploads the results to a database in the cloud. That database also pulls in measurements from my scale which records daily weight measurements.

Lots of financial data is being recorded with no effort from me. Banking and credit card activity is being tracked. With a little bit of effort I pull that info into my accounting software making a lot of my accounting information simultaneously more extensive and effortless. I used to have to do this by entering each transaction into financial software. It's waaaay easier this way!

Do these things make a difference? If you want to control a parameter like weight, activity or making good use of work time, the first step is to measure it. The more effortless that can be, the more likely it will get done. Of course, it needs to be used with care, not coercion. But when used correctly it is a strong motivator.

Data and education

Lots of data has come available through assessment testing. That's great, but the more valuable data is formative. However, formative assessments are often non-electronic. So, like requiring hand entry of financial data, it's less likely to get used. What to do? Take every opportunity to go electronic.

• Give formative quizzes online so they can be automatically scored, aggregated and compared.
• Take periodic online polls to collect data on students' feelings toward their work: too easy? too hard? challenging? boring? irrelevant?
• Collect stats on the kind of errors made on assignments that must be graded subjectively. Can we mark progress?
• Collect stats on facts learned throughout a semester or year to show growth in knowledge.

Posted at 8:29 AM (permalink) 0 Comments View/Leave Comment

Monday, October 24, 2011

A/B testing for effective consequences

Ever wonder why junk mail looks the way it does? I'm looking at a letter from a credit card company that says on the envelope, "50,000 Bonus Points - That's $500 Toward Your Next Vacation." Inside there are several individually folded pieces of paper. There's variation in the fonts used. Some text is bolded. There's a P.S. There's a call to action: a form to fill out to apply for the card. A prepaid return envelope that's stamped "Priority Processing."

None of this is accidental or done on a whim. It's all been tested for effectiveness. Added features that increase the success rate (in this case, the number of people who apply for the card) has been measured. The features are there because they work.

The simplest and most common form of testing of this kind is A/B testing. Randomly divide your batch into two sets, A and B, and make them identical except for one feature, for example, set A has a P.S. and set B does not have a P.S. Then keep track of whether you got better response from set A or B. (Actually, that's an easy one: tests have shown that the P.S. is among the text most likely to be read in the letter.)

A/B testing for effective consequences

Let's say you're seeing too many tardies in your school. What is the most effective consequence? Talking to the student? Lunch detention? After school detention? Try A/B testing.

First, define "effective consequence." Let's say consequence A is deemed more effective if the students with that consequence have fewer tardies over the subsequent two weeks than students with consequence B.

Second, randomly select the consequence. Flip a coin: heads, give consequence A, tails give consequence B. Keep track of the consequence assigned to each student so you can give them the same consequence if they're tardy again.

Keep track of the outcomes. (Shameless plug: with a product like our Discipline Manager, this happens automatically).

After a while, look at the outcomes. Did consequence A perform clearly better than consequence B? For example, if talking to the student was clearly more effective than lunch detention, make it your new policy. Now, is it also better than after school detention? Run another A/B test.

What if the numbers are nearly the same? Then it doesn't matter. Either just pick one as your policy or assign either consequence as the mood strikes you. (For the sake of simplicity, you're probably better off choosing one and sticking with it.)

Why bother?

Why bother with all this? If you're intent on improving your school, you need to know if the changes you make are actual improvements or just changes. Collect some data and know.

Posted at 9:19 AM (permalink) 3 Comments View/Leave Comment
Keywords: continuous improvement

Friday, October 21, 2011

The goal of education is not knowing, it's doing

Posted at 4:53 PM (permalink) 0 Comments View/Leave Comment

Wednesday, October 19, 2011

Improvement beyond the mousetrap

If you build a better mousetrap the world will beat a path to your door.

Or so the saying (inaccurately) attributed to Emerson goes. It also reveals a bias when thinking about innovation: that innovation involves making a better product. But that's not always the case.

Apple revolutionized the music business. Some say they saved the music business when they were seeing declining CD sales and increasing (free) online music sharing. The music executives' main reaction was to start suing their customers for piracy for downloading copies of songs from their friends without paying for them. Suing your customers is rarely a good business strategy.

Apple's iTunes changed the music business. First, customers could buy music by the track rather than having to buy an entire album. Second, the tracks were inexpensive, about 99 cents. While that's not free, it's pretty cheap which made it competitive with with free (but pirated) music. Third, it was fully integrated with iPods.

iTunes was not a better mousetrap (although the iPod was). iTunes was a better means for merchandising and distribution of music. Merchandising and distribution don't have the sex appeal of product improvements, of better mousetraps. However, as the iTunes example demonstrates, it can be very important, it can save an industry.

Where will the most important innovations in education be? The primary goal is to improve student performance so we are naturally drawn to improving the "mousetrap" of education, teachers teaching in their classrooms. And there are many opportunities for such product innovations.

At the same time, just as innovation in music merchandising and distribution changed that business, changes other than classroom changes can improve education. Three foundational opportunities:

• Behavior management - while not the goal of the school, student performance will suffer without orderly classrooms and orderly school environments.
• Relationships - again, not a goal of the school, but supportive relationships between teachers and students and between staff and administration can make a big difference for student performance.
• Curriculum - although many aspects of curriculum are specified in state learning standards, it is still critically important that teaching conform to learning standards and align what is taught in first grade with what is taught in second grade and so on, to make the most efficient use of instruction time.

Improvements in these areas are the iTunes of education complementing the iPod of education, learning in classrooms.

Posted at 9:24 AM (permalink) 2 Comments View/Leave Comment
Keywords: continuous improvement

Friday, October 14, 2011

Getting organized for improvement

We all have more to do than we can actually get done, right? What do you do? Here's a summary of what I've found most effective.

• Recognize that while the immediate goal is accomplishing things on a to-do list, that's not the ultimate goal. The ultimate goal is getting the most important things done.
• Ultimate goals are difficult because they typically involve long-term projects that may never rise to a level of urgency yet they are the most important to accomplish. Extra structure is needed to accomplish them.
• Define your projects to accomplish long-term goals. Make an outline, mind map, task list or description of the project.
• Create an infrastructure for the project including a file folder for paper documents, an electronic folder for electronic documents and an email folder for emails on the subject. Why? You're going to get distracted and will need to recreate your context some time in the future. Let your infrastructure help you do that.
• Put your next step for each project on your to-do list. I learned the idea of the next step from David Allen's Getting Things Done. The next step is a simple action to take which will make a small step of incremental progress on the project. The brilliance of the next step is that because it is small and actionable, you're more likely to do it when you see it on your to-do list. That creates progress, progress creates enthusiasm and enthusiasm over the long term gets the important work done.
• Keep a journal. You need to keep learning and improving and a journal is the best way I've learned to accomplish that.
• Record what you've learned. A journal augments your memory. If you're like me, you'll need to solve the same problems again in the future and by then chances are good that you will have forgotten the solutions you've found. I use a simple Word file with dates for entry headings. I can easily search the entire journal (which now goes back years) by date or by text string.
• Reflect on what you've done. Reflection is the basis of improvement. The journal is a great repository where you can record issues, concerns, progress and solutions. Alternatively, you may prefer to collect your reflections for projects within your project infrastructure. For example, I have an ongoing project to improve our effectiveness at tradeshows. Immediately after a tradeshow we do an after-action review (reflection) and the results go into the Tradeshow Project infrastructure. Teachers typically have a long-term project of improving their lesson plans and their reflections might best go into their lesson plan system.

I started talking about having more things to do than we can actually accomplish. And then I gave you more to do. Crazy? No. I've described tools to help you do what's important. Let the other stuff go. You don't improve either for yourself or for your organization by doing more unimportant work. Improve by doing more of what's important.

Posted at 12:00 AM (permalink) 2 Comments View/Leave Comment
Keywords: continuous improvement

Wednesday, October 12, 2011

The after-action review

The U.S. Army has a valuable tool for improving performance in complex environments: the after action review (AAR).

AARs are conducted in an atmosphere of openness and honesty to discuss what actually happened and how it might be improved next time. The point of the AAR is to identify strengths and weaknesses from the point of view of the soldiers and leaders who were there. The feedback compares the actual outcome with the intended outcome.

After-action reviews (from A Leader's Guide to After-Action Reviews):

• Are conducted immediately after each event
• Focus on intended objectives
• Focus on soldier, leader and unit performance
• Involve all participants in the discussion
• Use open-ended questions
• Are related to specific standards
• Determine strengths and weaknesses
• Link performance to subsequent training

The general format of after-action reviews is to answer the following:

• What was supposed to happen?
• What happened?
• Why it happened and how to improve?

Why AARs are valuable

Military events are complex, meaning that they typically involve dealing in real time with unexpected events. Complex events cannot be pre-scripted before the fact. They depend upon everyone understanding how to improvise as situations arise yet still accomplish the mission. AARs are a technique for learning from reflection.

How we use AARs at Edclick

When we at Edclick have a major event such as attending a tradeshow, we conduct an AAR afterward. It might consist of a few minutes of discussion or an email discussion addressing the questions above. It's easy, it's useful review, and it leads to better performance next time.

How AARs can be used in education

Do lesson plans always go as planned? Of course not. A few minutes reflection shortly after class can be of immense help in improving teaching, classroom management, school discipline and other events around the school. Reflection on intent and results is the basis of RtI (Response to Intervention). It is a key to continuous improvement.

Posted at 12:00 AM (permalink) 2 Comments View/Leave Comment
Keywords: continuous improvement, after-action review

Monday, October 10, 2011

Build a better checklist

Why is the service at a Ritz-Carlton hotel noted to be better than the service at most other hotels? At Ritz-Carlton hotels, the staff works according to better checklists.

Basically, all hotels provide guests with a room and a bed. They then differentiate from one another by the quality of the furniture and the view out the window. And they further differentiate by the quality of service.

It's easy to see how to differentiate on quality of furniture. But how, exactly, does one go about differentiating on quality of service?

Nearly all hotels provide their staff with checklists of how to do their jobs. For example, there will be a checklist for how to clean a room. Quality of service is determined by the adherence to the checklist and by the quality of the checklist. Ritz-Carlton hotels have developed more effective checklists that include not only the standards for making the bed but what to do when a guest happens to ask for something unusual.

How can we create better schools? I suggest it must be done in much the same way that Ritz-Carlton makes a better hotel: more attention to exactly how the learning experience can be improved, and then replicating those improvements throughout the school so that they happen predictably and reliably.

Does your school have standards for high quality lesson plans? Standards for contacts with parents? Standards for dealing with behavior issues? And are your standards continuously evaluated for effectiveness and improved when needed?

Posted at 12:00 AM (permalink) 0 Comments View/Leave Comment
Keywords: continuous improvement

Friday, October 7, 2011

Don't forget the dumb stuff

I'm reading a really interesting book with the least interesting sounding name" The Checklist Manifesto / How to get things right.

We humans aren't very good at remembering all the little details in our daily tasks. As our tasks have become increasingly complicated (involving many steps) and increasingly complex (involving many variables that cannot be anticipated), we tend to make dumb mistakes. Dumb mistakes are when we do something wrong even though we know better.

How can we avoid dumb mistakes? In aviation it was done with pilots' pre-flight checklists. The checklists remind them to check all the little details that they knew should be checked but sometimes overlooked.

In surgery, deaths due to complications (which occur about 150,000 times per year in the U.S.!) have been reduced signficantly by applying the same simple idea: checklists. In this case, surgical nurses have the checklists and are empowered to stop the surgery if the doctor skips a step.

In building construction, ambitious skyscrapers do not fall down despite the fact that they are so complicated to build that there is no one person who comprehends all variables, many that conflict with others, that must be gotten right in order for the building to stand and function properly. How do they do it? Checklists of requirements and sequences and checklists of required discussions among the sixteen trades (architectural, structural, plumbing, electrical,...) that must occur by set dates to ensure that decisions are made on issues that arise that satisfy each of trade's requirements.

How might this apply to the continuous improvement of teaching?

• Lesson plans are essentially pre-flight checklists of how a lesson will be presented. In practice, many lesson plans, however, are sketchy at best, consisting of little more than a textbook section to be covered and a homework assignment. Those overly simplified plans often overlook some of the dumb stuff like, are the most important learning objectives being addressed? and is the lesson optimized for student learning rather than being optimized for teacher presentation?
• Behavior improvement plans and individualized education plans are typically checklists. They add organization and specificity to student interventions.

Simple checklists are the most common means for defining a repeatable process. They help us avoid overlooking the dumb little details that can often lead to failure.

Posted at 9:54 AM (permalink) 0 Comments View/Leave Comment
Keywords: continuous improvement

Monday, October 3, 2011

Nourishing student enthusiasm

Continuing the discussion of The Progress Principle.

The research showed that intrinsic motivation blossomed with a few gestures of interpersonal support. They are called the four major nourishers.

1. Respect.
2. Encouragement.
3. Emotional support.
4. Affiliation.

The first three can be rare in a business environment but are common in schools. Occasionally one will find a teacher who treats students disrespectfully, but it is relatively rare.

Affiliation is the actions that develop bonds of trust, appreciation and affection among those working together. Developing affiliation is particularly important when the group is not located together. In a business environment, it means building bonds with folks located in remote offices and frequent travelers. In a school, it means taking extra time to build bonds with online students and with students who may share only one class. Bond among members of a collaborative group reduce interpersonal conflicts and encourage the free flow of ideas. So, have some fun together!

Posted at 12:00 AM (permalink) 0 Comments View/Leave Comment
Keywords: progress