Edclick

Edclicking Keyword Cloud

By Dr. Harry Tennant

Comments: Edclicking

To the blog

Enter a comment

Tuesday, February 13, 2018

Continuous Improvement

This series of posts comes from a paper, Responsibility, Motivation and Engagement: How To Develop Learners Using Behavior Manager. It describes how Edclick’s Behavior Manager combines three essential capabilities.

  1. Proactive PBIS tools to prevent student misbehavior
  2. Reactive interventions and processes for misbehaviors to minimize loss of instruction time and keep students in school
  3. Data and tools for continuous improvement

Schools and districts often bring in Behavior Manager with the goal of making things quicker and easier. And it does that. Many times, they keep addressing their behavior issues in the same way before Behavior Manager, but now, it’s online. Doing that misses a huge opportunity.

Behavior Manager collects and stores a lot of data. The data can be explored in a variety of different ways and it often reveals insights that were unanticipated.

Typically, when people look through the data available in systems like Behavior Manager, they are looking for the good news. What can I show my superintendent that will impress her? What will I show my staff? Showing off the good news feels good. And often that’s what the recipient wants to see. But there is something much more powerful to pull from data like this: where are the opportunities for improvement?

Many people are reluctant to look for opportunities for improvement because they think of them as problems and they think problems are bad. But if you’re going to do better than you’re doing now, it will be because you found a problem and fixed it. Identifying problems is good. Problems are where all the progress comes from. Problems are opportunities…the opportunities for improvement.

Finding opportunities for improvement is only half the job. The other half is finding and implementing the improvement.

How to explore data in Behavior Manager

Common behavior issues are described by the behavior dashboard.

The quickest way to get started with data in Behavior Manager is with the dashboards. Dashboards show a summary of the most commonly sought behavior metrics. The dashboard above gives a quick overview of behavior issues, where they come from and what has been done about them. The graphs suggest some immediate questions. Why do some teachers have a lot more office referrals than others? Are we providing the best interventions for those students who have the highest number of referrals? What about the out-of-placement consequences? Are there better interventions for keeping those students in class? What about the overall numbers? Should we be satisfied with these numbers? (Hint: no) If not, what are we doing about it? (Hint: you should be working an explicit behavior improvement plan)

 Another useful report is the behavior analytics report. The analytics report to the right is showing the number of out-of-placement days assigned in the school year until January 30. The numbers are compared for this year (red) to previous years (pink). The declining trend looks good. Do we know why it is declining? Is it random or because of something we’re doing right? Should we be happy with this year’s total or should it be half that? What are we going to do about it?

The civil rights dashboard displays key behavior metrics. It compares the percentages of special populations to the overall percentages in the school population.

The civil rights dashboard in this example shows that we need to understand what is going on with consequences  versus race. About 10% of our students are African American but 17% of the out-of-placement consequences assigned are African American. Do we know why?

The query and reporting components of Behavior Manager allow us to ask more specific questions of the data and drill down for more detail and to look for evidence for or against theories we may have.

Taken together, this data suggests opportunities for improvement. It also helps us answer the question, are the steps we’ve taken for improvement working?

Building a culture of continuous improvement

The most important shift is to develop the mindset that our goal is to always be improving. Improving what? Improving everything. And the fundamental metric to improve is student achievement.

Key: Be better than we were yesterday!

In order to improve the school, involve everyone with making improvements all the time. Make a common question be, how are we better today than we were yesterday? From lots of small improvements come big changes.

If a cynic answers, we’re not better today than yesterday, then ask about last week or last month. Pretty soon even the most resistant among us must admit that stagnation is not a winning strategy.

Keep in mind that not every experiment will work. Some attempts at improvement will fail. But it’s still a form of progress to keep trying. Remember Thomas Edison’s response when asked if he had failed to make a practical light bulb:

I have not failed. I’ve just found 10,000 ways that won’t work.
-- Thomas Edison

Procedure improvements

Most improvements will be small. You might notice that when students pass papers to the person behind them that there tend to be little disruptions. One student just throws the papers over his head. Another jams the papers into the face of the person behind. You could reprimand or it may be better to change the paper passing procedure to pass papers to the right instead of to the back. If that procedure change helps, make it stick. The aggregation of many small procedure changes results in big behavior changes.

Encourage teachers to share their procedure improvements. A useful new procedure in one class is likely to be useful in others.

Data for insights into common issues

A number of questions are frequently asked in schools. Behavior Manager provides convenient data for insight into the answers. Here are some examples.

How many office referrals are made weekly?

Which students are the most common offenders?

Which teachers make the most referrals?

How are referrals distributed throughout the day?

Where to offences occur?

What are the most common offenses?

How many out-of-placement consequences are being assigned?

Are office referrals biased by race? By ethnicity? By gender? By English proficiency? By special ed status?

Are out-of-placement consequences biased by race? By gender? By English proficiency? By special ed status?

Are serious crimes being reported?

Are there allegations of bullying, intimidation or harassment? How are they distributed over sex, race, nationality and religion?

Are there weapons violations?

If there seems to be racial bias, are certain individuals responsible for it?

How do key success indices compare from the current school year to previous school years? Do the indices indicate improvement?

These questions and more are addressed in the dashboards, civil rights report and behavior analytics report. To dig deeper into these questions or to ask other questions, Behavior Manager includes query capabilities through the behavior population reports, behavior record query and behavior records by intervention.

“That’s funny…”

One of the great benefits of capturing a digital trail of events in schools, is that it can lead to facts and insights that would otherwise be hidden. How should we take advantage of this opportunity?

The science fiction and science fact author, Isaac Asimov wrote

The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka!” (I have found it!) but “That’s funny…”

That is what the dashboards, reports, analytics, histories, drill downs and the rest are about. Some folks look at that sort of data and think of it only in terms of a snapshot of the status of the school. Yes, it is that. But more important are the opportunities for improvement that it may be revealing.

Here are some general questions to ask.

What are your most important metrics?

A few numbers will be the most important indicators of how your school is doing. What are your key indicators for concerning student behavior? Office referrals per week? Tardies? Out-of-placement consequences? Harassment complaints? The first step is to decide what is important to you and 1) start watching those numbers regularly and 2) make it a specific, scheduled task to take steps to improve them. If you feel that those key numbers are unimprovable, pick the set of next most important indices and improve them.

Does the data contradict your gut feel? In other words, is it funny?

If you’re looking to improve, you’re looking for surprises. If something looks funny, for whatever reason, follow up on it. These are often the sources for the greatest insights.

Keep in mind that you might be surprised in two ways. The numbers may be worse than expected. Or, you may be surprised that the numbers are better than expected. Surprisingly good numbers often point the way to your best opportunities for improvement. Whatever caused that surprisingly good result, identify it and do more of it!

Why are the aggregate numbers this high (low)? Why not lower (higher)? How can we make it lower (higher)?

Pick a data point. For example, pick the number of out-of-placement consequences assigned. Too often we look at a number like that and think that, if it is close to what it has been in the past, then it is okay. With the mindset of continuous improvement, the question should always be, how could we make that better, even if just by a little bit? Small incremental improvements add up to large improvements.

Is there a trend?

Look for trends by week, month, grading period and year. Do you see a pattern? If you do, what do you think causes the pattern? If you see no pattern, why not?

Am I seeing the effect I expected?

When you make a change, what effect do you expect to see? For example, if you’re implementing hallway sweeps between classes to reduce tardies, how much of a reduction in tardies do you expect to see? Are you seeing it? Is it more or less than expected? Has the improvement been worth the effort? What should we try next?

Are there correlations? Do the numbers for a subpopulation differ from the overall aggregate numbers?

This is what drilldown is all about. Compare overall aggregate numbers for whatever you’re investigating to the numbers for subpopulations. We do that in the dashboards for the most common metrics and subpopulations by race, gender, special ed, LEP. Do the differences suggest opportunities for improvement?

Don’t get too hung up on the data.

The goal is to improve. Ideas for improvement will come from all directions, not just out of the data. Solicit ideas from teachers, students and parents. But then figure out how you can measure the current baseline and then the effect of change. Without data, you’re just another person with an opinion.

How to create and track improvements with Plan-Do-Check-Act cycles in Behavior Manager

It is best to think of improvement as a repeating process. It’s not just one-and-done, but each improvement is built upon previous improvements and provides opportunity for further improvements.

Also, improvement should be based on the scientific method. We should be specific about what we intend to change and what measureable effects are expected. After we try it, measure the results. Did we see the measured improvement that was expected? After trying this improvement, what should we do next? Implement the improvement? Try an alternative?

This approach to improvement is captured in the simple Plan-Do-Check-Act (PDCA) cycle.

Plan – Assess the current situation and hypothesize how it might be improved. What can we measure before and after that would show improvement?

Do – Do the planned improvement. This is a test so implement it as quickly and easily as possible.

Check – Measure the data captured during the Do phase. Did you see the expected improvement? Why or why not?

Act – If the change was an actual improvement, then incorporate it into your process. If not, decide what to do next. Modify the plan and try again? Abandon this approach?

Behavior Manager includes a tool for documenting improvement opportunities and the PDCA cycles to test and implement them.


 

Posted at 12:00 AM Keywords: Responsible-Motivated-Engaged , Behavior Manager 0 Comments

  No comments. Be the first to enter a comment.

Enter your comment

Your name



To fight spam, please enter the characters in the image.