Good morning. This is Jeff Anderson. I'm in my office in Cincinnati, and I wish I was live with you guys, but the morning of the learning session will be rounding, probably, at the time you guys are talking and sharing. I appreciate Clara and the rest of the QNET team helping me and letting me record some education for you guys and get you started on a conversation about variation and learning from each other. Just for a few minutes, I wanted to talk to you guys about funnel plots and this special kind of control chart that helps us identify individuals or systems or groups that are falling outside of the common cause variation that we talked about when we last talked about variation. So this is cleverly labeled bases loaded bottom of the ninth, because I'm going to use a baseball analogy here to try to explain something about funnel plots. The objectives of the next few minutes are to understand what I call the physiology of a funnel plot, to be able to look at data that's being presented using a funnel plot and interpret what it means, and then begin to review data from QNET to inform learning strategies so that we can learn from each other. So what is a funnel plot? Well, we've talked in the past about shoe heart charts or control charts, and you'll remember that control charts give us some information about data. Control charts will tell us something about the central tendency of the data, usually a median, but will also give us upper and lower control limits of the data. These upper and lower control limits tell us what data is within and without statistical process control. One of the additional pieces of information you need to know about funnel plots as a special kind of control chart is that they're organized by N. So the Ns, whether it's an individual person, the Ns of that person's performance or the Ns of a system's performance, the lower to the left side of the funnel plot you are, the smaller the N, and to the right, the higher the N, and we'll give you a couple of examples of this. This is useful for correctly identifying outliers, those who are special causes in the system, and those might be outliers above the system or below the system, depending on which direction you want to go. Those might be high performers or low performers. And it'll help us really distinguish between those who are outside the system and those who are inside the system. So I want to give you a baseball analogy to help you understand funnel plots and what they say and what they don't say. This is entirely fictional, I will tell you, and I can't remember when I first made this story why I used the University of Michigan baseball team, except that it was so long ago that nobody probably remembered anyone or any stories from this team. And that helps when you're making things up that nobody actually knows the facts. But anyway, so the story goes that you're in the last game of the championship series and it's the bottom of the ninth inning and bases are loaded and the game is tied and there are two outs and you have the opportunity to choose who's going to be up at the plate for your team to hit the ball. And so if you look at this list of players here from A to Z and their batting average, and remember for those of you who aren't baseball fanatics, batting average is the percentage essentially of hits. So someone with the batting average of .470, like player A, hits the ball 470 times out of 1,000, which is much better than someone who hits the ball 200 times out of 1,000. So your job as a manager is to pick who should hit the ball next. So who do you want to bat? You can look at your data this way and say, these are my highest, players with the highest batting average. So 470, 380, 480, down to even this guy Z who is, or X who is 667. Those seem really high. So those might be the batters that you want to hit to send up to the plate. But what if you had a little bit more information about these batters? So now I've combined the batting average with the number of times they've actually been at bat. And you can see that there's a really important piece of information here. Your person who's batting 667, player X, has only been to base, to the plate six times, whereas the person with, like L, who's hitting 360 has been to bat 250 times. So a funnel plot, one of the things it's going to do is combine your actual output of the graph with your ends to give you even more information. And what essentially we do mathematically is we'll combine that batting average, use the number, use the end, so the number of times at bat, to create a standard error that then gives us the upper control limits and lower control limits for the chart. So here's the batting average funnel plot for those players. And if you look at the players that were listed over here, these are players. So if you look at the Y axis, this is my batting average, and the X axis is the number of times at bat. And you can ask yourself, are these our best batters? Well, you guys know something about control charts now. So you know that this is your median, this is your central tendency. And in this control plot, these are your upper and lower control limits. So what you should be saying to yourself is, well, those players all lie within the control limits, so they are all part of the same system. And that tells us that statistically, this player here batting at 600 is no different statistically than this player here who's batting somewhere around 280 because of statistical process control limits. And those limits are dependent for every individual value on the end that is used to create that value. So are any of the batters better or worse than the rest? So I'll let you look at the chart and answer that question for yourself. And what you likely said was, yeah, this guy is the only one who's inside or outside or who is outside the control limits. And in this case, he is statistically a worse batter. He is the worst batter on the team, even though his batting average might be a little bit higher than this person over here. Because of the number of times he's been at bat, we know that he is statistically different than the rest of his teammates. So you now have more information to answer that question. Just a little side note, and again, these people weren't members of this University of Michigan team, but it is interesting to have a real person example to show you how you might be misled. This is Kevin Moss, and Kevin Moss joined the major leagues with the Yankees in 1990. He had one of the best starts in MLB history, hitting 10 home runs in his first 72 at bat, giving him a really great batting average. And that was in 1990. Kevin Moss ultimately was released by the Yankees four years later as a complete bust. So judging his first 72 at bats and his batting average did not tell you that he was going to be a long-term success. As opposed to this guy, Derek Jeter, who was a draft pick in 1992 and had a very average first season with the Yankees and ultimately went on to the Hall of Fame as one of the best hitters in the game. So again, we look at data all the time, charts of our own data, and make judgments like this. And as you guys are getting more educated and smarter about data and how to visualize it, you'll be less likely to fall into these kind of traps. So let's move on to a couple of our funnel plots from QNET. And I wanted to go over these as a preface for you guys learning from each other. So this is the control chart for genetic testing in Tetralogy of Fallot. Now each one of these blue dots is one of the centers in QNET. And the blue dot represents the percentage of patients who received appropriate genetic testing in that system. The ends, as you'll see along the x-axis, go from small at 33 patients to large at 372 patients on the right side. And what you can see is that there are a number of sites that fall within control limits. So that means that they are very similar to other sites within the system. There is one site who is outperforming the system, meaning that their results are much better than the system as it is. And there are a number of sites whose performance is lower than the system. So if we were going to create a plan for how to learn from each other, we might ask this site here what their processes are, and we might compare them to the processes at these other sites. That is one way to plan and move forward. Moving on to body mass index measurement in pediatric cardiac patients, again, a funnel plot here. You'll notice that this funnel plot looks a bit different than the last one. And the reason for that is the ends. So again, this center here gave us information about 40 patients. This center here gave us information about 18,000 encounters, I guess, in this case. So the bigger the end, the smaller the control limits, the more narrow the control limits. But even though this is a somewhat skewed metric, we see lots of centers that are performing very near 100%. One thing that we might do with this chart is say, well, how can we help this center, this center, this center in reaching their goals? There might be lots of solutions in these centers here. It might be good to find a center for UIC that is similar in size and practice scope that's up here and pair them, pair them, pair them to find some solutions for these centers. One of the great things about collaborative learning and doing quality improvement in the way that we're doing it here with this network is that we all care about ourselves and our center and our patients, but we also care about raising all boats. And so getting these centers the tools that they need to change to be high-performing should be something that we all want to do. And then if you look at the appropriate counseling among patients with a BMI greater than 85%, this is a more similar pattern to the Tetralogy of Fallot metric. First thing I'd point out is that our median here is 50%, so as a system, we're only halfway doing this for all of our patients. So even though all of these centers fall within control limits, we shouldn't say, well, that's good enough just because they're in the system and statistically in the system. I think our goal is to move this whole system upwards. Now having said that, there are a couple of centers who are outperforming the system, and if this center is only at 70% now, they're still outperforming the system. And we all might learn from these two centers, and we all might learn ways that we can move this system up. These centers here might even learn from these centers who are a little bit higher performing in the system. So these charts really are an opportunity for us to learn what's going on now, but then plan for what's going to happen as we learn from each other and purposefully go about making changes to the system to improve our outcomes. So this is just one tool that we can use, and as we continue on as a learning network, we'll talk about the other control charts that we might use and start putting our data into those control charts. So with that said, I think at this point we're going to move along to learning from some of our high performers. So I'll sign off there.

funnel plots

control chart

performance variation

QNET data

system improvement