Hi, everyone. Thank you so much for joining this session. My name is Carl Mingus, and I'm a member of the data analytic team at the Yale School of Medicine. I was heavily involved in the development of the ACC Cath PCI quality of care composite over the past several years, and I'm excited to tell you more about why this is really and truly a one-stop shopping opportunity for improving quality in your sites. This slide presents a description and objectives of your presentation in this session in and of itself. I'm here to tell you more about the rationale for the metrics and why they were included, why different weights were applied, and how to learn how to evaluate the composite score and speak to the performance with your administrative and clinical teams. So some of the language will get jargony, but I will try to keep it as layman's terms as possible so it's really easy to convey what you are presented in terms of the quality composite and translate that into quality improvement into your own institutions. The learning objectives for today's session is to discuss how the composite metric demonstrates that quality of care was provided and to identify one stakeholder who might benefit from receiving updates on the facility's composite measure score. This slide simply demonstrates that the timeframe from which we have been undergoing the Cath PCI quality composite has been for quite a while, since 2010, when a feasibility phase was introduced, followed by an implementation plan and the execution for quality composites and public reporting. This was an ACC Board of Trustees directive as of 2009, so it has been quite some time in the works. So this slide illustrates the considerable amount of time that has gone into the development of the Cath PCI quality composite. You can see that the kickoff to this project started back in 2016, and here we are in 2022 where there is the trial phase and development for the Cath PCI quality of care composite. There were obviously numerous committees that were involved throughout this process as illustrated in this timeline, including the Metrics and Reporting Methodology Committee, various steering committees, including CSQC, Board of Directors, and it also went through a public comment period. Importantly, the MRM Committee, or the Metrics and Reporting Methodology Committee, had a very direct role in overseeing the methodological approach to reporting this measure through the Voluntary Hospital Public Reporting Program and making recommendations to the CSQC about advancing this forward. The CSQC also had a role in leading and overseeing the key activities related to the development and implementation of the Cath PCI quality composite. Where we are at now in terms of the phase of development is really a piloting phase, and we'll tell you more about the future of the Cath PCI quality composite at the end of the presentation. So this slide presents the composite metric selection. As indicated in the table below, there are six overall metrics that were included in the Cath PCI quality composite. There were many that were assessed and considered. The MRM Committee really drove the charge in terms of identifying which metrics to use, and they voted on them and also presented them for public comment to seek feedback from the public about which metrics should be included, the weights associated with them, et cetera. So it was a very thoughtful process through which we identified the following six metrics. We have three outcome and three process metrics that were indicated. The first one is metric one. This is PCI and hospital risk-adjusted mortality for all patients. Some of you may be saying, well, this one's being retired, so how is it still in here? That's a good point. At the time of this recording, we are still in the evaluation assessment process to determine which hierarchical model makes the most sense to include as to assess the risk of mortality for the composite measure. So stay tuned. ACC will update you with the new risk-adjusted mortality metric that will be included in replacing metric one. Metric four is PCI within 90 minutes. This is among patients with STEMI and is a process measure. Metric 38 is guideline medications prescribed at discharge. This is a process measure. Metric 39 is PCI in hospital risk-adjusted acute kidney injury or dialysis for all patients. This is an outcome measure. Of note, this one will also be updated to a hierarchical model, and this will take place likely at the end of this year or early next year. Metric 40 is PCI in hospital risk-adjusted rate of bleeding events for all patients, an outcome measure as well. And then finally, we have metric 45, cardiac rehab referral, which is a process measure. So taken together, these elements and these metrics truly represent an overall quality of care that the patient is receiving as a result of their care at your institution. And it is a pretty holistic viewpoint, including both process and outcome measures. So the quality of care composite includes both process and outcome measures, as just indicated. It is derived from facility-level metric performance, not individual level. It's at the facility level. It's an important thing to know. It's reported as a rolling four-quarter value only. So we don't have quarterly trends available, but it is a rolling four-quarter value that is available to capture the quality of care from a broader time period. And finally, the quality of care composite right now is being released as beta. So this really means that it's a way to kind of introduce the metric to participants, to gauge interests, for you to take a look at quality improvement performance and overall assess the value of the CAF PCI quality composite in your institution. So now onto the eligibility criteria. So these are really the rules in which a site must engage in order to be able to receive the CAF PCI quality composite and be considered as part of it. So first, in order to report a composite, sites must contribute all six metrics. So in the previous slides, I showed you the six metrics that were included. If, let's say, one of those, the site does not have enough sample to report that metric or fails to report it, then that site would be excluded from the CAF PCI quality composite. So they must contribute all six. And then finally, if there were no eligible cases within a metric, the hospital would not be eligible for the composite metric. So that sort of follows the same logic. Overall, in analysis of 2019 data, we found that 9.3% of sites were excluded. And in terms of the metric reported, you can see in the table to the right there that those that reported six metrics were 1,477, those that reported only five or 114, and far fewer hospitals reported only three or four metrics. So with regard to directionality of metrics, well, first, it's important to recognize that metrics chosen for this quality composite were reported in different ways. So in some, a higher score would be perceived as better, whereas in others, a lower score is better. So we call this the directionality of the measure. So for example, less mortality is better, giving more patients a cardiac rehab referral, discharge is conceived as better. So in order to roll up the metrics with different directionality into one output, this required that we sort of leveled the playing field for the metrics so they each have a similar distribution. And that was achieved through rescaling as per the formula indicated here. The higher scores of overall performance were indicative of better quality. We conducted several sensitivity tests to show similar distributions with regard to the pre-scaling and then the rescaling. In terms of the final score interpretation, obviously, higher is better. We indicated on a 0 to 100 scale, and that is what the overall number would indicate along with its confidence intervals. So this slide demonstrates several descriptive statistics about the overall model, so including all sites. And so the central tendency by metric is indicated by each of the individual metrics. I will present a mean and a standard deviation, those two go together, as well as a median and an IQR. So risk-adjusted mortality is one of the many metrics that we have that has a star next to it, meaning that it had to be rescaled. So generally, lower is preferred. However, to coincide with the directionality, we did have to rescale as described in the previous slide. So all the outcome metrics are indicated. They are rescaled accordingly. In terms of risk-adjusted mortality, overall sites did really well. 95 was their overall score with a plus or minus standard deviation of about a 3.8. This is a very tight distribution, so not too much variability, as also indicated by the median at 95.7 is the median score and an IQR. So basically, being the 25th to the 75th percentile of 94 to 97 percent of a score. The PCI within 90 minutes, this is a process measure. The mean here was about 92. Again, pretty well performing overall. However, there was quite a bit of variability with a standard deviation of plus or minus 10.2. The median score for PCI within 90 minutes was 94.6. So again, many sites were achieving that goal with an IQR of 89 to 98. Discharge medications. So this is another process measure. The mean score was just about a 96. So again, overall sites performing well with fairly modest variation of about a 5.8 plus or minus. The median for discharge medications is a 97.8, and the interquartile range is being 95 to 99. On to an outcome measure, acute kidney injury. So this was among the sort of lower performing metrics. We had a mean of 80.7 in terms of the score for acute kidney injury. The standard deviation was 9.1. And then we had a median of 81 with an IQR of 75 to 86. So quite a bit of variability as well. Risk standardized bleeding. Overall sites similarly do not perform as well on this outcome as say risk adjusted mortality. The mean was 74. The standard deviation was plus or minus 12.5. So pretty much a lot of variability there. The median was 76 with interquartile range of 68 to about 83. And then finally we have cardiac rehab. And so the overall score on this was the lowest at 69.7% of sites achieving the cardiac rehab goal at discharge. But there was quite a large standard deviation here. So plus or minus 33 points. The median was 84.3. So much different than the mean here, which is a better measure actually of central tendency given the variability of this metric. And the IQR ranged from about 51 to about 96. So again, quite a large distribution. So for those of you who are more visual learners, this is just another way of displaying the data that was on the previous slide. So what's presented here are all medians with their interquartile ranges. So as you can see, for example, we talked about cardiac rehab having a lot of variability. So the median for this metric, as indicated here, was right above the 80 mark with the IQR being what's in this box. So from 25th to the 75th percentile. And so you can just see that sort of the metric with the least variation was risk adjusted mortality as indicated by a very small box with the median line being in the middle there. Here's with PCI within 90 minutes. Discharge medications, again, similarly a smaller distribution. Acute kidney injury and risk areas bleeding look pretty similar in terms of their distribution as well. So before I show you the weightings, I want to illustrate the process by which the weightings were established. So our overall goal was to achieve a clinically meaningful balance between weights of process and outcome metrics in the composite. So generally speaking, determined within the ACC scientific team that clinically outcomes ought to be weighed more heavily than process measures. And this was largely done by the MRM committee and comprised several votes along the way. The distribution of the metrics were examined in the previous slide, and this showed that there were two outcome and one process measures that had significant variability, and thus we need to take that into consideration. We developed overall five unique weighting scenarios and arrived at one, which I'll be presenting in the next slide. We determined that composite scores increased significantly as more weight was placed on the outcome, and if you go back and take a look at the mean and medians, that might make sense. So presented here are the CAF PCI quality composites weighting of the overall metrics. As a reminder, the MRM committee approved the weighting scenario displayed here with more weight associated with the outcome metrics. So here we can see at the top, the outcome measures comprise 35% for metric one or in-hospital risk-adjusted mortality. Metric 39 got 20% of the overall weight towards the overall score, and metric 40, PCI and hospital risk-adjusted bleeding rates had 20%. The process measures were such that 10% went towards PCI within 90 minutes. Metric 38 comprised the composite of guideline medications prescribed at discharge had 10% of the weight, and then cardiac rehab referral, the one with the greatest variability and the lowest score, only had 5% of the overall weight. So in general, all of these percentages add up to 100, which illustrates the quality composite measure overall score. So as illustrated in the previous slide, you saw that the outcomes accounted for 75% of the overall weight for the composite. Process measures accounted for 25%. We saw the risk-adjusted mortality had the most weight at 35% contribution, and the cardiac rehabilitation referral had the least weight at 5% contribution. Overall, the composite metric descriptive statistics are illustrated in this slide. We can see that the mean score of the overall composite based on these weightings was 86.6. The standard deviation was about plus or minus 4.4. The median was 87 with an interquartile range ranging from 84 at the 25th percentile to about 90 at the 75th percentile. So overall, a fairly decent distribution, not too wide, and it encompassed the overall composite descriptive statistics. So now onto understanding what everything we've talked about today actually populates in your own internal environments and how to interpret it. So of note, this is a score. That's the interpretation. So this given institution is indicated on the left of the slide is the beta number 55 quality of care composite for the CAP PCI registry, and to the right is the actual range of performance that this site achieved. So their rolling four-quarter performance rating was an 87.2, which was slightly below the 50th percentile. It's important to note that this score would indicate that about 87% of the time the quality of care as assessed in the CAP PCI registry is achieved, and it also goes to demonstrate that it could be assessed against the benchmark. That's the reference point. So a reminder that the output is a score and that it should always be assessed against the benchmark. So now that you have visually seen how a score is interpreted, you understand that it's on a zero to 100 point scale, that there's various weights for various metrics that roll out to the overall score. Now let's talk about how this composite can be utilized in your site. So one is to promote quality improvement, right? The goal is to assess overall quality of care of the CAP PCI registry. That's why this talk is entitled One-Stop Shopping. It's to think about how you can best present the quality of care that your institution is delivering. So that is the number one goal of the composite. A secondary potential goal of the composite is for it to be valid for public reporting, because it would be more easily understood by stakeholders than perhaps looking at each individual metric alone. So let's talk a little bit about what that means. We would use a star rating nomenclature, 1, 2, 3, 4 stars, much like CMS does. We would score against a pre-specified threshold or benchmark. That means that every site could conceivably get four stars, which would be the highest number allocated if they met that benchmark. So that's the exciting thing about this quality composite is that it is movable and adaptable to an institution's own efforts to improve quality. So now that we have introduced the concept that public reporting could be in the works for this composite measure of quality, I'd like to reiterate some of the ACC's public reporting strategic efforts that have actually existed for the past 12 years. So in the publication indicated below, the main goals of the public reporting strategic effort that ACC has delivered has been to promote quality through public reporting, to allow for performance measures to have scientific validity, meaning that they will undergo NQF approval and be submitted for NQF review. They'll be developed in partnership with physicians, very much done in the case of this quality composite, such that most of the scientific team comprises physicians. We will standardize data elements and uniform submission process across all public reporting programs. That also has been achieved here and would be applied to quality composites going forward with regard to the star ratings. Reporting should occur at the appropriate level of accountability, of course. And then we'd like to include a formal process for evaluating the impact of the program on the quality and cost of healthcare, including assessment of unintended consequences. So of course there would be a review of the quality of care composite in the space of public reporting, and this would be issued for all sites to voluntarily public report. So where are we in the rollout of the composite measure trial phase? In January of 2022, we went live on hospital dashboards, as many of you know. The measures included the term test to indicate that it was indeed a test measure. From January to June of 2022, participants had the opportunity to review their data, work with their teams to decide whether or not they wanted to opt in to the quality composite or not. Bi-monthly calls were also conducted by the NCDR to all participants to engage them in training and how to understand the quality composite. All questions were also documented within ACC's CRM sales force. In July of 2022, comments or questions were available for the steering committee, CSQC, to review. The test terminology was then removed from the measure title. And then going forward, we have to update metric one with the new mortality metric after that is decided by the scientific teams and also eventually update the acute kidney injury metric as well once that is rolled out. So in summary, the composite measure was strongly endorsed by the American College of Cardiology and all of its committees and various scientific groups. We perceive it to be an overall reflection of quality of care provided for sites that participate in the CAF PCI registry. We feel that it's a valuable measure for stakeholder engagement and helps us to demonstrate the quality of care that your hospital delivers in a frankly a very digestible way. It will ultimately enhance public reporting efforts by the college and has been received extremely well by participants. And of course, we will continue to monitor this for any concerns that might come up. Thank you very much for your time and your commitment to quality improvement in your institutions. If you have any other questions regarding this presentation, feel free to contact NCDR at ncdr at acc.org. And thank you again.

ACC Cath PCI

quality of care composite

metrics

weights

performance

public reporting