WEBVTT 00:00:10.986 --> 00:00:15.095 position:50% align:middle - [Nancy] Welcome everyone to NCSBN's Virtual Conference 00:00:15.095 --> 00:00:16.710 position:50% align:middle on our Approval Guidelines. 00:00:16.710 --> 00:00:19.570 position:50% align:middle And I'm here today with Jan Hooper. 00:00:19.570 --> 00:00:24.850 position:50% align:middle She was the Chair of our Education Metrics and Outcomes Committee, and as many of you 00:00:24.850 --> 00:00:30.390 position:50% align:middle know, she's been a seasoned education consultant at the Texas Board of Nursing. 00:00:30.390 --> 00:00:34.172 position:50% align:middle You came right about the same time that I came to NCSBN, correct? 00:00:34.172 --> 00:00:36.053 position:50% align:middle And that was 2002. - [Jan] 2002. 00:00:36.053 --> 00:00:38.513 position:50% align:middle - So we've known each other for a long time. 00:00:38.513 --> 00:00:42.631 position:50% align:middle She was the Chair of that Metrics and Outcomes Committee, and she's going to 00:00:42.631 --> 00:00:46.703 position:50% align:middle give us a little bit of the regulatory perspective today. 00:00:46.703 --> 00:00:51.573 position:50% align:middle Whereas Brendan Martin, who is our associate director of research 00:00:51.573 --> 00:00:56.583 position:50% align:middle is going to talk about the quantitative study, that wonderful 5-year annual 00:00:56.583 --> 00:01:02.289 position:50% align:middle report study that all of you helped deliver your annual reports for. 00:01:02.289 --> 00:01:08.339 position:50% align:middle He's going to talk about that as well as a brand new exciting performance indicator 00:01:08.339 --> 00:01:10.389 position:50% align:middle that we're going to talk to you about. 00:01:10.389 --> 00:01:16.158 position:50% align:middle And then, you know, in the background is Josephine Silvestre, many of you know her. 00:01:16.158 --> 00:01:18.410 position:50% align:middle She helps with emails, etc. 00:01:18.410 --> 00:01:23.199 position:50% align:middle And she has just been my partner in crime in all of this. 00:01:23.199 --> 00:01:30.173 position:50% align:middle This has been at least two years and going and we're just so exciting to be able to 00:01:30.173 --> 00:01:32.983 position:50% align:middle present everything to you today. 00:01:32.983 --> 00:01:36.313 position:50% align:middle Before we get started, I just want to thank our interactive 00:01:36.313 --> 00:01:37.515 position:50% align:middle services department. 00:01:37.515 --> 00:01:41.833 position:50% align:middle They have been excellent in helping usto put this together. 00:01:41.833 --> 00:01:48.086 position:50% align:middle This is, you know, not our first virtual conference, we had another one in 2015 00:01:48.086 --> 00:01:53.423 position:50% align:middle distance education and that one went very well and we're just convinced that 00:01:53.423 --> 00:01:56.013 position:50% align:middle today it'll go even better. 00:01:56.013 --> 00:01:59.192 position:50% align:middle So the PowerPoints will be available to all of you. 00:01:59.192 --> 00:02:05.557 position:50% align:middle We will send you the PDF after, you know, we're done with the conference. 00:02:05.557 --> 00:02:14.147 position:50% align:middle So you will also be able to get continuing education units 4.2 if you're registered, 00:02:14.147 --> 00:02:16.961 position:50% align:middle we will send you the evaluation if you send it back. 00:02:16.961 --> 00:02:18.897 position:50% align:middle We'll send you your certificate. 00:02:18.897 --> 00:02:21.628 position:50% align:middle If you're not registered, some of you maybe are in a room with a 00:02:21.628 --> 00:02:27.303 position:50% align:middle bunch of people, just send an email to Qiana Macintosh and she's on your 00:02:27.303 --> 00:02:32.306 position:50% align:middle Powerpoint there and, you know, say you want CEs, and she'll be glad 00:02:32.306 --> 00:02:35.397 position:50% align:middle to make sure you get them. 00:02:35.397 --> 00:02:40.157 position:50% align:middle So I've already introduced everybody to you, this the agenda that we'll be 00:02:40.157 --> 00:02:41.757 position:50% align:middle talking about today. 00:02:41.757 --> 00:02:47.847 position:50% align:middle We'll begin talking about the literature review and then the Delphi. 00:02:47.847 --> 00:02:51.967 position:50% align:middle With the Delphi, some of you have heard those presentations before. 00:02:51.967 --> 00:02:56.237 position:50% align:middle I know that I spoke at the scientific symposium about our results, 00:02:56.237 --> 00:03:01.050 position:50% align:middle I've spoken on some of the education calls about some of these results. 00:03:01.050 --> 00:03:06.370 position:50% align:middle And these results we're really just kind of going to summarize so that we can then 00:03:06.370 --> 00:03:09.900 position:50% align:middle show you how they fit into the approval guidelines. 00:03:09.900 --> 00:03:14.160 position:50% align:middle And then Brendan is going to give the report of the annual report study 00:03:14.160 --> 00:03:18.120 position:50% align:middle and that one I don't think many of you have heard before. 00:03:18.120 --> 00:03:23.500 position:50% align:middle And we're going to show you also an annual report template that we have derived 00:03:23.500 --> 00:03:28.450 position:50% align:middle from that study that maybe some of you can use for your annual reports. 00:03:28.450 --> 00:03:33.011 position:50% align:middle I'm going to go over them, the site visit study that was analyzed 00:03:33.011 --> 00:03:37.591 position:50% align:middle by Alison Squires as I'll talk to you about, the results of that, I think, 00:03:37.635 --> 00:03:40.288 position:50% align:middle Jan, were really very interesting. 00:03:40.288 --> 00:03:47.349 position:50% align:middle You know, it was kind of the quantitative went into some of the details of what you 00:03:47.349 --> 00:03:50.368 position:50% align:middle tend to see, NCLEX Pass Rates and those kinds of things. 00:03:50.368 --> 00:03:55.358 position:50% align:middle But the site visit study went into some of the minutiae of things that they really 00:03:55.358 --> 00:03:57.828 position:50% align:middle do see, don't you think, on some of those visits? 00:03:57.828 --> 00:03:58.388 position:50% align:middle - Yes. 00:03:58.388 --> 00:03:59.398 position:50% align:middle - Yes. 00:03:59.398 --> 00:04:04.725 position:50% align:middle So, and then we'll also, of course, go into the approval guidelines and then 00:04:04.725 --> 00:04:09.435 position:50% align:middle how you might use some of these approval guidelines in your boards of nursing. 00:04:09.435 --> 00:04:13.415 position:50% align:middle As you can see, we're going to try to keep this rather conversational, 00:04:13.415 --> 00:04:18.275 position:50% align:middle we're going banter back and forth a bit because we know four hours is a long time 00:04:18.275 --> 00:04:21.495 position:50% align:middle and we want to keep it interesting and interactive for you. 00:04:21.495 --> 00:04:24.375 position:50% align:middle And along with that, we want your help. 00:04:24.375 --> 00:04:30.120 position:50% align:middle We want you to send us questions, comments, we have a little area in the end 00:04:30.120 --> 00:04:33.148 position:50% align:middle where we would really like to hear your perspective. 00:04:33.148 --> 00:04:38.338 position:50% align:middle But also we have a couple Q&A sessions, so we really would like to hear from you. 00:04:38.338 --> 00:04:42.408 position:50% align:middle We don't want 10 minutes of nothing or where we make up the questions. 00:04:42.408 --> 00:04:48.008 position:50% align:middle So please bear with us with asking some questions. 00:04:48.008 --> 00:04:50.998 position:50% align:middle These are the objectives for today. 00:04:50.998 --> 00:04:57.248 position:50% align:middle We're going to discuss the literature review findings, we're going to explain 00:04:57.248 --> 00:05:02.832 position:50% align:middle how the NRBs will be able to use the annual report and site visit templates and 00:05:02.832 --> 00:05:07.272 position:50% align:middle then we're going to describe how you might use the approval guidelines. 00:05:07.272 --> 00:05:11.962 position:50% align:middle And remember, as we go through this, we do have that wonderful new exciting 00:05:11.962 --> 00:05:16.982 position:50% align:middle performance indicator that we're going to talk about, and Brendan is going to show 00:05:16.982 --> 00:05:18.912 position:50% align:middle you a demonstration on how that'll work. 00:05:18.912 --> 00:05:22.447 position:50% align:middle And I know none of you have probably heard about that before, it's very new 00:05:22.447 --> 00:05:25.346 position:50% align:middle and it's very exciting. 00:05:27.322 --> 00:05:31.666 position:50% align:middle Before we get started in some of the...well, I'm going to start out with a 00:05:31.666 --> 00:05:36.903 position:50% align:middle literature review, I just want to go through a little bit of the background. 00:05:36.903 --> 00:05:42.557 position:50% align:middle Our charge was very, if you remember, Jan, it was a little verbose. 00:05:43.822 --> 00:05:51.027 position:50% align:middle It was several lines long so we've kind of melted it down to develop a legally 00:05:51.027 --> 00:05:55.287 position:50% align:middle defensible approval process, and that's what the board wanted. 00:05:55.287 --> 00:05:58.907 position:50% align:middle They wanted something was legally defensible. 00:05:58.907 --> 00:06:05.284 position:50% align:middle A few of the boards were getting some questions about what they were using 00:06:05.284 --> 00:06:09.651 position:50% align:middle especially if they were using NCLEX as a sole measure. 00:06:09.651 --> 00:06:13.699 position:50% align:middle And so they wanted something that you could defend, 00:06:13.699 --> 00:06:15.921 position:50% align:middle something that was evidenced-based. 00:06:15.921 --> 00:06:21.471 position:50% align:middle Now, I will tell you, in our first meeting of the Metrics and Outcomes Committee, 00:06:21.471 --> 00:06:26.921 position:50% align:middle we did have an attorney come in, and he told us, you know, what they would expect 00:06:26.921 --> 00:06:28.481 position:50% align:middle for being legally defensible. 00:06:28.481 --> 00:06:32.197 position:50% align:middle And our attorneys have worked with us all along the way. 00:06:32.197 --> 00:06:37.037 position:50% align:middle And one of that is it has to be evidence-based, thus going 00:06:37.037 --> 00:06:38.917 position:50% align:middle through those three studies. 00:06:38.917 --> 00:06:44.057 position:50% align:middle And also, there needs to be consensus of the education, not only the regulatory 00:06:44.057 --> 00:06:46.097 position:50% align:middle but the education community. 00:06:46.097 --> 00:06:50.637 position:50% align:middle And, you know, I think we found that, you know, with everything that 00:06:50.637 --> 00:06:52.477 position:50% align:middle we have done. 00:06:52.477 --> 00:07:00.109 position:50% align:middle So our first meeting of our committee was in December of 2017 if you remember that, 00:07:00.109 --> 00:07:05.861 position:50% align:middle Jan, we had a great group of committee members, didn't we? 00:07:05.861 --> 00:07:10.432 position:50% align:middle And, you know, we started with doing some data collection. 00:07:10.432 --> 00:07:15.205 position:50% align:middle We thought the quality indicators, the evidence would be out there. 00:07:15.205 --> 00:07:21.055 position:50% align:middle So we sent some surveys out to the Boards of Nursing. 00:07:21.055 --> 00:07:25.655 position:50% align:middle We had all of the accreditors individually on a call. 00:07:25.655 --> 00:07:29.825 position:50% align:middle We thought, "If they have educational criteria, they must have evidence 00:07:29.825 --> 00:07:30.947 position:50% align:middle in back of them." 00:07:30.947 --> 00:07:33.942 position:50% align:middle We were very surprised to hear, "No, they don't." 00:07:33.942 --> 00:07:39.332 position:50% align:middle What they do though is rely on the U.S. Department of Education because that's 00:07:39.332 --> 00:07:43.141 position:50% align:middle where they're accredited so they rely on their outcomes. 00:07:43.141 --> 00:07:47.339 position:50% align:middle So we went to the U.S. Department of Education, and remember 00:07:47.339 --> 00:07:50.620 position:50% align:middle we had an expert on there, Dr. Matsudaira. 00:07:50.620 --> 00:07:55.557 position:50% align:middle He was actually part of planning the iPads for the federal government. 00:07:55.557 --> 00:08:00.026 position:50% align:middle And we said, "You know, where is your research?" 00:08:00.026 --> 00:08:05.794 position:50% align:middle For I remember they use employment rates and licensure pass rates and graduation 00:08:05.794 --> 00:08:09.054 position:50% align:middle rates and really nothing. 00:08:09.054 --> 00:08:14.094 position:50% align:middle What he said is in order to get funding, they need to graduate, 00:08:14.094 --> 00:08:18.993 position:50% align:middle they need to get a job and they need to be licensed if that's part of the profession, 00:08:18.993 --> 00:08:22.274 position:50% align:middle so they really didn't have any evidence either. 00:08:22.274 --> 00:08:26.628 position:50% align:middle So we then had to go forward with collecting evidence ourself. 00:08:26.628 --> 00:08:31.874 position:50% align:middle As you know, we did our literature review and then three studies. 00:08:31.874 --> 00:08:36.404 position:50% align:middle So our Delphi and then our 5-year annual report and our 5-year 00:08:36.404 --> 00:08:37.654 position:50% align:middle site visit study. 00:08:37.654 --> 00:08:41.604 position:50% align:middle So I've kind of put it together right here in this PowerPoint. 00:08:41.604 --> 00:08:46.079 position:50% align:middle I really like the summary of this PowerPoint that shows how everything 00:08:46.079 --> 00:08:51.716 position:50% align:middle goes together to provide us with an evidence-based approval process 00:08:51.716 --> 00:08:56.596 position:50% align:middle which is exactly what our board of directors wanted. 00:08:57.324 --> 00:08:59.932 position:50% align:middle So we're going to start talking about the literature review. 00:08:59.932 --> 00:09:04.902 position:50% align:middle I don't think any of you have read the literature review, and, you know, 00:09:04.902 --> 00:09:07.255 position:50% align:middle Jan was asking me, "Well, are we going to publish this?" 00:09:07.255 --> 00:09:10.923 position:50% align:middle And we are going to publish this, I'm not quite sure when, 00:09:10.923 --> 00:09:12.984 position:50% align:middle and I'm not quite sure on what format. 00:09:12.984 --> 00:09:16.173 position:50% align:middle Will it be in a supplement for the JNR, I'm not sure, 00:09:16.173 --> 00:09:17.822 position:50% align:middle or will it be individual article? 00:09:17.822 --> 00:09:20.705 position:50% align:middle So we'll keep you up to date on that. 00:09:20.705 --> 00:09:24.775 position:50% align:middle But the literature review, you can see we use the databases that most 00:09:24.775 --> 00:09:27.615 position:50% align:middle of the nursing education researchers use. 00:09:27.615 --> 00:09:32.688 position:50% align:middle The keywords to remember, Jan, our committee coming up with keywords. 00:09:32.688 --> 00:09:37.270 position:50% align:middle They weighed in a lot and we had a lot more keywords than this to begin with 00:09:37.270 --> 00:09:39.958 position:50% align:middle but that's always what happens. 00:09:39.958 --> 00:09:44.068 position:50% align:middle And then we knew we had to use the Gray literature. 00:09:44.068 --> 00:09:49.048 position:50% align:middle We looked at the accreditation literature and there's a lot out there. 00:09:49.048 --> 00:09:52.868 position:50% align:middle We looked at other organizations, we didn't just stay in nursing. 00:09:52.868 --> 00:09:56.248 position:50% align:middle We looked in engineering, we looked in medicine, 00:09:56.248 --> 00:10:00.918 position:50% align:middle we looked in pharmacy, OT, the national academies, 00:10:00.918 --> 00:10:05.616 position:50% align:middle had some reports out, remember that big report on the accreditation? 00:10:05.616 --> 00:10:11.377 position:50% align:middle So we did have to include the Gray literature. 00:10:11.377 --> 00:10:15.077 position:50% align:middle So you can see these are our research questions. 00:10:15.077 --> 00:10:21.607 position:50% align:middle The first one being, should first-time NCLEX pass rates be the sole indicator 00:10:21.607 --> 00:10:29.797 position:50% align:middle that you use, and then what are some of the additional factors that we might use? 00:10:29.797 --> 00:10:34.422 position:50% align:middle And is there anything in the literature about warning signs? 00:10:34.422 --> 00:10:35.762 position:50% align:middle So we looked at that. 00:10:35.762 --> 00:10:40.402 position:50% align:middle Now remember, we had to keep this evidence-based in order 00:10:40.402 --> 00:10:42.472 position:50% align:middle to be legally defensible. 00:10:42.472 --> 00:10:49.841 position:50% align:middle So to do that, we went through a lot of...and actually this was our regulatory 00:10:49.841 --> 00:10:52.892 position:50% align:middle scholar and residence who helped us with this. 00:10:52.892 --> 00:10:58.514 position:50% align:middle And we went through a lot of different levels of evidence and we finally went 00:10:58.514 --> 00:11:04.978 position:50% align:middle to the Johns Hopkins one because it looks very strongly also at the gray literature 00:11:04.978 --> 00:11:07.339 position:50% align:middle and we knew we had to include that in this. 00:11:07.339 --> 00:11:14.219 position:50% align:middle So it rates the literature up to one, grade one is randomized control trial 00:11:14.219 --> 00:11:15.170 position:50% align:middle to grade five. 00:11:15.170 --> 00:11:22.443 position:50% align:middle And that would be, you know, an expert writing about something. 00:11:22.443 --> 00:11:26.489 position:50% align:middle And then grade four would be more like consensus of experts. 00:11:26.489 --> 00:11:36.461 position:50% align:middle So we leveled all of our literature that we had and we ended up with 74 articles. 00:11:36.461 --> 00:11:40.089 position:50% align:middle So, again, this is just a highlight of some of the findings. 00:11:40.089 --> 00:11:45.531 position:50% align:middle But for question number one, you know, first-time pass rates. 00:11:45.531 --> 00:11:51.711 position:50% align:middle Well, we did find that many of the professions use pass rates and looking 00:11:51.711 --> 00:11:57.190 position:50% align:middle at their quality, but we also found that they have reported that the 00:11:57.190 --> 00:11:59.110 position:50% align:middle outcomes are just more complex. 00:11:59.110 --> 00:12:01.971 position:50% align:middle You can't just look at outcomes. 00:12:01.971 --> 00:12:07.915 position:50% align:middle There is a specialized accreditation organization, and they did a survey. 00:12:07.915 --> 00:12:12.785 position:50% align:middle They found out that about 84% of professions that do have licensure 00:12:12.785 --> 00:12:19.095 position:50% align:middle certification exams do use them but it's part and parcel of everything that 00:12:19.095 --> 00:12:20.375 position:50% align:middle they look at. 00:12:20.375 --> 00:12:26.023 position:50% align:middle And then, interestingly, you know, those test preps that they have, Kaplan 00:12:26.023 --> 00:12:32.037 position:50% align:middle and whatever, there was a study across professions that looked at test preps 00:12:32.037 --> 00:12:34.793 position:50% align:middle and they found no difference in pass rates. 00:12:34.793 --> 00:12:39.993 position:50% align:middle Now, I know a lot of times I've pushed test preps, but what they...you know, 00:12:39.993 --> 00:12:43.857 position:50% align:middle it was just one study, so I don't know if that'll hold in the 00:12:43.857 --> 00:12:50.793 position:50% align:middle long run, but they did find what they thought was that longterm learning is 00:12:50.793 --> 00:12:56.410 position:50% align:middle reflected by your education and that the test preps just don't look at that. 00:12:57.893 --> 00:13:04.383 position:50% align:middle Then, you know, we did find that the studies do not support predictor exams. 00:13:04.383 --> 00:13:09.140 position:50% align:middle We had a national study that looked at that, we also had another statewide study 00:13:09.140 --> 00:13:10.760 position:50% align:middle that looked at that. 00:13:10.760 --> 00:13:16.440 position:50% align:middle There was one state study out of Oregon, maybe somebody from Oregon is on, 00:13:16.440 --> 00:13:20.859 position:50% align:middle that looked at second and third-time pass rates and they actually found that 00:13:20.859 --> 00:13:28.080 position:50% align:middle in 180 days, 95% of their students would pass, so they looked at that. 00:13:28.080 --> 00:13:33.908 position:50% align:middle But, you know, the conclusion was there was really no evidence out there 00:13:33.908 --> 00:13:40.428 position:50% align:middle to support solely NCLEX pass rates for the quality of the program. 00:13:40.428 --> 00:13:45.488 position:50% align:middle And so when we talk about quality of a program, Jan asked a really important 00:13:45.488 --> 00:13:48.785 position:50% align:middle question, and what do you think she asked me? 00:13:48.785 --> 00:13:50.638 position:50% align:middle You know, "Well, what do we mean by quality?" 00:13:50.638 --> 00:13:52.858 position:50% align:middle So I'm going to let Jan answer that question. 00:13:52.858 --> 00:13:56.691 position:50% align:middle - Well, when I think of quality, I think of a product or a service 00:13:56.691 --> 00:14:03.333 position:50% align:middle or an education and it is what the consumer expected and more. 00:14:03.333 --> 00:14:07.743 position:50% align:middle And when we look at education and think about the quality of education, 00:14:07.743 --> 00:14:12.073 position:50% align:middle it has to be a longterm value to the person. 00:14:12.073 --> 00:14:18.993 position:50% align:middle We have our rules that they tell us what should be the essential elements 00:14:18.993 --> 00:14:20.325 position:50% align:middle in an education program. 00:14:20.325 --> 00:14:24.533 position:50% align:middle They cannot function if they don't carry out according to our rules. 00:14:24.533 --> 00:14:30.538 position:50% align:middle But looking at quality indicators tells us the difference between using 00:14:30.538 --> 00:14:33.635 position:50% align:middle the essentials and going beyond. 00:14:33.635 --> 00:14:38.685 position:50% align:middle So often it may mean the difference of having the good pass rate or having 00:14:38.685 --> 00:14:41.565 position:50% align:middle students who can get their jobs. 00:14:41.565 --> 00:14:46.025 position:50% align:middle I think also it tells us about best practice and that seems to be something 00:14:46.025 --> 00:14:47.395 position:50% align:middle we're all looking for these days. 00:14:47.395 --> 00:14:49.795 position:50% align:middle - Right, right. 00:14:49.795 --> 00:14:55.679 position:50% align:middle So for question number two then, because we didn't find licensure pass rates went 00:14:55.679 --> 00:15:01.565 position:50% align:middle we are looking at any additional quality indicators and, of course, 00:15:01.565 --> 00:15:06.155 position:50% align:middle so normal ones come up in the literature, employment rates, graduation rates, 00:15:06.155 --> 00:15:09.511 position:50% align:middle which is very similar to retention rates. 00:15:09.511 --> 00:15:15.825 position:50% align:middle And employment rates just are not reliable because you never know what the regional 00:15:15.825 --> 00:15:17.475 position:50% align:middle employment rate is. 00:15:17.475 --> 00:15:20.519 position:50% align:middle So the literature really doesn't support that. 00:15:20.519 --> 00:15:25.354 position:50% align:middle It went to then graduation or retention rates, but even that, there was one large 00:15:25.354 --> 00:15:31.318 position:50% align:middle study in higher education that looked at 210,000 students and they found what 00:15:31.318 --> 00:15:37.146 position:50% align:middle really made the graduation rate was the characteristics of their students. 00:15:37.146 --> 00:15:43.973 position:50% align:middle You know, were they affluent, how did they do on the SAT or were they 00:15:43.973 --> 00:15:48.243 position:50% align:middle needing to work while they were in, what were their high school grades? 00:15:48.243 --> 00:15:53.780 position:50% align:middle It really didn't have anything to do with, you know, what the quality of the 00:15:53.780 --> 00:15:58.073 position:50% align:middle education was because it's where those students went to school. 00:15:58.073 --> 00:16:03.879 position:50% align:middle So there was however a better study or I mean another study that at least showed 00:16:03.879 --> 00:16:08.609 position:50% align:middle that what you do does make a difference and what they did is looked at students 00:16:08.609 --> 00:16:10.299 position:50% align:middle with similar characteristics. 00:16:10.299 --> 00:16:13.869 position:50% align:middle So they all had the same kinds of characteristics and then they used 00:16:13.869 --> 00:16:18.659 position:50% align:middle different strategies and they found that if you went above and beyond, 00:16:18.659 --> 00:16:21.379 position:50% align:middle you did have better graduation rates. 00:16:21.379 --> 00:16:28.286 position:50% align:middle So I think that is one thing that the programs can be very happy about. 00:16:28.286 --> 00:16:33.033 position:50% align:middle But neither graduation rates or employment rates, you know, 00:16:33.033 --> 00:16:35.825 position:50% align:middle are that reliable. 00:16:35.825 --> 00:16:40.979 position:50% align:middle So the literature found, kind of convincing really, two big national 00:16:40.979 --> 00:16:44.725 position:50% align:middle studies and you'll find out when you hear from Brendan, that we found it in our 00:16:44.725 --> 00:16:49.235 position:50% align:middle quantitative study as well, pretty convincing, public schools 00:16:49.235 --> 00:16:55.215 position:50% align:middle over private schools and in either private for-profit or not profit, 00:16:55.215 --> 00:16:57.185 position:50% align:middle we don't know what that reasoning is. 00:16:57.185 --> 00:17:01.962 position:50% align:middle We're hoping maybe if we get more data, if all of you participate in our annual 00:17:01.962 --> 00:17:06.092 position:50% align:middle report data collection, we'll learn why we don't know 00:17:06.092 --> 00:17:08.922 position:50% align:middle but it certainly is very convincing. 00:17:08.922 --> 00:17:16.152 position:50% align:middle Also, you know, semesters, larger cohorts but again, 00:17:16.152 --> 00:17:20.322 position:50% align:middle if we have more data, we can control for variables, and we might find out why 00:17:20.322 --> 00:17:22.112 position:50% align:middle or what's going on. 00:17:22.112 --> 00:17:26.230 position:50% align:middle Again, national study, higher percent of full-time faculty, 00:17:26.230 --> 00:17:30.618 position:50% align:middle and you'll see when Brendan gives his report, we found that as well. 00:17:30.618 --> 00:17:36.079 position:50% align:middle No use of, you know, those standardized progression exams, 00:17:36.079 --> 00:17:42.873 position:50% align:middle ATI, HESI if they didn't use those, they you know, had better outcomes now. 00:17:42.873 --> 00:17:49.089 position:50% align:middle The caveat may be a lower program put them in place, and they didn't have a chance 00:17:49.089 --> 00:17:51.073 position:50% align:middle to impact the outcomes. 00:17:51.073 --> 00:17:55.153 position:50% align:middle So, you know, we're not saying nobody should use them but, you know, 00:17:55.153 --> 00:17:57.273 position:50% align:middle those were just what the findings were. 00:17:57.273 --> 00:18:01.957 position:50% align:middle For curricular components, quality clinical experiences really came 00:18:01.957 --> 00:18:08.954 position:50% align:middle out loud and clear and this has come out in everything that we've looked at. 00:18:08.954 --> 00:18:12.387 position:50% align:middle Clinical judgment has come out in the literature, and, Jan, 00:18:12.387 --> 00:18:15.260 position:50% align:middle you are also the Chair of the NCLEX Committee. 00:18:15.280 --> 00:18:16.897 position:50% align:middle You've been around a lot at NCSBN. 00:18:16.897 --> 00:18:21.357 position:50% align:middle - I'm very happy to know that that came up at a high percentage as being 00:18:21.357 --> 00:18:22.977 position:50% align:middle a quality indicator. 00:18:22.977 --> 00:18:27.952 position:50% align:middle And today, we seem more interested in critical thinking and clinical reasoning 00:18:27.952 --> 00:18:32.492 position:50% align:middle and clinical judgment because that's going to be on the next generation NCLEX. 00:18:32.492 --> 00:18:37.752 position:50% align:middle We know that new nurses have to make very important clinical judgments, 00:18:37.752 --> 00:18:42.373 position:50% align:middle so to be able to measure their ability will help to determine that they're ready 00:18:42.373 --> 00:18:46.962 position:50% align:middle for practice, ready to be safe competent practitioners. 00:18:46.962 --> 00:18:48.482 position:50% align:middle So I'm glad to see that one there. 00:18:50.009 --> 00:18:53.822 position:50% align:middle - And I was glad to see coursework and quality and safety, you know, 00:18:53.822 --> 00:18:59.252 position:50% align:middle I've worked with Houston group for a while now and a lot of literature came 00:18:59.252 --> 00:19:03.865 position:50% align:middle up especially in the areas of delegation and I'm not just talking about one study 00:19:03.865 --> 00:19:07.745 position:50% align:middle but I'm talking about a lot, interprofessional communication, 00:19:07.745 --> 00:19:13.502 position:50% align:middle leadership and time management and all of those are part of QSEN 00:19:13.502 --> 00:19:15.085 position:50% align:middle or quality and safety. 00:19:15.085 --> 00:19:20.325 position:50% align:middle So that was all, I think, very positive that came up. 00:19:20.325 --> 00:19:26.299 position:50% align:middle So also in question number two, we looked at faculty qualifications 00:19:26.299 --> 00:19:31.355 position:50% align:middle in the literature, we kind of thought more would come in on that. 00:19:31.355 --> 00:19:34.288 position:50% align:middle It was really more on the non-research literature. 00:19:34.288 --> 00:19:36.588 position:50% align:middle The same with program evaluation. 00:19:36.588 --> 00:19:38.844 position:50% align:middle I think the education consultants really 00:19:38.844 --> 00:19:43.165 position:50% align:middle find a systematic evaluation to be very important. 00:19:43.165 --> 00:19:47.745 position:50% align:middle It was in the non-research literature but I don't think there've been many studies 00:19:47.745 --> 00:19:51.113 position:50% align:middle looking at a program that does and doesn't and seeing the outcomes. 00:19:51.113 --> 00:19:54.505 position:50% align:middle But you have found that very valuable with your programs, right? 00:19:54.505 --> 00:19:56.705 position:50% align:middle And then accreditation. 00:19:56.705 --> 00:20:02.132 position:50% align:middle Accreditation in one of the studies didn't have a positive effect. 00:20:02.132 --> 00:20:06.623 position:50% align:middle But on the other hand, they only had 6% of their programs that weren't accredited. 00:20:06.623 --> 00:20:10.186 position:50% align:middle NCSBN, our committee did, remember the Metrics and 00:20:10.186 --> 00:20:11.636 position:50% align:middle Outcomes Committee? 00:20:11.636 --> 00:20:15.646 position:50% align:middle We looked at every single program and then we looked to see and we, you know, 00:20:15.646 --> 00:20:20.026 position:50% align:middle looked at the program codes to do that to see if they were accredited and we found 00:20:20.026 --> 00:20:26.306 position:50% align:middle strong significant difference if they were accredited or not based on their 00:20:26.306 --> 00:20:28.636 position:50% align:middle NCLEX pass rates. 00:20:28.636 --> 00:20:32.846 position:50% align:middle So then, you know, we wanted to ask about warning signs in the literature. 00:20:32.846 --> 00:20:34.807 position:50% align:middle We found no research on warning signs. 00:20:34.807 --> 00:20:40.406 position:50% align:middle I guess where most of the writing came from, came from JNR. 00:20:40.406 --> 00:20:46.006 position:50% align:middle You know, the warning signs came from regulators, this is what all of you see. 00:20:46.006 --> 00:20:53.006 position:50% align:middle Jan, do these look like warning signs that you see normally in your programs? 00:20:53.006 --> 00:20:54.376 position:50% align:middle - Definitely, definitely. 00:20:54.376 --> 00:21:02.065 position:50% align:middle And looking at the ones on high faculty turnover and high administrator turnover, 00:21:02.065 --> 00:21:06.727 position:50% align:middle this is really a clue that things are not going well. 00:21:06.727 --> 00:21:12.747 position:50% align:middle If there are a lot of faculty turnover, then either the work situation is not 00:21:12.747 --> 00:21:20.254 position:50% align:middle compatible with life or the expectations are too high and faculty leave. 00:21:20.254 --> 00:21:24.167 position:50% align:middle And the new faculty who come in then need to be oriented maybe with not 00:21:24.167 --> 00:21:30.702 position:50% align:middle as much practice in teaching so it weakens the educational process for the students. 00:21:30.702 --> 00:21:35.039 position:50% align:middle And the same thing with director, a strong director is a key 00:21:35.039 --> 00:21:37.176 position:50% align:middle to a good program. 00:21:37.176 --> 00:21:40.830 position:50% align:middle We had one program in five years, they had eight directors, 00:21:40.830 --> 00:21:45.336 position:50% align:middle so, you know, a problem and that program didn't, of course, last. 00:21:48.060 --> 00:21:53.726 position:50% align:middle - You know, I think also poor clinical placements, poor clinical and what really 00:21:53.726 --> 00:21:57.276 position:50% align:middle came out very strong in these studies and the literature 00:21:57.276 --> 00:22:01.458 position:50% align:middle was quality clinical education. - Yes. 00:22:01.458 --> 00:22:08.028 position:50% align:middle - So in the end, after that literature review, we found out that more study 00:22:08.028 --> 00:22:11.848 position:50% align:middle is warranted, we needed more study on this. 00:22:11.848 --> 00:22:18.886 position:50% align:middle So we went to the board and we said, "Maybe we need to do the studies ourself." 00:22:18.886 --> 00:22:25.579 position:50% align:middle And the board agreed, and we went forward with our Delphi, our annual report 00:22:25.579 --> 00:22:27.268 position:50% align:middle and our site visit studies. 00:22:27.268 --> 00:22:32.926 position:50% align:middle And I have to say the boards were wonderful in cooperating with all three 00:22:32.926 --> 00:22:36.336 position:50% align:middle of them because, you know, in the Delphi, they were participants 00:22:36.336 --> 00:22:42.420 position:50% align:middle and then, you know, provided us with those annual reports and site visit documents 00:22:42.420 --> 00:22:44.966 position:50% align:middle which we know was not easy. 00:22:46.815 --> 00:22:50.996 position:50% align:middle - So we're going to go into now the National Delphi. 00:22:50.996 --> 00:22:55.841 position:50% align:middle For the Delphi study, if you remember, some of you were probably 00:22:55.841 --> 00:22:57.240 position:50% align:middle part of the Delphi study. 00:22:57.240 --> 00:22:59.761 position:50% align:middle We sent out emails. 00:22:59.761 --> 00:23:06.947 position:50% align:middle We sent it out for the...you know, we had those people that worked in 00:23:06.947 --> 00:23:08.875 position:50% align:middle the clinical with new graduates. 00:23:08.875 --> 00:23:15.980 position:50% align:middle For that, we use the ANPD Organization, which really is an organization 00:23:15.980 --> 00:23:17.750 position:50% align:middle for clinical educators. 00:23:17.750 --> 00:23:22.752 position:50% align:middle So we use them, we got a list from them, and we sent out emails if 00:23:22.752 --> 00:23:24.150 position:50% align:middle they were interested. 00:23:24.150 --> 00:23:27.970 position:50% align:middle We sent out emails to the education consultants were they interested, 00:23:27.970 --> 00:23:34.915 position:50% align:middle and to educators from a list we have from the NCLEX codes. 00:23:34.915 --> 00:23:41.955 position:50% align:middle So we sent those out and you can see we had a fairly good response rate with all 00:23:41.955 --> 00:23:43.205 position:50% align:middle three of the groups. 00:23:43.205 --> 00:23:48.595 position:50% align:middle We had 50 education consultants, which I just thought was wonderful. 00:23:48.595 --> 00:23:55.955 position:50% align:middle And what we were doing is looking at consensus, what you do with a Delphi is 00:23:55.955 --> 00:24:01.105 position:50% align:middle send out first an email, which we did to everybody, however you do it, 00:24:01.105 --> 00:24:06.505 position:50% align:middle it doesn't have to be an email, sometimes it's paper and pencil. 00:24:06.505 --> 00:24:10.685 position:50% align:middle And we asked them, you know, "What are the quality indicators that you 00:24:10.685 --> 00:24:14.935 position:50% align:middle would save on a program that graduates safe and competent nurses? 00:24:14.935 --> 00:24:19.665 position:50% align:middle What are the outcomes of a program or what outcomes would you like to see? 00:24:19.665 --> 00:24:25.545 position:50% align:middle And then also what are the warning signs before a program begins to fall?" 00:24:25.545 --> 00:24:30.958 position:50% align:middle And before we even got started, we decided on, and this was based on the 00:24:30.958 --> 00:24:36.150 position:50% align:middle literature and really that's more based on Robert's rules, that we would have to have 00:24:36.150 --> 00:24:37.643 position:50% align:middle with all three of these groups. 00:24:37.643 --> 00:24:43.559 position:50% align:middle Remember, they're quite variant, a 67% consensus. 00:24:43.559 --> 00:24:45.780 position:50% align:middle So that was defined for us. 00:24:45.780 --> 00:24:50.110 position:50% align:middle And then we sent out the survey to see what they, you know, 00:24:50.110 --> 00:24:58.040 position:50% align:middle thought and then we got it back and did some great content analysis on it. 00:24:58.040 --> 00:25:01.884 position:50% align:middle And again, our scholar in residence helped us with some of that. 00:25:01.884 --> 00:25:05.891 position:50% align:middle And then we sent it out for consensus. 00:25:05.891 --> 00:25:10.601 position:50% align:middle And believe it or not, every single one of the factors had a 00:25:10.601 --> 00:25:15.251 position:50% align:middle consensus of over 67%, which is what we had set. 00:25:15.251 --> 00:25:17.701 position:50% align:middle So everything was included. 00:25:17.701 --> 00:25:22.441 position:50% align:middle And I think we were very, very lucky with that because some people 00:25:22.441 --> 00:25:27.141 position:50% align:middle doing Delphi's go on for maybe four or five different surveys trying 00:25:27.141 --> 00:25:29.081 position:50% align:middle to get consensus. 00:25:29.081 --> 00:25:31.760 position:50% align:middle So here are some of our results. 00:25:31.760 --> 00:25:37.890 position:50% align:middle You can see I'm really happy with the first two, quality and safety integration 00:25:37.890 --> 00:25:38.752 position:50% align:middle in curriculum. 00:25:38.752 --> 00:25:41.975 position:50% align:middle You know, how can you get betterfor regulation than that? 00:25:41.975 --> 00:25:43.600 position:50% align:middle That is wonderful. 00:25:43.600 --> 00:25:48.210 position:50% align:middle And then critical thinking and clinical reasoning skills. 00:25:48.210 --> 00:25:51.830 position:50% align:middle And, Jan, I know you just talked about clinical judgment, aren't you happy to see 00:25:51.830 --> 00:25:53.522 position:50% align:middle this on the Delphi as well? - Absolutely. 00:25:53.522 --> 00:25:57.670 position:50% align:middle - And I feel like critical thinking is the starting place and that's in process 00:25:57.670 --> 00:26:01.019 position:50% align:middle with clinical reasoning and then the outcome is good clinical judgment. 00:26:01.019 --> 00:26:01.849 position:50% align:middle - Yes. 00:26:01.849 --> 00:26:07.495 position:50% align:middle And, you know, if you think about it, it's the educators agreeing with those that 00:26:07.495 --> 00:26:13.566 position:50% align:middle work with new nurses and the regulators, so this is excellent. 00:26:13.566 --> 00:26:18.549 position:50% align:middle You know, faculty role model professional behaviors. 00:26:18.549 --> 00:26:20.769 position:50% align:middle Jan was going to talk a little bit about that. 00:26:20.769 --> 00:26:21.339 position:50% align:middle - Yes. 00:26:21.339 --> 00:26:26.519 position:50% align:middle I read a study a few years ago about medical school, and they found that 00:26:26.519 --> 00:26:32.249 position:50% align:middle in medical school, the young doctors were not learning to make proper diagnoses 00:26:32.249 --> 00:26:37.049 position:50% align:middle because they weren't asking enough questions and doing enough assessment. 00:26:37.049 --> 00:26:42.089 position:50% align:middle And so there was the critical thinking, it was the reasoning, and they realized 00:26:42.089 --> 00:26:47.419 position:50% align:middle that critical thinking is caught not taught. 00:26:47.419 --> 00:26:51.599 position:50% align:middle So they were watching their professors and learning from them and learning 00:26:51.599 --> 00:26:53.699 position:50% align:middle about their thinking. 00:26:53.699 --> 00:26:58.259 position:50% align:middle So to me, that's the key in nursing education today. 00:26:58.259 --> 00:27:03.012 position:50% align:middle Faculty must talk more about their thinking, explore the thinking of the 00:27:03.012 --> 00:27:09.572 position:50% align:middle nursing student to help them improve their thinking to make those good judgments. 00:27:09.572 --> 00:27:11.933 position:50% align:middle - Wow, you should write an article on that. 00:27:11.933 --> 00:27:14.225 position:50% align:middle That is great. 00:27:14.225 --> 00:27:18.362 position:50% align:middle And then, you know, clinical experiences with actual patients as I've said before, 00:27:18.362 --> 00:27:22.122 position:50% align:middle those quality clinical experiences are really important. 00:27:22.122 --> 00:27:26.182 position:50% align:middle And this has come up time and time again in the Delphi study as well as some of our 00:27:26.182 --> 00:27:28.322 position:50% align:middle other studies and the literature. 00:27:28.322 --> 00:27:34.288 position:50% align:middle Then a systematic process to address student practice errors. 00:27:34.288 --> 00:27:37.799 position:50% align:middle I was particularly happy to see that because as many of you know, we have the 00:27:37.799 --> 00:27:43.148 position:50% align:middle safe student reports study and there we have about 200 programs involved in that. 00:27:43.148 --> 00:27:47.178 position:50% align:middle And it's just a database for them to keep track of their student errors 00:27:47.178 --> 00:27:48.288 position:50% align:middle and near misses. 00:27:48.288 --> 00:27:53.638 position:50% align:middle But it helps them then to compare it to the aggregate to see where they might be 00:27:53.638 --> 00:27:57.408 position:50% align:middle making improvements and to think about it and track it. 00:27:57.408 --> 00:28:01.601 position:50% align:middle And that is exactly what the Delphi found here. 00:28:01.601 --> 00:28:06.431 position:50% align:middle And then faculty demonstrate current clinical practice. 00:28:06.431 --> 00:28:11.524 position:50% align:middle You know, as I had said earlier, we had some experts on our course, 00:28:11.524 --> 00:28:12.531 position:50% align:middle the metrics and outcomes, 00:28:12.531 --> 00:28:16.741 position:50% align:middle And you probably remember the call we had, Jan, with Joan Kavanagh. 00:28:16.741 --> 00:28:21.561 position:50% align:middle She's from the Cleveland Clinic and, you know, really that transition 00:28:21.561 --> 00:28:24.111 position:50% align:middle of practice is very important for her. 00:28:24.111 --> 00:28:29.411 position:50% align:middle And so she talked about one day bringing in the faculty rather than the students 00:28:29.411 --> 00:28:33.462 position:50% align:middle and having them take the medication test that all of their students take. 00:28:33.462 --> 00:28:35.930 position:50% align:middle And several faculty failed. 00:28:35.930 --> 00:28:38.312 position:50% align:middle And she said, "What's up guys? 00:28:38.312 --> 00:28:40.139 position:50% align:middle You're the ones teaching, how come you're failing 00:28:40.139 --> 00:28:41.942 position:50% align:middle and your students are passing?" 00:28:41.942 --> 00:28:44.772 position:50% align:middle And they said, "Well, some of us haven't been in the clinical 00:28:44.772 --> 00:28:46.042 position:50% align:middle for 15 years." 00:28:46.042 --> 00:28:49.259 position:50% align:middle And she said, "Well, that shouldn't be." 00:28:49.259 --> 00:28:55.022 position:50% align:middle And so you'll see in our guidelines, one of our guidelines is at least in the 00:28:55.022 --> 00:28:59.302 position:50% align:middle last five years, they should have been in clinical experiences. 00:29:00.492 --> 00:29:10.073 position:50% align:middle So next in the Delphi is the consistent leadership. 00:29:10.073 --> 00:29:13.195 position:50% align:middle And, Jan, tell us a little bit your experience 00:29:13.195 --> 00:29:17.289 position:50% align:middle with consistent administrative leadership. 00:29:17.289 --> 00:29:23.753 position:50% align:middle - Well, we often find when we visit a program with issues that there is a 00:29:23.753 --> 00:29:26.125 position:50% align:middle constant turnover of the leader. 00:29:26.125 --> 00:29:30.400 position:50% align:middle And we find that many new leaders of nursing programs do not know 00:29:30.400 --> 00:29:31.948 position:50% align:middle how to be a leader. 00:29:31.948 --> 00:29:36.618 position:50% align:middle They're placed in this position and they're wondering how to handle it. 00:29:36.618 --> 00:29:40.718 position:50% align:middle So I know in our orientation, we're starting to put more content and 00:29:40.718 --> 00:29:42.711 position:50% align:middle about being a leader. 00:29:42.711 --> 00:29:48.072 position:50% align:middle But schools need to be able to provide opportunities for their 00:29:48.072 --> 00:29:53.878 position:50% align:middle directors to go to leadership training and to constantly be upgrading their 00:29:53.878 --> 00:29:55.847 position:50% align:middle skills in leadership. 00:29:55.847 --> 00:30:01.201 position:50% align:middle - That's interesting, other education consultants could do that as well. 00:30:01.201 --> 00:30:05.901 position:50% align:middle You know, I also think the pattern of NCLEX pass rates and, of course, we knew 00:30:05.901 --> 00:30:10.882 position:50% align:middle that would be there, but, you know, when you'll hear from Brendan that it's 00:30:10.882 --> 00:30:13.006 position:50% align:middle kind of a lagging indicator. 00:30:13.006 --> 00:30:17.820 position:50% align:middle Other things have happened first and then NCLEX pass rates begin to fall. 00:30:17.820 --> 00:30:20.471 position:50% align:middle Have you found that, Jan, with programs? - Yes. 00:30:20.471 --> 00:30:24.779 position:50% align:middle - And I guess we're always looking backwards but we require our programs 00:30:24.779 --> 00:30:28.031 position:50% align:middle to write a self-study when their pass rate drops. 00:30:28.031 --> 00:30:32.333 position:50% align:middle And so they are looking backward to see what happened, looking at the differences 00:30:32.333 --> 00:30:37.583 position:50% align:middle between the students, the curriculum, the faculty, and they do recognize areas 00:30:37.583 --> 00:30:40.183 position:50% align:middle where they need to have quality improvement. 00:30:40.183 --> 00:30:45.793 position:50% align:middle So it does relate to what has happened rather than what's going to happen. 00:30:45.793 --> 00:30:46.993 position:50% align:middle - Right. 00:30:46.993 --> 00:30:51.783 position:50% align:middle And you'll hear that also from Brendan, and that's another reason that NCLEX pass 00:30:51.783 --> 00:30:55.782 position:50% align:middle rates shouldn't be the only thing you look at because by that time, the program 00:30:55.782 --> 00:30:58.072 position:50% align:middle has a lot of problems. 00:30:58.736 --> 00:31:03.353 position:50% align:middle Then for the quality indicators, you know, one of the things that we're beginning 00:31:03.353 --> 00:31:06.513 position:50% align:middle to see patterns with already, we've looked at the literature, now we're 00:31:06.513 --> 00:31:12.504 position:50% align:middle looking at the Delphi is this consistent full-time faculty as opposed 00:31:12.504 --> 00:31:14.223 position:50% align:middle to adjunct faculty. 00:31:14.223 --> 00:31:17.249 position:50% align:middle We saw that as well in the literature. 00:31:17.249 --> 00:31:20.761 position:50% align:middle You know, we're seeing patterns here, which was very important 00:31:20.761 --> 00:31:22.783 position:50% align:middle when you're developing evidence. 00:31:22.783 --> 00:31:27.213 position:50% align:middle I assume that you see that as well when you're out there with programs. 00:31:27.213 --> 00:31:27.933 position:50% align:middle - Yes. 00:31:27.933 --> 00:31:32.694 position:50% align:middle And we're concerned because we see in the numbers of full time and part-time faculty 00:31:32.694 --> 00:31:36.214 position:50% align:middle that more programs are relying on more adjuncts. 00:31:36.214 --> 00:31:41.944 position:50% align:middle And that means there are fewer seasoned faculty who are not just teaching 00:31:41.944 --> 00:31:47.054 position:50% align:middle but they're looking at the curriculum, they're in the evaluation plan. 00:31:47.054 --> 00:31:52.394 position:50% align:middle And so the part-time people don't get involved in that side of the nursing 00:31:52.394 --> 00:31:54.634 position:50% align:middle program and so many lose out. 00:31:54.634 --> 00:31:58.794 position:50% align:middle But full- time faculty as you know from this simulation study that are teaching 00:31:58.794 --> 00:32:03.400 position:50% align:middle didactic and clinical provide the best instruction. 00:32:03.400 --> 00:32:07.370 position:50% align:middle - And, you know, again, I think it's really interesting 00:32:07.370 --> 00:32:09.420 position:50% align:middle to see these trends. 00:32:09.420 --> 00:32:13.120 position:50% align:middle So now we go to the warning signs. 00:32:13.120 --> 00:32:18.160 position:50% align:middle Remember, in the literature, we didn't really see much in terms of research 00:32:18.160 --> 00:32:24.900 position:50% align:middle on warning signs but now we are seeing some certainly lack of consistent and 00:32:24.900 --> 00:32:28.484 position:50% align:middle prepared clinical faculty 100%. 00:32:28.484 --> 00:32:32.707 position:50% align:middle You rarely see 100% on Delphi. 00:32:32.707 --> 00:32:34.979 position:50% align:middle And, you know, limited clinical experiences 00:32:34.979 --> 00:32:37.593 position:50% align:middle like in that quality clinical experience, stands out. 00:32:37.593 --> 00:32:40.966 position:50% align:middle Poor leadership, that's there as well. 00:32:40.966 --> 00:32:42.870 position:50% align:middle NCLEX pass rate trend. 00:32:42.870 --> 00:32:49.273 position:50% align:middle But complaints to Boards of Nursing, now, we haven't seen that yet. 00:32:49.273 --> 00:32:51.496 position:50% align:middle Do you keep track of complaints, Jan? 00:32:51.496 --> 00:32:53.446 position:50% align:middle How would the board do this? 00:32:53.446 --> 00:32:59.065 position:50% align:middle - We actually started a log a few years ago where we log in compliants as we 00:32:59.065 --> 00:33:02.177 position:50% align:middle receive them with no names. 00:33:02.177 --> 00:33:07.767 position:50% align:middle We do have the school name, the complaint itself, and how we 00:33:07.767 --> 00:33:09.213 position:50% align:middle help resolve it. 00:33:09.213 --> 00:33:14.202 position:50% align:middle And we do see a lot of complaints, but when we see multiple complaints about 00:33:14.202 --> 00:33:19.950 position:50% align:middle one program, especially from students, we know there's an issue 00:33:19.950 --> 00:33:22.407 position:50% align:middle because it is a red flag. 00:33:22.407 --> 00:33:26.402 position:50% align:middle - What are some of those complaints about, if I might ask? 00:33:26.402 --> 00:33:29.770 position:50% align:middle I mean, you can keep the school name out. 00:33:29.770 --> 00:33:33.687 position:50% align:middle - Many complaints are because the students all of a sudden toward the end of their 00:33:33.687 --> 00:33:37.670 position:50% align:middle program are required to make a certain score on a standardized exam. 00:33:37.670 --> 00:33:41.533 position:50% align:middle And the policies are changed so that if they don't make that score, 00:33:41.533 --> 00:33:43.557 position:50% align:middle they may not pass. 00:33:43.557 --> 00:33:45.487 position:50% align:middle We hear that a lot. 00:33:45.487 --> 00:33:51.577 position:50% align:middle We hear complaints about faculty behavior and we have no rules about rudeness or 00:33:51.577 --> 00:33:54.077 position:50% align:middle behavior problems with faculty. 00:33:54.077 --> 00:33:59.697 position:50% align:middle So we always let the director know of complaints so that they know what we're 00:33:59.697 --> 00:34:04.721 position:50% align:middle hearing and to see if they can do something to change that. 00:34:04.721 --> 00:34:06.171 position:50% align:middle - Okay. 00:34:06.171 --> 00:34:07.012 position:50% align:middle Interesting. 00:34:07.012 --> 00:34:09.482 position:50% align:middle And so you have a regular log 00:34:09.482 --> 00:34:10.900 position:50% align:middle of those complaints. - Yes, we do. 00:34:10.900 --> 00:34:13.562 position:50% align:middle So you might talk to Jan. 00:34:13.562 --> 00:34:15.762 position:50% align:middle - It's the way for us to be accountable. 00:34:15.762 --> 00:34:17.474 position:50% align:middle - Yes. 00:34:17.474 --> 00:34:23.372 position:50% align:middle And, you know, now we see it's a red flag gets in the literature. 00:34:23.372 --> 00:34:29.722 position:50% align:middle So going forward with more warning signs, you know, some of these 00:34:29.722 --> 00:34:30.850 position:50% align:middle kind of make sense. 00:34:30.850 --> 00:34:35.011 position:50% align:middle Again, we talk about I don't know how long, many times we can talk 00:34:35.011 --> 00:34:36.911 position:50% align:middle about administrative attrition. 00:34:36.911 --> 00:34:40.294 position:50% align:middle We've talked about literature review, Brendan will talk about it, 00:34:40.294 --> 00:34:42.404 position:50% align:middle and we've talked about... - Just on administration attrition, 00:34:42.404 --> 00:34:44.902 position:50% align:middle I counted the new directors we approved. 00:34:44.902 --> 00:34:48.291 position:50% align:middle For one physical year, last year, there were 65. 00:34:48.291 --> 00:34:49.756 position:50% align:middle - Oh my gosh. 00:34:49.756 --> 00:34:51.954 position:50% align:middle And other boards have told me that too. 00:34:51.954 --> 00:34:57.607 position:50% align:middle They've said it's just a huge problem and now, you know, we have it in evidence. 00:34:57.607 --> 00:35:03.886 position:50% align:middle But I thought it was interesting practice is unwilling to host clinicals experience. 00:35:03.886 --> 00:35:10.972 position:50% align:middle I found this when I was teaching in clinical, there was another university 00:35:10.972 --> 00:35:13.616 position:50% align:middle that wanted to come in and have their students there at the same 00:35:13.616 --> 00:35:15.260 position:50% align:middle hospital I had students. 00:35:15.596 --> 00:35:20.826 position:50% align:middle And the faculty member would literally sit in the break room and have coffee the 00:35:20.826 --> 00:35:24.146 position:50% align:middle whole time and assign her students to the nurses. 00:35:24.146 --> 00:35:28.868 position:50% align:middle And finally, the hospital said, "No, you know, you don't pay us. 00:35:28.868 --> 00:35:33.927 position:50% align:middle We can't instruct your students," and they didn't allow them to come. 00:35:33.927 --> 00:35:42.538 position:50% align:middle Now if you looked at this, would you ask the students or would you go 00:35:42.538 --> 00:35:47.348 position:50% align:middle to clinical sites or how would you get that information? 00:35:48.914 --> 00:35:50.528 position:50% align:middle - That's a good question. 00:35:50.528 --> 00:35:54.738 position:50% align:middle I think I would first talk to the people in the program to see what had happened 00:35:54.738 --> 00:35:59.465 position:50% align:middle to create a negative response from the clinical agency. 00:35:59.465 --> 00:36:03.841 position:50% align:middle But then it might be worth a visit to the clinical agency because we rely 00:36:03.841 --> 00:36:09.101 position:50% align:middle on clinical settings to provide that kind of practice for our students. 00:36:09.101 --> 00:36:13.011 position:50% align:middle - And I think some boards do make visits to clinical settings, but, 00:36:13.011 --> 00:36:14.393 position:50% align:middle you know, I know a lot don't. 00:36:14.393 --> 00:36:20.811 position:50% align:middle It does seem like you could get a fair amount of information by doing that. 00:36:20.811 --> 00:36:25.011 position:50% align:middle And then the curriculum is teaching to the NCLEX again. 00:36:25.011 --> 00:36:31.294 position:50% align:middle I was at a meeting once and somebody actually was on the podium talking 00:36:31.294 --> 00:36:36.617 position:50% align:middle about how their curriculum was the NCLEX detailed test plan. 00:36:36.617 --> 00:36:40.703 position:50% align:middle And, you know, I raised my hand and said, "Well, you're really teaching for that 00:36:40.703 --> 00:36:43.649 position:50% align:middle first year in practice and that's about it." 00:36:43.649 --> 00:36:45.967 position:50% align:middle Have you had experience with anything like that? 00:36:45.967 --> 00:36:49.927 position:50% align:middle - Occasionally, we'll see one of the program goals for a program is that their 00:36:49.927 --> 00:36:52.117 position:50% align:middle students all pass the NCLEX. 00:36:52.117 --> 00:36:57.472 position:50% align:middle So we know they're teaching to help them pass that test and that's the focus rather 00:36:57.472 --> 00:37:02.764 position:50% align:middle than teaching the student to be a good nurse the rest of their career. 00:37:02.764 --> 00:37:04.474 position:50% align:middle So it is a problem. 00:37:04.474 --> 00:37:09.214 position:50% align:middle I think that for programs that really teach to the NCLEX and have those high 00:37:09.214 --> 00:37:13.956 position:50% align:middle scores on standardized tests as a graduation point, maybe those students 00:37:13.956 --> 00:37:20.214 position:50% align:middle do go and pass the NCLEX but that doesn't really tell me it was a quality program. 00:37:20.214 --> 00:37:24.744 position:50% align:middle - Right, because you're teaching for their career and not for just passing 00:37:24.744 --> 00:37:26.624 position:50% align:middle the NCLEX. 00:37:26.624 --> 00:37:32.490 position:50% align:middle So then, on the Delphi, we looked for outcomes and, you know, 00:37:32.490 --> 00:37:38.220 position:50% align:middle some of the outcomes that were given were a little bit hard to collect. 00:37:38.220 --> 00:37:42.894 position:50% align:middle One of the things we did ask on the email was any outcomes that you think 00:37:42.894 --> 00:37:47.217 position:50% align:middle the Board of Nursing could collect but we didn't get a lot of that. 00:37:47.217 --> 00:37:52.395 position:50% align:middle Now, we have NCLEX pass rates and certainly you can get that, 00:37:52.395 --> 00:37:57.860 position:50% align:middle but again there's that relationship with the clinical partners. 00:37:57.860 --> 00:38:01.809 position:50% align:middle Would you suggest what's going in to the clinical? 00:38:01.809 --> 00:38:07.079 position:50% align:middle I would think you could get some of that from having an interview with students. 00:38:07.079 --> 00:38:08.839 position:50% align:middle - Yes, you can. 00:38:08.839 --> 00:38:11.650 position:50% align:middle Students are very candid when you're interviewing them. 00:38:11.650 --> 00:38:13.459 position:50% align:middle They will tell you what's going on. 00:38:13.459 --> 00:38:18.119 position:50% align:middle But I'd like to say one thing that, in our state, we are concerned because there's a 00:38:18.119 --> 00:38:25.859 position:50% align:middle gap between what education is producing that may be practices and CEOs ready. 00:38:25.859 --> 00:38:32.515 position:50% align:middle And so we are currently involved in having a world cafe this month where practice and 00:38:32.515 --> 00:38:37.985 position:50% align:middle educators will be speaking to each to try to resolve some of these issues. 00:38:37.985 --> 00:38:45.395 position:50% align:middle This has become such a big event that we now have joining us the Texas Organization 00:38:45.395 --> 00:38:50.615 position:50% align:middle of Nurse Leaders and the APRN Organization because they want to be a part of this. 00:38:50.615 --> 00:38:54.744 position:50% align:middle So, obviously, it's a big concern. 00:38:54.744 --> 00:38:57.195 position:50% align:middle - And I'm going to be there as well. - You going to be there? 00:38:57.195 --> 00:39:00.690 position:50% align:middle - I'm going to be talking about our transition of practice study. 00:39:00.690 --> 00:39:05.939 position:50% align:middle And then preparedness for an interprofessional practice, again, 00:39:05.939 --> 00:39:11.029 position:50% align:middle I think that was going to be very difficult to collect because how do you 00:39:11.029 --> 00:39:15.889 position:50% align:middle measure the preparedness for interprofessional? 00:39:15.889 --> 00:39:20.639 position:50% align:middle The graduate satisfaction with a program, it's hard to get that back. 00:39:20.639 --> 00:39:24.429 position:50% align:middle You can have that in your rules but then it's really hard for the programs to get 00:39:24.429 --> 00:39:28.360 position:50% align:middle it back and sometimes they have a response rate of three or something 00:39:28.360 --> 00:39:32.510 position:50% align:middle and, of course, it's all the people that love the program or people they knew 00:39:32.510 --> 00:39:34.252 position:50% align:middle so I don't know how reliable it is. 00:39:34.252 --> 00:39:38.957 position:50% align:middle Graduation rates, we have talked about, not really the best. 00:39:38.957 --> 00:39:43.402 position:50% align:middle We will be putting it on our annual report template, we're going to try to study 00:39:43.402 --> 00:39:45.689 position:50% align:middle it more, but at this point... 00:39:45.689 --> 00:39:51.605 position:50% align:middle And, again, employment rates came up but, you know, not really as reliable 00:39:51.605 --> 00:39:53.087 position:50% align:middle as it could be. 00:39:53.087 --> 00:39:58.223 position:50% align:middle So the last one was history of discipline with the Board of Nursing. 00:39:58.223 --> 00:40:04.726 position:50% align:middle And you see it...I mean, 67% reached our...so it came up as one but 00:40:04.726 --> 00:40:07.252 position:50% align:middle it's still the lowest one of all. 00:40:07.252 --> 00:40:11.012 position:50% align:middle History of discipline with a board, that's gotta be hard to collect, right, Jan? 00:40:11.012 --> 00:40:12.412 position:50% align:middle - Yes, yes. 00:40:13.377 --> 00:40:21.952 position:50% align:middle I know that our enforcement department is supposed to give us when a student is 00:40:21.952 --> 00:40:28.652 position:50% align:middle disciplined within the first year after graduation but the discipline activity 00:40:28.652 --> 00:40:32.533 position:50% align:middle takes longer than that, so we really don't get that information. 00:40:32.533 --> 00:40:37.125 position:50% align:middle - Right, and not only that but it's usually much more egregious than 00:40:37.125 --> 00:40:43.093 position:50% align:middle what you might want to see a good practicing nurse do that would come to the 00:40:43.093 --> 00:40:47.568 position:50% align:middle board so that's probably nothing that they could use either. 00:40:47.568 --> 00:40:54.153 position:50% align:middle So that, folks, is the end of the National Delphi, and it's about time for a break. 00:40:54.153 --> 00:40:58.713 position:50% align:middle We're going to have a break for 15 minutes until 1:00 Central Time. 00:40:58.713 --> 00:41:00.727 position:50% align:middle I don't know what time you're on. 00:41:00.727 --> 00:41:05.955 position:50% align:middle So think about some questions and the answers that you are...not answers, 00:41:05.955 --> 00:41:12.197 position:50% align:middle just questions that you would want to send us and we would be able to answer 00:41:12.197 --> 00:41:14.865 position:50% align:middle later on in that session. 00:41:14.865 --> 00:41:21.125 position:50% align:middle Coming up next is Brendan Martin after the break at 1:00 Central Time. 00:41:21.125 --> 00:41:26.105 position:50% align:middle And he's going to talk about the annual report study as well as giving you that 00:41:26.105 --> 00:41:29.904 position:50% align:middle demonstration of our brand new performance indicator. 00:41:29.904 --> 00:41:33.319 position:50% align:middle And we don't even know if that term for it will stick. 00:41:33.319 --> 00:41:37.709 position:50% align:middle I kind of like performance index too, so we'll see if that sticks. 00:41:37.709 --> 00:41:41.797 position:50% align:middle But thank you very much for your attentiveness, and we'll see you back 00:41:41.797 --> 00:41:43.950 position:50% align:middle at 1:00 Central Time.