Nick Humensky: Alright. Thank you, Jorge.
Nick Humensky: Good morning, everyone. My name is Nick Uminski, and I'm here with my colleague, Dr. Cindy Williamson.
Nick Humensky: Tell you a little bit more about ourselves in just a moment. But 1st I want to thank everyone for being here today. We're very excited to showcase our collaborative dashboard effort, and hope that you walk away with some some practical insight and knowledge about how you could apply similar lessons to a project that you may be working on
Nick Humensky: first.st Cindy is going to start out by telling you about how this project came to be, and then she's going to discuss how her team had a desire to move towards a more dynamic approach to analyzing evaluation data. And then I'm going to take over and I'll discuss the technical implementation and some of the details that I carried out to make this dashboard come to life.
Nick Humensky: or these dashboards come to life, and then Cindy will end us up with a discussion about how this project helped her team align with campus and system level goals.
Nick Humensky: So I am a data developer on the institutional effectiveness team under the greater academic affairs, research and student success team at the University of Tennessee System Administration.
Nick Humensky: I handle a lot of dashboard creation
Nick Humensky: back end data modeling automations ad hoc data requests things like that.
Nick Humensky: Cindy.
Cindy Williamson: So Jorge already mentioned it. But I'm the director of accreditation and assessment at Utc, and also our institutional accreditation liaison. So the data that we are talking about today is important for both of those things, accreditation and assessment. Our office is responsible for lots of data collection. So lots of surveys, including course, learning, evaluations, faculty rating of administrators, and then also responsible for the institutional
Cindy Williamson: outcomes. Assessment process. That is, part of our saccoc requirements, but also is driven by the need to support students and and make sure that we're doing what we should
Cindy Williamson: for them.
Cindy Williamson: Okay? So this all started.
Cindy Williamson: innocently enough. We were in. I think it was a power bi workgroup Zoom Meeting.
Cindy Williamson: and I asked if we could get some help with power via because we had talked about in my office, wanting to be able to present data in a way that our stakeholders could interpret it, they could access it fairly easily.
Cindy Williamson: so started with a conversation between
Cindy Williamson: you know Jimmy and Nick's office. Rachel was there, and a couple of others, and then our office at Utc. So we really just described what we were thinking about, and
Cindy Williamson: then also talked as as groups back and forth about you know, questions that came up, or ideas that we had along the way.
Cindy Williamson: and we all know how important communication is. And when we were planning this presentation, I I it just made me think about. Yes, that's how this all started was clear communication, answering questions. Making sure that everybody understood the need. And then, on our end, making sure that we understood what was possible and how how we could make these dashboards look you know, based on
Cindy Williamson: power bi parameters.
Cindy Williamson: So we're able to you know, communicate openly and talk about the project. But then
Cindy Williamson: there's so much data, all the data. So it was interesting to to kind of step back and think about our data from a different perspective. We decided to start with our outcomes, assessment information course, learning evaluations and senior exit exams, and we'll we'll go into a little more detail on those here. In just a second
Cindy Williamson: already mentioned communication, we had an excellent cycle of communication back and forth.
Cindy Williamson: We answered questions, provided data.
Cindy Williamson: And I've mentioned this already. But we thought about what we were doing as part of our data collection process and presenting that data from a place that we don't often
Cindy Williamson: get to be. You know, a lot of times we're distracted by the process. How are we gonna make this happen.
Cindy Williamson: and here's what we want. Let's get it. But really taking a step back and thinking about possibilities, and the best way to make this
Cindy Williamson: it well, make it an efficient process, but also something that our stakeholders could use. When they needed to. So really, thinking about the the bigger picture in order to make data informed decisions across campus.
Cindy Williamson: So the 3 dashboards that we started with were the Ets proficiency profile, which is our senior exit exam.
Cindy Williamson: It's required that all graduating seniors take this exam.
Cindy Williamson: We do 3 administrations of it. Well, 3 semesters outcomes assessment information.
Cindy Williamson: That's a lot.
Cindy Williamson: Our office. We have a team. Well, we have 5 people and 3 who work specifically on outcomes, assessment review and feedback.
Cindy Williamson: We report, we report these data multiple times a year, and then, of course, also report to Saccoc as required. And I left out that our our senior exit exam is also reported to Tec. So not just something that we use at our institution, but as part of our quality assurance, funding.
Cindy Williamson: course, learning evaluations. Yes, results are available to faculty department heads, deans, administrators, other stakeholders. But we wanted a way
Cindy Williamson: for these stakeholders to easily access what they need. What we have not right now is fine. Our platform is just fine. But the reporting capability is limited, and so we thought that a dashboard would be a great way to present that information in a way that can be interpreted by those who need who need to use the data.
Cindy Williamson: Okay, this is where my sob, story starts.
Cindy Williamson: so senior exit exam. Results were previously presented in excel. We'll show you some examples in a minute. But
Cindy Williamson: yeah, just don't cry. Okay. They're bad.
Cindy Williamson: and the way it's presented, which you'll see, it's not easy to interpret results. And
Cindy Williamson: we want this data to be used, not just you know, by OA for T Hec purposes. But this is a lot of good information that we can use programmatically, departmentally. You know, in order to make changes, make improvements, data informed decisions.
Cindy Williamson: so yeah, we really wanted to make sure that those who needed
Cindy Williamson: that maybe even didn't know they needed it. But those who we thought should be using the data would have access to it.
Cindy Williamson: This is it. I'm sorry. It's graphic.
Cindy Williamson: it's it's terrible. This is what has been on our our website for several years. We usually have at least 5 years of data up
Cindy Williamson: at a time. So imagine as a department head or faculty member or dean, you know, trying to make sense of this information. It's there we have proficiency scores. We have, you know, mean test scores mean scores for each subject on the senior exit. Exam. But
Cindy Williamson: it's it's it's not fun. And I know that there have been difficulties in people accessing the results and getting what they need. But just like with these other data sets, we were constantly
Cindy Williamson: and probably more for course, learning evaluations than anything else, but providing data through, you know, from ad hoc requests. So we had our process. But then, on top of that, we we have multiple requests for the data in a different format.
Cindy Williamson: Our outcomes assessment data.
Cindy Williamson: This is a relatively new you know, data set that we had. We have worked for about 10 years.
Cindy Williamson: To get to where we are now with outcomes assessment. So we've improved that process.
Cindy Williamson: I mentioned that there are 3 of us who work with the outcomes, assessment information and really 2 fantastic outcomes, assessment management analysts who review every piece of outcomes assessment information for every program, department, college office, everybody across campus, academic student support, administrative.
Cindy Williamson: So we're talking about around 230 separate areas, separate offices or departments, and we have about 220 outcomes assessment contacts that we send information to send data to send our assessment results to our feedback.
Cindy Williamson: So we do provide specific, detailed feedback. We'll see an example of that here in just a second. But
Cindy Williamson: we want to know where and how assessment occurs. Yes, we want these areas to highlight strengths and weaknesses in order to see where they can focus their efforts and where more resources may be needed. There may be a need for changes to the curriculum
Cindy Williamson: changes to their particular assessment process. All kinds of good information that can be gained
Cindy Williamson: from this outcomes assessment data.
Cindy Williamson: So this isn't as bad as the senior exit data, but the the 1st screenshot you see at the top, the top on the right. That is our internal spreadsheet. So we had a spreadsheet, have still have a spreadsheet that lists all departments, all programs
Cindy Williamson: and then contacts for those departments, and then across the top you'll see that the columns are. It's kind of hard to see on here, I'm sure, but the columns include deadlines, dates where certain pieces of our outcomes assessment information is
Cindy Williamson: so when it's supposed to be reported, so we make notes on here, follow up with those who don't have what what they're supposed to have at the time.
Cindy Williamson: And then that second screenshot is our outcomes assessment rubric. So that's 1 of the big improvements that we've made.
Cindy Williamson: and that's been in the past 3 or 4 years. It's a rubric that has dimensions aligned with each requirement for the outcomes assessment process. So each outcome that's developed is supposed to have specific information. And so that's what we're looking at with that rubric.
Cindy Williamson: Nick, you can go on to the next one. So we have that rubric feedback that we provide. But then also we provide
Cindy Williamson: more qualitative feedback. So specifically like, here's your score. But here's specifically what we were looking for and what we didn't see. Or here's where you did well keep doing that. So we send all this feedback via email. We also post it in anthology planning, which is our platform that we use for outcomes assessment data collection.
Cindy Williamson: and this information is, yes, helpful to individual departments, but we were thinking that having a dashboard would make it more accessible for the larger units. So the the colleges and units
Cindy Williamson: they could access the information. And then, like I already said, you know, make data, informed decisions, improvements. And those responsible for outcomes assessment
Cindy Williamson: typically only see their information, their feedback, that we provide. So this is a way to for multiple departments to have some kind of comparison as as far as what they're saying for themselves.
Cindy Williamson: And everybody's favorite data collection process is course, learning evaluations.
Cindy Williamson: it's always a hot topic, I'm pretty sure, on every campus across the country, probably across the world anyway. So currently, administrators, deans, department heads faculty can access results through the platform. I mentioned, though, that it is not user friendly and
Cindy Williamson: the reports that can be generated through that platform are are lacking. So we have a ton of ad hoc requests. That come through
Cindy Williamson: for this cle data in in different formats.
Cindy Williamson: and there is comparative data in that platform. But that's 1 of the main things that I think we're asked about even though they can access it. It's not easy to access and having something right there already available, ready for people to use without making a separate request. I mean, that just just sounds like.
Cindy Williamson: you know, the ultimate way to be with your your data. Processes. But so
Cindy Williamson: lots of reasons that cle data is important. Jorge actually hit on that. And in his introduction, this is this is for faculty. They can use it in different ways, including, you know, during their promotion and tenure efforts.
Cindy Williamson: These next couple of slides are examples again. Still, not as bad as senior exit exams. But it's not. It's not great. We you can see that there's so many different ways. We're asked to format the results
Cindy Williamson: on the left hand side. You'll see that it's by
Cindy Williamson: cle question, and then also about term
Cindy Williamson: and then on the right hand side, we've also got it broken down by response type and question.
Cindy Williamson: Then on the bottom, it's, you see, the comparison data there. So we have the program, the department, the college, and then the university. But that's also broken down by semester
Cindy Williamson: and then this next one
Cindy Williamson: is is broken down. Even further, we have as part of this request. Well, both of these requests. I think we
Cindy Williamson: provide individual instructor data. And and that's for the department, right? That's not something that that's going out to everybody. But we have the app, the mean scores by course, modality also by term.
Cindy Williamson: And then we've got the response
Cindy Williamson: averages means by term and by question. So anyway, you get the picture like this is just a lot of data and having it displayed in a way that is functional, that can be interpreted and used is is what we needed.
Cindy Williamson: And I told Nick that this is probably an
Cindy Williamson: an underestimate of how much time this is actually saving us. When you think about updating spreadsheets with new data multiple times a year. Senior exit, we only do once a year. But then outcomes assessment. There's 3 or 4 updates a year that we do to that data. And it's aligned with those deadlines that that they have throughout the cycle. And then, Cle, data is is updated all the time as well.
Cindy Williamson: at the least 3 times a year, once for each term.
Nick Humensky: Alrighty.
Nick Humensky: Thank you, Cindy, for that.
Nick Humensky: Now I'm going to take over, and I'm going to walk you through the pieces of the technical implementation, and how we got this dashboard to be
Nick Humensky: alright with any project like this or any sort of dashboard project. It's always good. It's a good idea to start out with a solid requirements gathering meeting.
Nick Humensky: So we have 3 dashboards, we know. And for each I want detail on the following items.
Nick Humensky: data source. So where is this coming from? Is it from a software? Is it a flat file or some sort of Api that we can connect to? Also does the structure of the data change over time. For example, do the number of columns or fields shrink or grow?
Nick Humensky: What is the frequency in which you're going to be receiving the data. Is it monthly, yearly or by term?
Nick Humensky: This helps me decide whether or not I want to have the data refresh pretty much instantly if it's very frequent, or have it set up on some sort of schedule to refresh the data more periodically, if it's less infrequent.
Nick Humensky: what is the volume of the data. So each time that you receive a batch in the cadence of the frequency, how many rows are coming in with it? Is it hundreds, thousands, millions.
Nick Humensky: Spoiler, alert, we went with storing the data on
Nick Humensky: sharepoint and excel files. And what this volume of data piece helped me decide was.
Nick Humensky: if I wanted to keep the data all in one spreadsheet and just append rows to it. Or if there's just too much too many rows and a lot of transformations that needed to happen to where I just wanted the stakeholder to go in and be able to drop a file and have power bi refresh it.
Nick Humensky: And to that matter, the transformations I like to get with the stakeholder and understand exactly the process that they have where they take the data from the raw format to the presentable format. I like to mimic their actions as closely as possible in whatever automation tool I'm using in this case, it's power query. It's similar to what Justin said yesterday. If you were on for the panel discussion, our job is to help the stakeholder
Nick Humensky: not worry about the data or any of the manual pieces. Let us take care of all of the automation so that you can focus on what your area of expertise is.
Nick Humensky: And then finally, security.
Nick Humensky: Are there any sensitive pieces of information that we want to hide or protect?
Nick Humensky: Or is there any sort of row level security that we want to implement, such as
Nick Humensky: keeping departments from seeing each other's data.
Nick Humensky: And so with that.
Nick Humensky: this is sort of a high level piece or screenshot of what I have for all 3 of the dashboards the course learning, evaluation, and the senior exit exam. Dashboards both come from an external software.
Nick Humensky: Unfortunately, there was no way to directly connect to them and get automatic access to the data. So we had to go with the next thing. The next best thing, which is a export to a tabular data source such as a spreadsheet.
Nick Humensky: The outcomes assessment dashboard is just a manually created spreadsheet maintained by Cindy's team
Nick Humensky: for the frequency of the data. Both the Senior exit Exam and outcomes assessment dashboards
Nick Humensky: are updated about once a year. The outcomes assessment might be Updated 2 or more. But the course. Learning evaluation is a different story that is updated once per term.
Nick Humensky: Again, for the volume outcomes assessment and senior exit exam are similar. It's just hundreds high, hundreds of rows. And then course, learning evaluation is a massive data set. It's got tens of thousands of rows in each file that's added to the sharepoint folder, as far as transformations go, outcomes. Assessment and senior exit exam are pretty simplistic. You're just adding new rows to the existing spreadsheet. But the course, learning evaluation requires some complex transformations, and we'll talk about that in a moment
Nick Humensky: and for security. But the course learning, evaluation and the senior exit exam dashboards have some names in them. The the free text questions might have
Nick Humensky: the professor names in the course, learning evaluation, and the senior exit exam records the students, 1st and last names in in separate fields whenever they take the exam, and we take measures to protect that which I'll showcase in a second
Nick Humensky: with all of that in place. I determined that the easiest and best way to move forward with this would be to store the data in excel files in a structured manner in a sharepoint folder.
Nick Humensky: So
Nick Humensky: you have the General Document library that comes with standard sharepoint sites underneath that I created a folder called assessment data, dashboard data sources. And I created this to house everything under there and separate it from any other projects that they might have within their team's sharepoint site, but I wanted them to be able to have access to the site so that they could go in and drop the data files in there
Nick Humensky: underneath that I have a folder for each of the 3 dashboards, and then you'll notice
Nick Humensky: there are right underneath the level, right underneath the dashboard folder are data files
Nick Humensky: for each of the dashboards, and then I also created a separate dimensional folder
Nick Humensky: that will allow you to drop in any sort of if you're familiar with standard data modeling or reporting
Nick Humensky: acumen dimensional tables just provide additional context
Nick Humensky: and labels for the factual data.
Nick Humensky: So now that I have the data stored, it's time to start pulling it into power bi. And whenever you create a connection to data and power bi, you want to make sure that you're setting up authentication and privacy levels appropriately.
Nick Humensky: Here's a reiteration of the security. You have the names in both the course, learning, evaluation and the senior Exit Exam. Nothing really an outcomes assessment. It's just department names and scores
Nick Humensky: for the authentication on all 3. I wanted to do Microsoft credentials. It makes the most sense because everyone who's going to be using this as part of the University of Tennessee system, so they will be on the Microsoft tenant
Nick Humensky: for privacy levels.
Nick Humensky: I went with
Nick Humensky: private, organizational and private. There's 4 different privacy levels. Whenever you're connecting data in power. Bi, there's none.
Nick Humensky: public, organizational and private.
Nick Humensky: The reason I went with private for course, learning, evaluation and senior exit exam is because of the names. And what private means is that it just isolates that data. Query, that specific data query
Nick Humensky: from being merged or appended to any other data sets and creating some sort of unwanted inference about maybe the student
Nick Humensky: so the private isolates it from being connected or merged with an even other private data sets
Nick Humensky: for outcomes. Assessment. I went with organizational. That's a step up from private. It's just for more trusted internal resources. I didn't. Blanket set everything to private, because I wanted to be intentional about my decisions of
Nick Humensky: the level of privacy that I set for the dashboards.
Nick Humensky: So with the dashboard or with the senior exit exam and outcomes assessment.
Nick Humensky: it's pretty simple. You just connect to a single spreadsheet, and then there's minimal transformations that need to be done to it. But the course learning evaluation is a monster, and I'm going to show you
Nick Humensky: some of the steps that I took to be able to create make this digestible format for power. Bi. Now, here we have a it's a kind of a cumbersome looking spreadsheet, but I'll I'll describe it the best I can.
Nick Humensky: This is just one terms, file that comes from Cindy and her team. For one term. Of course, learning evaluation survey data. Within each file there are different sections or templates of data responses. And that's just for the different colleges that
Nick Humensky: they might have different templates, which just means they have some additional questions that they ask for the students within that particular particular program.
Nick Humensky: They all start out like this where there's it skips a few rows and
Nick Humensky: it has a little header here. It says raw results for template brsc. That's just the honors college at Utc. It'll skip a row, and then it has the
Nick Humensky: headers down here like this, and then so all the responses
Nick Humensky: and brsc usually has a couple of 100 rows. I did this little row skip, to try and show
Nick Humensky: how like a different section in the data without having it scroll out and being extremely scrunched.
Nick Humensky: So this right? Here is the
Nick Humensky: last row of the brsc data.
Nick Humensky: and then the next section starts raw. Result, raw results template for course, evaluation skips row
Nick Humensky: more more data, and there's a varying number of sections within each spreadsheet. There is a varying number of rows.
Nick Humensky: and there's a varying number of columns like I mentioned, because some might have additional columns or additional questions.
Nick Humensky: So if you notice
Nick Humensky: starting at this fall, 2020 right here and all the way to the right are all question columns, and it could go to column
Nick Humensky: Z. It could go to column A, B,
Nick Humensky: and as such, when you pull this into power. Bi, you can't simply filter out the blanks and just stack them on top of each other. We have to do a little transformation.
Nick Humensky: Do some unpivoting and get an equal number of fields so that you can create one long, narrow table that's more easily digestible by power. Bi
Nick Humensky: and I included this little graphic
Nick Humensky: to help you visualize what unpivoting is. If you don't know what it means. I'm sure everyone knows what it means, but I wanted to include it just in case someone might not be familiar with unpivoting.
Nick Humensky: Let's say you have a
Nick Humensky: survey that's sent out. And these 3 people respond to the 3 questions. Here you have these descriptive fields, the dimensional fields.
Nick Humensky: such as their name and department? And then you have all the questions that they answered the question. Headers are the attributes, and then the respective values that they answer for each question below
Nick Humensky: unpivoting. What that does is. It transposes the column headers down to a single column like this.
Nick Humensky: And then the matching values.
Nick Humensky: This does create create some duplication and redundancy in the descriptor fields. But it's necessary for this particular data set. And it's a much easier format to pull into power bi to work with.
Nick Humensky: So this is what I this is what I had in. This is just one of the sections that I would work on in the Cle. Dashboard. So I keep all the
Nick Humensky: columns between term and professor name, and then anything to the right of that is going to be an attribute column, and I have to use some complex power query to be able to dynamically figure out where the end is, and then unpivot it.
Nick Humensky: And so here's what all the power query steps for that look like. I know this is a lot, and I don't expect you to read, and really
Nick Humensky: I don't expect you to read all of it. But I have this in here for just a little heads up for best practices in terms of naming your steps when you're working in power. Query.
Nick Humensky: You want to be as descriptive and specific as possible to help yourself whenever you come back in several months at a time to remember exactly what's happening in the power query. And also if someone else inherits the dashboard, they'll be able to go in and troubleshoot a step, because they'll be able to see what's happening in the steps.
Nick Humensky: I left
Nick Humensky: these 1st few steps as their general, the generic names that are generated by power, bi or power query. For a couple of reasons. One. These are automatically generated whenever you connect a sharepoint file to power Bi. They don't really do anything for the purposes of troubleshooting. It's just creating some additional helper files to pull the data in. And
Nick Humensky: also I wanted to showcase how these generic step generic name steps don't really mean anything so filtered hidden files. One? What does that mean? Well,
Nick Humensky: yeah, you want to be specific, because there could be multiple filtered rows, filtered rows, one filter rows, 2 things like that. And so everything below that
Nick Humensky: and over to the right are all the steps that I created for this particular data set in order to unpivot all the columns, all the sections, and combine all the files together.
Nick Humensky: And so, after all is said and done, you have one huge table that has.
Nick Humensky: I think, upwards of 3.9 million of rows or 3.9 million rows.
Nick Humensky: That's quite a lot of data to work with. And because the
Nick Humensky: the value column is going to have a combination of both. The free text answers as well as the numeric answers.
Nick Humensky: it's quite difficult to perform mathematical Dax calculations on fields like that, unless you want the measures to be extremely cumbersome
Nick Humensky: as such. I separated the big table that's aggregated into 3 fact tables, and I did this to make the Dax easier, but group them logically together. So it's a little easier and more clear to work with. So I have a table that has groups together all of the likert questions. And that's all of those questions where you rank between strongly disagree or strongly agree.
Nick Humensky: We have fact text, and that's just all the free text responses and questions. And then we have.
Nick Humensky: I just call this counts. It's any other remaining question. They're pretty much multiple choice questions, such as.
Nick Humensky: what modalities did you use to be successful in this course? Choose all that apply items like that.
Nick Humensky: And then this 4th one over here is a separate data sort set that comes from the same tool as the surveys. But it's just a different spreadsheet. It's for the response rates, and I have that separately. But I
Nick Humensky: am able to connect them all to the dimensional tables based on term and college and department connections like that.
Nick Humensky: So once I have the data in power Bi, and you start creating measures. And you have all that good stuff
Nick Humensky: you want to start worrying about the visual styling. And so I got some branding assets from Cindy, including the Utc approved colors and their hex codes. And then I also asked for some vector images, Logos of her office, so that I can include that on the dashboard.
Cindy Williamson: Hey, Nick? I just wanna add in really quick this is kind of funny. So when Nick asked for the graphics or Logos, I asked him which ones he wanted, and he told me, but I had no idea what he was saying. Like the words did not, so I was like, well, which ones are those, and he's like, just send them all. Just send them all to me.
Cindy Williamson: So.
Nick Humensky: Yeah, they these were helpful to add that sort of personal touch to the dashboards.
Nick Humensky: So here is an initial iteration of the outcomes assessment dashboard. It's called. I initially had it called the Assessment Review. But it's now the assessment or the outcomes assessment dashboard where I had across the top the all the different filters for the dashboard. So that way I could create as much real estate in the body area, keep it wide, and have all the visuals large, so that people can see them.
Nick Humensky: But I did the same thing for the senior exit exam. Where I had the filters across the top
Nick Humensky: for the course, learning evaluation. There were too many filters to have it horizontally across the top, so I had to make them vertical, and I didn't want to have 2 dashboards where
Nick Humensky: they were the filters were horizontal, and then one was vertical. I wanted everything to be consistent.
Nick Humensky: so I took some time to think it through. And this is the format that I ended up coming up with
Nick Humensky: is the following.
Nick Humensky: I have a narrower navigation bar on the left where I'll have some buttons, such as a filter button to show and hide a filter pane so that you can apply filters. But you don't have to have a consistent, persistent filter pane on the left that takes up space where you could have more visuals.
Nick Humensky: and then other page navigation icons, and then a button for a tutorial that'll show you how to walk through the dashboard and use it
Nick Humensky: up on the top. We have a header section. That's where I have the dashboard title, a label on when the data was last updated, and then I also will have a text box that contains
Nick Humensky: which filters have been applied, so that after you hide the filter pane you can understand and remember what the context is that you're looking at. So you don't have to keep toggling the pane back on, back and forth.
Nick Humensky: And then just a main data section.
Nick Humensky: Also fun fact. I found out that you can create backgrounds and pretty looking visuals in Powerpoint. This background was actually created in Powerpoint. You can do a little more complex, and fancy visuals, such as the gradient borders and transparency effects within Powerpoint, and then save the slide as an Svg. And set that to your canvas background and do an Svg. Just so it doesn't lose the quality whenever it scales and expands to a bigger size.
Cindy Williamson: Hey, Nick, one other quick thing. So it was interesting, as Nick was doing all this stuff. When he would find something new or create something. He he would, you know, message me and be like, can you get on real quick? I got something to show you. I just found this out or so it was like, I don't know. It's just kind of refreshing to to know that like this was exciting right on both
Cindy Williamson: things. This wasn't just like me being excited about what what the final result was. Gonna be so.
Nick Humensky: Definitely.
Nick Humensky: And so just to reiterate, here's that initial iteration of the outcomes assessment dashboard the before.
Nick Humensky: And then here's the after. It's a little more clean looking with the
Nick Humensky: gradients from the approved Utc colors, and
Nick Humensky: Yeah, yeah, right? It's it's super fancy. So over here. This is the
Nick Humensky: label that shows when it was last updated. Here is the text box that shows the filters that are applied, and this is the filter button that shows and hides the filter pane. Another little thing that, I added, is this number here to just add another visual cue to remind you how many dimensions have been filtered. The outcomes assessment doesn't have any other pages. It's just one. So there's nothing here, but there is the how to use button down here. I'll show you what that looks like in a moment.
Nick Humensky: Here's the senior exit exams data, same format, similar format.
Nick Humensky: this one actually has multiple pages. So right now, you're viewing the scores. But there's also a little radar gauge, but button to go to the proficiency profile.
Nick Humensky: and finally the the big one, the course learning evaluation.
Nick Humensky: This has the most pages, and
Nick Humensky: on the front home page. I wanted to include the the the minimum required
Nick Humensky: measures that Cindy asked for. Just so it's right front and center for people to be able to go in and look at, and slice and dice. However, they see fit with the filters
Nick Humensky: whenever you click on the filter button. This is what it looks like pops up, and you can adjust the slicers.
Nick Humensky: You can see that they are applied up in the text box behind a little transparent background, just to
Nick Humensky: show that something is currently in the front foreground of the visuals.
Nick Humensky: and then you can hide it.
Nick Humensky: And then this is what the tutorial looks like when you click on this, how to use button down here.
Nick Humensky: It brings up this page that says, you know, welcome to the dashboard. I want a tour, or you can just click no and go right back into working. But whenever you click this, the following happens, it starts a little sequential coach mark one of those sort of self guided tutorials on how to use the dashboard. I didn't include a screenshot for every one of them, but I have it walked through
Nick Humensky: a few pieces for the navigation bar.
Nick Humensky: and then it'll walk through the filters. It'll walk you through how to use the filter page. And then the
Nick Humensky: this little section is for the data, and it just walks you through looking at the data.
Nick Humensky: So we're really excited to get these out and rolling.
Nick Humensky: and I'm going to pass it back to Cindy to discuss the final strategic alignment steps.
Cindy Williamson: Wanted to mention to Nick that
Cindy Williamson: That help feature in the dashboards. Dashboards, I think, is something that's going to be really helpful.
Cindy Williamson: of course we're there. Nick's there. Whoever is there to answer questions. But I think, having kind of a tutorial like Walkthrough situation is.
Cindy Williamson: it's gonna be really nice. So yeah, strategic alignment.
Cindy Williamson: When we 1st started this, I obviously did not.
Cindy Williamson: Excuse me, I obviously did not think about it on
Cindy Williamson: on this level, right about this collaboration between campus and system. But I am glad to have had the opportunity to think about it in that way, because this is something that's really important.
Cindy Williamson: of course, ideally, we want goal alignment, right between the system utc our office, any office on our campus but that cross campus collaboration, I think, is super important.
Cindy Williamson: So open data sharing across departments and units and then also system wide. Yes, we report data to system. Yes, we report to T. Hec, but I think that reciprocal data transfer. And while in this case, like what Nick was doing wasn't technically, you know, data transferred. It was a back and forth process. That
Cindy Williamson: required efficient communication in order for us to you know, both get what we need for our offices for our
Cindy Williamson: you know, employers to get what we need. So
Cindy Williamson: I think when we think about that clear communication and sharing data. It's not just the data that we're sharing. There's you know, contextual information that we can provide. And then the next level is like talking about the process involved, for all of this.
Cindy Williamson: But the sharing of data is is helpful for us. Yes, but also our stakeholders, and that communication really facilitates solid partnerships, I think, across the system and between institutions. So we can work together and collaborate. We can share ideas about processes, projects on some level. We all do many of the same things, regardless of which institution we're at
Cindy Williamson: and then having the conversations that are are beneficial to everyone involved. And then, of course, being able to meet requirements for our creditors. Programmatic, institutional also for T, Hec, and then for the individual campus campuses to be able to provide what system needs to.
Cindy Williamson: I mean, you can go to the next slide, Nick. But I wanted to mention to elizabeth Pemberton is on here. I'm gonna call you out, Elizabeth. But she reached out to us when she saw
Cindy Williamson: that this was going to be one of the sessions for the ie. Summit, and that just like spoke to me. I was like, yes, we will talk about all day about this. So Nick and I met with her, and so she had a little sneak, sneak, peek of what we were going to be talking about. But that's exactly what we want from this right?
Cindy Williamson: so
Cindy Williamson: aligned goals talking about continuous improvement. But that data gives us a way to make informed decisions, to guide our planning, not just seeing what our planning needs to be or where we think we need to go. But
Cindy Williamson: yeah, there's just so much to it and the data that we have in our new dashboards. They're not live yet. But stay tuned but this they absolutely facilitate continuous improvement and individual faculty staff. Again. Department heads. Deans.
Cindy Williamson: vice chancellors, you know, can really leverage the data for enhancement of their programs and services and make sure that they're allocating resources in the appropriate places. And then I'll mention Sacco. See? One more time. But our creditor
Cindy Williamson: wants to know that we're following our processes and that we're we're continuously improving. And I think something like this is exactly
Cindy Williamson: something, you know. One thing that we can use to provide evidence of that continuous improvement.
Cindy Williamson: so yeah, we can provide artifacts. We can provide assessment results. But I think, having them in a way to
Cindy Williamson: a way that someone from outside our system, our campus can look at it and answer their own questions is is super helpful.
Cindy Williamson: And Dick and I also talked about strategic alignment across levels. So yes, we have ut system. We have an overarching strategic vision. We have priorities that have to be well communicated, and then campuses can align with those priorities. So then, down to the campus level, we have our strategic plan.
Cindy Williamson: That is, in alignment with system priorities and then executing our plan and making changes at the campus level.
Cindy Williamson: one more way to show evidence of continuous improvement, but also it gives a place for individual departments, units areas, whether it's colleges or divisions. It gives them a place
Cindy Williamson: to align what they're doing. We we want to see that all the way through our organizational structure. So if we're talking about assessment and accreditation, accreditation, yes, start there. But think about what that means, as you you know, look higher in the organization, or.
Cindy Williamson: you know, up to the system.
Cindy Williamson: Okay, so what's next? We are excited. Oh, well, I am. I didn't ask Nick. If he was excited or not
Cindy Williamson: be, we're going to keep collecting data. Of course we're going to update our dashboards. We will make them live. But as part of that process. We've also talked about a plan for informing campus about the dashboards, so not just putting them out there and saying, Hey, they're here. But you know, holding potentially short information sessions and
Cindy Williamson: talking about how the dashboards can be used, what the you know, specific context around the data. Nick, you want to go on to the next one,
Cindy Williamson: and then we'll continue to collaborate. I've already got some ideas for additional dashboards. Of course we gotta get these out and
Cindy Williamson: get people using him first, st but I I don't think there will be anybody who is.
Cindy Williamson: It's not happy with with what we've come up with. But yeah, making sure that faculty staff students, administrators, everyone understands what they're looking at when they go to these dashboards and we we do already have that dashboards. But at Utc as does system and across the system. But
Cindy Williamson: I think these are different than than what has been expected in the past in the past, and we want to be sure and
Cindy Williamson: have a balance between what we feel like is needed, and then what's wanted?
Cindy Williamson: I think this will be a continuous process.
Cindy Williamson: Once we close the loop it opens right back up. And you know, we just keep going with this. But
Cindy Williamson: initially, I think additional dashboards that we'll think about developing our faculty rating of administrators, employment and placement data super important right now. And we actually finally, Ashley, I'm looking at you finally have a process in place. That
Cindy Williamson: is
Cindy Williamson: most efficient. We we have more and better data, regarding employment and placement than we've ever had. So then, also, as part of that you know, quality assurance funding
Cindy Williamson: several of the things that we've talked about are part of quality assurance, funding, and I think being able to display that so that others can see. That's another one of those things that
Cindy Williamson: just like I've reached about outcomes assessment. These things can't be done in isolation. You know.
Cindy Williamson: campus needs to know about quality assurance funding. Yes, it's part of performance funding, and we know, you know.
Cindy Williamson: money is tied to that right. But even even past that I'm I'm thinking about how different programs score on their program reviews or, you know what our senior exit exam data looks like?
Cindy Williamson: it's just so much institutional satisfaction. I'm not going to go on and on. But there's lots of opportunity there, I think. And this has been a great process so far. And I hope that Nick will keep going with me on this. But I I think we've had a a
Cindy Williamson: fabulous collaboration.
Nick Humensky: I agree.
Cindy Williamson: I think I saw a couple of questions like in the chat, but I haven't gone back in there.
Rachel Borashko: Yeah, thank you so much, Cindy and Nick, that was a great presentation. I really appreciated that. I'll invite anybody to also come off mute. If anybody has any questions that they would like to share with the presenters. We do have a couple in the chat. At least one to everyone, and then a couple sent just to me.
Rachel Borashko: Feel free. If you have any questions also to drop them into the chat. Now you can send them to everybody or to me. I can keep your if you're embarrassed about your question. If you think it's a dumb question, it's not a dumb question, and if you really want to send it just to me, I can keep your identity confidential.
Rachel Borashko: Does anyone want to come off mute to ask any questions.
Ashley Ludewig: I have one. So I used to be faculty at another institution. And so I've seen the faculty side of this data many many times, and getting these isolated reports of my students reviews of the course that are just one at a time. They're not put in any conversation with each other unless you choose to do it. So even on a 1 to one basis like, it's difficult to make use of that data. And so when I look at the scale of this.
Ashley Ludewig: it's just mind blowing, and so awesome. And I'm wondering I know they're not live yet. But have you gotten to do some testing or like Beta, work with, like the faculty and administrators, at least a handful of them who will be using this? And and what do they think? Or do they see the same potential that that you and Nick see in this.
Cindy Williamson: That is coming. We have previewed these dashboards to Provost Hale and so he is certainly on board with this. And yeah, I think
Cindy Williamson: we've gone back and forth with some changes we want to make. And so once we get to a point where we're comfortable. You know, with what we have.
Cindy Williamson: That's an excellent point. Yes, share with faculty department heads, Deans. They're all looking for something different from from each of these dashboards, right? So making sure that it's there and they can access it.
Cindy Williamson: Yeah, that's super important.