27 Nov In order to analyze the data to answer your evaluation questions, you will need to collect the data and store the data in a database. This week you will be

Okay. All right. Well, welcome to Week five. I noticed just the two of us here, but people are watching online. So I always record, of course. This is going to be a very busy week. I want to remind you that just two weeks ago, you had the assignment with just two sentences in it. So there was an easier week. I know some people wrote more than two sentences, but I did prefer to see it in that concise here's one sentence, here's maybe three sentences. This week will be a lot more work. It just simply is, and again, I'm sorry, it falls during this week of Thanksgiving, but that's the design. We're going to go over the assignments for this week, and then I'm going to really push you're watching the videos that are on your Week five page in your module. They are prepared by doctor Hansen who was here before me. They're truly excellent. One thing I want to make clear is that we're only reviewing in this class on program development, one program. Her very first video refers to two programs, and you are not assessing two programs. It's for a capstone course, and it's not something you have to concern yourself with when you watch the video. But other than the two programs, you'll just be using one. Everything is the same and very, very helpful. She even shows you how to copy. As you're making up your data, this week creating your data, you won't have to make up 100 lines of data, even though you'll have supposedly 100 subjects or participants in this clients in this study. What we have this week is a discussion, which is like every other week's discussion, it is going to help prepare you to do the assignment. But the assignment will not be a written assignment this week. It won't be furthering the report yet. It will be designing a database, making a database. Best as I understand from your other courses, when you have evaluated material in SPSS X, you already are given a database. Here we want you to learn how to develop the database, so that if you go into the real world and you're given data by a client after you've designed the program evaluation, you can build that database. You can name the variables. The things that will be very helpful to you later in the analysis and you can set up the database, so it answers the questions you're interested in. So there's no written assignment or furthering of the report this week. There is the building of the database, which is a lot of work. So prepare yourself, leave yourself time to watch each of the six videos, which are 7-15 minutes each. There's none longer than 15 minutes. The first one is really the longest. As you watch them, we'll show you how to build your database throughout the week, and then how to check that the data in your database is correct and so on. Lynette and I were talking before class, don't overwhelm yourself by watching all the videos at once. Watch the first two. Then after you've built your database, move on to video three, and four and five and six. Do it slowly, and you'll be fine. Watch a ten, 15 minute video, do that portion of the work. Watch a ten minute video, do that portion of the work. And it'll get easier as the week goes on. The bulk of the work is really in that first and second video that you'll watch this week in week five module. Okay, Let's get started with your discussion, which, as always, you'll turn in by Wednesday night at first, and then you'll give feedback to each other as I will give it to you. And so the discussion is quite like every other week. It's a time to sort through much of what you're going to have to handle in this week's assignment, which is the building of a database. In order to analyze the data, which will be next week's assignment, you won't analyze the data this week. To answer your evaluation questions, you will need to collect the data and store the data in a database. Remember, we're not really collecting the data, we're making it up. This week could be creating a database in SPSS to store the data from the measures that were selected in Week four. Discuss the following issues that need to be considered when setting up the database for data entry. What are the scales of measurement of the items on the measures. What are the scales of measurement of the items on the measures? Well, it depends what variables you're talking about. When you're talking about the demographics for the study, we're often looking at what are called nominal scales, and you'll need to know that to set up the database. For example, if you're just looking at categories of information, like what sex one is, then you would be using nominal scales of measurement. If you are looking at an ordinal scale, you're looking at something like a chard scale. Now, the chard scale is one that has order, but the spaces between points may not be equal. When you have a Likert scale, the responses may be not at all, a little bit, somewhat and very much. Because it's a liker, we don't know that those numbers, those spacings are actually completely equal, and so we call them an ordinal scale. You'll also have ratio and interval scales, where the difference between say, answering a one and a two is equivalent, and it is equal to the difference 7-8. But for the most part, in this database. You're going to be using nominal scales for things like sex, race, what therapy you took and so forth, and you're going to be using ordinal scales, which are Licart scales. You'll also be using possibly if you're doing substance abuse, a dichotomous scale, which is when they take the urine test and they pass it or they fail it. But that two can be scaled. We can use it for regression and correlation. You're going to enter that as a scale type of measurement for the items on a dichotomous question. The number of days somebody has been in therapy can be nominal or scale. You'll have to decide that and what's the best route of thinking for your particular scale. Just remember, the nominal scales are going to be categories, the interval scales, I don't want to call it an interval, sorry, the ordinal scales, it's going to be your chord scale. Then anything that you have a continuous measurement for is going to be more like your ratio or interval scales. I know you've studied this all before, but we'll refresh it in the videos you watched this week too. The next thing you have to look at in the discussion question or item two is, are there any reverse scored items that need to be rescored before data can be analyzed? The thinking, particularly 20 years ago or so, when many of the scales you're going to use were validated and made reliable, was that you wanted to make sure that you asked some questions in the affirmative I feel healthy today and some questions in the negative, which I feel unwell today. If you're trying to test healthiness, you've got to take that question, I feel unwell today and reverse score it so that always the high numbers represent healthiness. What you do when there's a reverse scored item or any item that must be entered as such is you're going to go into SPSS, At the place where the reverse scoring takes place, again, this will be shown in the data entry video that you'll watch first. You'll go in and you'll reverse score the items on your scale that are noted as reverse scored. Let's say it's a zero to two scale. Every zero now equals two. Every one still equals one, and every two now equals zero. There'll be a place to enter that information. You always want to rescore your items that need to be rescoed before you do the adding up and summing, before you do the scoring instructions for any scale. First you rescore, then you follow the scoring instructions. Okay. Now, one thing I want to remind you or pre remind you before you've watched this video on data entry is you're going to be naming every one of your variables. As you name say you have nine items on your scale. If the seventh item is re scored, then you want to call that seven r. Not just item seven, but item seven that was rescoed. It will help you later, when you're going through to try to correct any errors or notice any mistakes, you'll say to yourself, Oh, Number seven was supposed to be rescoed. Did I enter that originally? Did I make that clear in my database when I named and laid out the information for each variable? Or you'll notice when you're looking at the frequencies, and you suddenly see and we'll talk about the frequencies in a second. You suddenly see a number that doesn't look rescoed in that seven R item. Okay. You rescore before doing any of your scoring. The next thing to ask yourself in the discussion for the particular measure, again, that you've picked out or maybe measures. Some of you have picked two, even though I strongly recommended one. Are there any subscales or total scales that need to be calculated? That is to say, If you did a scale that has four subscales. You need to create a, a single number for that subscale that's all added up after the recoding after reverse scoring and come up with a single number for that. Again, that will be shown in one of the videos how to do that. But think ahead in discussion. Are there any subscales or are there total scales that have to be calculated? The way this is going to work is, for the first scale you use, and for many of you, that's the only scale you use. You have to enter individual data for each of the 100 people clients. If you had a nine item scale, you would need to enter nine items for client one, nine items for client two, nine items for client three, and so on. Then you have to have built in that re scoring. Now, if you have subscales or if you used a second scale, you're going to want to come up with some total scores. You're not going to want to enter all the data yourself. Again, we're making up this data. Now instead, you could say, Okay, for the second scale, instead of entering nine items for every single person, maybe the second scale is a ten item question scale. Now you would just enter their total score across the ten items, what's their score? Think ahead so you don't give yourself more work. Are there subscales to be calculated? Are there total scores for additional scales that need to be calculated? Fourth thing to address in the discussion, how can the variables in the database be labeled to best identify the correct items and avoid future errors? You may be surprised to hear my answer here. That follows on what doctor Hansen says, for the most part, don't use labels. For the most part, if you have named your variables well, you don't need additional labels. I know in prior classes at Kaiser, you've had points taken off sometimes if you didn't use labeling. I will not take points off if you don't use labeling because it actually tends to mess things up in SPSS. As I saw doctor Hansen's comments in the videos, I agreed thoroughly. So We're not going to worry about labeling. You should avoid that or use it extremely carefully. It's just further define your variables. But if you named your variables well, Bec one R. That's the depression scale or maybe Be D, one. Be depression item one, reverse scored. The next one, BEC two, Bec D two, BC D three, BC d4r because it's reverse scored. Then you don't need the labels. The labels end up being what gets spit out in SPSS, and it really can confuse the analysis. I'm going to go with doctor Hansen's recommendation, which is avoid using labeling or use it extremely carefully. She'll show it to you in the videos, but I don't think it's what you want to do. The last important question in the discussion is, how can errors in data entry be avoided or caught to ensure valid results. Here you're going to use your frequencies. That's the second video that's shown this week in week five. It's going to help you see on B depression one. I'm just making that up. But on the very first item, B depression one r, the frequencies show a number of people who scored one, a number of people who scored two, a number of people who scored three, and four and five, and then somebody who scored 49. Whoops. 49 is outside of the scoring for the BC depression inventory. You'd want to go back into your data base and find that 49, you'll say, Oh, that's lined up with client number 16. How do I go to client 16 and correct that 49 to make it a one, two, three, four or five. I'm going to very quickly share my screen L et me get this right first. I shouldn't say very quickly, but I do want to save time this week because a lot of it is time you have to put in watching these videos and prepping. I just want to align you up for the work and then send you to your week's work with the holiday here. I want to show you content. Now. Okay. Is everybody seeing week databases? Lanette, just give me a thumbs up if you're seeing a week five, great. Okay. Et's see if I can make this any bigger. Okay. The very first video is going to be about setting up the database, naming the variables, putting your demographics in, putting the scores of your scales in, and then just a score scale, a scale score if you did a second variable. This is the week I'm hoping none of you did a second variable. Then you'll go over to the second video. It actually spills over from Video one, and it'll start to talk about frequencies. Actually, I got ahead of myself. We're still in data entry, setting up the database, excuse me. We're still in setting up the database in videos one and two, how to set it up before you create your numbers. Then you'll start creating your numbers in your data entry. This we are going to work on. She wants to talk. You'll see that the data entry we'll talk about now how to put numbers in so that you have everything represented for your scales from each of your 100 clients. You won't have to make up the whole 100. She's going to show you how to make up ten and then copy it 90 more times. We're still going to talk about the variation you have to have in data entry and so on. But finally, when you get to the fourth video, it will deal with frequencies. Why am I bringing up the fourth video now? Well, because the fifth question in your discussion this week asks you to think about how can you avoid or detect errors in your database, one way to detect errors in your database is to run frequencies. That's your fourth video this week. It's quite brief, but it'll help you make sure that all your scales are entered for each client, each participant within the range of that scale? If you have a scale 1-10, you shouldn't see a zero anywhere in that row? If you have a scale from One to 15, you shouldn't see a 250 in there. We can accidentally drop zeros and put numbers in and so on. Frequencies will help you look. Item by item. Did I enter numbers that make sense for that item? If it's only a scale of one to five, a six doesn't make sense, and that'll pop out at you from your running of the frequencies. So again, the first two videos this week are setting up the database, just getting it ready, with the names of each of the variables, with the possible responses for each of the variables, whether it's a nominal or categorical scale like sex and race, and so on, describing the demographics, or whether it's scale, that's an interval scale, that's a nominal scale, and so on. Then you'll move into actually entering the data in this 15 minute video. Frequencies is shorter than that, as I recall. Then you'll do the reverse items and subscales, which is only the variables. I'm making her talk. And then the subscale scoring, which we've been talking about will be in the last seven minute video. So watch a video. Do that chunk of the work. Watch another video. Do the next chunk of the work. Do a couple videos a day and the chunks of the work, and you'll do very well. The data entry takes a good bit of time, even with the copying, prepare yourself for that 15 minute video one day and the data creation. It's not really data entry, so much as data creation, but in the real world, it would be data entry. Okay. Now that we're talking a little bit about data entry, let me remind you, you're going to make up 100 responses, 100 clients. You're going to need that n of 100 to show whether the program works or not. If the program really works, you may need as many as 100 people to show that it works. Remember with lower numbers in statistics, if you just took five people who went through the program. You could get some really weird numbers, maybe the two most extreme people show up in that first five numbers of people, and that could throw off showing that the program really does work, or it could show that the program works and it really doesn't. Again, if the extreme results are in another direction. Better that you take 100 people. That's the number we're hoping to get between your pre your intake and discharge dates. As I mentioned, the video will show how to enter the ten lines and then how to copy that ten lines nine more times, and then you'll end up with 100. When you enter the first ten lines, and this will be repeated in the video, but I just want to highlight it here. Make sure there's variability or what we call variance between subjects for computation. You don't want to say that subject number one answered the BC by responding one, two, three, four, five on five items. Client two responded one, two, 345, and client three responded one, two, 345. Client four, you're going to want to say client one responded one, three, two, five, four, and client two responded five, three, one, four, two. We want to throw in that variation or variance because we want it to be realistic data. It's not the case that in the real world, that if you brought ten different clients in, they would all respond to the B in the exact same way. Item one, the same way, item two, the same way, item three, the same way, and so on. Again, normally, we don't make up the data. Normally we receive the data from the folks who administered the scales at the agency we're looking at. But We would have to have people intake, given therapy, discharged, a pretest to protest, and we've got five, six weeks here to get the bulk of the material done for this course. We're trying to get you to learn how to build a database and then analyze what's in it with the framework or a mimicking of the real world. We're going to mimic real life here, we're going to have a real program in place, but we're not going to have real data in place. A couple of other things I wrote down to remind you here, make sure you run those frequencies. That's the fourth video. Again, not too long to find errors, things that are out of range. This is a quick, easy step so that data ends up in the range it's supposed to be in for each of the scales that you selected. The other thing to remember to do in SPSS, save often, like when you're writing, if you enter five lines of data, save it. Five more lines of data, save it. If you have a 90 item scale, and for each client, you're putting in 90 responses, save it after each client. If you have a ten item scale, you could save it after five clients. Because God forbid you had to go back and enter, I would only be that number of things. But if it's 90 items per client, save it each time. Then each time you save a copy block of ten more clients, save. Save often in SPSS. My final notes, and then I'm going to double check with you that you understand to watch these videos. Remember what we're going to come back and do is then watch a video on reverse scoring and subscales, and then subscale scoring. Just building the database this week, not doing any assessments, but we need to have those subscales in there, the reversed items scored so that next week when you run the analysis, you've got this nice cleaned up database. And My final notes, name each variable really well. When you're in that space, that you're naming each variable down that row of variables, name it beautifully. A little taste of the scale, the number of the item, and then whether it's been reverse coded or not. If you do use the labels column, make sure it's really unique. I'm going to tell you to avoid that labels column, but do make sure it's really unique. Lannett and I were speaking before class. Don't forget how good the layered stats site is, L A E RD, the stat site. That's a place to go for help, SMI, and as are the videos that are set up for this week. Yeah. I think those are the main points I wanted to make. This weeks a lot less of watching my lecture and a lot more of watching the prepared videos, the week five videos. Then I'm going to stop sharing here. I'm going to pull my file back up. Then you go on to the assignment this week. Using those five videos, here's what you have to do for your assignment, different than a written assignment, different than continuing the report. Here you attach the SPS S file that contains the data for your program evaluation. You can create the hypothetical data for this assignment, but the variables should reflect the items in the actual assessment measures that you have selected. That's what we've been talking about tonight, creating data in a beautiful database for what would match up well with the measures you selected. Be sure to follow the scoring instructions for your instrument or instruments, including reverse scoring items and calculating subscales and or totals based on the scoring instructions. Also, save the SPV file of the frequencies you run for each variable at subscale. I need you to attach the spss do sv, which will have all the data. Then I need you to attach the sps S SPV, which will have the frequencies for each variable and subscale. Then finally, attach the instruments with scoring instructions. Let me see the actual instrument you're using and the actual scoring instructions so that when I go through, I can say, Yeah, this does all work out. These are the data that should be entered for this scale, this subscale, et cetera. These reverse scored items are reverse scored and so forth. It's going to take me a little while to grade next week. I've been a little bit speedier. Probably will be a little slower this week too with the holiday and some days off from school. But In general, next week's going to be a busy week for me because this week is a busy week for you. Do you have any questions for me? I know a lot of the questions will probably come after you watch the videos, but if you have anything you need to clarify now, jump right in. No, I probably have questions with the sub scale portion at maybe after the video, maybe. Yeah. I would suggest watching the video first and seeing if it maps on with what you expect for a subscale. Okay. How many subscales do you have? They have listed three subscales. Okay. Yeah. Okay. So when you're entering your data, be thoughtful about baby putting an additional letter or two on that remind you which item is going to be in those subscales. Okay. So if you say item let's say it's the SEL one R because it reverse scored, and then it was something like conduct. Then you might put a CD. That was the conduct subscale. So as you name your variables, do a good job there. But overall, you're going to probably come out pretty smooth with the subscales given that there's only three. How many pieces of data each is entered for each of your clients? Well, they have how many items on your scale overall? There are 26 questions. Okay. That's very manageable. And then there's three subscales within there. Is there in the scoring instructions, anything that says additional items? No. It has the actual subscales. It has the numbers of what the subscales are, like numbers three, five, seven, eight, and 20, or problem focused, ten, 12, Yeah, and so forth, like that. So put a PF for problem focused at the end of that variable name, and it'll help you just have a prettier database in the end. Yeah. And then you'll have to follow the instructions for how to create the subscale score. There are probably additional items because you said you had 26 items, and there's three subscales, right? Mm hm. So three doesn't divide easily evenly into 26. So there's some extra items that just have nothing to do with any subscale. Don't worry about them. Don't make them an extra subscale. Just enter them as SEL three. And that might be all it is. It's SCL three. It's not reverse scored, and it's not part of any subscale. Okay. I'll re read. And rite. Write me, text me. Texting, I think is good over a holiday weekend. I'm not getting on as often as I should. And so definitely text. But e mail too, if it's easier for you. Okay. You can always text me that you've sent an e mail, right? So sometimes easier to explain everything in an e mail than in a short text. But you can always say, doctor Wade, I need to hear back from you. I sent you an e mail. I sent it Friday morning, and it's Friday night, and I haven't heard from you, something like that. Actually, I got a plane ticket, so I'll probably text you a book. Okay, got you lucky to get one. Great. Great. Enjoy your holiday. That's fabulous. You. Thank you. Okay. Do you see you have any other questions now? No, not yet. Okay. Then I'll see you and just start watching those videos. Thank you. Thank you for your time. I appreciate it. Yeah. Thank you, Lent. Bye bye. Bye bye.

,

2

Week 4 discussion

Student’s Name

Institution Affiliation

Date

Participants to Be Evaluated

Participants to be measured will include those who have, in one way or another, been actively engaged with the MHA programs, whether through direct participation in stigma reduction or increased access to mental health services. Subjects will be selected using stratified random sampling to ensure a balance in demographic representation according to age, socioeconomic status, geographic location, and past experiences with mental health services. This helps ensure that the assessment captures the diverse experiences of targeted groups of the program. However, stratification can be resource-intensive, requiring very careful planning to identify and recruit participants from all relevant subgroups.

Staff Administering the Evaluations

The assessment will be carried out through MHA's trained staff or the contracted assessors. These shall carry out various surveys and interviews and gather information using observational approaches. Evaluators should be trained to ensure consistency and minimize evaluator biases when collecting data. Experienced personnel may increase validity and grant more weight to the assessment, although, in this case, requiring more training may elevate the cost and lengthen project timelines.

Sample Size and Assessment Schedule

The sample will be targeted to ensure a minimum of 300 participants per program for statistical significance. This number creates a reliable comparison across subgroups and program sites. The participants will be assessed at three key points: at baseline (before intervention), immediately following the implementation of the program, and six months post-intervention. This longitudinal approach provides insights into both the immediate and sustained impacts of MHA programs. While this design realizes valuable data longitudinally, the impact of participant dropout, especially in long-term follow-up measurements, may jeopardize the reliability of such findings.

Validity and Reliability of Instruments

The tools to be used in the assessment include but are not limited to the Mental Health Knowledge Schedule (MAKS) and the Patient Health Questionnaire (PHQ-9) which have been pretested. MAKS captures(stigma-related knowledge and attitude) while the PHQ-9 focuses on mental health outcomes corresponding to the severity of depression. The validity of the two tools used here has lots of support. For example, the works of Singleton et al. (1998) show that self-report questionnaires such as MAKS have been determined to possess construct validity since they ranked high on the criterion measure of validity In a similar vein, works carried out by Kroenke et al. (2001) reveal that the PHQ-9, a self-administered inventory that is popularly used in mental health research has strong scores in Construct validity. Besides, these tools have a powerful reliability; for example, PHQ-9 is said to be 0.89; which reveals that this tool has recorded high levels of Cronbach’s alpha. That is why their application at MHA may require certain modifications to meet the organization’s goals and objectives more accurately.

Cost and Accessibility of Instruments

Cost is a very sensitive parameter when it comes to decisions regarding which tools would be most suitable for evaluation. PHQ-9 is also in the public domain and therefore relatively cheaper for this evaluation. On the other hand, adaptations by MAKS may need permission or charge for this sort. If cost becomes an issue then other freely available tools such as the Generalized Anxiety Disorder (GAD-7) scale can be looked at as options. However, using public-domain instruments usually involves a low cost, but it is unclear whether they can be customized according to MHA’s specific program requirements, and whether they are suitable for this kind of task at all (Torjesen, 2022).

Conclusion

Meanwhile, utilizing a balanced implemented, stratified random sampling valid, and reliable equipment and a long-term research timetable will allow MHA to avail themselves of the measurement of programs for effective outcomes. This will provide important information on matters bearing stigmatization and improvement of the health system in a sampled and cost-effective manner. This will further improve MHA’s capacity to adapt programs in practice and be better placed to meet the needs of society’s needy groups( Fink, 2015).

References

Fink, A. (2015). Evaluation fundamentals: Insights into program effectiveness, quality, and value

(3rd ed.). Thousand Oaks, CA: Sage.

Torjesen, I. (2022). Access to community mental health services continues to deteriorate, survey

finds. BMJ: British Medical Journal (Online), 379, o2585.

https://doi.org/10.1136/bmj.o2585