[COPLAC-IR] FW: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Lashley, Brian (Institutional Research)
LashleyB at easternct.edu
Thu Jun 26 07:53:40 CDT 2014
Hi everyone,
You can delete this email if you didn't join the NSSE COPLAC consortium, even if you did NSSE on your own. This regards the additional items that the consortium participants included, not the modules you may have done.
You might or might not remember hearing from me in February that there was a slight glitch with two of our additional items, where the response options were mistakenly changed to an agreement scale instead of a satisfaction scale. No one panicked at that time, nor should they have. It was a simple matter of alerting NSSE to fix the response options for those two questions, which they did.
The emails below are a recent dialogue between myself and the NSSE team. Louis provides a pretty detailed accounting of the effect of the glitch, or lack thereof, on the two affected items. I'll let you have a look and just let me and/or Louis know if you have any questions.
Brian R. Lashley, Ph.D.
Assistant Director of Inst. Res.
Eastern Connecticut State Univ.
[PIR logo_20%_April 2014]
From: Rocconi, Louis Marshall [mailto:lrocconi at indiana.edu]
Sent: Wednesday, June 25, 2014 2:49 PM
To: Lashley, Brian (Institutional Research)
Cc: Gonyea, Robert Michael; Sarraf, Shimon Aaron; Brooks, Jennifer Lynn
Subject: RE: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Hey Brian,
Please do share. We will move forward with the consortium reports as described below. If there is strong objection from the COPLAC team regarding the reports, please let me know by July 3 in order to have sufficient time to implement.
Thanks,
Louis
From: Lashley, Brian (Institutional Research) [mailto:LashleyB at easternct.edu]
Sent: Wednesday, June 25, 2014 11:13 AM
To: Rocconi, Louis Marshall
Cc: Gonyea, Robert Michael; Sarraf, Shimon Aaron; Brooks, Jennifer Lynn
Subject: RE: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Hi Louis,
Thanks very much for this information, you guys have handled this really well.
Your findings on how the two versions were responded to aren't very surprising to me. The student worker who brought this to my attention said he could tell what was meant, and answered accordingly.
If it's OK I could forward your email to the COPLAC list so everyone knows. And personally I think your suggestion about combing the tables & footnoting is fine. I'm sure other folks will be fine with that.
Brian R. Lashley, Ph.D.
Assistant Director of Inst. Res.
Eastern Connecticut State Univ.
[PIR logo_20%_April 2014]
From: Rocconi, Louis Marshall [mailto:lrocconi at indiana.edu]
Sent: Tuesday, June 24, 2014 4:44 PM
To: Lashley, Brian (Institutional Research)
Cc: Gonyea, Robert Michael; Sarraf, Shimon Aaron; Brooks, Jennifer Lynn
Subject: RE: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Brian,
I have done some analyses with the COPLAC data examining the distributions for the items that were mislabeled and wanted to share with you the findings. About 31% of the COPLAC respondents received the incorrect response frame (1,607 students in 10 out of the 14 institutions). From examining the frequency distributions, it appears as though students answered the two versions of the question similarly (see frequency distributions below). Also, examining skip patterns showed that students who received the incorrect response frame did not skip these questions at a higher rate than those who received the correct version (fewer than 1% of COPLAC respondents skipped these items in both the correct and incorrect versions). How you would like this to be presented in the consortium reports? Given the similar distributions, I recommend that we combine the two versions into one variable in the report and add a footnote that says the following: "About 31% of students at 10 of the 14 COPLAC institutions received a mislabeled version of this item (Response options were incorrectly given as "Strongly disagree" to "Strongly agree"). Analysis of response distributions demonstrates that students answered both versions similarly. Upon request, NSSE will provide a data file which identifies students who received the mislabeled version."
The following table shows the percentage of students at each institution who received the incorrect version:
101709 University of Montevallo
21%
107071 Henderson State University
55%
129215 Eastern Connecticut State University
35%
148654 University of Illinois Springfield
0%
161226 University of Maine at Farmington
0%
163912 St. Mary's College of Maryland
24%
174251 University of Minnesota, Morris
67%
178615 Truman State University
46%
207722 University of Science and Arts of Oklahoma
36%
210146 Southern Oregon University
0%
226833 Midwestern State University
32%
232681 University of Mary Washington
56%
235167 The Evergreen State College
0%
240426 University of Wisconsin-Superior
33%
Thanks,
Louis Rocconi
Assistant Research Scientist
Center for Postsecondary Research
Indiana University
COP1407 Overall, how satisfied are you with faculty enthusiasm for teaching General Education courses (i.e., the courses outside your major or primary academic program)? {INCORRECT VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
1 Strongly disagree
54
3.4
3.4
3.4
2 Disagree
210
13.1
13.1
16.4
3 Agree
987
61.4
61.4
77.8
4 Strongly agree
249
15.5
15.5
93.3
9 Not applicable
107
6.7
6.7
100.0
Total
1607
100.0
100.0
COP1407 Overall, how satisfied are you with faculty enthusiasm for teaching General Education courses (i.e., the courses outside your major or primary academic program)? {CORRECT VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
1 Very dissatisfied
87
2.5
2.5
2.5
2 Dissatisfied
284
8.0
8.0
10.5
3 Satisfied
2220
62.8
62.8
73.3
4 Very satisfied
573
16.2
16.2
89.5
9 Not applicable
373
10.5
10.5
100.0
Total
3537
100.0
100.0
COP1407 Overall, how satisfied are you with faculty enthusiasm for teaching General Education courses (i.e., the courses outside your major or primary academic program)? {COMBINED VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
Valid
1 Very dissatisfied
141
.1
2.7
2.7
2 Dissatisfied
494
.3
9.6
12.3
3 Satisfied
3207
2.1
62.3
74.7
4 Very satisfied
822
.5
16.0
90.7
9 Not applicable
480
.3
9.3
100.0
Total
5144
3.3
100.0
COP1408 Overall, how satisfied have you been with the faculty enthusiasm of your courses in your major or primary academic program? {INCORRECT VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
1 Strongly disagree
20
1.2
1.2
1.2
2 Disagree
55
3.4
3.4
4.7
3 Agree
652
40.6
40.6
45.2
4 Strongly agree
801
49.8
49.8
95.1
9 Not applicable
79
4.9
4.9
100.0
Total
1607
100.0
100.0
COP1408 Overall, how satisfied have you been with the faculty enthusiasm of your courses in your major or primary academic program? {CORRECT VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
1 Very dissatisfied
38
1.1
1.1
1.1
2 Dissatisfied
160
4.5
4.5
5.6
3 Satisfied
1515
42.8
42.8
48.3
4 Very satisfied
1689
47.7
47.7
96.0
9 Not applicable
141
4.0
4.0
100.0
Total
3543
100.0
100.0
COP1408 Overall, how satisfied have you been with the faculty enthusiasm of your courses in your major or primary academic program? {COMBINED VERSION}
Frequency
Percent
Valid Percent
Cumulative Percent
Valid
1 Very dissatisfied
58
.0
1.1
1.1
2 Dissatisfied
215
.1
4.2
5.3
3 Satisfied
2167
1.4
42.1
47.4
4 Very satisfied
2490
1.6
48.3
95.7
9 Not applicable
220
.1
4.3
100.0
Total
5150
3.3
100.0
From: Sarraf, Shimon Aaron
Sent: Tuesday, February 25, 2014 1:23 PM
To: 'Lashley, Brian (Institutional Research)'
Cc: Rocconi, Louis Marshall; NSSETM1; Brooks, Jennifer Lynn
Subject: RE: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Yes, feel free to forward my email to other consortium members, Brian. Thanks for understanding. You are correct. Students saw the incorrect response options up until this past Friday evening for two survey questions, and then it was corrected as shown below in the second snapshot. Let me know if we can provide any additional information.
All the best,
Shimon
Incorrect Response Options Adminsitered:
[cid:image001.jpg at 01CF2F29.DC987FA0]
Corrected Response Options:
[cid:image003.png at 01CF2F2A.44686D20]
From: Lashley, Brian (Institutional Research) [mailto:LashleyB at easternct.edu]
Sent: Tuesday, February 25, 2014 12:58 PM
To: Sarraf, Shimon Aaron
Cc: Rocconi, Louis Marshall; NSSETM1; Brooks, Jennifer Lynn
Subject: RE: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Hi Shimon et al,
The error is regrettable but far from fatal. We'll deal with it. Everything is fine.
Would it be OK for me to forward your email to the other COPLAC NSSE contacts? And do I understand in point 3 below that issue is that some students saw one set of response options (the agreement scale) while other students saw another set (the satisfaction scale)? So technically there have been two different sets of additional items.
Thanks very much for your help.
Brian R. Lashley, Ph.D.
Assistant Director of Inst. Res.
Eastern Connecticut State Univ.
Gelsi-Young Hall room 248
Willimantic, CT 06226
860-465-5596
From: Sarraf, Shimon Aaron [mailto:ssarraf at indiana.edu]
Sent: Tuesday, February 25, 2014 11:57 AM
To: Lashley, Brian (Institutional Research)
Cc: Rocconi, Louis Marshall; NSSETM1; Brooks, Jennifer Lynn
Subject: NSSE 2014 Council of Public Liberal Arts Colleges consortium item set
Hi, Brian:
As I believe you've already heard from Mark, we did confirm that the response option text to two consortium items were inaccurate. NSSE staff are very sorry this error was not caught before your administration began. We have multiple checks in place but for several reasons it was not identified; needless to say, our checking process will change because of this.
I just want to provide a brief summary of where we are currently:
1) Approximately 1,600 students submitted responses to your consortium item set as of late Friday (distributed across 10 of the 14 schools in your consortium)
2) There is a lot more data that has yet to be collected. By the time the error was fixed this past Friday evening (2/21/14), 15 recruitment messages had been delivered to students at the 10 affected schools. Five institutions had only their invitation delivered while the other five also had their 1st reminder delivered. Collectively, there are 55 more recruitment messages to be delivered across all 14 schools.
3) We plan to conduct an analysis to help determine best ways to provide the data and reports in August. Once we have looked at the data closely during the late spring and summer, we will reach out again to provide an update.
Again, we regret that this occurred and will work hard to prevent it from ever happening again. If you have any questions, please do not hesitate contacting me by email or phone.
All the best,
Shimon
Shimon Sarraf
Asst. Director for Survey Operations & Project Services
National Survey of Student Engagement
812.856.2169
nsse.iub.edu
From: Lashley, Brian (Institutional Research) [mailto:LashleyB at easternct.edu]
Sent: Thursday, February 20, 2014 2:22 PM
To: NSSETM1
Subject: Eastern Connecticut State University, 129215
Good afternoon Team 1,
I just got some feedback from a student that the response options to our additional questions (consortium items uploaded on 9/12) are confusing. For example, it sounds like there were agreement scale response options for a question asking how enthusiastic the faculty are. Can you have a look? And maybe let me see how that section appears to the student?
Thanks!
Brian R. Lashley, Ph.D.
Assistant Director of IR
Eastern Connecticut State U
Willimantic, CT 06226
lashleyb at easternct.edu<mailto:lashleyb at easternct.edu>
860-465-5596
From: NSSE Team 1 [mailto:nssetm1 at indiana.edu]
Sent: Monday, February 10, 2014 10:22 AM
To: Lashley, Brian (Institutional Research)
Cc: nssetm1 at indiana.edu<mailto:nssetm1 at indiana.edu>
Subject: NSSE Administration Launch Coming Soon (Eastern Connecticut State University, 129215)
Dear Brian,
We are writing in anticipation of the launch of your NSSE administration, scheduled to begin next week. Please review the important information below.
After your administration begins, the dispositions summary on the Institutional Interface will detail responses. You can track how many students have completed the survey, for example, and whether or not any email messages are returned as undeliverable. You can also download a file detailing which students have completed the survey and when, which can be used by you for awarding incentives.
We strongly recommend that you monitor these details closely. We will be monitoring response rates, too, and will automatically stop administrations with evidence of technical issues.
The document below details technical specifications to prepare for message delivery and should be shared with IT professionals on your campus.
https://websurv.indiana.edu/nsse/interface/img/PreparingforMessageDelivery.pdf
Please contact us if you have questions.
Project Services Team 1
812-856-5824
nssetm1 at indiana.edu<mailto:nssetm1 at indiana.edu>
________________________________
To access the Institution Interface: https://websurv.indiana.edu/nsse/interface/login.cfm
You were sent a copy of this message because you are listed as a campus contact for Eastern Connecticut State University's participation in the 2014 NSSE (National Survey of Student Engagement). If you believe you should not be receiving these messages, please contact us.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 13488 bytes
Desc: image001.png
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 13931 bytes
Desc: image003.jpg
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0002.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.jpg
Type: image/jpeg
Size: 25120 bytes
Desc: image005.jpg
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0003.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 10498 bytes
Desc: image004.png
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image006.png
Type: image/png
Size: 10467 bytes
Desc: image006.png
URL: <http://lists.onenet.net/pipermail/coplac-ir/attachments/20140626/f1f5cb7f/attachment-0005.png>
More information about the COPLAC-IR
mailing list