As I have mentioned previously, Trinity is considering changing from the current student load of ~5 courses each semester to a load of ~4 courses each semester. This alternate configuration is called a 4-4 student load as each student normally takes 4 courses, each of which is 4 hours of credit. A related proposal is to reduce the teaching load from the current 3-3, which each faculty member teaches three courses each semester, to a 3-2 teaching load where faculty alternate between 3 and 2 courses. In general I am opposed to both of these changes, but I have to admit that my opposition is based largely on thought experiments and imagined consequences instead of empirical data.
This weekend I got the chance to talk to someone who teaches at Southwestern University. They made the change from a 5-5 student load and 3-3 teaching load to 4-4 and 3-2 a few years ago. So this faculty member has direct experience with both of these systems. I wanted to record what we talked about and her perspective of that change here, because I felt that she had some very good insights.
Lack of Student Flexibility
The #1 problem that she described was something I hadn't even thought of, a lack of student flexibility in scheduling. In a 4-4 student load situation, students really need to take 4 courses each and every semester. The reason being that it isn't feasible for most students to go up to 5 courses when each one is four hours, and if you have more than one or two semesters with only three courses, you won't graduate on time.
All faculty know that occasionally students get in over their heads or sign up for courses they really aren't prepared to take. Under a 4-4 scheme, these students really can't drop those courses without pushing back their graduation. In the case where students choose to register for only 3 courses originally and take a light load, they have an even worse problem if it turns out that one course causes them problems because then dropping to two courses can cause problems related to full-time enrollment for the year. That leads to all types of financial difficulties for most students.
Under the category of lacking flexibility, Southwestern also runs into problems when it comes to transfer students and transfer credit. Given the challenges of enrolling students, transfers are potentially very important to many liberal arts schools. How do you count the 3-hour credits that most transfer students will come in with? Similarly, many Trinity students take summer courses away from Trinity and the same it true for Southwestern. We can't give students 4-hours of credit for a 3-hour summer course taken elsewhere. So we might check off a requirement for them, but they run into problems when it comes to total hours. Here again you can have students who fail to graduate on time because they don't have the right number of hours. With a 4-4 configuration you simply lose the flexibility for students to go slightly above the normal requirements to offset deficiencies.
Too Few Courses
Closely related to the problem of student enrollment flexibility is the problem of course offering flexibility. The faculty member I talked to noted that her department (a STEM department) was forced to reduce their major to 10 courses. This reduces the number of electives that students take as part of the major and how many electives can be offered. Not only are there fewer faculty teaching slots for electives, students don't take many so it is hard to get a critical mass of students to validate offering them.
An odd side effect of having majors cut down to 10 courses was that some departments bend the rule by hiding requirements in prerequisites. In particular she mentioned that the physics department, in order to get under the 10 course limit, doesn't explicitly list any math requirements. Instead, they have math courses as prerequisites on certain physics courses, making them implicit requirements. I know that Trinity highly frowns on implicit requirements, and the University Curriculum Committee typically rejects any such proposal. However, some fields truly do have a need to include more courses, especially when outside requirements are included.
Caps on majors or just the limits to courses could cause problems for things like theses as well. The CS department at Trinity does a 3-semester honors thesis track. There is no way we can do three semesters when students only take four courses each semester. This is definitely one of those situations where two, four hour courses are not even close to the equivalent of three, three hour courses.
The last problem presented by reduced course flexibility with the change made at Southwestern is in the inability to offer short seminars and the like on topics of interest. There modified system does not nicely support the equivalent of one and two hour seminars or independent studies. This can make it much harder to support undergraduate student research.
Courses Didn't Increase in Difficulty
The primary arguments for the 4-4 student load is that students are overburdened by having five courses each semester and that courses could be more rigorous if students only took four. I have always felt that this argument falls flat. Students spend a lot of time doing things outside of academics. If faculty members really want their students to dedicate more time to their course, they simply need to make the course harder and find ways to enforce that students really do the work. It might not be easy or even obvious how to do it, but that is what needs to be done. If faculty can't find ways to enforce students doing the work, moving to the 4-4 model isn't going to help.
Indeed, the Southwestern faculty member said that my fears match what has happened there. Few faculty have actually made their courses more rigorous. What is worse, because most of the courses went to 4-credits without going up to 4-hours, she feels that students are actually spending less time working on academics. Why? Because students now only have 12-hours in class. So when they look at their schedule they see even more "free time" and they tend to book it for things like jobs, sports, or other extra-curricular activities. Once they have done that, they truly don't have the time to complete extra rigour even if faculty members step up and make their courses more rigorous.
The reality is that you have to change that campus mentality toward courses and course work and that is more important than how many courses students take or how many hours they meet. Apparently Southwestern is experiencing most of what I see as the worst possibilities of moving to a 4-4 and virtually none of the benefits. However, because they went down to a 3-2 teaching load, faculty see a benefit so it will be nearly impossible to switch back.
Adjuncts and Conclusions
One last odd problem that Southwestern has run into is that challenge in hiring adjunct faculty. That can be a challenging process in many departments when asking them to teach a three credit course. Asking them to teach a four hour course makes it harder. If they are teaching a course that only meets three hours, but is supposed to have a harder workload, it is very unlikely that they will require the desired level of effort.
The general conclusion from this faculty member was that she couldn't find anything good to say about the 4-4 student load at Southwestern. Only the negatives of the change have been manifest in the implementation. The same is almost true of the 3-2 teaching load with the minor exception that there is some small benefit to having the freedom of picking when the 2-course semester is done. However, in practice Trinity does not appear to be extremely strict about making certain every faculty member teaches three courses every semester so this is not really a practical benefit.
Thanks for this. I wonder if experiences are different by department, field, or even individual. Has there been a general survey of faculty (and students) at Southwestern? I also wonder about the basis for some of the observations: for instance, he/she argues that few faculty have actually made their courses more rigorous; but how does he/she *know* that?
ReplyDeleteSincerely,
Tom
Tom,
DeleteI have to admit that I drew a random sample of one. It was a chance meeting and we happened to get on the topic of the 4-4. So there is no statistical significance here in that regard. I do not know of any survey, though I will say that this has gone to people at SU and I have had a dialog with at least one person who feels very differently. I have written to a number of other STEM faculty members in the hopes that I could acquire a bit more information.
How do any of us know how rigorous our colleagues courses are? We never sit in on them or go through full syllabi and look at teaching materials except perhaps in formal reviews. Instead, we get a "feel" for the difficulty of courses by putting together multiple student reports on things. The only other way is to rely on RateMyProfessor.com.
Mark
Thanks, Mark. But if we're going for a sampling of faculty at Southwestern, why poll only STEM faculty? Why not everybody in the College of Arts and Sciences? Am I missing something?
DeleteIn your response, you indicate that the data for the subheading "courses didn't increase in difficulty" seems to have been compiled from either 1) student reports or 2) ratemyprofessor.com. Is there evidence that either method was employed by the faculty member with whom you spoke? More to the point, is there evidence that either method is appropriate for assessment? (I would like to think that we're *not* basing the bachelor's degree at Trinity on ratemyprofessor.com, but perhaps I've an irrational bias.)
Might there be superior ways of assessing this curricular change? (Were there formal departmental surveys, for instance?) At least one alternative (and to me, less subjective) method of assessment would be the collation of syllabi (both pre- and post-curricular change) to ensure that substantive changes were both considered and employed. I assume that that method was employed at Southwestern, though I don't really know; it would be good to find out.
Sincerely,
Tom
Tom,
DeleteMy discussion and the input were very informal. This was not a formal assessment. That would require a lot of buy-in and time/effort from the faculty and administration at Southwestern. I can't make that happen, so the best I can do is talk to individuals.
I have been talking to a second person in the social sciences. Her feeling is that everything is great with the 4-4. I want to specifically sample STEM faculty because I know at Trinity many of the opinions about moving to the 4-4 are drawn along what you might call divisional lines. I expect that most non-STEM faculty at Southwestern are happy enough with the changes and I don't have the time to do a full survey of their faculty. Not that they would reply if I did. I am hoping that I could get feedback from a few people though and I specifically want to know the experience of the faculty there who are in positions similar to many who oppose the change here. So far, no one has responded to me at all.
It might be interesting to collect syllabi, but I don't think that would be very conclusive. I think a lot of the difficulty of a course is in the details. My syllabi might look much like those of many of my colleagues, but that doesn't imply our courses are of equal difficulty.
Mark
P.S. - If I were going into conspiracies, I have to admit that I wonder a bit about why I am not hearing anything from people. It could be that they are busy, but even the social science professor who wrote quite a bit earlier this week withdrew from conversation today. That could honestly be because she has a lot of work to do. I can certainly understand that. However, this blog also made it to the Provost at Southwestern and the second/third hand report I got about his e-mail was that this blog post made him very unhappy. So there is at least a small fear in my head that I won't hear from people at this point because of that. It might be not be rational or realistic, but that fear exists.
Hi, Mark,
ReplyDeleteI don't follow the argument concerning syllabi at all. Are we really proposing that ratemyprofessor.com is a superior guide to assessing course rigor than systematically comparing syllabi from pre- and post-curricular review? If the difficulty of the course is in the details, then why not ask for *detailed* syllabi that clearly show the increased expectations in a 4-4 world? Syllabi aren't the only yardstick of rigor, of course, but certainly they contribute; and to me, they would be far more useful than haphazardly collected student reports.
As for the postscript, I'm not quite sure what to say. We've been replying to each other in about a day, but I suspect that most professors work on a far more leisurely schedule, especially during term time. And I would hope that Southwestern has already prepared (or is preparing) a report on their change: that would save us all a lot of time.
Tom,
DeleteI know that there are problems in trusting student reports on their learning, but student reports on difficulty of courses are something that I am inclined to trust. On the other hand, I think there is little correlation between a syllabus and course difficulty. Maybe that is more true for the fields I teachin, like CS and Physics, but I could add extra assignments and tests while making a course easier or go the opposite direction. The syllabus really tells you very little about how challenging I make the material, only what topics are covered and how many assessments are done.
I'm still waiting to hear back from other faculty. If I hear from them you can certainly expect to see a blog post on that topic.
Mark
Hi, Mark,
DeleteI'm not saying that student evaluations shouldn't be part of the equation, but not a single student report has actually been cited in our discussion so far--just one professor's *interpretation* of what seems to be elusive data. (I prefer to draw my own conclusions: give me data, or give me death! =).
As for syllabi, well, I assume (perhaps wrongly) that Southwestern professors were asked to revise their syllabi after the curricular change; my guess is that Southwestern professors who took the time to do so would be surprised (and perhaps outraged) to discover that there's in fact little correlation between their newly-revised syllabi and course rigor. (What would be the point of revision, then?) Again, I assume that Southwestern has implemented assessment measures for their new curriculum, and it would behoove us to see if there's a report that takes into account a number of instruments, including student evaluations, course syllabi, honors theses, and the like. That's only fair to our colleagues at Southwestern, and useful to us as we consider various proposals here at Trinity. -- Tom
Tom,
DeleteI welcome more data. However, I don't exactly have the ability to force replies from large numbers of people at Southwestern. I would love to see assessment with lots of data from both faculty and students. Without that though I have to go on what information I do have available. If you have the time to contact a swath of the SU faculty I'd love to hear how they respond.
As I understand it, people did have to resubmit syllabi. I expect we would have to do the same thing here at Trinity. The honest truth though is that not all 3-hour courses are equally difficult. Nor are all 4-hour courses. Two courses with equivalent syllabi can have different levels of difficulty depending on who is teaching, the details of their testing, and how they grade. The syllabus just happens to be the best proxy you can get prior to the course being offered. I would argue that after the course has been offered, student opinion would be more accurate than looking over the syllabi.
Mark
I guess I fail to see how students would know if a course were more rigorous in its revised iteration than its first unless they took it both times, especially since comparative evidence from syllabi has been discounted. (Is that indeed happening? Are students taking courses twice? Why?) If not, how can students be offering worthwhile comparative assessments if they never took the original course? After all, that's the crux of the original argument, that "few faculty have actually made their courses more rigorous." -- Tom
DeleteThat is a very good point about students not taking courses multiple times. One single student can't really do a fair assessment. Instead, you have to average over the feedback from many students and hope you can work on the assumption that the quality and capabilities of the admitted students doesn't change too much.
DeleteI think what this gets at it that is if very hard for anyone to make an honest judgement about the change in the difficulty of a course. I think the only way that would have really worked would be to have an outside faculty member closely observe the course before and after the change. They would basically need to audit the course twice. I think it is pretty clear that is prohibitively expensive. The only other alternatives are to rely on faculty self-reporting (that is very similar to what happens in the syllabus review) and having a statistically significant student response sample.
So, if I'm reading this correctly, your argument is that comparative assessment can only be derived from faculty self-reporting and a statistically significant student response sample. In the example in the original blog post, the student sample is, as you put it, "random" and "not statistically significant" and there is no faculty self-reporting. (There's some faculty other-reporting.) So why include that argument in the first place? I'm confused.
DeleteNo Tom, my point is that any truly accurate approach is nearly impossible to do, so you might as well gain what you can from flawed metrics and what you can get. In this case, whether Southwestern ever does a full assessment and makes it public isn't really important. Trinity will have voted on our decision before then. So I want to collect as much information as I can before we vote.
DeleteIf you don't like the approach I take, please feel free to collect your own information using something you think is more appropriate and share it.
Also, I specifically said that comparative assessment would be best done by having an outside observer actually audit the course before and after the change. That just happens to be prohibitively expensive. Please don't put words in my mouth.
Hi, Mark. I thought it was clear that the phrase "can only be derived" meant "can only *feasibly* be derived" since you had argued that outside auditors, while theoretically possible, were to be ruled out as prohibitively expensive. I apologize for any confusion.
DeleteI was, and remain, uncomfortable with the approach of the blog because it seems based entirely on anecdote: there's a later reference to student reports, but those have yet to surface. Now if anecdote is the best we can do, then that's the best we can do; but I think it's premature to draw conclusions concerning Southwestern's curriculum based solely on anecdote--and anecdotes from a single individual at that! Moreover, the strongly negative tone of the original blog might well seem offensive to at least some happy "stakeholders" (egads, I hate that phrase) of the revised Southwestern curriculum, including faculty, students and alumni.
Now if an institution truly deserves some knocks, then, sure, go ahead and malign it. I'm all for that. But I think the provost at Southwestern has every right to be "unhappy": it must seem to him/her that their curriculum has been unfairly characterized and judged in a public form, based on the most slippery of evidence. -- Tom
Tom,
DeleteThe blog is based on anecdote. It says so at the very top. However, as I have stated multiple times, I have no ability to collect anything more. I can't force a large number of SU faculty or students to complete surveys or do interviews for me. (I will note that I still haven't heard from any of the other SU faculty I wrote to.) However, I don't want to be going completely uninformed so I work with what I can get.
It sounds to me like you are arguing that we shouldn't listen to any single faculty member, even when it is made clear that is where the information comes from. In fact, I would go further to say it sounds to me like you feel the opinion of this faculty member should be silenced because a Provost might disagree with it. Surely that isn't true. If it isn't, then what is really your point in these comments?
Mark
I welcome the opportunity to clarify my argument. I was attracted to this blog post because it brought to my attention something that I hadn't really before considered: that Southwestern had recently made the switch from 5-5/4-4 and so their situation might provide a good analogy to Trinity as it considers a similar switch. (There are differences between the two institutions, of course, and those factors should be weighed.)
DeleteI’m persuaded that it’s indeed beneficial to talk to our colleagues at Southwestern and I applaud the impulse to collect such data. For me, however, a single anecdote isn’t particularly meaningful unless it’s contextualized within a range of views: otherwise, it’s hard to determine whether it’s a mainstream observation or an outlier. Now, if it were observed by a large swathe of faculty members at Southwestern, even anecdotally, that 1) rigor decreased or remained flat 2) that it’s difficult to hire adjuncts 3) that this all was a bad idea, etc., then I would be the first to conclude that the data are pretty grim, even if anecdotal. (I would still prefer corroborating evidence, if available: would Human Resources or Chairs have hard numbers on hiring adjuncts, for instance?) I further offered that comparative course syllabi, gathered from before and after the switch, might provide corroborating evidence of rigor (or lack thereof): at least that’s the sort of non-anecdotal data I was hoping might be available to complement or strengthen the anecdotal evidence.
Now, it could be that there is no non-anecdotal evidence and that my hopes are dashed. We might then indeed be forced to draw our conclusions about Southwestern’s curricular change based just on the observations of our colleagues there. I would, however, urge caution in formulating and publishing conclusions from such anecdotal evidence until we felt a substantial enough data set had been compiled. Otherwise, we risk alienating or upsetting the members of the Southwestern community who might be proud of their new curriculum in specific, or their university in general.
Tom,
DeleteThe Provost at Southwestern appears to be setting up a visit for the CCCR to visit and talk about their move to the 4-4. (I'd like to hope that this blog had something to do with that.) In the letter describing that, something very interesting was mentioned. Southwestern chose not to do a formal evaluation of the change based on the perception that it was going very well. So I fear you will only get anecdotal information from them. As I understand it, they didn't go through the process that would be required to do more. This truly worries me because the move to the 4-4 was apparently driven in large part from the top. That could make faculty members who don't like it unwilling to speak up publicly, and without a formal review, there wouldn't be an effective, and anonymous way for them to communicate their problems.
I have also still not heard back from any of the five STEM faculty members I contacted via e-mail. I have seen nothing so far that makes me feel that this worked well for STEM and that the process hasn't marginalized the faculty and departments from those departments. Perhaps the CCCR will be able to get a more clear picture. However, if they only meet with people who are selected to speak to them, and they can't get information from people who feel they can speak freely and anonymously, it isn't clear they will get a complete picture through the visit.
Mark
Tom,
DeleteIn matters of policy, the burden of proof rests upon the shoulders of those who propose changes to the status quo. In the context of Trinity University's course load, advocates of the 4:4 model have the burden to supply the sort of evidence that you asked Mark to supply. Specifically, this includes (a) non-anecdotal evidence that the current 5:5 model at Trinity is causing problems for our students, and (b) non-anecdotal evidence (perhaps from institutions like Southwestern) that 4:4 models solve those problems.
In other words, all of the evidentiary requirements you have placed upon Mark's shoulders in this comments thread should actually be placed upon the shoulders of those who are proposing that we change the 5:5 model at Trinity. Every single one of them. "Anecdotal evidence is not sufficient." Check. "Premature to draw conclusions about the failure of a model without hard data." Check. "Evidence that the status quo is (or is not) sufficiently rigorous." Check.
At this point, the only data we've seen as part of the 4:4 proposal is one survey of faculty which suggests -- at best -- that faculty opinion is highly divided. Granted, there are slightly more people who favor the pure 4:4 than who favor the pure 5:5 proposal, but this isn't exactly hard data.
You make terrific points about the importance of basing policy decisions on hard evidence and non-anecdotal data. I hope you will make those same points on the floor of the Faculty Assembly when we discuss the 4:4 proposal.
Aaron