This blog post, as with many of the others I have written recently, focuses on the efforts of some to change the standard student workload at Trinity to 4 courses/semester each counting for 4 credits (a 4-4 load) instead of the current 5 courses of 3 credits. This proposal normally also has attached to it a change in teaching load so that faculty teach 3 courses one semester and 2 the next instead of 3 courses every semester [*]. This change will have a lot of implications, but one interesting point to note is that, if I paint with overly broad strokes, most of the support for this comes from the humanities and social sciences, while most of the opposition comes from the STEM and pre-professional departments [**]. Something I have discussed with others, and which I truly believe to be the case, is that the departments supporting this move, along with a proposed change to our common curriculum, do so at their own peril. I want to elaborate on this idea here in part to record it and perhaps to bring it forward for broader discussions.
The move to the 4-4, and the new curriculum proposal that will come with it have one inevitable consequence, students will take a smaller number of courses and will get less diversity in the departments they are exposed to. The 4-4 makes that first point unavoidable. The second point happens because our current breadth requirements would be impossible to maintain under the 4-4 and the new proposal, which I should point out I support, does not require students to take courses in as many different departments.
So why do I think that this should bother the faculty in the humanities? Well, to pick on one particular major, how many people enter college saying they want to major in Religion? Of course, Religion isn't alone. There are a number of majors that only get majors by having them take introductory courses and fall in love with the topic. Some are even in STEM. The Geoscience department comes to mind as an example. This alone should probably give those departments second thoughts, but I am sure they can easily say that it will improve their departments in other ways so it is worth it.
What I feel they ignore though is the changing national view on the role of higher education. More pressure is being put on the idea that students should major in something that will get them jobs. This goes along with the complaints that students are leaving college with too much debt and then they aren't able to find jobs. This is why we see proposed legislation that will force colleges to report earnings of graduates by major.
I am personally already feeling some of the fallout of the change in how students are picking majors. I have 35 students in my CS2 courses at Trinity. We haven't had that many students go on to the second semester in a decade. CS is one of the few areas where people hear about there being lots of jobs and not enough people to fill them these days. It might still be true that no one goes to college thinking they want to major in Geoscience, but if they look at the incomes associated with that major they can probably be convinced.
What about other departments like Religion and History? I expect they are going to take a hit from this in the next few years as more and more students enter college thinking about their earning potential. Even if the Wyden-Rubio legislation does not pass, they have access to things like the "What's it Worth?" report for Georgetown. They will also find things like this Forbes article entitled "The 10 Worst College Majors", which slams most humanities majors for high unemployment and low incomes. So why would these departments support changes that are going to likely reduce the number of students coming into their classrooms and hence their ability to attract majors? Obviously I don't have an answer to that because it makes no sense to me, but I want to dig a bit deeper to make it clear what could be at stake for these departments.
Let's pick the top two departments from the top of the What's it Worth salary survey and some comparison humanities departments and look at faculty counts.
Engineering Science - 9
Computer Science - 7
Art and Art History - 12
History - 11
Religion - 9
Right now the humanities departments have as many or more faculty than the majors that will inevitably be getting a lot more attention if students and parents really start looking at ROI on a college education. (I know that liberal arts are all about the additional benefits of being broadly educated, but I promise you that parents are not blind to the dollar signs when they are looking at colleges for their kids. As such, this can't be ignored no matter how good the arguments are that some things matter more than money.) So the move to the 4-4 and the new curriculum will contract the distribution requirement and pull students out of almost every department, making major counts more important. Then, if we really do get a push toward majors that lead to jobs it seems to me that a lot of the humanities departments might have a hard time filling their classrooms and justifying their current faculty counts.
Note that I'm not wishing this on them or saying this is a good thing. Quite the opposite. By opposing the 4-4, I'm pushing to keep students taking a broader distribution of courses and make it so that even if more students choose to major in the sciences, the humanities will retain reasonable enrollments. I'm writing this because I don't think most of the humanities faculty see it that way and I'm wondering what they expect to see happen with their total head count in the next five years if they pass the 4-4 and the new curriculum.
[*] This isn't really a drop in teaching load. The current system has faculty teaching a total of 18 credit hours each year. The altered system would have them teaching 20.
[**] This is an overly broad generalization. My guess is that this post will be read by at least one Communications professor who doesn't favor the proposal. It is also problematic for Music. In addition, I'm sure a few STEM faculty are supportive. However, in general this dividing line holds.
Thursday, February 21, 2013
Wednesday, January 2, 2013
Loop Performance and Local Variables in Scala
This post was motivated by Local variables inside a loop and performance. I have to admit that the results of that post didn't really surprise me. For the code that they had written I felt like any reasonable compiler should have made the stack frame for the whole function big enough to hold those variables so that there was no additional work done for local declaration inside of a loop. What interested me though was whether the same would be true in Scala.
Why Scala Might Be Different
The for loop in Java is just a dressed up while loop. It is useful because it gives you places to put all the main components of a loop so that you are less likely to forget them. However, when you convert it to the assembly level (or bytecode level) it should have a conditional branch at the top and a non-conditional branch at the bottom with little additional overhead. In Scala however, the for loop is better described by the name "for comprehension". In a sense, there is no for loop in Scala. Instead, uses of for get translated to calls to higher order methods like map, flatMap, filter, and foreach. As an example, consider this for loop in Scala.
for (i <- 0 until runs) {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
This gets translated to the following:
(0 until runs).foreach(i => {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
This very simple example doesn't really show the power that is gained by this translation, but the capabilities of for comprehensions really do add a lot to Scala. Some of these features include simple data parallelism with parallel collections and concise syntax for Futures in the Akka framework. However, this greater power does come with some overhead in terms of performance. This change also makes it less clear to me if local variables in for loops would be significant.
Test Code
The following code shows how I modified the Java from Peter's blog post. The initial calls to the test functions are there to allow the JIT to go through the code before any timing is done. I also went a bit further and had each function call happen multiple times so that I could get a standard deviation in addition to average values. There are two functions using the for loop and two using the while loop.
object LocalVars {
def main(args:Array[String]) {
testInsideForLoop
testOutsideForLoop
testInsideWhileLoop
testOutsideWhileLoop
printResults("In for loop:",Array.fill(10000)(testInsideForLoop))
printResults("Out of for loop:",Array.fill(10000)(testOutsideForLoop))
printResults("In while loop:",Array.fill(10000)(testInsideWhileLoop))
printResults("Out of while loop:",Array.fill(10000)(testOutsideWhileLoop))
}
def printResults(header:String,results:Array[Double]) {
println(header)
val average = results.sum/results.length
printf("Average = %.3f ns\n", average)
val aveSqr = results.map(x => x*x).sum/results.length
printf("Standard dev = %.3f ns\n\n", math.sqrt(aveSqr-average*average))
}
def testInsideForLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
for (i <- 0 until runs) {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
(System.nanoTime() - start).toDouble / runs
}
def testOutsideForLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
var x, y, times = 0
for (i <- 0 until runs) {
x = i % 12
y = i / 12 % 12
times = x * y
counters(times) += 1
}
(System.nanoTime() - start).toDouble / runs
}
It is clear that where you declare local variables doesn't matter at all as all variations are inside the error bars for the measurement. However, there is a reasonable performance penalty for using the for loop. The ScalaCL project provides one approach to getting around this. Since ScalaCL is not up to date with 2.10 I didn't test this out. Earlier testing under 2.8 showed me that it generally did bring the performance of for loops in line with while loops. I would love to see the ScalaCL project get more support both for this reason and to improve the OpenCL collections.
More Questions
I have two problems with what is done here. First, my Scala code is not at all Scala-like. Part of this is because Peter picked a problem that is best solved in a very imperative way. It would be interesting to pick a problem that also has a nice functional solution and then play with that a bit. This would also make it possible to try seeing how using parallel collections could be beneficial.
A more significant question was posed in a discussion on Google+ by one of my former students, Will Shepherd. This is the question of what happens with memory if the variables are a bit more complex. Here again, the chosen example code holds us back a bit as there doesn't seem to be much need for non-primitive variables here. However, the results could be extremely different if the variable were to hold an array or some type of mutable object.
Perhaps someone can suggest a problem that would make some sense in exploring these two avenues.
Conclusion
My take away message from Peter's original blog, which I think hold true here as well is that your best bet is to write good code first, then worry about performance as needed. Declaring variables locally is just a good idea. Limiting scope is remarkably helpful in preventing bugs. So that is how you should write your code. If you find that your code isn't fast enough once you have it working, then you explore options for making it faster. In the case of local declarations of primitive variables in loops, I think it is clear that there isn't any point in even trying that route. However, if you do decide to start exploring optimizations, you need to be careful and meticulous. Re-benchmark after each "optimization" because it is possible that you might make a change which leads to uglier, harder to maintain code without getting any speed benefit at all.
Why Scala Might Be Different
The for loop in Java is just a dressed up while loop. It is useful because it gives you places to put all the main components of a loop so that you are less likely to forget them. However, when you convert it to the assembly level (or bytecode level) it should have a conditional branch at the top and a non-conditional branch at the bottom with little additional overhead. In Scala however, the for loop is better described by the name "for comprehension". In a sense, there is no for loop in Scala. Instead, uses of for get translated to calls to higher order methods like map, flatMap, filter, and foreach. As an example, consider this for loop in Scala.
for (i <- 0 until runs) {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
This gets translated to the following:
(0 until runs).foreach(i => {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
This very simple example doesn't really show the power that is gained by this translation, but the capabilities of for comprehensions really do add a lot to Scala. Some of these features include simple data parallelism with parallel collections and concise syntax for Futures in the Akka framework. However, this greater power does come with some overhead in terms of performance. This change also makes it less clear to me if local variables in for loops would be significant.
Test Code
The following code shows how I modified the Java from Peter's blog post. The initial calls to the test functions are there to allow the JIT to go through the code before any timing is done. I also went a bit further and had each function call happen multiple times so that I could get a standard deviation in addition to average values. There are two functions using the for loop and two using the while loop.
object LocalVars {
def main(args:Array[String]) {
testInsideForLoop
testOutsideForLoop
testInsideWhileLoop
testOutsideWhileLoop
printResults("In for loop:",Array.fill(10000)(testInsideForLoop))
printResults("Out of for loop:",Array.fill(10000)(testOutsideForLoop))
printResults("In while loop:",Array.fill(10000)(testInsideWhileLoop))
printResults("Out of while loop:",Array.fill(10000)(testOutsideWhileLoop))
}
def printResults(header:String,results:Array[Double]) {
println(header)
val average = results.sum/results.length
printf("Average = %.3f ns\n", average)
val aveSqr = results.map(x => x*x).sum/results.length
printf("Standard dev = %.3f ns\n\n", math.sqrt(aveSqr-average*average))
}
def testInsideForLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
for (i <- 0 until runs) {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
}
(System.nanoTime() - start).toDouble / runs
}
def testOutsideForLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
var x, y, times = 0
for (i <- 0 until runs) {
x = i % 12
y = i / 12 % 12
times = x * y
counters(times) += 1
}
(System.nanoTime() - start).toDouble / runs
}
def testInsideWhileLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
var i = 0
while (i < runs) {
val x = i % 12
val y = i / 12 % 12
val times = x * y
counters(times) += 1
i += 1
}
(System.nanoTime() - start).toDouble / runs
}
def testOutsideWhileLoop:Double = {
val start = System.nanoTime()
val counters = Array.fill(144)(0)
val runs = 1000 * 1000
var x, y, times, i = 0
while (i < runs) {
x = i % 12
y = i / 12 % 12
times = x * y
counters(times) += 1
i += 1
}
(System.nanoTime() - start).toDouble / runs
}
}
Results
I compiled the above code using using Scala 2.10.0-RC5 with the -optimise option. When I ran it I got the following output.
In for loop:
Average = 6.073 ns
Standard dev = 0.448 ns
Out of for loop:
Average = 6.084 ns
Standard dev = 0.507 ns
In while loop:
Average = 4.670 ns
Standard dev = 0.322 ns
Out of while loop:
Average = 4.669 ns
Standard dev = 0.320 ns
It is clear that where you declare local variables doesn't matter at all as all variations are inside the error bars for the measurement. However, there is a reasonable performance penalty for using the for loop. The ScalaCL project provides one approach to getting around this. Since ScalaCL is not up to date with 2.10 I didn't test this out. Earlier testing under 2.8 showed me that it generally did bring the performance of for loops in line with while loops. I would love to see the ScalaCL project get more support both for this reason and to improve the OpenCL collections.
More Questions
I have two problems with what is done here. First, my Scala code is not at all Scala-like. Part of this is because Peter picked a problem that is best solved in a very imperative way. It would be interesting to pick a problem that also has a nice functional solution and then play with that a bit. This would also make it possible to try seeing how using parallel collections could be beneficial.
A more significant question was posed in a discussion on Google+ by one of my former students, Will Shepherd. This is the question of what happens with memory if the variables are a bit more complex. Here again, the chosen example code holds us back a bit as there doesn't seem to be much need for non-primitive variables here. However, the results could be extremely different if the variable were to hold an array or some type of mutable object.
Perhaps someone can suggest a problem that would make some sense in exploring these two avenues.
Conclusion
My take away message from Peter's original blog, which I think hold true here as well is that your best bet is to write good code first, then worry about performance as needed. Declaring variables locally is just a good idea. Limiting scope is remarkably helpful in preventing bugs. So that is how you should write your code. If you find that your code isn't fast enough once you have it working, then you explore options for making it faster. In the case of local declarations of primitive variables in loops, I think it is clear that there isn't any point in even trying that route. However, if you do decide to start exploring optimizations, you need to be careful and meticulous. Re-benchmark after each "optimization" because it is possible that you might make a change which leads to uglier, harder to maintain code without getting any speed benefit at all.
Sunday, November 25, 2012
The Fallacy of "Deep" Courses and Workloads
Background
This is back on the proposal that is being considered at Trinity to change student course loads to 4 courses per semester, each worth 4 credits (4-4) and the associated change in teaching load to five courses each year with 3 one semester and 2 the next (3-2). I am writing this blog post because I am finally getting fed up with statements that I keep hearing about how some courses simply require more time than others, especially time spent outside of class. There is also an associated aspect of this, where the same people who make these statements insinuate that this extra time is needed for their courses because they are too "deep" for a normal 3 credits.
Why I am Upset
Here is the problem, you don't hear these things coming from STEM faculty. You know, the people teaching the science and math courses that most students consider to be the hardest courses on campus. The majority of STEM faculty seem to be quite happy with how things are. The push for change seems to be coming largely from the humanities and social sciences.
I guess what really offends me is that these statements imply that because I am happy with my course counting for 3-credits that somehow my course is easier and less deep. I'm sorry, but I have had many students tell me that the courses I teach are the hardest ones they take during their entire time at Trinity. My courses also keep students very busy outside of the classroom. My guess is that most of the students in my courses will spend more time on my class than on any of the other courses they are taking. In fact, when students can't do that they tend to wind up withdrawing from my course.
My courses aren't time consuming because they are full of busy work either. They are time consuming because students have to wrap their heads around completely new ways of thinking, and they have to learn to break down problems to levels they have never done before. Then they are forced to apply those capabilities and they are forced to make things work. My courses aren't fluff. They are rigorous and challenging and it is really getting on my nerves how so many faculty seem to be saying that their courses are harder than mine and hence need to count for more credits.
This doesn't just apply to my courses either. Anyone who tells you that courses like E&M, Quantum Mechanics, or Complex Analysis don't require deep thinking or much time spent outside of class clearly has no idea what he/she is talking about. They have obviously not spent time trying to picture a vector field flowing through a Gauss pillbox.
If people really want to get some data on what departments have more challenging courses, and which ones require more time and effort both inside and outside of class, there is a publicly available data set for that called RateMyProfessors.com. They list ease as a criteria. If you scan through it for faculty with an ease rating of 2.5 or less, you will notice that STEM faculty are extremely over represented. (So is the English department.)
(Here is a coding assignment for any of my students reading this. Write a program to scrape data from RateMyProfessor.com. I'm most interested in department and ease for this topic, though it would be good to have names so that people who aren't current faculty can be removed.)
Enforcing Work
My gut feeling is that faculty who think they need students to take fewer courses and want students to have more time outside of class really just need to find better ways of enforcing work done outside of class. The idea that students are booked solid with academics when taking five 3-credit courses is absurd. Trinity students spend lots of time doing many things that aren't academic. Give them more time outside of class and they will use it for a variety of non-academic activities unless you can enforce that they use it doing the work for your class.
The fallacy that STEM courses somehow require less time outside of class is absurd. The reality is that STEM courses typically do a much better job of enforcing that students actually do what they are supposed to do outside of class. I can give my students assignments and exercises and if they don't take the time to learn the material, they will be completely incapable of doing those assignments. No, my students don't read everything I assign. However, they will read enough to be able to write the programs I ask them to do. (Honestly, many students would probably spend a little less time on my class if they would do the reading up front instead of wasting hours trying to code before they understand what they are doing.) I know this is more challenging for many non-STEM courses, but that doesn't mean it is impossible.
The bottom line, as I see it, is that changing the number of hours/credits for a course doesn't make students do more work, and making new policies based on the idea that it will is a great way to reduce the quality of a Trinity education. If faculty want students to do more work on their course, they need to be inventive and creative and find ways to enforce students doing the required work. That is the only way to make change happen.
Feedback
I'd love to hear back from anyone who reads this and is willing to say which courses they had in college that challenged them the most or kept them busy the most. I'd also love to hear why. This is especially true for Trinity students.
(Update: I would like to thank Laura Gibbs for pointing out the National Survey of Student Engagement in the Google+ discussion of this post. Ideally students should be spending 2 hours outside of classes for every hour inside of class. This survey shows that students spend between 12 and 18 hours total prepping for class. In other words, they aren't even close to the 2-hour mark. If faculty want students to work harder, the reality is that most students have time in their schedules. They just need to be forced to take the time.)
Aside: A Useful Technique
I'll close with a technique that I learned in grad school for Amer Diwan who taught a course on program analysis. This course was all about reading journal articles. I have used this approach to good effect in similar courses at Trinity. Make students show up to class with written questions on the reading. If you don't want to do questions, have them write a short paragraph instead. Base the in class discussion on the questions students hand to you when they first walk in. Have a portion of the semester grade come from the quality of what the students write for this. Call students out when what they provide sucks and shows that they didn't really put in the effort.
This is back on the proposal that is being considered at Trinity to change student course loads to 4 courses per semester, each worth 4 credits (4-4) and the associated change in teaching load to five courses each year with 3 one semester and 2 the next (3-2). I am writing this blog post because I am finally getting fed up with statements that I keep hearing about how some courses simply require more time than others, especially time spent outside of class. There is also an associated aspect of this, where the same people who make these statements insinuate that this extra time is needed for their courses because they are too "deep" for a normal 3 credits.
Why I am Upset
Here is the problem, you don't hear these things coming from STEM faculty. You know, the people teaching the science and math courses that most students consider to be the hardest courses on campus. The majority of STEM faculty seem to be quite happy with how things are. The push for change seems to be coming largely from the humanities and social sciences.
I guess what really offends me is that these statements imply that because I am happy with my course counting for 3-credits that somehow my course is easier and less deep. I'm sorry, but I have had many students tell me that the courses I teach are the hardest ones they take during their entire time at Trinity. My courses also keep students very busy outside of the classroom. My guess is that most of the students in my courses will spend more time on my class than on any of the other courses they are taking. In fact, when students can't do that they tend to wind up withdrawing from my course.
My courses aren't time consuming because they are full of busy work either. They are time consuming because students have to wrap their heads around completely new ways of thinking, and they have to learn to break down problems to levels they have never done before. Then they are forced to apply those capabilities and they are forced to make things work. My courses aren't fluff. They are rigorous and challenging and it is really getting on my nerves how so many faculty seem to be saying that their courses are harder than mine and hence need to count for more credits.
This doesn't just apply to my courses either. Anyone who tells you that courses like E&M, Quantum Mechanics, or Complex Analysis don't require deep thinking or much time spent outside of class clearly has no idea what he/she is talking about. They have obviously not spent time trying to picture a vector field flowing through a Gauss pillbox.
If people really want to get some data on what departments have more challenging courses, and which ones require more time and effort both inside and outside of class, there is a publicly available data set for that called RateMyProfessors.com. They list ease as a criteria. If you scan through it for faculty with an ease rating of 2.5 or less, you will notice that STEM faculty are extremely over represented. (So is the English department.)
(Here is a coding assignment for any of my students reading this. Write a program to scrape data from RateMyProfessor.com. I'm most interested in department and ease for this topic, though it would be good to have names so that people who aren't current faculty can be removed.)
Enforcing Work
My gut feeling is that faculty who think they need students to take fewer courses and want students to have more time outside of class really just need to find better ways of enforcing work done outside of class. The idea that students are booked solid with academics when taking five 3-credit courses is absurd. Trinity students spend lots of time doing many things that aren't academic. Give them more time outside of class and they will use it for a variety of non-academic activities unless you can enforce that they use it doing the work for your class.
The fallacy that STEM courses somehow require less time outside of class is absurd. The reality is that STEM courses typically do a much better job of enforcing that students actually do what they are supposed to do outside of class. I can give my students assignments and exercises and if they don't take the time to learn the material, they will be completely incapable of doing those assignments. No, my students don't read everything I assign. However, they will read enough to be able to write the programs I ask them to do. (Honestly, many students would probably spend a little less time on my class if they would do the reading up front instead of wasting hours trying to code before they understand what they are doing.) I know this is more challenging for many non-STEM courses, but that doesn't mean it is impossible.
The bottom line, as I see it, is that changing the number of hours/credits for a course doesn't make students do more work, and making new policies based on the idea that it will is a great way to reduce the quality of a Trinity education. If faculty want students to do more work on their course, they need to be inventive and creative and find ways to enforce students doing the required work. That is the only way to make change happen.
Feedback
I'd love to hear back from anyone who reads this and is willing to say which courses they had in college that challenged them the most or kept them busy the most. I'd also love to hear why. This is especially true for Trinity students.
(Update: I would like to thank Laura Gibbs for pointing out the National Survey of Student Engagement in the Google+ discussion of this post. Ideally students should be spending 2 hours outside of classes for every hour inside of class. This survey shows that students spend between 12 and 18 hours total prepping for class. In other words, they aren't even close to the 2-hour mark. If faculty want students to work harder, the reality is that most students have time in their schedules. They just need to be forced to take the time.)
Aside: A Useful Technique
I'll close with a technique that I learned in grad school for Amer Diwan who taught a course on program analysis. This course was all about reading journal articles. I have used this approach to good effect in similar courses at Trinity. Make students show up to class with written questions on the reading. If you don't want to do questions, have them write a short paragraph instead. Base the in class discussion on the questions students hand to you when they first walk in. Have a portion of the semester grade come from the quality of what the students write for this. Call students out when what they provide sucks and shows that they didn't really put in the effort.
Saturday, October 20, 2012
Problems with the 4-4 Student Load
As I have mentioned previously, Trinity is considering changing from the current student load of ~5 courses each semester to a load of ~4 courses each semester. This alternate configuration is called a 4-4 student load as each student normally takes 4 courses, each of which is 4 hours of credit. A related proposal is to reduce the teaching load from the current 3-3, which each faculty member teaches three courses each semester, to a 3-2 teaching load where faculty alternate between 3 and 2 courses. In general I am opposed to both of these changes, but I have to admit that my opposition is based largely on thought experiments and imagined consequences instead of empirical data.
This weekend I got the chance to talk to someone who teaches at Southwestern University. They made the change from a 5-5 student load and 3-3 teaching load to 4-4 and 3-2 a few years ago. So this faculty member has direct experience with both of these systems. I wanted to record what we talked about and her perspective of that change here, because I felt that she had some very good insights.
Lack of Student Flexibility
The #1 problem that she described was something I hadn't even thought of, a lack of student flexibility in scheduling. In a 4-4 student load situation, students really need to take 4 courses each and every semester. The reason being that it isn't feasible for most students to go up to 5 courses when each one is four hours, and if you have more than one or two semesters with only three courses, you won't graduate on time.
All faculty know that occasionally students get in over their heads or sign up for courses they really aren't prepared to take. Under a 4-4 scheme, these students really can't drop those courses without pushing back their graduation. In the case where students choose to register for only 3 courses originally and take a light load, they have an even worse problem if it turns out that one course causes them problems because then dropping to two courses can cause problems related to full-time enrollment for the year. That leads to all types of financial difficulties for most students.
Under the category of lacking flexibility, Southwestern also runs into problems when it comes to transfer students and transfer credit. Given the challenges of enrolling students, transfers are potentially very important to many liberal arts schools. How do you count the 3-hour credits that most transfer students will come in with? Similarly, many Trinity students take summer courses away from Trinity and the same it true for Southwestern. We can't give students 4-hours of credit for a 3-hour summer course taken elsewhere. So we might check off a requirement for them, but they run into problems when it comes to total hours. Here again you can have students who fail to graduate on time because they don't have the right number of hours. With a 4-4 configuration you simply lose the flexibility for students to go slightly above the normal requirements to offset deficiencies.
Too Few Courses
Closely related to the problem of student enrollment flexibility is the problem of course offering flexibility. The faculty member I talked to noted that her department (a STEM department) was forced to reduce their major to 10 courses. This reduces the number of electives that students take as part of the major and how many electives can be offered. Not only are there fewer faculty teaching slots for electives, students don't take many so it is hard to get a critical mass of students to validate offering them.
An odd side effect of having majors cut down to 10 courses was that some departments bend the rule by hiding requirements in prerequisites. In particular she mentioned that the physics department, in order to get under the 10 course limit, doesn't explicitly list any math requirements. Instead, they have math courses as prerequisites on certain physics courses, making them implicit requirements. I know that Trinity highly frowns on implicit requirements, and the University Curriculum Committee typically rejects any such proposal. However, some fields truly do have a need to include more courses, especially when outside requirements are included.
Caps on majors or just the limits to courses could cause problems for things like theses as well. The CS department at Trinity does a 3-semester honors thesis track. There is no way we can do three semesters when students only take four courses each semester. This is definitely one of those situations where two, four hour courses are not even close to the equivalent of three, three hour courses.
The last problem presented by reduced course flexibility with the change made at Southwestern is in the inability to offer short seminars and the like on topics of interest. There modified system does not nicely support the equivalent of one and two hour seminars or independent studies. This can make it much harder to support undergraduate student research.
Courses Didn't Increase in Difficulty
The primary arguments for the 4-4 student load is that students are overburdened by having five courses each semester and that courses could be more rigorous if students only took four. I have always felt that this argument falls flat. Students spend a lot of time doing things outside of academics. If faculty members really want their students to dedicate more time to their course, they simply need to make the course harder and find ways to enforce that students really do the work. It might not be easy or even obvious how to do it, but that is what needs to be done. If faculty can't find ways to enforce students doing the work, moving to the 4-4 model isn't going to help.
Indeed, the Southwestern faculty member said that my fears match what has happened there. Few faculty have actually made their courses more rigorous. What is worse, because most of the courses went to 4-credits without going up to 4-hours, she feels that students are actually spending less time working on academics. Why? Because students now only have 12-hours in class. So when they look at their schedule they see even more "free time" and they tend to book it for things like jobs, sports, or other extra-curricular activities. Once they have done that, they truly don't have the time to complete extra rigour even if faculty members step up and make their courses more rigorous.
The reality is that you have to change that campus mentality toward courses and course work and that is more important than how many courses students take or how many hours they meet. Apparently Southwestern is experiencing most of what I see as the worst possibilities of moving to a 4-4 and virtually none of the benefits. However, because they went down to a 3-2 teaching load, faculty see a benefit so it will be nearly impossible to switch back.
Adjuncts and Conclusions
One last odd problem that Southwestern has run into is that challenge in hiring adjunct faculty. That can be a challenging process in many departments when asking them to teach a three credit course. Asking them to teach a four hour course makes it harder. If they are teaching a course that only meets three hours, but is supposed to have a harder workload, it is very unlikely that they will require the desired level of effort.
The general conclusion from this faculty member was that she couldn't find anything good to say about the 4-4 student load at Southwestern. Only the negatives of the change have been manifest in the implementation. The same is almost true of the 3-2 teaching load with the minor exception that there is some small benefit to having the freedom of picking when the 2-course semester is done. However, in practice Trinity does not appear to be extremely strict about making certain every faculty member teaches three courses every semester so this is not really a practical benefit.
This weekend I got the chance to talk to someone who teaches at Southwestern University. They made the change from a 5-5 student load and 3-3 teaching load to 4-4 and 3-2 a few years ago. So this faculty member has direct experience with both of these systems. I wanted to record what we talked about and her perspective of that change here, because I felt that she had some very good insights.
Lack of Student Flexibility
The #1 problem that she described was something I hadn't even thought of, a lack of student flexibility in scheduling. In a 4-4 student load situation, students really need to take 4 courses each and every semester. The reason being that it isn't feasible for most students to go up to 5 courses when each one is four hours, and if you have more than one or two semesters with only three courses, you won't graduate on time.
All faculty know that occasionally students get in over their heads or sign up for courses they really aren't prepared to take. Under a 4-4 scheme, these students really can't drop those courses without pushing back their graduation. In the case where students choose to register for only 3 courses originally and take a light load, they have an even worse problem if it turns out that one course causes them problems because then dropping to two courses can cause problems related to full-time enrollment for the year. That leads to all types of financial difficulties for most students.
Under the category of lacking flexibility, Southwestern also runs into problems when it comes to transfer students and transfer credit. Given the challenges of enrolling students, transfers are potentially very important to many liberal arts schools. How do you count the 3-hour credits that most transfer students will come in with? Similarly, many Trinity students take summer courses away from Trinity and the same it true for Southwestern. We can't give students 4-hours of credit for a 3-hour summer course taken elsewhere. So we might check off a requirement for them, but they run into problems when it comes to total hours. Here again you can have students who fail to graduate on time because they don't have the right number of hours. With a 4-4 configuration you simply lose the flexibility for students to go slightly above the normal requirements to offset deficiencies.
Too Few Courses
Closely related to the problem of student enrollment flexibility is the problem of course offering flexibility. The faculty member I talked to noted that her department (a STEM department) was forced to reduce their major to 10 courses. This reduces the number of electives that students take as part of the major and how many electives can be offered. Not only are there fewer faculty teaching slots for electives, students don't take many so it is hard to get a critical mass of students to validate offering them.
An odd side effect of having majors cut down to 10 courses was that some departments bend the rule by hiding requirements in prerequisites. In particular she mentioned that the physics department, in order to get under the 10 course limit, doesn't explicitly list any math requirements. Instead, they have math courses as prerequisites on certain physics courses, making them implicit requirements. I know that Trinity highly frowns on implicit requirements, and the University Curriculum Committee typically rejects any such proposal. However, some fields truly do have a need to include more courses, especially when outside requirements are included.
Caps on majors or just the limits to courses could cause problems for things like theses as well. The CS department at Trinity does a 3-semester honors thesis track. There is no way we can do three semesters when students only take four courses each semester. This is definitely one of those situations where two, four hour courses are not even close to the equivalent of three, three hour courses.
The last problem presented by reduced course flexibility with the change made at Southwestern is in the inability to offer short seminars and the like on topics of interest. There modified system does not nicely support the equivalent of one and two hour seminars or independent studies. This can make it much harder to support undergraduate student research.
Courses Didn't Increase in Difficulty
The primary arguments for the 4-4 student load is that students are overburdened by having five courses each semester and that courses could be more rigorous if students only took four. I have always felt that this argument falls flat. Students spend a lot of time doing things outside of academics. If faculty members really want their students to dedicate more time to their course, they simply need to make the course harder and find ways to enforce that students really do the work. It might not be easy or even obvious how to do it, but that is what needs to be done. If faculty can't find ways to enforce students doing the work, moving to the 4-4 model isn't going to help.
Indeed, the Southwestern faculty member said that my fears match what has happened there. Few faculty have actually made their courses more rigorous. What is worse, because most of the courses went to 4-credits without going up to 4-hours, she feels that students are actually spending less time working on academics. Why? Because students now only have 12-hours in class. So when they look at their schedule they see even more "free time" and they tend to book it for things like jobs, sports, or other extra-curricular activities. Once they have done that, they truly don't have the time to complete extra rigour even if faculty members step up and make their courses more rigorous.
The reality is that you have to change that campus mentality toward courses and course work and that is more important than how many courses students take or how many hours they meet. Apparently Southwestern is experiencing most of what I see as the worst possibilities of moving to a 4-4 and virtually none of the benefits. However, because they went down to a 3-2 teaching load, faculty see a benefit so it will be nearly impossible to switch back.
Adjuncts and Conclusions
One last odd problem that Southwestern has run into is that challenge in hiring adjunct faculty. That can be a challenging process in many departments when asking them to teach a three credit course. Asking them to teach a four hour course makes it harder. If they are teaching a course that only meets three hours, but is supposed to have a harder workload, it is very unlikely that they will require the desired level of effort.
The general conclusion from this faculty member was that she couldn't find anything good to say about the 4-4 student load at Southwestern. Only the negatives of the change have been manifest in the implementation. The same is almost true of the 3-2 teaching load with the minor exception that there is some small benefit to having the freedom of picking when the 2-course semester is done. However, in practice Trinity does not appear to be extremely strict about making certain every faculty member teaches three courses every semester so this is not really a practical benefit.
Friday, October 19, 2012
Automated Exercises for Code Tracing
One of my goals for my courses is to have students do a lot more exercises. I am already putting up videos to go along with the book material, but students need to apply what they watch. (There also needs to be a way to enforce that they watch/read things.) For writing code this isn't all that complex other than having a tool to do it. I have been planning to write such a tool. There are other tools that do this as well, but I don't know of any that use Scala. I want to write something that is language independent so that I can use it across all of my classes.
I also value having students trace code. Of course, if I just show students code and ask them what it evaluates to/prints when they are in their room, they can simply type it in and run it. That doesn't benefit them or force them to work on the skills I want them to develop. This post describes an idea I had in regards to this. If anyone happens to read this who writes CS educational tools, feel free to take the idea and implement it yourself. Part of the motivation for this idea came from the stack trace display in DrRacket.
The basic idea is that you have students indicate where the flow of control goes and possibly have them enter values for variables along the way. The goal is to force them to actually demonstrate that they followed the code. That way, even if they do type it in and run it, they still have to think things through.
The figure below shows a mock-up of what the simplest interface might look like. The instructor would provide the code and the student has to click on each line in the order that control passes through them. Not only does this force the student to think through what is happening, it can be verified automatically so that students can be told if they have the proper trace without taking additional instructor time. That is essential if you actually want to use this to force students to do things in a real setting.
The figure above is a rather minimal example of tracing as it only follows the control flow. For some problems this might be sufficient for demonstrating that the student knows what is going on. However, for many problems it would be advantageous to have the student also track values attached to names/variables. The figure below shows a very ugly, first draft of what such an interface might look like. This is something that would need a lot more interface work, but the idea is to show the same control flow information, but also have tables that show the values of variables and how they have changed over time.
This would work for recursive functions too. As shown here, you have a table with a large column for each variable/name. You could also have separate tables for the different stack frames. I can also picture a more dynamic representation where there is only one row drawn for each stack frame, and the program records the values in that for each step in the computation. Then the student or someone else looking at what the student did could select an arrow to see the value of the stack frame at that time.
Does anyone know of tools like this? Would anyone be interested in having something like this? Personally, I really like to get students to trace through code and I can't do nearly enough of it. I feel that a tool like this could be very beneficial in the CS1 setting and even in some parts of CS2 and Data Structures.
I also value having students trace code. Of course, if I just show students code and ask them what it evaluates to/prints when they are in their room, they can simply type it in and run it. That doesn't benefit them or force them to work on the skills I want them to develop. This post describes an idea I had in regards to this. If anyone happens to read this who writes CS educational tools, feel free to take the idea and implement it yourself. Part of the motivation for this idea came from the stack trace display in DrRacket.
The basic idea is that you have students indicate where the flow of control goes and possibly have them enter values for variables along the way. The goal is to force them to actually demonstrate that they followed the code. That way, even if they do type it in and run it, they still have to think things through.
The figure below shows a mock-up of what the simplest interface might look like. The instructor would provide the code and the student has to click on each line in the order that control passes through them. Not only does this force the student to think through what is happening, it can be verified automatically so that students can be told if they have the proper trace without taking additional instructor time. That is essential if you actually want to use this to force students to do things in a real setting.
Does anyone know of tools like this? Would anyone be interested in having something like this? Personally, I really like to get students to trace through code and I can't do nearly enough of it. I feel that a tool like this could be very beneficial in the CS1 setting and even in some parts of CS2 and Data Structures.
Thursday, October 11, 2012
Where I See Us Heading
Right now everyone seems to be debating politics and talking about why the candidate they dislike is so bad. A few people are also talking about why their candidate is any good. So I thought I would step to the side and write a few words about my view of what the near future holds, and why I think both major candidates, and indeed both major parties in the US, are pretty much worthless.
Everything centers on the economy and both parties say they want to get people back to work. My take is that both will fail, and the efforts they put into it are not only futile, but likely a waste of resources. The reason, is that technology has made many jobs obsolete, and it is going to make many more jobs obsolete in the near future.
I know that the standard economist response to technological unemployment is simply to point to the Luddite fallacy and say that technology has only created more jobs all through history. I have another blog post in preparation where I will argue that this is one place where you can't just go off history. Pure logic can show you that the Luddite fallacy breaks down at some point. So the only question is, are we at that point?
I would also point out that I don't think there is anything fundamentally bad about there not being enough jobs. There is nothing about the human condition which I feel implies we have to work to have a good life, at least not the type of work we do for money in the modern world. If you enjoy gardening keep doing it. However, that doesn't mean that humans have to be employed in the manual labor of growing food when they don't feel like it. Similar logic applies to many other areas.
Today
I write this post just after the September 2012 unemployment figures have come out. The U3 unemployment level for the US sits at 7.8%. This is actually a pretty significant drop from previous months. Here you can see a plot from Google's public data explorer showing this.
This seems like good news. However, the U3 doesn't count lots of people. The U6 is a much better measure and it didn't budge in September 2012, holding steady at 14.7%
The population has been changing though. The plot below of data from the US Census Bureau shows that the population simply keeps rising even as the labor force has completely flat lined.
In many ways the US is doing better than other parts of the developed world. I heard a report on NPR yesterday that Spain is up to 25% unemployment with the rate being over 50% for those in their 20s. I don't know if the "unemployment" they measure is closer to our U3 or U6, or if it is something completely different. However, those are pretty staggering number.
There are lots of reasons why things are this way today and technological unemployment is not the primary cause, IMO. However, it is growing in importance, and it is why I don't expect things to turn around. In fact, they will get worse. I think it will also break some standard economic ideas. For example, "trickle down" effects disappear if you stop paying money to people lower on the wage scale because you automate away all of their jobs. This line of reasoning could go into a whole different direction looking at things like median wages and quartiles vs. corporate profits, but that isn't the objective of this post.
While politicians and their fans can point fingers all they want about why the labor force isn't growing, none of them wants to tell the real truth. This is because the truth is that they are impotent to change things. Businesses today don't really need many people. They especially don't need people with the skills that most people in this country have. Instead, they can get more done by investing in technology and augmenting the employees that they have. Sometimes, that technology can even replace those people. I'll point the interested reader to two articles here. My Google+ feed is full of such things, but these two should be sufficient for this post.
First, I point to a recent article in the NY Times, "When Job-Creation Engines Stop at Just One". The rhetoric I hear from the GOP is that if we reduce the barriers to starting companies, those companies will make new jobs. I will point to this article and disagree. Most start ups are forming today with a lot fewer employers than they used to. I see no reason to believe that removing barriers to forming businesses would change this. It simply costs less to use technology and contract workers than it does to hire full time employees.
The second article is a bit older. It is called "9.2% Unemployment? Blame Microsoft.". It appeared on the Forbes site in the second half of 2011. Clearly the U3 unemployment rate has fallen since then, but as you can see from the charts above, that isn't because the labor force has grown. There is one particular quote from this article that I want to highlight.
Before I start looking forward a bit, I want to point out one more link. There is to a page at the Bureau of Labor Statistics called "Employment and wages for the largest and smallest occupations". I will refer back to the top table on this page in my points below. Note that the top 15 occupations in the US account for 27% of the labor force. If you were to wipe out most of these, even if not a single other job category was hit, our unemployment rate would look at lot more like that of Spain. So now let's look at why I think that is going to happen, and how technology that will come out at different points in the future is going to drive it.
1-5 years
What should probably scare you the most about that list of largest jobs is that the #1 item on the list is already under heavy assault. Unfortunately, the BLS list is for May 2011 and they haven't come out with a newer one. When this came out, there were still people employed at Borders. Some might argue that you can't point to a single employer because companies go under all the time. However, no one should argue that online sales aren't hitting retail sales in a huge way. The plot below from WolframAlpha only goes to 2009, but you can see the trend there and that was even longer before a giant like Borders fell.
Retail isn't going to go away, but it is going to shrink. Amazon doesn't employ people to tell you about their new products or run the cash registers. Speaking of running cash registers, that is #2 on the list and that is in the same boat.
Of course, some people like to go handle things and try them on. One might argue that makes things like clothing retailers safe. The fact that they aren't is brought home by this article on virtual dressing rooms from CES 2012. Within 5 years I fully expect that many people will be ordering clothes that fit them perfectly well, in fact better than what they could have gotten in a store, using a system like this.
Another area that is going to undergo significant change in the next five years is health care. I don't know how much it will hit the ranks of RNs who come in at #5 on the employment list, but if personalized health leads to fewer in-person visits, that will lead to reduced hiring for people to see those patients. This is an area that is about ready to "take off" as described in "When will data-powered personalized health hit ‘escape velocity’?" This is only one piece in a big puzzle that is going to dramatically disrupt health care. Given how much money goes into health care, and how big a problem that is for the government and every other segment of business, disrupting health care in a way that brings down costs has to be a net benefit, even if it includes putting a good fraction of the 2% of the labor force out of their current positions.
Right above RNs is food prep. It is a common joke to say that students who finish college with a degree in something that doesn't translate to a job will be asking, "Do you want fries with that?" On a more practical side, fast food employs a significant number of people and it often is a safe type of job position or a last resort for many people, including those without the skills to go on to other jobs. Don't expect that to be available in 5 years based on what you can read in "Hamburgers, Coffee, Guitars, and Cars: A Report from Lemnos Labs". The founder of Momentum Machines didn't go to the PR training school where they tell you that you never say your product is going to put people out of work, no matter how true that statement is. As a result, we get this great quote.
Since I mentioned Amazon above, and because they are so disruptive for the current workforce, I will close out the 1-5 year future with more about what they are doing right now. They bought Kiva systems so that they can completely automate their warehouses. That is opening the door for them to take the huge step into giving customers instant gratification. In less than 5 years, Amazon will have a warehouse in the nearest major metro to your house, and they will use heavy automation to get your orders to you fast. Next day will be the slow option. That will be just one more reason that retail will be less needed, as will retail employees.
5-10 years
Moving out to the 5-10 year range, robots beyond Kiva systems are going to start making an impact. Janitors are smack in the middle of the top 15 list. They, and others doing similar jobs in human environments, have been safe from automation because the robots of the past has been fast and accurate in controlled environments. They don't play well with humans and they don't deal well with the unexpected situations that inevitably occur in environments that humans spend time in. Those limitations are going away.
If you don't believe me, go to YouTube and search for robots cooking or robots folding towels. A lot of what you will find is research work taking place with the PR2 robots from Willow Garage. PR2 is probably the most broadly used robot in this space, but there are quite a few others. What they all have in common is advanced vision and AI systems that allow them to handle the unexpected. They are also built to work well with humans, unlike the robots used in places like automotive plants which will do serious damage to any humans who accidentally get in the way of what the robots are doing.
I have to push this out to the 5-10 year range because right now it is a bit too expensive. No one is going to put a PR2 in their house at $400,000. However, the Baxter system, at $20,000, is an example of how companies are working to bring that down. (Note that the Baxter spokespeople have done "proper" training and they assure you in the article that no jobs will be lost to their robots. Their reasons have some merit, but that doesn't change the fact that in the longer run their robots will prevent humans from being hired to do things.) There are several efforts that are looking to get personal robotics off the ground and they have some serious money behind them.
Construction and manufacturing don't make the top 15 list, but they are definitely significant in the US economy. I actually expect that a lot of manufacturing will be re-shored and that construction should tick up as well. However, the manufacturing will be done without humans either by employing robots like Baxter or more advanced, articulate machines, or through additive manufacturing/3-D printing. That is another field that is on the verge of taking off with the output quality for devices that are in the $2000 range improving dramatically this year.
3-D printing is going to touch construction too. Want a new house? Instead of hiring people to build it, there are people who are working it make it so you can print your house.
10-15 years
Beyond 10 years things get harder to predict, but as all of these technologies mature, the ability for companies to do things without hiring humans is going to grow dramatically. There is one big technology that is slated to become widely available in about 8 years, that I fully expect to hit two more parts of the top 15 job list in just over 10 years: autonomous cars.
Hopefully everyone reading this knows about Google's autonomous cars, which have now driven over 300,000 miles. Both Nevada and California have passed laws to make it legal for these cars to operate when they become commercially available. Since all the major car makers are working on competing technology, it will become commercially available. GM has said autonomous driving will be standard equipment by 2020. I don't think it will be many years beyond that before the jobs that involve driving simply disappear.
What To Do?
So if you buy my arguments from above, the real unemployment level in the US is going up and there is nothing that politicians can really do to prevent it. So what should we do about this? First off, don't ignore it. That is what politicians are currently doing. It makes sense. After all, who is going to vote for the guy that says the jobs are gone, they aren't coming back and, by the way, more are going to be disappearing. No one wants to hear that, even if it is the truth. That's why it falls to people like me who aren't running for political office to say this.
The reality is that we shouldn't bury our heads in the sand though. We need to address this challenge and try to come up with ways that society can be organized where everyone can live a decent life even if large fractions of the population have no jobs because there is nothing productive for them to do in production of good. This is the search for a post-scarcity social organization. I don't claim to have a great answer to that, and even if I did, I wouldn't make this post longer by placing it here. However, I truly believe that the right way forward involves dramatic social changes and that the most important thing one can do to make that happen is to get people thinking about this.
So think about this some yourself, post comments to discuss with me and others, then share this with anyone you feel is willing to think about it. Oh yeah, then vote for some 3rd party candidate to protest the two big ones. :)
Everything centers on the economy and both parties say they want to get people back to work. My take is that both will fail, and the efforts they put into it are not only futile, but likely a waste of resources. The reason, is that technology has made many jobs obsolete, and it is going to make many more jobs obsolete in the near future.
I know that the standard economist response to technological unemployment is simply to point to the Luddite fallacy and say that technology has only created more jobs all through history. I have another blog post in preparation where I will argue that this is one place where you can't just go off history. Pure logic can show you that the Luddite fallacy breaks down at some point. So the only question is, are we at that point?
I would also point out that I don't think there is anything fundamentally bad about there not being enough jobs. There is nothing about the human condition which I feel implies we have to work to have a good life, at least not the type of work we do for money in the modern world. If you enjoy gardening keep doing it. However, that doesn't mean that humans have to be employed in the manual labor of growing food when they don't feel like it. Similar logic applies to many other areas.
Today
I write this post just after the September 2012 unemployment figures have come out. The U3 unemployment level for the US sits at 7.8%. This is actually a pretty significant drop from previous months. Here you can see a plot from Google's public data explorer showing this.
Probably the best measure of what is happening with jobs is just looking at the labor force. Both political parties are promising to "make more jobs". That is what has to happen if you want to really drop unemployment. You have to have more jobs created than what is needed to balance the increase in population. Here is what Google public data explorer can tell you about the number of people working in the US. This is a plot of the labor force. Note that it hasn't budged since 2008.
There are lots of reasons why things are this way today and technological unemployment is not the primary cause, IMO. However, it is growing in importance, and it is why I don't expect things to turn around. In fact, they will get worse. I think it will also break some standard economic ideas. For example, "trickle down" effects disappear if you stop paying money to people lower on the wage scale because you automate away all of their jobs. This line of reasoning could go into a whole different direction looking at things like median wages and quartiles vs. corporate profits, but that isn't the objective of this post.
While politicians and their fans can point fingers all they want about why the labor force isn't growing, none of them wants to tell the real truth. This is because the truth is that they are impotent to change things. Businesses today don't really need many people. They especially don't need people with the skills that most people in this country have. Instead, they can get more done by investing in technology and augmenting the employees that they have. Sometimes, that technology can even replace those people. I'll point the interested reader to two articles here. My Google+ feed is full of such things, but these two should be sufficient for this post.
First, I point to a recent article in the NY Times, "When Job-Creation Engines Stop at Just One". The rhetoric I hear from the GOP is that if we reduce the barriers to starting companies, those companies will make new jobs. I will point to this article and disagree. Most start ups are forming today with a lot fewer employers than they used to. I see no reason to believe that removing barriers to forming businesses would change this. It simply costs less to use technology and contract workers than it does to hire full time employees.
The second article is a bit older. It is called "9.2% Unemployment? Blame Microsoft.". It appeared on the Forbes site in the second half of 2011. Clearly the U3 unemployment rate has fallen since then, but as you can see from the charts above, that isn't because the labor force has grown. There is one particular quote from this article that I want to highlight.
So here’s the cold, hard truth about why you're unemployed: most businesses don't need you any more. We can do just as much, if not more, without you.This is the real reason why politicians can make all types of promises about employment and jobs, but none of their promises will come true. There is no policy change that can make people more useful to businesses overnight. The closest you can come to that would be to make people more useful per dollar spent. That would be policies like removing forced benefits and dropping minimum wage so that employers can pay human employees significantly less than they do today. This is something the GOP would actually be willing to do. Even that is a stop-gap measure that will prove ineffective in the longer term. Education is far more beneficial if you go beyond a few years, but even that has limited effectiveness in the long run. Education is something the Democrats will support. Neither is going to push for the changes I think we need to deal with what I'm going to describe below.
Before I start looking forward a bit, I want to point out one more link. There is to a page at the Bureau of Labor Statistics called "Employment and wages for the largest and smallest occupations". I will refer back to the top table on this page in my points below. Note that the top 15 occupations in the US account for 27% of the labor force. If you were to wipe out most of these, even if not a single other job category was hit, our unemployment rate would look at lot more like that of Spain. So now let's look at why I think that is going to happen, and how technology that will come out at different points in the future is going to drive it.
1-5 years
What should probably scare you the most about that list of largest jobs is that the #1 item on the list is already under heavy assault. Unfortunately, the BLS list is for May 2011 and they haven't come out with a newer one. When this came out, there were still people employed at Borders. Some might argue that you can't point to a single employer because companies go under all the time. However, no one should argue that online sales aren't hitting retail sales in a huge way. The plot below from WolframAlpha only goes to 2009, but you can see the trend there and that was even longer before a giant like Borders fell.
Of course, some people like to go handle things and try them on. One might argue that makes things like clothing retailers safe. The fact that they aren't is brought home by this article on virtual dressing rooms from CES 2012. Within 5 years I fully expect that many people will be ordering clothes that fit them perfectly well, in fact better than what they could have gotten in a store, using a system like this.
Another area that is going to undergo significant change in the next five years is health care. I don't know how much it will hit the ranks of RNs who come in at #5 on the employment list, but if personalized health leads to fewer in-person visits, that will lead to reduced hiring for people to see those patients. This is an area that is about ready to "take off" as described in "When will data-powered personalized health hit ‘escape velocity’?" This is only one piece in a big puzzle that is going to dramatically disrupt health care. Given how much money goes into health care, and how big a problem that is for the government and every other segment of business, disrupting health care in a way that brings down costs has to be a net benefit, even if it includes putting a good fraction of the 2% of the labor force out of their current positions.
Right above RNs is food prep. It is a common joke to say that students who finish college with a degree in something that doesn't translate to a job will be asking, "Do you want fries with that?" On a more practical side, fast food employs a significant number of people and it often is a safe type of job position or a last resort for many people, including those without the skills to go on to other jobs. Don't expect that to be available in 5 years based on what you can read in "Hamburgers, Coffee, Guitars, and Cars: A Report from Lemnos Labs". The founder of Momentum Machines didn't go to the PR training school where they tell you that you never say your product is going to put people out of work, no matter how true that statement is. As a result, we get this great quote.
“Our device isn't meant to make employees more efficient,” said co-founder Alexandros Vardakostas. “It's meant to completely obviate them.”(I discovered this article through Martin Ford's blog post, "Fast Food Robotics - An Update".)
Since I mentioned Amazon above, and because they are so disruptive for the current workforce, I will close out the 1-5 year future with more about what they are doing right now. They bought Kiva systems so that they can completely automate their warehouses. That is opening the door for them to take the huge step into giving customers instant gratification. In less than 5 years, Amazon will have a warehouse in the nearest major metro to your house, and they will use heavy automation to get your orders to you fast. Next day will be the slow option. That will be just one more reason that retail will be less needed, as will retail employees.
5-10 years
Moving out to the 5-10 year range, robots beyond Kiva systems are going to start making an impact. Janitors are smack in the middle of the top 15 list. They, and others doing similar jobs in human environments, have been safe from automation because the robots of the past has been fast and accurate in controlled environments. They don't play well with humans and they don't deal well with the unexpected situations that inevitably occur in environments that humans spend time in. Those limitations are going away.
If you don't believe me, go to YouTube and search for robots cooking or robots folding towels. A lot of what you will find is research work taking place with the PR2 robots from Willow Garage. PR2 is probably the most broadly used robot in this space, but there are quite a few others. What they all have in common is advanced vision and AI systems that allow them to handle the unexpected. They are also built to work well with humans, unlike the robots used in places like automotive plants which will do serious damage to any humans who accidentally get in the way of what the robots are doing.
I have to push this out to the 5-10 year range because right now it is a bit too expensive. No one is going to put a PR2 in their house at $400,000. However, the Baxter system, at $20,000, is an example of how companies are working to bring that down. (Note that the Baxter spokespeople have done "proper" training and they assure you in the article that no jobs will be lost to their robots. Their reasons have some merit, but that doesn't change the fact that in the longer run their robots will prevent humans from being hired to do things.) There are several efforts that are looking to get personal robotics off the ground and they have some serious money behind them.
Construction and manufacturing don't make the top 15 list, but they are definitely significant in the US economy. I actually expect that a lot of manufacturing will be re-shored and that construction should tick up as well. However, the manufacturing will be done without humans either by employing robots like Baxter or more advanced, articulate machines, or through additive manufacturing/3-D printing. That is another field that is on the verge of taking off with the output quality for devices that are in the $2000 range improving dramatically this year.
3-D printing is going to touch construction too. Want a new house? Instead of hiring people to build it, there are people who are working it make it so you can print your house.
10-15 years
Beyond 10 years things get harder to predict, but as all of these technologies mature, the ability for companies to do things without hiring humans is going to grow dramatically. There is one big technology that is slated to become widely available in about 8 years, that I fully expect to hit two more parts of the top 15 job list in just over 10 years: autonomous cars.
Hopefully everyone reading this knows about Google's autonomous cars, which have now driven over 300,000 miles. Both Nevada and California have passed laws to make it legal for these cars to operate when they become commercially available. Since all the major car makers are working on competing technology, it will become commercially available. GM has said autonomous driving will be standard equipment by 2020. I don't think it will be many years beyond that before the jobs that involve driving simply disappear.
What To Do?
So if you buy my arguments from above, the real unemployment level in the US is going up and there is nothing that politicians can really do to prevent it. So what should we do about this? First off, don't ignore it. That is what politicians are currently doing. It makes sense. After all, who is going to vote for the guy that says the jobs are gone, they aren't coming back and, by the way, more are going to be disappearing. No one wants to hear that, even if it is the truth. That's why it falls to people like me who aren't running for political office to say this.
The reality is that we shouldn't bury our heads in the sand though. We need to address this challenge and try to come up with ways that society can be organized where everyone can live a decent life even if large fractions of the population have no jobs because there is nothing productive for them to do in production of good. This is the search for a post-scarcity social organization. I don't claim to have a great answer to that, and even if I did, I wouldn't make this post longer by placing it here. However, I truly believe that the right way forward involves dramatic social changes and that the most important thing one can do to make that happen is to get people thinking about this.
So think about this some yourself, post comments to discuss with me and others, then share this with anyone you feel is willing to think about it. Oh yeah, then vote for some 3rd party candidate to protest the two big ones. :)
Saturday, September 29, 2012
Appropriate use of "free time"
I am a big fan of the expression "Race against the machines". I really do think that is where the job market is these days. However, we are still in the early parts of the exponential explosion. For that reason, many people don't see this as a race against the machines. Instead, it is more of a race against other humans. That isn't simply perception, it is also largely reality. People can race against the machines for a while, but that isn't a race you can really win. The machines improve exponentially. By the time you realize you need to start running, there isn't much time left before you will be completely left in the dust. Instead, humans are racing one another for those positions that the machines can't yet do. They are also racing to positions that are currently machine free in the situations where their previous knowledge/capability falls behind in another area.
One of the other things that I have been watching a fair bit recently is the free availability of educational resources online. Just a few years ago, there really weren't all that many options when it came to learning most topics. You could buy a book or spend a lot of money looking for someone to teach you using traditional educational approaches. Some resources were available online, but they were expensive. Just between Khan Academy and the MOOC (Massive, Open, Online Course) sites (Coursera, Udacity, and edX), there is now a wealth of educational material present at a variety of levels.
So what does this have to do with free time? Well, if you spend all of your free time doing activities that don't lead to self-improvement, you are losing the race against other humans and the machines. I feel like the US society has a mentality that chases entertainment and leisure. There is no doubt that people need to have both entertainment and leisure in order to maintain good mental balance, but I worry that most of the entertainment and leisure that people strive for does nothing to improve them and that we have made entertainment the ultimate goal of our life quests. If you want to have any chance of not losing the race, you need to strive to improve yourself. More broadly, I think that we need to work on being a society that seeks for self-improvement, both physically and mentally instead of one that seeks entertainment. I mentioned some mental self-improvement options above. On the physical side, Fitocracy is one of many tools you can use to help motivate you to do exercise and in the end the goal is to turn self-improvement into something more akin to entertainment.
So who should be doing this? I would argue everyone. In my first blog post looking at the course I am taking at Coursera, I argued that MOOCs are great for continuing education for educators and many others. Students who want to go into more depth in a field or who find that they lack certain background in something should definitely consider turning to either MOOCs or resources like Khan academy.
These example groups skirt around the reality of the modern US and the fact there there is a large population of people who are currently losing the race and who really need to focus on self-improvement. That is the population of unemployed and underemployed. As it stands, the labor force in the US is below 50% of the population and dropping. That puts it back in the range it was in during the 1980s when many women still hadn't moved into the workforce.
In most segments of the economy, this means that there is very stiff competition for what positions there are available. That in turn means employers can be selective and only take the top people. If you aren't one of the top people right now, there is only one way to get there, that is through self-improvement. In the past this might have been some type of cruel catch-22 because self-improvement always came at a monetary cost, whether it was tuition for school or just the price of books you might buy. Today, those barriers are gone. Anyone can sign up for a MOOC for free and listen to lectures from some of the worlds greatest minds. If they start going over your head at some point during the semester, that's fine, you aren't contractually bound to go through the whole course and you lose nothing if you stop at the point where it becomes too difficult. Now the only cost is the cost of the time you take in doing these things.
Unfortunately, what I think is happening more often in our society is that instead of going for the challenge of self-improvement, people go for the mindless joy of Facebook games (and other similar substitutes). I can understand the temptation. I got sucked into a mindless FB game once and I have to admit that I am occasionally tempted to do such things when I feel overwhelmed. (The only reason I don't is because playing them wouldn't make me any less overwhelmed. In fact, while I was playing odds are good that more stuff would have gotten piled on top of me.) The thing is that FB games, pretty much uniformly, do not improve you or get you ahead in the race. As a result, the time that a person spends playing them causes that person to fall further behind those he/she is racing against.
Now let's be honest, I do believe that automation is causing technological unemployment, and in the long term, the nature of jobs and their role in our society needs to fundamentally change. The cynical side of me says that in the future we might actually want a large fraction of the population playing mindless FB games so that they stay happy and don't riot. However, we aren't to that point yet and we still live in a world where people need to work if they want to live a decent life. As such, everyone should be doing as much as they can to improve themselves to move ahead in the race. Everyone should take advantage of the new opportunities for education and other forms of self-improvement. They should do so first with the goal of making their own lives better. People that I know who are in the position of hiring others love seeing self-motivation and self-improvement. They will ask you in an interview what you have been doing with your down time, and if the answer isn't something they see as useful it is a major black mark on your application. However, I think there is a second reason.
When I look forward toward a world where most people don't need to work for things to run smoothly to run, I see a world where we need people to be motivated by different things. If everyone seeks mindless entertainment and leisure, I don't think think society will work well. It either falls apart completely, or we get a Wall-E type of scenario. However, if we can change our societal norms to become a society that strives for self-improvement, I think that a world where we don't need jobs works very well. People in such a society will continue to work to educate themselves. They will put effort into creative endeavors. Inevitably, they should still enjoy the occasional period of rest or activities that are purely for entertainment and leisure, but we need those activities to be what people do occasionally to recharge and to gain motivation for the next step in their life path, not to be the ultimate goal that people work towards.
One of the other things that I have been watching a fair bit recently is the free availability of educational resources online. Just a few years ago, there really weren't all that many options when it came to learning most topics. You could buy a book or spend a lot of money looking for someone to teach you using traditional educational approaches. Some resources were available online, but they were expensive. Just between Khan Academy and the MOOC (Massive, Open, Online Course) sites (Coursera, Udacity, and edX), there is now a wealth of educational material present at a variety of levels.
So what does this have to do with free time? Well, if you spend all of your free time doing activities that don't lead to self-improvement, you are losing the race against other humans and the machines. I feel like the US society has a mentality that chases entertainment and leisure. There is no doubt that people need to have both entertainment and leisure in order to maintain good mental balance, but I worry that most of the entertainment and leisure that people strive for does nothing to improve them and that we have made entertainment the ultimate goal of our life quests. If you want to have any chance of not losing the race, you need to strive to improve yourself. More broadly, I think that we need to work on being a society that seeks for self-improvement, both physically and mentally instead of one that seeks entertainment. I mentioned some mental self-improvement options above. On the physical side, Fitocracy is one of many tools you can use to help motivate you to do exercise and in the end the goal is to turn self-improvement into something more akin to entertainment.
So who should be doing this? I would argue everyone. In my first blog post looking at the course I am taking at Coursera, I argued that MOOCs are great for continuing education for educators and many others. Students who want to go into more depth in a field or who find that they lack certain background in something should definitely consider turning to either MOOCs or resources like Khan academy.
These example groups skirt around the reality of the modern US and the fact there there is a large population of people who are currently losing the race and who really need to focus on self-improvement. That is the population of unemployed and underemployed. As it stands, the labor force in the US is below 50% of the population and dropping. That puts it back in the range it was in during the 1980s when many women still hadn't moved into the workforce.
Unfortunately, what I think is happening more often in our society is that instead of going for the challenge of self-improvement, people go for the mindless joy of Facebook games (and other similar substitutes). I can understand the temptation. I got sucked into a mindless FB game once and I have to admit that I am occasionally tempted to do such things when I feel overwhelmed. (The only reason I don't is because playing them wouldn't make me any less overwhelmed. In fact, while I was playing odds are good that more stuff would have gotten piled on top of me.) The thing is that FB games, pretty much uniformly, do not improve you or get you ahead in the race. As a result, the time that a person spends playing them causes that person to fall further behind those he/she is racing against.
Now let's be honest, I do believe that automation is causing technological unemployment, and in the long term, the nature of jobs and their role in our society needs to fundamentally change. The cynical side of me says that in the future we might actually want a large fraction of the population playing mindless FB games so that they stay happy and don't riot. However, we aren't to that point yet and we still live in a world where people need to work if they want to live a decent life. As such, everyone should be doing as much as they can to improve themselves to move ahead in the race. Everyone should take advantage of the new opportunities for education and other forms of self-improvement. They should do so first with the goal of making their own lives better. People that I know who are in the position of hiring others love seeing self-motivation and self-improvement. They will ask you in an interview what you have been doing with your down time, and if the answer isn't something they see as useful it is a major black mark on your application. However, I think there is a second reason.
When I look forward toward a world where most people don't need to work for things to run smoothly to run, I see a world where we need people to be motivated by different things. If everyone seeks mindless entertainment and leisure, I don't think think society will work well. It either falls apart completely, or we get a Wall-E type of scenario. However, if we can change our societal norms to become a society that strives for self-improvement, I think that a world where we don't need jobs works very well. People in such a society will continue to work to educate themselves. They will put effort into creative endeavors. Inevitably, they should still enjoy the occasional period of rest or activities that are purely for entertainment and leisure, but we need those activities to be what people do occasionally to recharge and to gain motivation for the next step in their life path, not to be the ultimate goal that people work towards.
Subscribe to:
Posts (Atom)