Sunday, August 20, 2017

Want to make America Great? Go learn something new.

"Make America Great Again" is a slogan that all Americans are inevitably familiar with these days. Whether they love hearing it, or roll their eyes at it, everyone has heard it many time. While I am not opposed to the argument that America is currently great, the reality is that it doesn't feel very great for a lot of people because of the way our economy has changed. The US stock market has actually been on a very nice climb since around 2009. Nothing really special has been happening in 2017 other than perhaps we have a President who likes to claim credit for everything positive, even if he didn't have anything to do with it. The problem is that the gains from those stocks and the corporate profits that have driven them there aren't trickling down to a large fraction of the population.

Unfortunately, the "plan" from the Trump administration on how to fix things in America was largely to try to roll us back to the 1950s. He promised to bring back jobs in things like coal mining and manufacturing. Of course, those promises are all empty. You can't just roll back the clock and proclaim that the economy is going to work the way that it did decades ago. Things have changed. Technology has changed them.

There are a variety of reasons that lots of people feel left behind in the current US economy, but the one that really stands out to me is the skills gap. Technology has changed the economy, but not enough people have stayed up with technology to keep themselves relevant in the current job market. The manufacturing sector is an area often sited for people losing jobs, but the reality is that US manufacturing output is at an all time high. The thing is, the factories don't need as many workers, and the worker that they need are ones with higher level skills. This was highlighted in the article "In US, factory jobs are high-tech, but the workers are not".

As that article mentions, what US employees need is training, but US employers aren't prone to pay for it. Neither is the Republican leadership we currently have running most states as well as the federal government. There are some serious challenges with training everyone as well. A number of these were outlined in "Technology is setting us up for a training crisis".

The thing is, we really have two problems. The one we are feeling right now is that labor is losing out to capital in terms of employer spending, so money isn't trickling down, even when stock prices and company profits are really high. The other problem is that the lack of skilled people is setting up a really bad dynamic where the top companies become superstars. This problem was highlighted in the MIT Technology Review article, "It Pays to Be Smart". That article had the much more informative subtitle of , "Superstar companies are dominating the economy by exploiting a growing gap in digital competencies".

This article addresses the fact that for a number of years now, economists have worried about a stall in the growth of productivity. This is significant, productivity growth has been the key driver of economic growth and the increase in societal wealth for about as long as humans have had organized civilization. This slowing of productivity has also seemed very counter intuitive because during this time digital technology has grown by leaps and bounds and has changed our lives in many very fundamental ways. What "It Pays to Be Smart" discusses is a different analysis that looks only at the top performing companies in each segment of the economy. It turns out that the top companies still have great productivity growth. All the other companies are lagging behind though, and putting a drag on the economy. Why is that? Well, the top companies are doing a really good job of harnessing digital technology, while everyone else struggles to do so in an effective way.

Why is it that the top companies can use technology efficiently while most other companies struggle? To my mind the answer is once again the skills gap. In this case, a shortage of people with sufficient skills to really harness technology in a way that improves productivity. To put it another way, the small companies suck at technology because there are only so many people with the skills, and the big companies have the money to attract or buy out that small population of people. There are certainly other factors like the growth in the importance of data and the fact that large pools of data are worth a lot more than small pools of data (some of these are discussed by the Economist in "The world's most valuable resource is no longer oil, but data"), but it seems to me that the real challenge for small businesses that want to succeed is that they have to somehow attract and retain good people with high end skills who can bring the efficiencies of modern digital technology to bear. This would be a lot easier to do if there were more such people.

Based on this reasoning, the fundamental problem holding back our economy today isn't anything related to foreign competition or immigrants, it is a lack of people with the right skills to drive the economy forward. So if you believe that America has lost its greatness, and you want to actually do something the make America great again, go out and learn something new. Thanks to the multitude of online learning sites (see links below for some), it has never been easier or cheaper to pick up new skills. It does take effort, but if you really want to improve America, sitting around complaining about the loss of the jobs of old isn't going to do it. Putting in the effort to learn new skills yourself is the activity that I truly believe will have the greatest long term benefit, and unlike many other things, your learning is something you have control over.

Online Learning Sites

I'm listing some of the big ones here. If you think I missed one, let me know in the comments and I'll add it.

Sunday, June 18, 2017

How will technological unemployment impact birthrate?

This blog post is a short exploration of a thought that hit me about how birth rates might be impacted by the changes in employment caused by future technological change. I consider technological unemployment to be a significant issue as AI and robotics continue to become more capable. I'm not going to support that belief in this post, as that is many posts worth of material. I will assume that to be the case here, and briefly explore one impact of it that I hadn't thought of until recently.

Birth rates have been decreasing in the developed world so much that in many countries, the death rate is larger than the birth rate. According to the CIA World Fact book, there were 26 nations in 2014 for which this was the case. Japan (-1.8 net) and Germany (-3.1 net) are the ones most commonly mentioned, but there are many others. A nice treatment of this can be found at https://ourworldindata.org/fertility/. There are a variety of reasons for this. Historically, the first was probably lower infant mortality rates. The more significant ones to me though are the ones that center on the increase in women's rights. As women gain more control over their their reproductive rights, they generally choose to have fewer children. In developed nations we are also seeing a decline in marriage rates and more women entering the workforce. Both of these tend to further reduce the birth rate. It is the last factor mentioned that I want to focus on, as it is the one most related to technological unemployment.

It is worth taking a few sentences to explain why these thoughts popped into my head. I often debate the future impacts of technology with a friend who we will just call by his last name, McLane. He doesn't believe that technological unemployment will be an issue, and one of the many reasons he has sited is articles about there not being enough people to work in countries that have low population growth. He is correct that there are a number of countries providing incentives for people to have kids because they are worried about not having enough people to care for the elderly and keep the economy moving forward. Given the variety of reasons for lower fertility rates, I can indeed imagine a day when the global population goes into a tailspin simply because there are no kids born.

So let's assume that McLane is wrong and we get technological unemployment and have to do something, possibly a basic income, so that a large fraction of the population can live a life with dignity is a world where they can't do anything to earn a living wage because everything they have the skills to do can be done better and cheaper by machines. What happens to the birth rate at that point?

I'm going to by optimistic and assume that in such a world, women have at least equal power in society relative to men. I also assume that women maintain general control over their reproductive rights, and generally have all the freedoms of men. (I can actually imagine women having more power at that time as there are indications that technological unemployment so far has hit men harder than women, so if more women have jobs than men for any period of time, they could be the gender with greater social power.) Still, you get a different dynamic when people aren't chasing careers. Right now both men and women often put much of their personal lives on hold until they are well established in a career. If we have a social structure where machines are doing so much of the work that people live comfortable lives without having careers, that changes.

One of the standard arguments that I hear "against" technological employment is that people get lots of meaning from their jobs. I use quotes, because that isn't really an argument for why it won't happen, just something that could cause problems if it does. Personally, I think that people can find meaning in lots of other things, like hobbies or families and friends, if they don't need to work to live a comfortable life. Let's be honest, lots of people today have rather meaningless jobs and they already get much more of the meaning for their lives from other activities.

Given that, I can see a scenario where the birth rate goes up, and people decide to spend a lot more of their time raising children and focusing on family life. Of course, I can also imagine a world where everyone is just watching things on their screens, they have little physical contact with other humans, and the birth rate continues to sink. I'm wondering what other people think? If technology takes away the need to chase a career, will birth rates in the developed world start to rise again, or will we be so far into a situation where we enjoy our technology that the idea of having kids and a family will continue to decline?

Wednesday, April 19, 2017

Does Big Data Make Centralized Control the Better Option?

I grew up in the waning years of the Cold War. I remember doing a project in High School on the possibility and repercussions of nuclear war. I got to see the fall of the Berlin Wall and many other events that went along with the crumbling of the world's other superpower as their system of government, communism, proved to be inferior to the combination of democracy and free market capitalism.

One of the standard arguments I recall for why communism failed was that it was inefficient and wasteful, particularly when it came to the allocation of resources. A standard example was that local farmers knew better when to plant and harvest than the central government, but that in the USSR they had to do those things when the federal government told them to. By contrast, in the US, farmers would make decisions based on their experience to try to maximize their yields. They did that because that was how they made money. A year of bad yields would be an economic hardship in the best case and possibly lead to losing the farm.

A thought that struck me recently is that this particular argument against communism might not apply anymore. The reason is that we have moved into the era of "big data". In fact, I wonder if centralized control might not have the advantage these days, assuming that centralized control does things in an optimal way.

To illustrate this, consider the farming example I just gave. The local farmers have a lot of experience they can fall back on, but these days I'm guessing that even better decisions for allocating resources could be made using a data set that included crop yields across an entire nation for many years, combined with detailed weather data during that same time and other potentially relevant information.

Consider this article about a company that uses predictive AI to place orders in advance to improve shipping efficiency and reduce the number of returns. It is just one of many examples of how computer systems can now make decisions much better than humans are capable of because they can pull in much larger quantities of data.

Of course, the key here isn't centralized control, and I'm sure that many people would argue that centralized control still fails because it lacks the motivation to do well that individuals have. In that regard, this isn't a call to switch to communism. Instead, the key here is the data, and this is an area where government can play a role. Even better than one big centralized decision making process is a system where everyone has access to all the relevant data, and they can all try out various ways to process it to make optimal decisions.

In that regard, I think that government could play a role by making data available and helping different groups to make their data accessible and consistently formatted so that it can be more broadly used. This doesn't just apply to crops with data on weather, planting dates, harvesting dates, and yields by location, this could also be useful for a lot of data related to health including air and water quality and potentially consumption habits. I don't have a full mental image of what exactly this looks like in a broad sense, and I can clearly see challenges related to privacy issues. Still, I feel that we need to push for making more data generally available so that individuals and companies can utilize it to make better decisions. Regardless of where the control comes from, the way forward for efficient decision making is clearly availability of useful data.

Wednesday, January 4, 2017

Performance of N-Body Simulation Styles in Scala

I spend a lot of time doing N-body simulations. My working simulation code is written in C++. The reason for that is simple, speed. It is a serious number crunching code and even written in C++ runs can take months to complete. I would love to do these simulations in some other language that I enjoyed writing code in more, but I'm not willing to give up all that much speed in order to make that happen. A number of years ago a student and I explored the performance of a Java version of the code, and we found that we could get performance within 10-20% of the C++ version, but doing so required doing a number of things in a way that made the Java almost as challenging to work with as the C++. These days, I'd really love to be able to convert my code to Scala, as I find it much more enjoyable to code in, but I'd have to be certain that I can get performance close that what I'm getting in C++. My guess is that in the end I'll wind up updating my C++ code to use features of C++11, 14, and 17, but I still like to explore the possibilities of using a different language.

There are lots of ways that you can write N-body simulations in Scala. Some of them take advantage of the expressivity of the language to produce shorter code that is easier to read and generally less bug prone. However, there are reasons to believe that those approaches might also tend to be slower. For a while I've been meaning to explore different approaches to see how much of an impact it really has on the speed of execution. While there aren't all that many people who do N-body simulations, my expectation is that the results of these performance tests will apply to many other numerically intensive applications.

Array of Mutable Classes

In C++, one represents particles with a struct and makes sequences of these using arrays, vectors, or valarrays. Everything is generally mutable. This first version of the code in Scala is written to mirror the C++ code. Of course, there is a huge difference in the memory model. When you make a sequence of a simple struct in C++, the entire memory for all elements is laid out in a single contiguous block. The way this code is written in Scala, the array is an array of references to instances of MutableBody and each of those contains two references to instances of MVect3. This matters because caching is vitally important on modern computers and contiguous memory accesses have much better caching performance. In this simple example, odds are that because all of the objects are allocated in order at the beginning of the program, they wind up being contiguous in the heap, but there are still multiple levels of references that have to be followed, which are likely to impose some performance cost.

This code really does look similar to what the C++ code for this type of thing looks like, unless you happen to have a library set up for SIMD vector operations or expression templates for the 3-D vectors. Unfortunately, that isn't a good thing. Not only is this code verbose, I can speak from experience that it is bug prone. It is just too easy to have one of those x, y, or z fields typed wrong and the resulting error is very difficult to diagnose and track down.

Array of Immutable Classes

Given the capabilities of Scala, a much "better" way to write this is to use immutable classes and add symbolic methods to a 3-D vector class so that you can simplify the math expressions and make them less error prone. Of course, this change means that you are doing a lot of memory allocation making small objects for all of the 3-D vectors.

Clearly this code is shorter and more readable. It is still using the same basic approach with a mutable array for storage, but the ability to use mathematical operators on vectors improves the code significantly. It is worth noting that in the mutable version you can make operations like +=, but given the math that is being done, they aren't all that useful unless you are making temporary values to store scaled versions and such.

Arrays with Value Class Wrapper

Now we are going to get a bit more interesting and produce a version that tries to give us contiguous memory without hurting program readability and without making lots of temporaries. Basically, this is an attempt to make a version of the code that has the best chance of being speed competitive with the C++ code while still being reasonably readable. The real key areas of this code are lines 5-29 where we make a number of arrays of doubles and then define a value class. The value class is a newer feature of the Scala language that is stack allocated and has no more overhead than a primitive, while allowing you to have nice OO syntax. Instances of it are stored as just primitives on the stack and the methods wind up being static methods in the JVM, so there is no memory or speed overhead of object allocation. In order to use a value class, we have to put the whole thing in an object declaration and not a class declaration. Value classes can't go inside of other classes because then they would be non-static inner classes and that would mean they have the overhead of keeping a reference to their enclosing instance. They could go at the top level, outside of all other declarations, but then they wouldn't have access to the arrays. Putting both the arrays and the value class in an object declaration gives us what we need here.

Unfortunately, value classes are limited in Scala because they aren't really supported by the JVM at this time. The main limitation is that they can only have a single primitive value in them. There are plans to add value types to Java and the JVM. Once that happens, which probably will be Java 10, then you could make classes with three doubles in them that are stack allocated and arrays of them would be memory contiguous, giving you behavior much more similar to C++. Until that time, code like that above can give you a reasonable approximation.

Purely Functional Approaches

Of course, the real key to Scala is the functional aspects, something that none of the above examples took advantage of. The version that used immutable classes still mutated an array that stored references to those objects. I tried two different approaches to doing things functionally that are shown in the code below. The first approach is pretty much just like our immutable class version, but it uses a Vector and the updated method. Even without speed tests, we can feel confident that this is not an efficient approach, but it is pretty much required if we want to do the minimum number of distance calculations where each pair is only visited once. The second version, called forSim2, does twice as many distance calculations, but this allows it to be written in a much more succinct manner because we produce a full sequence with the accelerations to each particle without every mutating them. As it happens, this is the approach that we need to take if we parallelize the code to prevent race conditions. I will explore that more in a future blog post.

This code uses the ImmutableBody and the immutable Vect3 class shown earlier. That makes the code more compact than the non-mutable versions. Those unfamiliar with fold methods might find forSim2 difficult to read, but in general a fold can be used to replace a loop that would accumulate a value in a mutable way. Note that like the previous code, this is put in an object, but for completely different reasons. This code isn't stateful, so there is no reason to have a class. The state is created by the initBodies method and then passed into the forSim methods, which return modified values. This code never stores the state locally. This is a significant difference from the earlier code and should probably be expected for a functional approach.

Timing Results

So how do these different versions perform? We tested all of them using Scala 2.12.1 both with no optimization flags and with -opt:l:classpath -opt:closure-invocations -opt:simplify-jumps -opt:copy-propagation -opt:redundant-casts -opt:box-unbox -opt:nullness-tracking -opt:inline-global. The timing results are shown in the following table.

OptimizationStyleAverage Time [s]rms [s]
NoneValue Class0.7040.004
Mutable Class0.9040.006
Immutable Class1.2660.020
Functional 18.330.09
Functional 22.3430.048
OptimizedValue Class0.6790.001
Mutable Class0.6820.001
Immutable Class1.1910.026
Functional 18.550.11
Functional 22.3250.023

It is good to see that in many ways, my intuitions as to which versions would be fastest proved correct. Without optimization, the version that uses a value class and contiguous blocks of memory in arrays is clearly the fastest, followed by the mutable classes, then the immutable classes, then the purely functional approaches at the end. Note that the second functional approach takes nearly twice as long as the immutable class version. Since it is doing twice as many distance calculations, this indicates that the overhead of being functional is small and the speed difference is basically due to reorganizing the math. I will note again that this reorganization is also required for parallelization, so I would speculate that in parallelized versions, the second functional approach will be comparable to the immutable class version. The first functional approach is a clear loser. Calling updated so frequently on the Vector type is clearly inefficient. I will note though that I tried changing Vector to Array, and the resulting code was so slow for the first functional approach that I never saw it complete. On the other hand, using an array, even if it isn't mutated, seemed to slightly improve performance for the second functional approach.

It is also interesting to note that optimization benefited the mutable class version significantly, making it nearly comparable to the value class version.

Comparison to C++

We will close out this post with a comparison to C++. After all, we can't know if it is reasonable to use Scala for numerical workloads if we don't explore how it compares. I wrote a version of the code in C++ that was basically an adaptation of the mutable class version using a std::valarray for storage. I compiled it using g++ with the -Ofast compiler flag. I also made a separate application in Scala that only ran the value class version and had both the C++ and the Scala time 1000 steps of integration. The result was that the C++ generally completed in 6.22 seconds and the Scala completed in 6.88. Both were consistent across multiple runs. So the final result is a 10% advantage for C++. That isn't a huge advantage, and my guess is that most applications would be happy to live with that for the advantages that they get in coding in Scala over C++ in terms of developer productivity. Granted, that was with g++ I expect that using the Intel compiler or some other highly optimizing, and not free or open source, compiler will produce even faster executables for C++.

For me, the bottom line is that if you hear people say that Scala (or Java/JVM) are slow, there are good odds that they haven't really tested it compared to other languages/platforms. For straight number crunching applications, feel free to point them to this blog post to show them that the difference in speed really doesn't have to be all that large. My guess is that using macros, it would also be possible to create Scala code that has the readability of the immutable Vect3 class with symbolic methods, but without the speed cost for allocating lots of temporary values. This would be akin to expression templates for that purpose in C++. Maybe I'll take a stab at creating that code and write a blog about it as well. I also look forward to the Java 10 JVM adding value types, as I believe that they have the potential to significantly improve the performance of numerical code in all JVM languages.

Other Code

Here is the code for C++ and the timing in Scala in case anyone wants to be able to run everything.

C++ Code

Scala Timing

Sunday, January 1, 2017

Performance of Scala for Loops

One of the interesting features of the Scala programming language is that the for loop is basically syntactic sugar for calls to collection methods like foreach, map, filter, and flatMap. They are often called for-comprehensions and they have a lot more flexibility than what you get from a standard for loop in most languages or for-each loops in the languages that have them. The fact that they get compiled to calls to these other methods also means that Scala for-loops can be used on things that aren't collections like Options and Futures.

Unfortunately, this flexibility has often come at a price. As I noted in an earlier post, Loop Performance and Local Variables, using a for loop produced code that was significantly slower than using while loops. That was back in 2013, and I was using Scala 2.10. With the release of Scala 2.12, I really wanted to see if this was still the case. The primary change in 2.12 was to make Scala compile to Java 8 bytecode using all the new features of the Java 8 JVM. One of the main additions in Java 8 was lambda expressions. Since the foreach, map, filter, and flatMap are higher order methods that take functions, compiling to Java 8 lambdas seemed like it might improve performance. This post looks at testing that hypothesis.

Previous Code

We start by repeating the previous test that was written to look at where variables are declared. I took the same code as used before and simply ran it with three different versions of Scala, both with and without optimization. The following table shows the results.

VersionOptLoopVar LocAverage Time [ns]Time Deviation [ns]
2.10.6 forIn3.4100.141
Out3.7000.293
whileIn2.8230.270
Out2.8610.311
-optimizeforIn3.7120.588
Out3.8560.653
whileIn2.8950.252
Out2.8810.279
2.11.8 forIn3.2210.351
Out3.6660.397
whileIn3.0230.510
Out2.8480.243
-optimizeforIn3.3890.402
Out3.0140.120
whileIn2.8580.287
Out2.8480.254
2.12.1 forIn3.1540.354
Out3.4560.407
whileIn2.7650.167
Out2.7510.139
See BelowforIn3.2610.321
Out3.2790.549
whileIn3.1410.207
Out3.1640.264

All runs were done using Java 1.8.0_111 for the runtime. For 2.12, they added a lot of different optimization flags to the compiler. The values used for the timings in this post are -opt:l:classpath -opt:closure-invocations -opt:simplify-jumps -opt:copy-propagation -opt:redundant-casts -opt:box-unbox -opt:nullness-tracking -opt:inline-global. There is enough scatter here that it is hard to draw really strong conclusions. It appears that the while loop still has an advantage, but the percent difference in speed seems smaller across all the "current" compilers than what had been seen back in 2013. I put current in quotes because while 2.10 is older, 2.10.6 is a fairly recent release and the Scala team backports things when it makes sense, so there are good odds that 2.10.6 is incorporating optimizations of the for loop that weren't present in the earlier version of 2.10 I had been using in 2013.

N-Body Simulation

The problem of building multiplication tables was rather contrived as a simple example that worked well for testing the declaration locations of variables. If people are going to actually make their code uglier putting in while loops in place of for loops, it would be good to see if it matters on a somewhat more realistic example. For this I decided to do a simple first-order numerical integrator of bodies using gravity. This is a problem that involves a lot of number crunching in loops and which happens to be at least related to things that I write for my research, so it seemed like a good place to test performance.

The code used for this test is shown below. For the purposes of this post, what really matters is the forSim and whileSim methods. These have multiple loops including one area where they are triply nested. I store all the values in mutable arrays and then use a value class to access the elements in an object-oriented way. I chose this approach as there is minimal overhead from object allocation, potentially better cache performance, and I have a feeling that it is faster than other approaches, though testing that is a matter for later posts.

Here is a table giving the timing results for this code again the same three compilers.

VersionOptLoopAverage Time [s]Time Deviation [s]
2.10.6 for0.6660.002
while0.6670.029
-optimizefor0.6600.012
while0.6580.001
2.11.8 for0.7160.009
while0.6690.007
-optimizefor0.6750.006
while0.6560.001
2.12.1 for0.6990.003
while0.6830.001
See Abovefor0.6760.001
while0.6830.003

Note that for this code, there is very little difference between a for loop and a while loop. These tests were very stable in their timing results and while building up the tests I ran them multiple times and found little variation. It really doesn't appear that 2.12 did anything to help with the difference between for and while loops in either of these examples, but in this one, there really isn't a significant difference in any version. What does that mean? As with so many things dealing with performance, you should write clean code that runs first. Once you have that, and you are tweaking things for performance, you might consider changing your inner-most loops from for loops to while loops, but it is quite possible that it won't matter.

I also feel compelled to note that the for loop version is much easier to parallelize than the while loop version because of the ease of switching to a parallel collection. I haven't done it here as one must make some alterations to prevent race conditions, but that is something that I might also explore in a future post.

Variables in for Comprehensions

There is one caveat to the conclusion that for loops don't hurt performance in the larger example. In the forSim method shown above, the variables pi and pj are both declared inside of the inner most loop. The for comprehension in Scala allows variables to be declared in the "header" section of the loop. When I first wrote this code, I declared pi between the two generators and pj right after the generator for j. One wouldn't think that this would matter much, but it did. Having the declarations up in the header instead of the body cause this code to run roughly 2.5-3x slower than when they were put as the first lines in the body of the for loop. I don't have an explanation for this behavior and I haven't explored the generated bytecode to see what might be causing it. However, based on this result, it is probably worth not using the variable declaration capabilities of for comprehensions if performance is critical to your application.