Preparing to be a third world country

Lots to rant about this week. Today its going to be about US economic development policy.

As in, we have none.

I have had the opportunity to talk to people from many different countries and governments about startup funding for a new business in the past year or so. Its a software infrastructure project that has the potential to have a major developmental impact on the countries electronic entertainment industry.

Canada, Singapore and Scottland all have programs where they will pay 50% of the salaries of engineers hired in their countries to pursue this project.

Germany and the EU have programs where they will pay for the costs of commercializing university research.

What about the US? The US has the attitude that anything that is useful for industry should be funded solely by industry. In other words, we have no policy of support for the development of our high tech industry in this country. At all.

This is frightening. We have effectively lost all manufacturing in this country. When I was at Sun a common idea floating around was that this was okay because we would always be *the* center of intellectual property and new ideas.

I thought that was hubris then, I think its provably false now. With other countries actively investing in their high tech industries the way we once invested in a space program I am afraid our time at the top of the heap is coming to an abrupt close. Of the three top game consoles today, the only one that was created in the US i the one that doesn't make money. I am referring of course to the XBox360 and the huge sums of money Microsoft has poured into buying the number two console spot.

We have Apple still making innovative product, but that innovation is almost immediately copied and then riffed on by companies in Asia. I wouldn't peg our hopes to that particular horse. Apple's lead is primarily in industrial design, and thats a talent that can be developed anywhere with a little bit of effort. And at still less then 10% of the PC market and a seriously threatened position in mobile computing, they are hardly a drop in the world market bucket. The other 90% of the PC market? Thats already majoratively owned by Korean and Chinese computer makers.

Other signs of our retreat into third world status are the growing gulf between the rich and the poor and the evaporation of the middle class. With every step in that direction we are look less and less like a modern industrialized nation.

They say a laurel wilts the fastest when its sat upon. And we have planted our posteriors smugly upon ours for far too long. Either we snap out of it and remember that a rising tide rises all boats, and a falling one drops them, or we cede our place in the world to countries that need no reminding.




- Posted using BlogPress from my iPad

The Dysfunctional Family of Open Source

So, I have had the honor this week of working with a set of some of the smartest, most cutting edge game researchers in academia this week putting together a research agenda for the National Science Foundation.

This has given me a chance to immerse myself in a world I normal only touch on the edges, and in the process something has struck me. It is one of many many places I have been where there is an institutional double standard on open source software.

One of my frustrations in life has been how badly we do information transfer in this country. There are all sorts of interesting research projects in our universities that could lead to either interesting one off games or potentially whole new tools or genres. That transfer however seldom happens and the reasons come down to economics. The researchers need funding to do their research. The universities need funding as well. But the game industry isn't like the telephone industry where a few big companies dominate the market and are brimming with money to throw at Universities. Even the biggest game houses are really just large collections of individually accounted for projects, each of which has to make an immediate decision about where dollars for the currently in development game goes. If it doesn't make THIS game cheaper or better NOW, there is no room in the budget for it.

But where is this double standard? The answers is that university researchers *love* free and open source game engines. They find them incredibly useful in doing their jobs. But raise the idea that they should in turn open source their software artifacts and you better duck.

This is not specific to academia. I've seen this double standard over and over. Lots of people use open source software today without any interest in returning the results of that use to community ownership. Big software companies will use open source components freely in their products, but have no interest in giving away the products of their own labors.

But it seems to me a fundamentally dysfunctional relationship. As a socialist revolution, it seems pretty one-sided, opportunistic, and ultimately unsupportable.




- Posted using BlogPress from my iPad

The power of connotations

Its funny how even intelligent engineers can be easily swayed by the connotations associated with a name.

Periodically, we get asked "why does Darkstar (now RedDwarf) use pessimistic concurrency, wouldn't optimsitic concurrency be better? Optimism is better then pessimism right?"

Well, not necessarily. Unbridled optimism can be a very bad thing. Optimism about the housing market created the economic situation we are all in now. But more to the point, the connotations of the words "optimistic" and "pessimistic" don't really say anything about how the software functions. As it turns out neither is "better" then the other. It depends on what you are trying to optimize, where and when.

To quote wikipedia:

"In the field of relational database management systems, optimistic concurrency control (OCC) is a concurrency control method that assumes that multiple transactions can complete without affecting each other, and that therefore transactions can proceed without locking the data resources that they affect. Before committing, each transaction verifies that no other transaction has modified its data. If the check reveals conflicting modifications, the committing transaction rolls back.^[1]

However, if conflicts happen often, the cost of repeatedly restarting transactions hurts performance significantly; other concurrency control methods have better performance under these conditions."

So called pessimistic concurrency checks for conflicts when an object is first accessed for read or write. If there others out there using the object in a conflicting manner, it can cause the thread to pause while it waits for the other thread to finish its use of that object. In the event that there is such a contention, this is a good thing. It means you don't waste CPU doing calculations based on a 'stale' object and throwing all that work away at the end and having to start all over again. This reduces CPU load. While the blocked thread is waiting, other threads get to use the CPU. The end result is that more users can be processed in parallel with less total CPU usage.

However that check does come with some small cost. In an environment where you are accessing hundreds or thousands of objects in a thread this way, it can add up. Thats where so-called optimistic concurrency comes in. It doesn't take those costs but just acts like the object is always free. It keeps its own copy and, at the end, checks for consistency. If it finds a conflict, it dumps all its investment and starts over. (No government bailout applies.)

SO, which is better? It depends on your expected usage. If you expect a high volume of data accesses to data processing with very little contention, optimistic concurrency makes sense.

In Darkstar/RedDwarf however where we expect maybe a few dozen data access per thread, and where we expect our data processing (game logic) to be of significant cost, its better to be safe (and pessimistic) then sorry.