Archive for software

Where to now Mac?

I’ve been looking at the Mac product line and I don’t see a roadmap to 64-bit machines anywhere. Moving Macs from PowerPC to Intel was a step backwards for applications that require lots of memory. There were 64-bit PowerPC Macs, but nowhere do I see any plans for the new Intel Macs to address anything other than 32 bits and that’s actually pretty troublesome to me. As far as I know, they are going to kill off all PowerPCs, both 32 and 64 bit. Mid-range PCs these days have 1GB of memory, so very shortly high end Macs are going to have problems addressing more than 4GB of memory. This seems like a pretty serious flaw in their product strategy. Maybe there’s something I’m missing… I know that there are Intel extensions which allow their 32 bit chips to address more than 4GB of real memory with some virtualization(they call them Physical Address Extensions) but that’s already kind of hacky. PAE generally limits each process to 4GB of memory, which is pretty annoying if you just want one big process with a huge chunk of memory. I’ll do some more digging to see where the Mac is going, but it does seem like a problem for high end Mac users in the short term.

Comments off

VMWare

After listening to that talk on Solaris Zones I went back and played with VMWare again, and I was again impressed with how well it works. I installed VMWare Player and a packaged Kubuntu(Ubuntu with KDE) and it was working with full networking and sound right out of the box(Windows host).

It got me to thinking that maybe software vendors could package their stuff into VMWare execution environments instead of shipping real appliances or regular software packages. For instance Tango works on the Proofpoint appliance and they ship a rackmount server with all their software on it. Rather than having to deal with hardware, they could just ship something that runs under VMWare. There’d be a bit of a performance penalty, but it’s not much.

The big benefit is that the vendor can design a clean environment to work with. Then they can always know the environment the customer is running. No more valuable support time spent tracking down what changes or configuration oddities a customer has. Customers shouldn’t be touching the virtual machine anyways and the vendor can completely lock them out if necesary. It also makes debugging customer problems a lot easier. Got a problem that’s hard to replicate? Just have the customer send you back the whole virtual machine they’re using. It can even solve some porting problems. Want to port from Linux to Windows or vice-versa? Don’t bother, just have the customer run your OS of choice within theirs(this isn’t practical for most applications but it depends). The vendor also doesn’t have to worry about supporting specific OSes(but of course VMWare must support that OS), hardware sizing requirements, hardware support contracts. Obviously this doesn’t work for all kinds of software, but it’d work for most application software. Think of how developers use Java as an abstraction layer, this would be an even higher level of abstraction.

I’m not saying it’s a great idea, it is just a thought.

Comments off

MySQL Insert Performance

I ran some tests on MySQL insert performance and thought I’d share the results.

The MySQL client and server are version 5.0.18. The client and server are otherwise idle 1 CPU rack mount servers running Linux 2.6.9, nothing fancy.

These performance tests are designed to test insert speed. Because these are micro-benchmarks the data should not be relied on too heavily for a real world application test. However the data for this test is clear.

Here’s the test script
The table is relatively simple with no indexes.

mysql> desc small;+-------+-----------+------+-----+-------------------+----------------+| Field | Type | Null | Key | Default | Extra |+-------+-----------+------+-----+-------------------+----------------+| id | int(11) | NO | PRI | NULL | auto_increment || value | int(11) | YES | | NULL | || time | timestamp | YES | | CURRENT_TIMESTAMP | |+-------+-----------+------+-----+-------------------+----------------+3 rows in set (0.00 sec)

The test only inserts one the value into the table and does it consecutively for 1 million iterations. An example statement is:

insert into small(value) values(1)

The load data uses mysql’s LOAD DATA statement to do a large batch load. The LOAD DATA does a batch load, so in the test script I write to a file first(a memory file in /dev/shm/) and then upload that file to MySQL. The LOAD DATA command requires a special privilege on the client side, in Perl it’s done by specifying the mysql_local_infile parameter like this in Perl:

my ($dsn) = "DBI:mysql:database=test;host=athena;mysql_local_infile=1";

Results

Test Time
Prepared Statement 222
Batch Prepared Statement(batches of 100) 21
Unprepared 192
Batch unprepared(batches of 100) 18
LOAD DATA 8

The big surprise is that prepared statements are actually slower than just directly executing the query. Probably has something to do with more overhead for no benefit on statements this simple. I really hope it’s better with complex queries.

Batch statements are a huge win, with a medium-large batch size giving a 10x speed increase.

LOAD DATA is the fastest, because among the things it does is to lock the table, and only rebuilds the index once. It really blows away everything else in speed. It’s a little bit trickier to set up, but it’s worth it.

I should have also run some tests with INSERT DELAYED.

Comments (2)

Solaris 10 Zones

I saw a presentation on Solaris 10′s Zones virtualization today, and I came away somewhat unimpressed. They use a different model than other virtualization products like Xen and VMWare. Their model is to re-use the same kernel among all zones(each zone is an instance of the OS more or less). The process id space is shared between all processes, but the IP port space is not.

Compared to the other models, this means that you can run a lot more virtual machines because you only need memory for all the processes run in each zone. With VMWare and Xen, you need to allocate a large contiguous block of memory which is as large as the amount of real memory you want to emulate. ie if you have 3 virtual machines which each wants 512MB of memory, but in each VM the kernel and processes require only 50MB of memory in the Solaris Zone model, you only really need 150MB of memory(3*50), but in the VMWare and Xen models, you’ll pretty much need 1546MB(3*512) of memory.

However, the drawback of the zones model outweighs the benefits to me. The major drawback is you can’t use different kernels or OSes between the virtual machines. There are some other oddities, like they claim the global and local zones can have different user ids and password files, but a user in the global zone which has the same userid as a user in a local zone can act as that user in the local zone. I’d prefer a completely hidden virtual machine abstraction.

Comments off

Dreamhost and Gallery 2.0

I’m impressed with both Dreamhost and Gallery 2.0. I had an old Gallery 1.x with all my photos on chen.net which was hosted by Funkware. I moved everything to Dreamhost, but I didn’t want to re-install Gallery. Dreamhost has a one-click install process for Gallery 2.0, so I figured I’d try it out. It was technically several more steps than one click, but it went very smoothly. I was even more impressed with how well the upgrade of my 1.0 Gallery went, about all I had to do was to specify the location of my old files and it took care of almost everything. Lots of new features in Gallery 2.0, and more importantly a lot more extensibility so it can use all sorts of plugins and modules.

Oh and Google completely reamed my PageRank. This happened a month or two ago, but I was just mentioning it now because I was completely looking for a blog post from a few months ago and I was totally unable to find anything despite hitting the keywords and the title pretty close. Yahoo found the post immediately. Thus we can conclude that Yahoo is now the superior search engine based on my sample size of one.

Comments off

Prosper.com

Ugh. Another lost post. This time it was firefox that crashed on me.

I like the idea behind Prosper.com quite a bit. Like HedgeStreet it provides an innovative person-to-person financial service, in this case personal lending. Prosper.com will handle all of the loan servicing, including credit checks, payment schedule, and potentially getting a collection agency to make sure debtors make good.

For borrowers, you request a loan and provide some information including your credit history and then tell the system how much interest you are willing to pay.

Then lenders bid down the amount and price they are willing to loan to you. If a borrower with an excellent credity history requests $10k at %6, you might have a dozen lenders offer $1k each at %5. Then the system would take the lenders and potentially consolidate them into a single loan at the highest rate required to satisfy the sum total of the loan. eg. If the bids were $5k at %3, $2k at %4, $3k at %5, and $7k at %6, then the first three bids would be accepted because they have the best rate(for the borrower). The rate would be %5(highest of the bids) if I understand it correctly.

There are a couple cool things. First you can(and definitely should) diversify your lending power to mitigate the risk of a single bad borrower. And on the borrowing side there is a reputation system(defined by joining a member of a group) which helps lower the borrowing rates that people might have to endure otherwise due to bad credit or something like that. I’m still a bit fuzzy on the borrowing group system.

Anyways, this business model is great. Completely scalable, highly leveragable, no marginal cost. Prosper.com charges a %1 loan origination fee. They also currently limit loans to 3 years and $25k, but I’m sure they’d expand that if they are successful.

It also presents interesting opportunities for enterprising individuals. Borrowing and lending currently require a person who is a U.S. resident with a bank account and a social security number. However I could see some Chinese kid getting his cousin who is a US resident to set him up an account with a few hundred dollars and then start arbitraging loans. Of course the %1 commission is going to eat into it, but it can’t be that hard to make sure you’re doing better than that. Reminds me of the old banker’s 3-6-3 rule. Borrow at %3, lend at %6, golf at 3. I admit I’m tempted to leverage my good credit to try doing some trading here, but I’m too lazy to actually follow through on it.

Comments off

Dreamhost migration complete

I was able to move over the domain name to dreamhost.com successfully. I think everything looks good, but send me email if anything looks more broken than it used to.

Yesterday there were new screenshots of KDE4 and also Nat showed me this Touchscreen Interface. It’s hard to get excited about KDE4 when you see something as cool as that touchscreen stuff. The one danger of a touchscreen UI like that is that it requires a lot of movement to do it. It would tire your arms out to work on that all day.

I’m seeing a lot of URLs from You Tube lately. It’s a pretty good service from what I’ve seen. It’s very web 2.0, user-created content, tagging, viral networking. I wonder what their bandwidth bills are like though.

Comments off

Moving from Earthlink to Dreamhost

After some frustrations with Earthlink, I’ve decided to switch my hosting company to Dreamhost.com. They come highly recommended from a couple of different people, and they are very UNIX developer friendly. They have features like PHP, mysql, email forwarding, unlimited subdomains, unix shell access, and they are very cheap, only $10/mo if you prepay for a year.

They also have a nice policy where they are always giving you more bandwidth and storage. You start with 20GB storage and 1TB bandwidth per month, but they also add to that capacity 160MB storage and 1GB bandwidth every month. I’ve never heard of another web hosting company doing this for you automatically, but it shows they are customer oriented. People use a webhoster and they know that the costs per MB should always drop, but usually the customers have to complain to get a better deal.

I’ll start by moving DNS and mail forwarding to dreamhost and maybe later I’ll move the blog and gallery.

If you sign up with them, tell them that chen.net referred you and I’ll get a small bonus (like $10) out of it.

Comments off

Google and Frappr

Google is an interesting dichotomy. They are very closed about their plans and future direction, but when they do open up their systems, they do an excellent job with the APIs and documentation for developers. Their APIs for AdWords,Maps and Search are very good.

That brings me to a popular new application, Frappr. I’d describe it as a simple group mapping tool. You create a group and everyone who joins your group can put their location on a map, so you can see where they all live. It’s not too complicated(and honestly it’s not that amazing either), but it is the kind of thing that can be built relatively quickly on top of Google Maps because their APIs are good.

It’s things like these that could be very troublesome to Microsoft. Microsoft’s application dominance is based on having the most developers who are loyal to their tools. As web based applications become more popular, they are in danger of losing their developer base, which means they are in danger losing their applications. Microsoft’s web application tools are not bad, but they are very Microsoft-centric causing some developers to think twice when using them in mixed environments.

Comments off

Civ 4

Civilization is probably the only game I’ve ever played in four different versions. I don’t buy too many games, but I like the series enough that I figured this couldn’t be too bad. The game design is extremely deep. There is more to this game than any game I’ve ever played. This is good and bad. You can invest a huge amount of time in developing optimal strategies and learning how the game works, but it also takes a long time to play a game. A full game can easily take 6-10 hours. Thankfully, there’s now a Quick mode which is probably twice as fast, but that is still a long time. I have to give this game a thumbs up because it’s got such a great legacy and it lives up to it. I do have to say though, that even though the design is great, the game itself is programmed badly. I’m surprised they released it because it leaks memory so quickly you can barely play an hour or two before you have to restart the game to make it fast again. And it crashes more than I would expect, bad programming again. I hope they put out a patch soon.

Comments off