Data migration

We're almost ready to start the data migration. It will take a while, though, and we'll need to test it online ourselves. Getting close! Scott will have more migration details when it's ready for you.

Good news: the Google App Engine team did some digging and uncovered the issue that's been causing the site to be slow. Bad news: it won't be an easy fix. Expect the slowness to continue (on-and-off) for a couple weeks until I can crush it.

The sun has gone down and the moon has come up

I've been breathing vocab definitions and seeing traditional variants when I close my eyes for just under a month, now. I'm tantalizingly close to finishing all the data preparation needed before we push the new stuff live. When I finish that, on Saturday, we'll see how long it takes to finish debugging and do the data migration.

We could potentially hit my estimate and have it ready in a week. (Potentially!) The bodacity, dear users, that you will experience when it goes online--mwah! (That's me kissing the imaginary bodacity, to show how delicious it will be, like a fine spaghetti with some fine Kraft Parmesan cheese on it.)

It burns with the power of shark

My right ring finger, that is. Why? From typing on 6524 definitions in the past two weeks. Excluding the 3544 entries in HSK4, which I'll correct later, that means I'm 86% done fixing up all the words in all our lists. My three week estimate from a week ago thus happily becomes a two week estimate.

Some of these definitions we have from other lists are really bad, and others are unceasingly hilarious. Like this one:
滔滔不绝 - tāotāobùjué:
dash along; give it mouth
(saying) gushing; torrential; non-stopping
pouring out words in a steady flow
unceasing torrent (成语 saw); talking non-stop/gabbling forty to the dozen

Hehehe, gabbling forty to the dozen!

能歌善舞 - néng gē shàn wǔ: good at signing and dancing
I read this and immediately imagined a large, Eastern European diplomat doing Riverdance-style jigs all day long while signing treaties, contracts, and other important documents, with throngs of fly honeys ooh'ing and aah'ing around him.

游击 - yóujī:
guerrilla warfare
guerrilla
shortstop
(n) guerrilla warfare

Turns out 游击手 (yóujīshǒu) means shortstop. I don't think 游击 does, though.

Meanwhile, George has added the extra couple hundred characters with the Chef, and he and Scott have designed and built anew even more parts of our interface. The drool flows...

In other news, "review only" mode seems to be causing a lot of problems of late. It'll be a lot different in the new system, so for now, don't try going review-only--always have one list on "adding and review."

Making it Go Fast

Over the last few days, we've had Skritter slow down markedly every so often, and it looks like it's due to Google App Engine being slow for long periods of time during the day. This is bad for practicing, since each stroke to be recognized needs to be processed by the server. So when we can, we'll make the flash do the recognition to make practice faster in general, and especially when the servers are being sluggish. Also, the server status page shows an anomaly during these slow times, so Google seems to know about the problem and hopefully will fix it soon. You can see the current status of our servers at:

http://code.google.com/status/appengine/

Click the cell for 'Serving' and 'Now' and you can see when the latency is abnormally slow.

Getting there

So in order to get all the vocab lists into the new format, we need to do the big definition consolidation, too, it turns out, in addition to all the other stuff we were going to do later but had to do first. Which means it's taking even longer, but will be even better, than heretofore dreamt.

George has been doing the first pass on this work for the past two months, and he just finished today. I started reviewing/correcting them last week, and I'm 40% done. I'm trying to go as fast as I can, but it's exhausting. At least I'm learning a lot of Chinese while doing it.

There are yet several handfuls of data migration and debugging tasks to do before we can put everything online, after that's done. I've been abysmal about estimates before, but assuming we don't discover anything else that needs to get done at the same time, let's guess it'll be ready in three weeks. Pretty brutal, I know. But it will yet be worth the wait!