leŭksman

Ok, Rob’s starting to clear out some of the FlaggedRevs setup requests, now that we’ve cleaned up some of the configuration files.

For today we’ve set up en.wikibooks.org as requested.

If everything’s going smoothly there, we’ll start chugging through the rest over the coming days.

I’ve got MediaWiki on the live Wikimedia sites all up to date at r43514 now.

We’d been updating some extensions individually over the last couple weeks, but full updates were held back as general code review got a little behind during the lead-up to the fundraiser and our staff meeting last week… I’m hoping to get us on a regular weekly update schedule, probably Tuesdays since my experience is that Mondays end up totally unproductive.

I did have to pull back the Special:Search redesign for the moment; it’s looking *awesome* but has a few glitches still, which I’m hoping we can resolve before putting it live.

See my comments on issues I noticed in the revert.

I’m thinking we should start making more active use of branches for experimental/iterative development like this, where existing features in core are majorly refactored and need some iterations of testing before going live.

We try to keep our trunk code ready-to-run at all times, so when something in trunk is not quite ready yet we end up rolling it back (which requires tracking down multiple changes and reverting all of them) or else rushing fixes so we can get an update pushed out.

The SVN server was updated to 1.5 a while ago, which is apparently a little handier at branch merging, but branching still is kind of awkward in SVN. Any good recommendations on SVN-friendly DVCSs? I know some folks use SVK or a GIT-SVN bridge for doing various local development, but how easy is it to share a development branch among multiple developers over time this way?

This is shaping up as a busy week at Wikimedia HQ… We’re starting our year-end fundraiser, our remote staff are flying into town for an all-staff meeting, and to top it off there was this election thingy going on…

Over the last few weeks we’ve all been scrambling to get things ready. The fundraiser landing pages are beautiful, our donation tracking system seems to actually work (thanks David!), and the site notice banner doesn’t have any scrolling marquees…

Thanks to the tireless work of Tomasz and Trevor, we now have a workable system for previewing notice banner updates and scheduling them before they go live — which should make things much smoother as the designs get updated over the course of the fundraiser season.

We started live-testing the notice Monday on test.wikipedia.org:

By Tuesday morning, Rand and the rest of the fundraiser team were rushing to fix up the donation pages in response to review and broken-link testing, and we started deploying notices live — first to Wikibooks, then Wikipedia in Japanese, Polish, French, and German.

After a few hours of testing and tweaking, we stuck the notice on full on all Wikipedias… And saw a prompt decline in site traffic as several of our proxy cache servers threw themselves into some kind of overload failure.

This being a) the start of our fundraiser and b) the night of the US Presidential election, we thought we’d better stay in and fix it instead of going out for a tech department dinner…

I canceled our reservation and ordered some Indian delivery while Tim and Mark sorted out the servers, and we kept a sharp eye on things as the evening continued…

The overload may have been triggered by extra hits to backend web servers for the images in the notice; to be on the safe side we moved them over to the upload/media servers before redeploying.

Around the time the networks started calling the election for Obama, we were ready to put the notices back in. We were seeing some load spikes on backend web servers, but these seemed to be from legitimate page rendering, most likely related to the heavy editing traffic for election results.

As McCain conceded the election, we started filtering in the fundraising notice for about 1 in ten hits, making sure it wasn’t breaking anything.

By the middle of Obama’s victory speech, we’d ramped it up to 100% of hits, and started to see the donations rolling in.

We’re seeing definite positive response — people are certainly turning out to support Wikipedia! We’re also getting feedback on the layout of the notice itself, and are making some tweaks in response. For starters we’re changing the “Hide” link to a more accurate “Collapse”; like last year’s the link doesn’t completely hide the notice, just collapses it to a smaller version.

Thanks everybody for your support (and I hope you voted, too, if you were eligible!) — we’ll try to keep things smooth on our end and keep the fundraiser classy.

Apparently we’re having one of those election thingies again…

or… not

23381 Oct 16 13:32 lighttpd.conf
11085 Jun 18  2007 lighttpd.conf~
11085 Jun 18  2007 lighttpd.conf.before-brion-broke-it
11250 Apr 11  2008 lighttpd.conf.before-mark-broke-it

It’s Bug Monday again!

This week’s theme is JavaScript issues… so let’s get cracking! There’s bugs to confirm, patches to review, and fights to fight!

Some highlights…

Software:

• The Collection extension for PDF generation from wiki pages has been enabled on all Wikibooks wikis for further testing. It remains experimental, but is pretty handy already!
• Welcome new dev Trevor Parscal at Wikimedia’s San Francisco office! Trevor is cutting his teeth on an edit draft saving/recovery extension.
• We’ve caught up on code review and restarted standard update schedule of MediaWiki on the cluster. There have been a few hiccups affecting some bot tools, mostly fixed.

Cluster issues:

• Tim’s been fixing up some of our monitoring systems and cleaning up some backlog on crashed/weirdly configured machines.
• We’ve continued to see some intermittent issues with image thumbnail rendering since re-establishing a normalized configuration for the image scaler machines. Tim has been poking at these to sort the problems out.
• There have been issues with timeline generation, not yet resolved, probably caused by changed configurations of ploticus and/or fonts on new web server installations.

In the data center:

• Some issues with our primary image file server — a kernel crash during debugging, and a disk replacement (RAID wins again!)
• Rob and Mark have been setting up our new expansion data center in Tampa, at Switch & Data. This gives us additional space and cooling power needed to expand our server base, which means more disk space for media files and backups. We like backups!
• Mark has researched plans for our future data center migrations in Amsterdam, which we’ll be finalizing over the coming weeks.

At the office:

• David Strauss has been visiting us in SF a lot to get our CiviCRM donation-tracking backend ready for the fundraiser.
• Tomasz has been fixing up CentralNotice with multiple message scheduling and a localization helper interface in preparation for the fundraiser. Coordination with designers and the CRM stuff is ongoing.
• Welcome new dev and office IT folk Ariel Glenn! Ariel’s a longtime Wikipedian and Wiktionarian. At the moment Ariel’s working on some of the IT backlog at the office, getting the lay of the land and beefing up our backup procedures.

Updated spelling 2008-10-27

We finally replaced my fiancée’s ancient PC with a shiny new Dell laptop. While ordering, I couldn’t help myself and tossed in a Inspiron Mini 9 for myself:

This little cutie weighs in at just 2.26 pounds, less than half of my MacBook’s hefty 5 pounds. I’ve found that the Mini is much more back-friendly than my MacBook; I can painlessly lug it to the office with my laptop bag slung over my shoulder (easier for getting on and off the subway) instead of nerding it up in backpack mode.

The top-end model I picked packs 16GB storage and 1GB RAM running on a 1.6 GHz Atom processor — far more powerful than the computer I took with me to college in 1997. Admittedly, my iPhone also beats that computer at 8GB/128MB/300MHz vs 6.4GB/64MB/266MHz.

The compact form factor does have some impact on usability, though. The 1024×600 screen sometimes feels too tight for vertical space, but they include a handy full-screen zoom hotkey for the window manager which opens things up.

The keyboard feels a bit cramped, and some of the keys are in surprising places (the apostrophe and hyphen are frequent offenders), but it’s still a lot easier to type serious notes or emails on than the iPhone. I had to disable the trackpad’s click and scrolling options to keep from accidentally pasting random text with my palms while typing…

The machine shipped with a customized Ubuntu distribution which is fully functional; they include a “friendly” launcher app which can be easily disabled, and even the launcher doesn’t interfere too badly. The desktop launch bar that’s crept into Gnome nicely handles my “I need Spotlight to launch stuff with the keyboard” fix. Firefox works fine (after uninstalling lots of Yahoo! extensions), Thunderbird installed easily enough, and I even got Skype to work with my USB headset! (AT&T’s international roaming charges can bite me…)

The biggest obstacle for me to use this machine every day is my Yojimbo addiction. I use Yojimbo for darn near everything — random notes, travel plans, budgeting, grocery lists, recipes, encrypted password stores, saving articles and documentation for future references. It’s insanely easy to use, the search works, I don’t have to remember where I saved anything, and it syncs across all my Macs. But… it’s Mac-only.

I’m trying out WebJimbo, which provides an AJAX-y web interface for remotely accessing your Yojimbo notes. It’s very impressive for what it does, but I’m hitting some nasty brick walls: editing a note with formatting drops all the formatting, but I use embedded screen shots and coloring extensively in my notes.

I’ve seen some reports of people hacking Mac OS X onto the Dell Mini — very tempting to avoid OS switching overhead. But I think if I really want that, eventually I should just suck it up and buy a MacBook Air. The form factor is the same as my MacBook (full keyboard, roomier 1280×800 screen), but at 3 pounds it’s much closer to the Mini than to my regular MacBook in weight, so should be about as back-friendly for the subway commute and air travel.

Of course, the Air costs $1799 and I got my tricked-out Mini for about $400, so… I’ll save my pennies and see.

Gave another talk, this time at the Online Community Report Meetup hosted by Tech Soup here in San Francisco.

As usual, my slides are available for download on my presentations page:

This went into a little more general background on wikis and community structure; we kind of got side-tracked by Wikipedia-specific questions and I didn’t get through all my slides in the couple hours we had.

Gave a talk last night at the Greater San Francisco Bay Apache LAMP Meetup Group, meeting in the historic mansion on the Sun campus in Santa Clara.

As usual, my slides are available for download on my presentations page:

Some of the usual background, plus I went into some more detail on MySQL replication issues. Lots of great questions from the crowd — they seem like a fun bunch!