Wednesday 22 July 2009

wine-patches the black hole of code?

The Problem

On Monday, I took part in the first Wine Bug Day (one of hopefully many). It was a great success, I managed to rope in some support from some friends specifically Sean and Yorik both of whom are Wine users, not Wine developers (yet!).

And it wasn't just us, there was of course Scott Ritchie who organized the thing, but also several users popped up in #winehackers asking for others to change the status of their bugs. It was all quite exciting in a weird kind of way.

It was because of this buzz of excitement that I continued my bug triaging efforts. I knew that Autocad 2008 was supposed to *almost* work, so I thought I'd download the trial and give it a spin in the latest Wine. Surprisingly it worked pretty well, once I'd installed the prerequisites using winetricks that is. The installation went smoothly but on the first load of Autocad all of the icons were black - I've seen this issue before and I knew that using a native gdiplus.dll would fix it, but of course this is a bug in Wine so I thought I'd look at little closer. Sure enough there were 2 gdiplus calls that Autocad uses which are just stubs in Wine: GdipCreateHICONFromBitmap and GdipCreateHBITMAPFromBitmap.

Now, this is where I confirmed (I kinda knew, and mentioned it in a previous blog post) that there is a real problem with sending patches to Wine. Google showed me these two links. The work has already been done before! But it's not in Wine...

I'm not criticizing the fact that these patches were rejected, even skimming over them I can see a few issues that could do with fixing before they are accepted. What I do get annoyed about is these people have spent hours of their lives developing something, they've gone through the effort of creating patches and emailing them to the wine mailing list, and the first patch didn't even get a reply! Yes, they could join #winehackers and chase for comments, yes, they could take on board the comments and resubmit, but why? They've already put in the extra effort to get the patches out there, isn't a bit unfair to expect them to jump through hoops to get the patches in to Wine? I think so.

I know how frustrating it is to get patches into Wine. In fact, of the patches I have submitted I think only one went in first time, and that was only because it was a simple addition to an existing structure. It's taken me 3 attempts to get even simple patches in, this is not encouraging for a developer looking for a project to devote their time to. Even I have just abandoned patches, simply because I couldn't be arsed to go through the loop again for a minor minor thing. So let's break down the actual problems:

  • Patches rarely receive feedback - positive or negative - on the wine-patches list

  • Patches that are rejected aren't labelled as such, or publicly given a reason for why they were

  • Hundreds of patches don't get in, that just need a tweak or two, that would REALLY improve the wine experience

  • These failed patches can't easily be seen by anyone unless they know exactly what to Google for

The solution

What we need is a public notice board for these failed patches. We need a way for everyone to see why a patch was rejected and we need a central point to find these patches so they can be cleaned and resubmitted. Time permitting that's what I intend to develop, and here's how it might work.

wine-patches holds every patch that is sent externally to Wine, whether it was accepted or not. Given a month and year, we can programmatically dig out a list of all the emails that were submitted by parsing the index page at{YEAR}-{MONTH}/thread.html - this is our master list. We can follow those links and also, in code, read the bodies of the patches. Some of them will be attachments, others will be embedded in the page. Either way, we can parse the majority of them, some emails won't have a patch attached we can strip these ones out. At this point we have a list of patches submitted, now we just need to know whether they were committed....

Fortunately, wine-cvs holds the answer. wine-cvs contains emails for every commit to the master repository. What we need to do is match the patches in our master list, to the ones in wine-cvs. If we can't find a close match then we can assume the patch never made it in. So, in theory we can generate a webpage with this information, with the possibility of devs submitting feedback against the patches so they can be cleaned up and resubmitted. We can go back pretty far in time with this, I would say at least 6-8 months back before the patches just become too old to be worth bothering with. Stay tuned...

I'd just like to clarify some things. Firstly, I'm not attacking Wine here I'm just pointing out that there is code that slips through the net on wine-patches that could be cleaned up and resubmitted if there was a central point to find these patches and a means to find out what was wrong with them. I'm not just aimlessly moaning either, I've stated that I hope to develop a web app to do just that.

Secondly, I have contributed to Wine A LOT over the last couple of years; submitting patches, application tests, contributing to discussions on wine-devel and aiding developers and community members like Thunderbird, Stefand and Yokozar with their contributions, and I hope to continue to do so. I am not new to Wine development.

Thursday 16 July 2009

2010 - The Year of Linux on the Desktop?

Every year for at least the last decade has been jokingly associated with being the "year of the Linux desktop". Optimism surely is strong in the Open Source world and it's only in hindsight do we realize how far short of the goal we have been. Even now, on new years day Linux geeks think to themselves, maybe this year will be the one. But of course it won't be, I mean, for a start to have a "year of Linux on the desktop" (YOLOTD) you need to actually quantify what that means.

The year of Linux on my desktop was back in 2005, when I installed Ubuntu Breezy and never looked back. But obviously that's not what the YOLOTD actually means, what it means is the year that a large percentage of desktops run a Linux based operating system. But what percentage? I guess enough to be noticed. Enough that the big developers start writing software for Linux because the audience they would be missing out on is just too large to ignore.

It's widely circulated that OSX market share is around 10% yet most developers are still developing for Windows only, with only certain products making it to the Mac desktop. That seems to imply that Linux needs more than that. I'd hazard a guess that Linux and OSX market share combined would need to hit 20% before people start really taking notice. At that point if you develop only for Windows, you are ignoring 1 in 5 potential customers.

If you were a developer in that situation, what would you do next? If you were smart you'd see which had the next highest market share after Windows and port to that platform. Aha, so to actually get a YOLOTD we don't need to compete with Windows, we need to compete with OSX. If Linux can get a higher market share on the desktop than Mac, then two things happen:

  1. The magical 20% margin for non-Windows desktops is broken

  2. Linux becomes the next target for developers

Nobody knows what Linux desktop market share is, I've heard figures vary from 1% to 8%, but for now, let's stick with a conservative 2%. So Linux desktops need an 8% increase to even "start" the YOLOTD. I believe that this will happen before the end of 2010 and here's my reasoning...

Ubuntu has become the largest desktop Linux distro in the world in a very short time. Each release adding more and more features, many for technical reasons (improved X server, Pulseaudio, automatic proprietary driver installation) but still many new users see it as too complicated, or confusing. But the last release of Ubuntu - somthing changed. A new feature was added which wasn't technical in nature, it wasn't to get something working, it was to make the experience more pleasant - the notification system.

The new notification system in Jaunty has set the ball rolling. Already for the Karmic release cycle we've already seen the 100 Paper Cuts project which in the first month or so has made real changes. The notification system is seeing lots of commits, the DX team is organized and getting work done, Canonical seem to actually have a plan. They definitely seem to be focused on a goal, and reading between the lines of Mark Shuttleworth's interviews that goal is set for completion in 10.10. A new sleek desktop that will just be pure class from boot onwards. If slicker than Mac doesn't win audience I don't know what will.

Then there is ChromeOS, Google's entry to the OS market, due out next year. Google has a habit of excelling in every venture - Picasa, Google Earth, Desktop Search, Android, Gmail I can't see this trend changing. ChromeOS will be based on Linux, so yes it counts and with Google's brand power, I believe they'll easily win 1-2% of the desktop market within a year, albeit likely via Netbooks.

Now, Linux's debut in the Netbook market was completely sabotaged by the limited crappy Xandros based OS of the eeePC which inevitably resulted in even worse copycat distros from other manufacturers meaning that by the time Ubuntu Netbook Remix came along, the Linux Netbook world was a barren wasteland with XP sitting over the bridge in green fields of victory. Things will change though, there is a place where Windows just can't go: ARM.

ARM Netbooks with long battery life and good performance will be out over the next year, with ChromeOS and UNR splitting the marketshare between them.

So here's my prediction; 2010 will be the YOLOTD, 2011 cementing it. Hopefully by 2012 balance will restored to the IT industry. It's good to be optimistic :)

Wednesday 15 July 2009

Thoughts on Wine's development model...

Scott Ritchie has just blogged about the need for a new Wine stable release. Everything he says in the post, of course, makes perfect sense. Both distributions and the Wine project itself would benefit from a new stable release. But I think the lack of releases actually stems from a more fundamental problem with Wine's development model as a whole.

The Wine repository gatekeeper is Alexandre Julliard, who is notoriously strict about the quality of the patches he accepts. To contribute code to Wine, you should mail a patch to the wine-patches mailing list, AJ and other developers watching the list will check your patches and give you lengthy feedback about how to improve your patch if it is not accepted. At least, I think that's the idea behind wine-patches. In reality, if your patch is rejected or accepted, you likely won't be notified at all. There only seem to be two ways to get feedback, firstly, if your patch happens to wonder onto the turf of a Wine developer (e.g. a subsystem they maintain), then you may get feedback from them, alternatively you can wait a day or two then pester AJ on #winehackers for feedback. The most common outcome if your patch is rejected though is that it disappears into they abyss of the wine-patches archives for an eternity of bit-rotting. This is a real shame, I would hazard a guess that 60% of patches which hit wine-patches are almost right and just need minor fixes, but are rejected. And I guesstimate 50% of those patches are never ever resubmitted and just disappear.

I believe the reason that it is so difficult to get patches into Wine is that there is NO development branch. OK, theoretically there is the 1.0.x branch which is stable and the 1.1.x branch which isn't, but releases are so rare that the stable branch is obsolete and the development branch is the branch you are supposed to use.

This leads to major problems aside from those mentioned in Scott's blog post. One of these such problems is the inability to make any major structural change to Wine.

As an example, recently a guy called Max spent time writing a DIB engine. DIB handling in Wine is quite slow, and the DIB engine is supposed to rectify that situation. Despite Max writing a working DIB engine which passed the Wine tests and showed speed improvements in several apps it was not accepted, the reasoning for which is quite sound; the design was wrong. But why was the design wrong? Why did Max spend ages writing something which he knew would never be accepted? Because to implement it properly would cause mass breakage because it required fundamental changes to Wine (specifically gdi32.dll). AJ won't accept patches which cause mass breakage because the development branch is essentially the ONLY branch.

Here's the catch 22. Wine needs regular releases to allow for big structural changes and Wine doesn't have regular releases because the criteria for the next regular release require big structural changes.

What's the solution? I think the best solution is to right now create a stable branch from the development branch. For the next 6 months, stablize that branch, add no new features just bug fixes. In the mean time open the flood gates for massive changes and not-quite-right-but-nearly-there patches into the development branch. After 6 months release a new stable version based on the stable branch and then once that release is done, sync the development branch to the stable branch to begin another 6 month stablization cycle.

This is exactly what Ubuntu does with Debian Experimental. At the start of each release cycle, sync from the unstable branch and spend 6 months stablizing it. Wine should do the same. This method has many benefits:

1. If a patch is sent that isn't quite right, then it can be accepted into the development branch anyway to be fixed up.
2. Big structural changes can happen
3. Wine development would be easier to get into, meaning more devs, meaning faster progress.

If this cycle was adopted, then some kind of semi-automated means of getting patches into wine-unstable would be nice, with AJ remaining in charge of wine-stable. I'm thinking of some website which displays the patches sent to wine-patches. Developers can vote a patch up or down and comment on it. If a patch gets 3 votes up, it automatically is committed, if it gets 3 votes down it is rejected. Wishful thinking I know, but that would be awesome ;)

Tuesday 14 July 2009

My newest project... a scene editor

So, over the last few months I've been planning an interesting 2D/3D platform game with a new camera concept, it's one of those ideas that won't go away, so I started implementing it using Irrlicht (which is a very nice engine btw) but I got stuck pretty quick when I went looking for a scene editor...

There are various ways you can generate scenes for Irrlicht, you could use Blender for example but that's overkill for me, all I want is a way to load models, place them on a heightfield and then save it. There is of course IrrEdit.. the closed, Windows only editor. Unfortunately this doesn't run under Wine, and after chatting with Stefan Dösinger one of the Wine D3D guys, it turns out, it's not easy to fix.

After Googling various methods, I thought sod it, I'll write my own. So that's what I'm doing, it's a simple, unique scene editor written in C++/OGL and wxWidgets. It's unique in that it behaves unlike any other editor I know, hopefully I'll have a preview in the next couple of weeks. The general idea is that everything is done via a single 3D view and a tree view which displays the scene graph. So far I have terrain generation (either a plain grid, or via a heightmap), object selection/hovering, a lot of underlying infrastructure and the scene tree view. Next on my list is camera movement, this is going to be particularly cool, because selecting an object will cause the camera to move so the object is focused ready for you to manipulate it. More updates to follow ;)

Oblig screenshot: