Software and hardware annotations 2011 November

This document contains only my personal opinions and calls of judgement, and where any comment is made as to the quality of anybody's work, the comment is an opinion, in my judgement.

[file this blog page at: digg del.icio.us Technorati]

111130 Wed Debian finally supports installing packages of diverse architectures

In this useful interview with the Debian project leader (Stefano Zacchiroli) he mentions that he has managed to push for the Debian package manager to be fixed so that it can handle packages with multiple architectures something that is a major limitation of DPKG with respect to the much better RPM.

As far as I can see Debian will keep the other major limitation of DPKG, which is the inability to have two packages with the same name but different versions installed, which is very useful for example for libraries.

It is notable that this project started in 2004 and after a few years of discussion and implementation it is coming to an end in 2011 (admittedly in part because it is not just the package tools that need upgrading, but very many packages).

This will be accompanied by somewhat sensible changes to the filesystem hierarchy to accomodate separate filesystem trees for multiple architectures, changes that resemble a return ancient practice before the IA32 monoculture happened, and after the Linux FHS introduced a very partial and very ill thought workaround.

111124 Thu Review of Philips 240PW9ES 24" LCD monitor

In 2009 I wanted a bigger better LCD monitor, after using two 17" LCD monitors for while which was a good arrangement but distracting because they had very different colour gamut and temperature, which was narrowly dependent on viweing angle, and in opposite ways for the two monitors; both monitors had such narrow viewing angles that moving the head a bit would shift colors (like for most laptop displays), and the cheaper one so narrow that the top and bottom of the screen had different color temperatures without moving the head. Both monitors also had 18 bit color palette which is a bit limiting, and some backlight bleeding and disuniformity.

In order to have good color fidelity with viewing angle independence an LCD monitor should be using IPS or PVA/MVA cells, which are more expensive than the usual TN cells. I also wanted a large display to compensate for using one display instead of two, and a tall one because I mostly edit text with it rather than watching movies, and a monitor with a good stand and the ability to remove it to mount the display on clampable arm like this (of which I had two for the two monitors).

Therefore I determined that I wanted a 24" 1920×1200 IPS or PVA/MVA display and business oriented monitor rather than a 22-23" 1920×1080 and consumer oriented monitor as the latter are more aimed at widescreen movies and have inflexible and often non removable stands.

There were quite a few such mid-high end monitors around, for example the Dell 2408WFP (PVA), HP LP2475w (IPS) but they all cost around £430-£450, which was around 2-3 times a lower end TN 1920x1080 23" monitor, and I was slightly hesitating until I found the Philips 240PW9ES model which was selling for around £360.

This monitor has the same IPS panel as the HP 2475w, manufactured by LG-Philips, and it cost less than the others simply because it had less features, which I did not much care about, so I bought it and I still have it. The good points are:

• The display quality is very good. This is mostly thanks to the very good LG-Philips panel (H-IPS, native 24 bit color range), but also to the bright and effective backlight, the matt surface, and a good range of contrast and brightness and color settings.
  Colors, depth of black, purity of white, smooth gradations between black and white, evenness of illumination, lack of bleeding, viewing angle are all very good, with an overall viewing experience that is impressively better than cheaper TN displays, and as good or better as displays in the same class.
• The electronics and firmware are quite good, with a sensible menu system, a good autosync system for VGA inputs, and even the very welcome ability to change the brightness of the power LED, while omitting pointless gimmicks.
  There is a useful range of display modes, in particular high brightness modes for image viewing and normal brightness for office work, where power consumption is rather lower, as well as giving a more restful viewing (the brightest mode is very bright indeed). My impression is that panel response time and input lag are also pretty good, with no obvious trailing or ghosting or delays during movies and games.
• Quality of build and design is quite good, as common for Philips products, with fairly slim borders, and a nondescript gray decor that does not distract from the display. It is controlled by a set of decent buttons that are conveniently located on the front, instead of being on the side or back (or both) as in other monitors. It does not get hot, having good ventilation, it comes in a very good box, and is light enough to be mountable on most or all monitor arms.
• The stand is very good, solid and unobtrusive, with a full range of adjustments, including portrait mode.
• The price point makes it very good value.

There are some questionable or negative points for me:

• There is a slight color gradient across the display, with slightly colder or greenuish colors (greenish) on the left and warmer or reddish on the right and perhaps slightly yellow in the middle. Other monitors based on the same IPS panel technology have been reported to be the same (even if I suspect it is actually the backlight). The overall effect is very hard to notice unless one has a full screen all white window, and even then it is not distracting.
• The W-CCFL backlight has an extended emission spectrum, which allows the display to have a gamut to be substantially wider than the sRGB standard and to cover essentially all of the Adobe RGB standard, with much wide ranges in greens and in particular reds. This would seem a significant advantage, but it means that without color management doing gamut normalization it displays rather more saturated reds (and greens) than it should, as most images are calibrated to sRGB. Under recent MS-Windows here is color management and it is easy to load the manufacturer profile to normalize the gamut, but this is not yet convenient under the X Window System. I have just put up with somewhat more intense greens and reds, and I mostly use the monitor for office work anyhow.
• The monitor has indeed relatively fewer features than more expensive models, in particular just two inputs, VGA and DVI, while more expensive monitors in the same class tend to have also HDMI and a second DVI input, and perhaps also DisplayPort and composite inputs. I don't care about this limitation. There is another lack of features that I don't care but is comical: while more expensive products in the same class tend to have a builtin powered USB hub, which can be convenient, typically with one uplink and 4-6 powered downlinks this product has a vestigial USB non-hub with one uplink and a single (unpowered) downlink. To me this seems pointless, but perhaps it had to have USB connectivity as a checklist item. Other missing features are speakers (which are usually terrible on other monitors) and flash card reader.
• The legends for the control buttons are printed in dark gray on medium gray and can be pretty difficult to read. Fortunately the OSD provides an on-screen guide, and dark gray on medium gray is better than many other monitors that have black-on-black button legends.
• While the default color temperature setup is reasonable, the sRGB mode to me seems terrible. But this may be just because sRGB has a limited gamut. Also in some versions of the firmware the movie mode seems to involve some bizarre sharpening which may or may not be useful.

Overall I am very impressed with the 240PW9ES, and I think it is amazingly good, and I have used several other monitors in the same class. For me the tradeoff of fewer features for a lower price has been a success, as I would not use those features anyhow, and the main features (image quality, build quality, stand quality, price) are very good.

Which is not surprising for a Philips product, which is one of the few brands that seems to indicate consistenly good engineering and good value, probably as the result of a corporate culture that focuses on substance more than appearance (the 240PW9ES like a lot of other Philips stuff looks very boring indeed, and I am happy with that).

Then there are the general advantages of its class: the height of 1200 pixels does help, a 24" screen is comfortably large (I can used it at a distance of 60-90cm), IPS does deliver much better and more convenient display experience.

Lesser monitors may look fine, but there is a definite improvement to monitors of the quality of the 240PW9ES, and I am still sometimes amazed to look at my monitor and just how good it is.

111120 Sun Main memory fragmentation and Linux huge pages

Depending on workload it happens fairly often that Linux based systems apparently freeze for noticeable, even long periods of time.

There must be some kind of resource overcommitment leading to long queues and thus high latency, and this must happen to a widely shared resource.

One surprising report shows that in some cases this is because of huge page allocations under memory pressure:

Once upon a time, one just had to assume that, once the system had been running for a while, large chunks of physically-contiguous memory would simply not exist. Virtual memory management tends to fragment such chunks quickly.

So it is a bad idea to assume that huge pages will just be sitting there waiting for a good home; the kernel has to take explicit action to cause those pages to exist.

That action is compaction: moving pages around to defragment the free space and bring free huge pages into existence. Without compaction, features like transparent huge pages would simply not work in any useful way.

My impression is that most of the time this is the disk, as the Linux disk elevator algorithms are far from optimal, and the disk becomes a critical resource on a desktop system usually because of the accumulation of unwritten pages in memory that are then bulk written to disk periodically.

The fix for that problem is to ensure that the amount of unwritten pages in memory is not proportional to memory size, which is the ludicrously silly default, but to to the speed of mass storage device, and I typically aim at no more than 0.5s to 1s worth of unwritten pages.

There are causes for long latencies, usually misguided optimization that batch operations, such as the laughable plugging mechanism in the Linux page cache, and some can be fixed. For example Con Kolivas has been working for a a while on a more responsive CPU scheduler, as the default one(s) because of an ancient UNIX tradition will give long running periods to background tasks.

The huge page story is not surprising in this context. As the article says, because of huge pages the Linux memory manager has to maintain pools of different sized blocks, but does not, which leads to the case where there are many ordinary pages free but they are not contiguous so a huge page cannot be allocated. This triggers a moving around of pages to reduce free page list fragmentation, and this moving around can involve pages waiting to be written out and thus long delays. This description is objectionable for several reasons:

• Huge pages are a rarely needed optimization, as they only minimize the number of TLB enties in use (which can be quite important, but only for some applications), and this optimization should not be automatically be used by the kernel, but only on application request, and the requesting applications should suffer the consequences.
• Moving around ordinary pages does not require any flushing to storage, because pages can be exchanged in memory as long as there is an empty page to use as buffer, or directly using XOR.
• Anyhow even if one makes pages available for reuse by flushing them to storage, the number of pages to be flushed should be a fraction of IO bandwith of probably well below a second of bandwith.

Indeed I have had very few problems with the system seemingly freezing since I have changed the parameters of the Linux page cache flushers to flush unwritten pages far more often than default, and I have written a Linux kernel patch to allow expressing the maximum number of unwritten pages directly instead of as a percentage of memory.