Software and hardware annotations q3 2005

• By unmounting the filesystem and then remounting it before each operationg the filesystem blocks are eliminated from the buffer cache. This greatly improves reproducibility, as it ensure that each test begins with no blocks already in memory. However, part of a filesystem design is how well it caches things, in particular metadata, so starting with no cached blocks is both less realistic and favours filesystems that don't cache that well.
• In the tests, for example the find one, the time taken is reported with time find... which does not fully report the time taken by the test; for example with find the atime of all inodes is updated, and this means than when find ends, there will be many updates inodes in the block cache yet to be written to the disk. Similarly for data blocks in time tar x.... The problem here is that is it pretty difficult to account reproducibly for the extra time to fully complete the operation, and that not accounting for it helps file system designs that cache more aggressively.

  	/* Use a smaller cluster for small-memory machines */
	if (megs < 16)
		page_cluster = 2;
	else
		page_cluster = 3;

int valid_swaphandles(swp_entry_t entry, unsigned long *offset)
{
	int ret = 0, i = 1 << page_cluster;
	unsigned long toff;
	struct swap_info_struct *swapdev = swp_type(entry) + swap_info;

	if (!page_cluster)	/* no readahead */
		return 0;
	toff = (swp_offset(entry) >> page_cluster) << page_cluster;
	if (!toff)		/* first page is swap header */
		toff++, i--;
	*offset = toff;

	swap_device_lock(swapdev);
	do {
		/* Don't read-ahead past the end of the swap area */
		if (toff >= swapdev->max)
			break;
		/* Don't read in free or bad pages */
		if (!swapdev->swap_map[toff])
			break;
		if (swapdev->swap_map[toff] == SWAP_MAP_BAD)
			break;
		toff++;
		ret++;
	} while (--i);
	swap_device_unlock(swapdev);
	return ret;
}

<?xml version="1.0"?>
<!DOCTYPE rss
 PUBLIC "-//IDN Netscape.com/DTD RSS 0.91//EN"
 "http://My.Netscape.com/publish/formats/rss-0.91.dtd">
<rss version="0.91">
</rss>

<?xml version="1.0"?>
<!DOCTYPE rss
 PUBLIC "-//IDN Silmaril.IE/DTD RSS 2.0//EN"
 "http://WWW.Silmaril.IE/software/rss2.dtd">
<rss version="2.0">
</rss>

<!ELEMENT channel
  ((title|link|description)+,
    (language|copyright
    |managingEditor|webMaster|pubDate|lastBuildDate
    |category|generator|docs|cloud|ttl|image
    |textInput|skipHours|skipDays)*,
   item+)>

<!ELEMENT item
  ((title|link|description)+,
   (author|category|comments|enclosure|guid|pubDate|source)*)>

<?xml-stylesheet type="text/css"
  href="style/rss.css"?>

<script xmlns="http://www.w3.org/1999/xhtml" type="text/javascript"
    src="style/rss.js"></script>

• After an extended period of really heavy web browsing it turns out that JFS by itself does not quite bring much improved responsiveness beyond the effect of having a freshly loaded filesystem.
• Setting the page clustering parameters seems instead to result in a large improvement in responsiveness when memory is tight and there is quite a bit of memory swapped out.

• MALLOC_TOP_PAD_: if the heap as to be grown at its end, add this much to the allocation in bytes.
• MALLOC_TRIM_THRESHOLD_: if the heap has more than these many free bytes at its end, shrink it.
• MALLOC_MMAP_MAX_: maximum number of blocks to allocate allocate via mmap.
• MALLOC_MMAP_THRESHOLD_: blocks of this size or larger (in bytes) are allocated via mmap.

So it provides scaling of the disk filesystem size and it allows you to make larger and larger filesystems without the fsck penalty, but it's not designed to add things like very, very fast, scalable directories (to Linux) to EXT2.
It's not designed to provide extremely efficient support for extremely large files and extent-mapped files and this type of thing that people have been talking about for EXT2. But really the goal is to provide that one piece of functionality alone.
And one of the most important goals for the whole project was to provide absolute, total, complete backwards and forwards compatibility between EXT2 and EXT3.

• mmap does not deal very well with non default block sizes that are smaller than the size of a page.
• If mmap sometimes cannot deal at all with file blocks being smaller than pages then file IO will fall back to read and write via the buffer cache. Perhaps it turns out that this is not well tuned because most people use page-sized file system blocks.
• There is something demented in either the VM subsystem or the memory allocator in GNU LIBC that handles very poorly the case where file system block size is smaller than the CPU page size. Recent versions of the allocator are supposed to turn large allocations into their own memory segments, and indeed pmap shows quite a few 4KiB, 8KiB and 12KiB anonymous mappings in existence for Konqueror (but there is a single 59KiB anonymous mapping, presumably the main allocator arena).
• For executables, which are presumably mmaped into memory on exec, the alternative is to read them into memory where they get reblocked into 4KiB pages that then get swapped out. The dreadful suspicion is:
  • copy-on-write applies to mmap'ed executables, which means all processes running the same executable share the same pages (minus the copy-on-write ones), whether or not they descend from a common ancestor;
  • for executables that have been read on exec that creates a distinct copy.

• JFS only allocates space for most metadata as needed, while ext allocates most statically and usually one makes sure it is overallocated.
• For the metada that ext3 allocates dynamically, the indirect blocks in the file space tree, JFS uses extents instead.
  While the number of indirect blocks is solely a function of the number of blocks in a file, the number of extents in a file under JFS depends also on how contiguous is the free space area.
  Since the JFS numbers above are for a load into an empty file system, in which the free space area is entirely contiguous, it is likely that most if not all files will be described by a single extent.

• JFS can be after all more space efficient than ext3 even when the latter has a smaller block size.
• If there are many large files, ext3 with a larger block size might take less space than with a smaller block size, because the internal fragmentation at the tail is less important, and many less indirect blocks are needed because the file is chunked in many less data blocks, or in other words a lower number of bigger fixed size extents.
• The available space reported in a JFS file system is not quite the same as that for an ext3 file system, as part of it will be taken by the metadata of newly added files, which is mostly preallocated for ext3.
• As the free space area becomes less contiguous, the JFS filesystem will have rising metadata overheads because of an increased number of extent descriptors used, up to the limit of needing an extent descriptor per each 4096 byte basic block.
• While reloading a file system into a freshly made filesystem does wonders for ext3 speed, it is likely that it also increases the available space under JFS, as files that previously needed several extent descriptors end up in a single extent.
• In order to increase the chances of proximity in allocating inodes and data blocks, ext3 should not be fully allocated, its available space should never fall too low; indeed I think that the default 5% reserve is way too low, considering the sevenfold slowdown over time possible. The same probably applies to JFS, and doubly so, as a larger available free space reserve raises the chances that longer contiguous extents are found, and therefore that both speed and space occupied are better.

However, the disk hasn't actually written out its cache yet. It lied to the OS / file system and said it had, but it hasn't, it's busy doing something else. Poof, the power goes out.
Now, the journal doesn't have our data, we've already cleared it out, and the file system, which is supposed to have been coherent because we fsynced it, is not, and it is now corrupted.
I have reproduced this behavior a dozen or more times on IDE based systems. The only way to stop it is to tell the drive to stop using it's write cache.

I've been using ReiserFS _EXCLUSIVELY_ since about 2.4.11 and I've never had a single problem. It's important to format with the defaults and not specify 'special' arguments to mkreiserfs or you can run into trouble.

• A PC with an Athlon XP 2000+ and 512MB.
• Linux kernel 2.6.13 with no gui, otherwise quiescent.
• A 4GB partition on a 160GB 7200rpm 100MHz ATA hard drive.
• A .tar.gz of a SUSE 9.3 root filesystem (3,132,712,960 bytes uncompressed, 173,759 entries), chosen because it contains a lot of small files and a number of fairly large files.
• The operations of restoring the filesystem, re-tar-ing it, finding a file based on a non-name property, and deleting all files in the filesystem.

• The system times above are probably not complete, as they related only the overheads directly incurred.
• The repack, find, delete times above are for cleanly loaded data; it would be interesting to see how performance would degrade after a lot of file additions and deletions, which would presumably result in a less optimal layout.
• The elevator used was the anticipatory one (except for UDF, for which cfq was used); some tests with the cfq elevator show slightly increased elapsed times, not surprising as the anticipatory optimizes throughput at the expense of latency, the viceversa for cfq.
• The numbers for ext3 were with data=writeback, but tests showed that with data=ordered the restore took only a little more time, so the latter, which is the default, is good. With data=journal the restore took 40% more time.
• The specific commands given were:

  # gunzip -dc /tmp/SUSE.tar.gz | (time tar -x -p -f -)
  # (time tar -c -f - .) | cat > /dev/null
  # (time find * -type d -links +500)
  # (time rm -rf *)

  and the execution of each was preceded by unmounting, flushing the buffer cache, and remounting; for restoring the archive to be restored was on a different drive.
• JFS (because of its OS/2 lineage) has a unique feature, case independent file name matching (which can only be enabled when the filesystem is created).
  This of course is not POSIXly correct, but it can be a (rather large) advantage for filesystems exported via Samba.
• The UDF filesystem crashed on deletion; also mkudffs supports blocks sizes of 1KiB, 2KiB and 4KiB, but the udf system module only supports 2KiB.
• The ReiserFS and Reiser4 were not tested at all.

• ext3 for a small desktop machine seems the best bet overall, and the choice is between a somewhat faster 4KiB block and a rather more space efficient 1KiB version. Considering the availability of a lot of tools (including MS Windows drivers) for ext[23] that are not available for other filesystem types, this impression is reinforced (050917 update).
  The one possibly large weakness of this filesystem is that directories are linearly scanned by default. There is now an option to have them as trees, but if directory size is an issue then I'd suggest JFS.
• UDF has been included as an oddity, for one thing it is not journaled; but then it is log based, which gives largely the same advantages.
  UDF amazingly is the smallest in code and still puts in a fairly credible performance. It is slow at writing, but probably this could be optimized; the UDF filesystem code feels less mature than the others.
• It is utterly amazing to me that the code for the XFS filesystem amounts to half a megabyte.
• JFS and XFS are a bit slower (but still quite reasonable), but other tests show they perform much better under highly parallel setups. Of the two JFS is slightly slower in elapsed time, and XFS has much higher CPU overheads. Those other tests show that XFS scales a bit better. I would still prefer JFS in most cases because of smaller code and because at high throughput rates higher CPU overhead could matter a lot more.

• As to filesystems, ext3 for most configurations (especially desktops, in particular if dual booting with MS Windows), but JFS for large partitions and/or for systems with several processors and RAID with many discs, and perhaps switch to XFS for really large numbers of processors and disc arrays (051003 update).
• As to elevators, cfq for desktops as it minimizes latency, as when throughput is more important (but is not good on subsystems with many heads, like RAIDs), and the deadline elevator for DBMSes (best for random access patterns); noop is good for storage subsystems with their own intrinsic scheduling.

• The ext2 usually has awesome performance for almost anything, but does not journal, so bad news for large filesystems.
• The ext3 has pretty good performance across the board except that since it uses kernel based coarse locking (050916 update), as locking being particularly essential for the journal, it does not scale well to highly parallel hardware and process configurations (presumably because locking the journal becomes a bottleneck, as ext2 scales well).
  It is probably by far the best for desktop style computers (050917 update); for larger systems it can result in very long fsck times in part due to having index based directories disabled by default (which is right, because ext3 is designed to be simple, and index based directories are sort of unnatural for it, even if they are now available).
• JFS and XFS both seem rather more scalable, because they are designed for high degrees of both internal (their own fine grained locking mechanism) and external (ability to work with several drives) parallelism.
  XFS seems even more scalable than JFS, but at lower levels of scale the advantage is eaten by the high overheads of XFS. XFS is also awesomely complex code, and even if it is mature, well tested code, that worries me.
• Elevator choice can have a truly dramatic impact on performance even greater than filesystem choice.

 USER     PRI  NI  VIRT   RES   SHR S CPU% MEM% Command
 pcg       15   0 64192 38788  8120 S  0.0  2.6 kicker [kdeinit]
 pcg       15   0 62412 38960  7076 S  0.0  2.6 konsole [kdeinit]
 pcg       15   0 60752 37108  7160 S  0.0  2.5 konsole [kdeinit]
 pcg       15   0 46908 27868 10480 S  0.0  1.9 konsole [kdeinit]

 USER     PRI  NI  VIRT   RES   SHR S CPU% MEM% Command
 pcg       15   0 34064 17656 13520 S  0.0  4.3 kicker [kdeinit]
 pcg       15   0 33200 17072 12508 S  0.0  4.1 konsole [kdeinit]
 pcg       15   0 32460 16124 12292 S  0.0  3.9 konsole [kdeinit]
 pcg       15   0 32452 16116 12292 S  0.0  3.9 konsole [kdeinit]

What we did witness and it seems to be the case with most of the media we tested, is that they all need a couple writes/erasures in order to "settle in" after which we had lower levels and fewer errors.

dvd+rw-format -force=full /dev/hdg

#!/bin/bash

: "'stdin' encrypted with a random password"

case "$1" in '-h')
  echo 1>&2 "usage: $0 [ARGS...]"
  exit 1;;
esac

if test -x '/usr/bin/aespipe' -o -x '/usr/local/bin/aespipe'
then
  NOISE_KEY="/tmp/noise$$.key"
  trap "rm -f '$NOISE_KEY'" 0
  dd 2>/dev/null if=/dev/random bs=1 count=40 of="$NOISE_KEY"
  exec 3< "$NOISE_KEY"
  aespipe -e aes256 -p 3
else
  export MCRYPT_KEY MCRYPT_KEY_MODE
  MCRYPT_KEY="`dd 2>/dev/null if=/dev/random bs=1 count=32 \
	       | od -An -tx1 | tr -d '\012\040'`"
  MCRYPT_KEY_MODE='hex'
  exec mcrypt -b -a twofish ${1+"$@"}
fi

Another issue is that the new 16x DVD-RAM media do not support a high overwriting cycle, which means that the discs will perform the best before 10,000 overwrites (100,000 for the 1x, 2x, 3x media).

~D(libc6~V2\.3\.5|!~D(libc6~V2\.3\.5))

Use uhci_hcd or ehci_hcd, but never both at the same time. ehci_hcd will work with all lo-speed ports, so uhci_hcd is then no needed.

> Alloced memory amount grew to ~2266 Mb (from 1433 Mb
> before) but allocation speed dropped significantly
> (several times).

Were you building the i686 glibc?  I.e. rpmbuild --target
i686 -ba -v glibc.spec?

stop using cmpxchg on multiproc systems, doesn't scale.. 10x slower on 4p, 100x slower on 32p.

A Poltergeist in My Plasma TV
LG's $5,000 set worked for a month. Then things got weird as the unit developed a disturbing "memory leak"

we expect that our malloc will find more bugs in software, and this might hurt our user community in the short term. We know that what this new malloc is doing is perfectly legal, but that realistically some open source software is of such low quality that it is just not ready for these things to happen

-adobe-courier-medium-r-*-*-*-100-0-0-*-*-iso8859-*

/*
 * XXX - we'll want this as long as bitmap instances are prevalent
 * so that we can match them from scalable fonts
 */
if (res.x_resolution < 88)
    res.x_resolution = 75;
else
    res.x_resolution = 100;

You can't use 'Package: *' and 'Pin: version' together, it's nonsensical.

Package: *
Pin: release o=Debian,a=sarge
Pin-Priority: 990

Package: *
Pin: version *sarge*
Pin-Priority: 990

CSound

Quite impressive free music synthesis system. It is quite ancient, but it is still being improved. It has two programming languages, one high level, one low level, and astonishingly musicians seem to be not bothered by the latter, which is assembler-level.

Kamaelia

The issue is massive multimedia streaming, with a 10 year horizon.

• BBC currently delivers 10-50,000 concurrent streams, 1-6 million streams served per day.
• With RealAudio they have to pay per stream licensing costs, and 1,000 times growth anticipated.
• Mass distribution: P2P and multicast. P2P starts slow, but multicast can be used to seed it much more quickly.
• CPUs are not growing in speed as fast as storage and network speeds.
• New codecs (like Dirac) needed for growth.
• Concurrency is the future, and it is easy.
• BBC is a broacaster, more precisely a program maker, however we have to develop technology. Making technology available helps with feedback and standards.
• Real has been used only because it was most scalable and platform independent at the time it was chosen, even if proprietary.
• Kamaelia is a componentized, based on an architecture with scheduler and other infrastructure, and then data flow. It has a plugin framework, and a nice GUI frontend for the pipeline/graphline.
• Concurrency can be tamed with read only stuff or single reader/single writer pipeline/graphline.
• All written in Python, with growing C/C++ bits.
• It is not very mature yet, so for production you may want to look into Fluendo's Flumotion which is based on the Twisted framework/ and GStreamer.
• It has been released as free software to expose it to actual usage and garner feedback.

The Rosegarden sequencer and Studio To Go!

Nice tutorial on how to use these MIDI tools.

Access Space

Access Space is an open lab for creative digital activities in Sheffield. It has been set up for creative artists (a euphemism for unemployed people, the speaker said) and they have both availability of free software based technology and courses and seminars.
Studio To Go! is a distribution packaged with smoothly working GNU/Linux audio applications, a bit like aGNUla's DeMuDi; a couple of users present said that they preferred it even to DeMuDi.

A presentation on open source film making

I started to be a bit busy with user enquiries, so I was not following that much. From what I could glimpse the two themes were that free software is one of the enabling technologies for a lot of grassroots film making. I was reminded of the StarWars Revelations movie, which is a pretty good example.

• Since C (and C++) were designed for static libraries, they have extremely poor (or nonexistent) primitives for source and binary interface handling. Since the UNIX style toolchains prevalent today were also designed for static libraries they also don't offer much to handle the issues created by shared libraries.
• In particular for Debian, since both its DPKG package manager and (in particular) its APT dependency manager not only do not handle the notion of dependency on an ABI as a consequence of the previous point, but also do not handle having installed multiple versions of the same package, which means that different ABI versions of a library must be packaged with different names.

That simple program uses 73 shared libraries that allocate a total of 13Mb of non-shared static data.

• Prelinking by the user means that all shared objects and binaries involved get modified, which means that all the checksums held by the package manager become invalid.
  Since I reckon it is pretty valuable to verify that executables and libraries have not been modified after installation, this is a major difficulty.
  As to this, Gentoo handles this difficulty:

  Current versions of Portage can handle, via prelink, the changing MD5sums and mtimes of the binaries.

  but this is easier because of the source-based nature of Portage.
• Prelinking actually would not help me a lot, because I use mostly KDE and its developers have implemented a completely different workaround to the same problem, which works well enough, even if it is less elegant.
  The workaround is to have most KDE apps as parts (shared objects) mapped into processes forked off running the kdeinit service, which already has mapped most of the KDE libraries. This means that the child process inherits the already mapped in and prelinked shared libraries of the parent. Thus in effect prelinking is done at the start of every session instead of statically, but that is good enough, especially as few systems run multiple sessions at a time.
• Some distributions have started using an additional technique to reduce dynamic linking cost, which is to mark entries internal to a shared object as not globally visible using the new -fvisibility option to GNU CC, which especially helps with C++ code.