Software and hardware annotations 2011 February

This document contains only my personal opinions and calls of judgement, and where any comment is made as to the quality of anybody's work, the comment is an opinion, in my judgement.

[file this blog page at: digg del.icio.us Technorati]

110219 Sat Ubuntu boot process issues

Because of a desire to run the same OS on my personal laptop and at work I have switched from CentOS 5 IA32 to Ubuntu LTS 10.04 AMD64 on my laptop.

Ubuntu could be called the MS-Windows of GNU/Linux distributions (even if there are some that are more visually similar) for a relentless focus on superficial ease of use often at the expense of structural integrity and maintainability.

Soon I noticed soon with ULTS10 the fragility of the boot process:

These issues seem to be a combination of the poor design of the new dæmon manager Upstart and of the the boot process graphical decorator Plymouth.

The poor design is that both seem aimed at making the boot process an entertaining and cool experience. Unfortunately that to me seems misguided because the boot process is critical and its design objective should be robustness achieved through simplicity and flexibility.

The big problems above are that it is essentially impossible to boot unless /sbin/upstart works, as the decade old ability to boot into any other program has effectively been lost, and also that Plymouth works there is little information that allows diagnosing potential startup issues.

One of the problems with Upstart is that the new style scripts it uses are written in its own minilanguage, and that they communicate back to it.

A related problem is that Upstart scripts contain both the run levels at which they should be activated or deactivated, which is configuration information, and their behavioral logic an embedded shell, which is not. A site may very well want to change the run levels for a dæmon while not touching the startup scripts, which is then impossible. Indeed as the latter gets updated by Ubuntu one has to manually apply run level changes again.

The run levels also often need changing because of the very objectionable Debian policy that all dæmons are activated by default as soon as they are installed, where instead a user often wants to install a daemon to be activated only manually.

Also Ubuntu seems to be nearly unable to boot without a suitably constructed initrd, and what it should contain is largely undocumented, just like the constraints on the whole boot process. Which seems to me like an MS-Windows style as long as it just works in most cases attitude.

Indeed at some point I ended up with a non working boot process because of some initrd issue and after spending a while on it trying to figure out the issue it was easier to just reinstall Ubuntu. part of the reason is that the boot ended in a prompt of the shell used in the initrd and with the display cleared just before that, so I could not see for what reason the initrd boot sequence had stopped.

These issues caused by a disregard for robustness and integrity in favour of shallow coolness, and short sighted disregard for the UNIX philosophy of simple flexible building blocks.

In particular trying to prettify something as critical as the boot process at the price of making it more brittle seems a very bad tradeoff to me.

However it is possible to fix soem of these aspect, and there are some suggestions here from another user with similar frustrations.

110205 Sat OpenSSL speed tests as a rough CPU metric

I have used for some time the OpenSSL encryption speed tests as a very quick and rough metric of CPU power, in particular to detect CPUs running at slower-than-nominal clock rates.

Also as a very rough example comparison, here on two very different systems I got, the first is a laptop with a 2.4GHz 2-CPU I3-M370 with 4GiB in 2 sticks of 667MHz DDR2 memory, running 64b GNU/Linux (ULTS10):

OpenSSL 0.9.8k 25 Mar 2009
built on: Fri Dec  3 22:53:56 UTC 2010
options:bn(64,64) md2(int) rc4(ptr,char) des(idx,cisc,16,int) aes(partial) blowfish(ptr2) 
compiler: cc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -O3 -Wa,--noexecstack -g -Wall -DMD32_REG_T=int -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2               1426.02k     2908.30k     3913.56k     4289.54k     4427.52k
mdc2                 0.00         0.00         0.00         0.00         0.00 
md4              43099.08k   137919.91k   356871.17k   581403.57k   714773.85k
md5              34283.22k   107119.23k   254004.31k   389428.23k   458459.82k
hmac(md5)        36410.35k   112629.06k   262120.11k   392695.47k   460689.04k
sha1             33042.83k    93120.87k   197200.31k   272451.93k   307539.74k
rmd160           24252.99k    62239.40k   121103.41k   158395.05k   174506.04k
rc4             307803.27k   324677.38k   333726.82k   334818.30k   341844.76k
des cbc          46614.00k    48493.81k    48755.20k    49053.71k    48925.35k
des ede3         18448.63k    18798.23k    18873.36k    18880.30k    18729.64k
idea cbc             0.00         0.00         0.00         0.00         0.00 
seed cbc             0.00         0.00         0.00         0.00         0.00 
rc2 cbc          25407.91k    26202.01k    26176.26k    26203.82k    26280.16k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00 
blowfish cbc     78086.28k    83939.68k    85415.77k    85305.71k    85172.22k
cast cbc         62535.17k    65110.23k    65699.50k    65934.64k    65907.37k
aes-128 cbc     106594.82k   142991.15k   154974.52k   160888.15k   160055.30k
aes-192 cbc      78842.35k   114011.67k   128751.99k   133131.26k   136488.58k
aes-256 cbc      71400.88k   101909.04k   111841.96k   116355.41k   115767.48k
camellia-128 cbc        0.00         0.00         0.00         0.00         0.00 
camellia-192 cbc        0.00         0.00         0.00         0.00         0.00 
camellia-256 cbc        0.00         0.00         0.00         0.00         0.00 
sha256           23047.33k    54581.36k    97954.30k   122539.24k   132071.42k
sha512           16399.32k    65366.76k   115776.85k   173225.53k   200878.76k
aes-128 ige     134575.97k   145034.10k   149180.84k   149657.09k   147622.57k
aes-192 ige     119246.64k   125384.49k   127382.36k   126727.02k   127251.80k
aes-256 ige     104084.83k   107931.05k   109439.66k   109311.32k   110822.14k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000124s 0.000012s   8083.6  86244.5
rsa 1024 bits 0.000612s 0.000033s   1633.3  30068.1
rsa 2048 bits 0.003889s 0.000117s    257.1   8515.6
rsa 4096 bits 0.027295s 0.000433s     36.6   2310.6
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000118s 0.000128s   8499.4   7796.5
dsa 1024 bits 0.000323s 0.000372s   3093.1   2689.3
dsa 2048 bits 0.001114s 0.001340s    897.3    746.0

The second system is a desktop with a 2.8GHz 3-CPU AMD Phenom X3-720 with 2GiB (in 2 sticks) of DDR2-800 memory, running 32b GNU/Linux (EL5):

OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
built on: Mon Dec 13 14:05:02 EST 2010
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) aes(partial) blowfish(idx) 
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -I/usr/kerberos/include  -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -Wa,--noexecstack -DOPENSSL_USE_NEW_FUNCTIONS -fno-strict-aliasing
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
md2               2841.84k     5983.00k     8301.65k     9178.11k     9508.18k
mdc2                 0.00         0.00         0.00         0.00         0.00 
md4              28763.04k    95398.19k   252457.93k   429939.03k   545827.50k
md5              23981.34k    77067.20k   199529.73k   327948.97k   403387.73k
hmac(md5)        25160.48k    81999.86k   207223.55k   334116.52k   408857.26k
sha1             20568.40k    57928.51k   124274.35k   172735.83k   195578.54k
rmd160           17491.41k    46406.85k    92204.97k   122070.67k   136052.74k
rc4             112408.82k   119566.95k   121361.15k   121932.46k   121935.19k
des cbc          18613.86k    19049.66k    19159.04k    19236.18k    19127.37k
des ede3          5388.87k     5447.66k     5453.31k     5458.60k     5469.53k
idea cbc             0.00         0.00         0.00         0.00         0.00 
seed cbc             0.00         0.00         0.00         0.00         0.00 
rc2 cbc          30136.70k    31357.01k    31679.23k    31719.08k    31842.30k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00 
blowfish cbc     38330.82k    39543.79k    39810.38k    39979.35k    39996.07k
cast cbc         48525.27k    50430.34k    50837.16k    51150.17k    51074.39k
aes-128 cbc      56536.60k    57076.42k    57992.87k    58107.56k    57991.74k
aes-192 cbc      49274.15k    49236.25k    49838.59k    50101.93k    49960.28k
aes-256 cbc      40785.97k    43417.56k    43508.48k    43688.28k    43627.16k
camellia-128 cbc        0.00         0.00         0.00         0.00         0.00 
camellia-192 cbc        0.00         0.00         0.00         0.00         0.00 
camellia-256 cbc        0.00         0.00         0.00         0.00         0.00 
sha256           18042.68k    42723.54k    75967.66k    95211.86k   102582.95k
sha512            4249.07k    17000.23k    24625.92k    33884.84k    38054.57k
aes-128 ige      57684.83k    60724.82k    61841.92k    61907.97k    62125.40k
aes-192 ige      49876.98k    51829.16k    52809.56k    52989.95k    52876.63k
aes-256 ige      43518.36k    45226.47k    45730.82k    46050.30k    46011.73k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000832s 0.000068s   1202.6  14611.3
rsa 1024 bits 0.004688s 0.000229s    213.3   4370.6
rsa 2048 bits 0.029703s 0.000826s     33.7   1210.8
rsa 4096 bits 0.206531s 0.003043s      4.8    328.6
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000730s 0.000822s   1370.8   1217.2
dsa 1024 bits 0.002360s 0.002703s    423.7    370.0
dsa 2048 bits 0.008328s 0.009862s    120.1    101.4

In encryption the laptop in 64b mode is over twice as fast as the higher-clocked desktop in 32b mode. A bit of that is that Intel I3 CPUs have better CPI, but most of it is the AMD64 architecture being more efficient than the IA32 one.

For a quick result I tend to run just the blowfish speed test, for eample here on a 2.5GHz Xeon E5420 running GNU/Linux EL5 64b:

# openssl speed blowfish
Doing blowfish cbc for 3s on 16 size blocks: 14888277 blowfish cbc's in 2.88s
Doing blowfish cbc for 3s on 64 size blocks: 4099300 blowfish cbc's in 3.01s
Doing blowfish cbc for 3s on 256 size blocks: 1034519 blowfish cbc's in 2.98s
Doing blowfish cbc for 3s on 1024 size blocks: 256851 blowfish cbc's in 2.99s
Doing blowfish cbc for 3s on 8192 size blocks: 32495 blowfish cbc's in 2.99s
OpenSSL 0.9.8e-fips-rhel5 01 Jul 2008
built on: Mon Dec 13 14:10:10 EST 2010
options:bn(64,64) md2(int) rc4(ptr,int) des(idx,cisc,16,int) aes(partial) blowfish(ptr2) 
compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -I/usr/kerberos/include -DL_ENDIAN -DTERMIO -Wall -DMD32_REG_T=int -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DOPENSSL_USE_NEW_FUNCTIONS -fno-strict-aliasing -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM
available timing options: TIMES TIMEB HZ=100 [sysconf value]
timing function used: times
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
blowfish cbc     82712.65k    87161.20k    88871.43k    87965.02k    89029.78k