Solaris, HP SmartArray and data corruption on shutdown

For quite some time I had been experiencing power-off problems on a HP Proliant server running Solaris (or OpenSolaris). Most of the time, the poweroff or init 5 commands will not cut the power and the machine will hang and stay up with fans spinning at full speed. The only solution was to manually power cycle or use LOM to shut the machine down. This has caused data corruption problems for me several times, specially when the HP SmartArray batteries get discharged.

Turns out there is a HP support document, SHUTDOWN PROCEDURE REQUIRED for ProLiant Server Running Sun Solaris 10 to Properly Flush Cache on Smart Array Controller Prior to Shutdown that precisely describes my situation. It seems it’s an interaction problem between Solaris and the cpqary3 drivers (2.1.0 or older) that causes buffers in the HP SmartArray controller to not be flushed on shut down. If the batteries drain, eventually data loss might occur. And precisely, since I was using cpqary3 driver version 2.1.0, I have just upgraded to cpqary3-2.2.0 to see if this solves my problems for once and all. We’ll see 🙂

HP Proliant DL180 G6 and ZFS (part V)

Once I got the OpenSolaris installation kicked I faced another problem. The computer has 3 x 1.5TB and 1 x 2TB disks installed. This is suboptimal when configuring a RAID volume but I thought that I could cheat here. The cheat consists of slicing the 2TB disk in a 1.5TB slice and using the remaining 0.5TB to store the ZFS rpool.

As nice as it sounds this turned out tobe a little bit tricky. More than I expected. The challenge was convincing the OpenSolaris installer to create one 0.5TB slice and another 1.5TB slice. But I could not: OpenSolaris creates a partition and creates a slice that fills it up entirely. So, I either got a 2.0TB partition and a 2.0TB slice or a 0.5TB partition and a 0.5TB slice. In the end, what I did is to create a 0.5TB partition leaving the rest of the disk unused. The OpenSolaris installer created a SOLARIS2 partition and, inside it, a 0.5TB slice for the ZFS rpool. Once the installation was finished, I booted from a Ubuntu 9.04 LiveCD. As usual, Ubuntu did not require any driver disk or updates: it recognized the HP Smart Array controller automatically. Then I used fdisk to destroy and recreate the SOLARIS2 partition but this time I recreated it to cover the whole disk. I had to make sure the partition was makred active or GRUB would not boot. Since OpenSolaris doesn’t like multiple partitions of SOLARIS2 (hexadecimal bf) type, this is the only way I can think of to do what I want.

Next step was to boot from the OpenSolaris x86 LiveCD, apply the driver update and check that fdisk reported a single Solaris partition that fills the entire 2TB disk. Then I summoned format to create a new s3 slice that covered the remaining 1.5TB. This is how the slices look like now:

Current partition table (original):
Total disk cylinders available: 60796 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0       root    wm       1 - 15198      465.69GB    (15198/0/0)  976623480
  1 unassigned    wm       0                0         (0/0/0)              0
  2     backup    wu       0 - 60795        1.82TB    (60796/0/0) 3906750960
  3 unassigned    wm   15199 - 60795        1.36TB    (45597/0/0) 2930063220
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
  8       boot    wu       0 -     0       31.38MB    (1/0/0)          64260
  9 unassigned    wm       0                0         (0/0/0)              0

Then, I only had to apply a label to the remaining 3 x 1.5TB disks and create the corresponding s3 slices on each of them. All s3 slices are about the same size:

Current partition table (original):
Total disk cylinders available: 60796 + 2 (reserved cylinders)

Part      Tag    Flag     Cylinders         Size            Blocks
  0 unassigned    wm       0                0         (0/0/0)              0
  1 unassigned    wm       0                0         (0/0/0)              0
  2     backup    wu       0 - 60795        1.36TB    (60796/0/0) 2930063220
  3 unassigned    wm       0 - 60795        1.36TB    (60796/0/0) 2930063220
  4 unassigned    wm       0                0         (0/0/0)              0
  5 unassigned    wm       0                0         (0/0/0)              0
  6 unassigned    wm       0                0         (0/0/0)              0
  7 unassigned    wm       0                0         (0/0/0)              0
  8       boot    wu       0 -     0       23.53MB    (1/0/0)          48195
  9 unassigned    wm       0                0         (0/0/0)              0

And finally,

# zpool create -f zfs raidz1 c9t0d0s3 c9t1d0s3 c9t2d0s3 c9t3d0s3
# zpool list
NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
rpool   464G  5.50G   458G     1%  ONLINE  -
zfs    5.44T   152K  5.44T     0%  ONLINE  -

HP Proliant DL180 G6 and OpenSolaris (part IV)

After all my failures trying to get this HP computer to work, I decided that perhaps OpenSolaris was the solution. And it turned out I was right. OpenSolaris x86 LiveCD uses a 64-bit kernel by default. The tricky part was to get the driver loaded before starting the OpenSolaris installation. And to my fortune, I read Installing OpenSolaris 2008.11 on a server with a HP Smart Array Controller. The post was extremely useful.

To keep it short, download the cpqary3 driver from the HP Web site. Or, if you trust in me (you should always download drivers from the manufacturer’s Web site), I also keep a copy of the 2.2.0 driver that can be downloaded from here.

Next,

$ cd /tmp
$ tar zxvf CPQary3-2.2.0-solaris10-i386.tar
$ cd CPQary3-2.2.0-solaris10-i386
$ pfexec touch /ADD_DRV_IGNORE_ROOT_BASEDIR
$ pfexec pkgadd -d ././CPQary3.pkg
The following packages are available:
1 CPQary3 HP Smart Array Controller Driver
(i386) 2.2.0, ...

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]:

Processing package instance from

HP Smart Array Controller Driver(i386) 2.2.0,...
Copyright 2009 Hewlett-Packard Development Company, L.P.
## Executing checkinstall script.
Using as the package base directory.
## Processing package information.
## Processing system information.
11 package pathnames are already properly installed.
## Verifying package dependencies.
## Verifying disk space requirements.
WARNING:
The /usr filesystem has 0 free blocks. The current installation
requires 158 blocks, which includes a required 150 block
buffer for open deleted files. 158 more blocks are needed.

WARNING:
The /usr filesystem has 0 free file nodes. The current
installation requires 26 file nodes, which includes a required 25
file node buffer for temporary files. 26 more file nodes are needed.

Do you want to continue with the installation of [y,n,?] y
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with
super-user permission during the process of installing this
package.

Do you want to continue with the installation of [y,n,?] y

Installing HP Smart Array Controller Driver as

## Installing part 1 of 1.
/kernel/drv/amd64/cpqary3
/kernel/drv/cpqary3
/kernel/drv/cpqary3.conf
/usr/share/man/man7d/cpqary3.7d
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
[ verifying class ]
## Executing postinstall script.

Installation of partially failed.

Even if the installation seems to fail the key consists of checking if the driver itself has been loaded:

$ modinfo
modinfo | grep cpq
 53 fffffffff79a6000  129f8 265   1  cpqary3 (HP Smart Array Driver Ver 2.2.0)

If modinfo shows a line like that it means the driver has been loaded and the OpenSolaris installer should see all the logical volumes.

HP Proliant DL180 G6 and Solaris 10 (part III)

And now it is the time to try Solaris 10 on this HP machine. The first thing to take into account is that the HP SmartArray P212 controller requires a closed-source binary from HP that is not distributed in the Solaris media. The cpqary3 driver can be downloaded from the HP Web site, although it is not very easy to find it out.

I downloaded the archived file for the cpqary3 driver, extracted its contents and burnt the ISO file to a CD-RW. Then I booted from the Solaris 10 DVD and applied the driver update. However, this didn’t work very well. The Solaris 10 DVD uses a 32-bit kernel which, apparently, does not like my >1.5TB disks. When the cpqary3 driver gets it prints a message to the console complaining that the detected disks have too many blocks for a 32-bit kernel. Very weird. I’m not entirely sure what a 32-bit kernel has to do with 64-bit LBA addressing but, so far, I haven’t been able to get Solaris 10 to recognize and get to like my disks.

So far, so good. VMware ESXi 4.0 didn’t work. Solaris 10 didn’t work either. Perhaps some of you, that are more experienced with Solaris than me, know how to fix this problem. In the mean time, I decided to take OpenSolaris for a spin. Fortunately, the final outcome with OpenSolaris has been a success. More on that later.

Nouveau: Open Source 3D acceleration for nVidia cards

Recently I stumbled on the Nouveau WiKi. Nouveau is a community-driven effort to develop open source X.org drivers for nVidia graphics cards:

“nouveau” [nuvo] is the french word for “new”.

The Nouveau community is reverse engineering the closed-source, privative nVidia driver. They claim that nothing currently works, but that’s not exactly true. For example, you can check in the status page that the 2D driver is working, but no 3D features are working yet.

Particularly, I find this proyect extremely interesting and fascinating. As a devoted fan of AIGLX and Beryl, I have been looking for fully-functional, performance-wise, open-source drivers for my nVidia graphics cards. The ATI open-source driver is an example of a nice job done good, but its performance is not as good as the closed fglrx driver yet.

I’m really looking forward to see what the Nouveau community can deliver and will help as much as I can, either by investing cash or by investing my testing efforts.

Basic backlight support for MacBook Pro

I made some modifications to the original bl1.c program from Nicolas Boichat. In summary, this modifications allow:

Specify an absolute brightness value

Let x be the absolute brightness value, such that 0<x<16, that is, the value must between 1 and 15, where 1 is the minimum brightness level before turning off the screen, and 15 is the maximum brightness supported by the LCD screen.

For example:

./bl1 15
./bl1 1

Specify an increment or decrement (delta)

Let d be the delta, and x the current brightness value, then the new brightness value y is y=x+d, and 0<y<16.

y = min(max(1, x + d), 15)

Source code

/*
 * Apple Macbook Pro LCD backlight control
 *
 * Copyright (C) 2006 Nicolas Boichat <nicolas @boichat.ch>
 * Copyright (C) 2006 Felipe Alfaro Solana <felipe_alfaro @linuxmail.org>
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful,
 * but WITHOUT ANY WARRANTY; without even the implied warranty of
 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 * GNU General Public License for more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program; if not, write to the Free Software
 * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 *
 */

#include <stdio .h>
#include <sys /io.h>
#include <stdlib .h>

void init()
{
	if (ioperm(0xB2, 0xB3, 1) < 0)
	{
		perror("ioperm failed (you should be root).");
		exit(2);
	}
}

int get_current_value()
{
	outb(0x03, 0xB3);
	outb(0xBF, 0xB2);
	char t = inb(0xB3) >> 4;
	return t;
}

int calculate_new_value(const char *arg)
{
	int val, new = atoi(arg);

	if (arg[0] == '+' || arg[0] == '-')
		val = new + get_current_value();
	else
		val = new;

	if (val > 15)
		val = 15;
	else if (val < 1)
		val = 1;

	return val;
}

int main(int argc, char** argv)
{
	if (argc > 2)
	{
		printf("Usage:n");
		printf("%s : read current valuen", argv[0]);
		printf("%s value : write value [0-15]n", argv[0]);
		exit(1);
	}

	init();

	if (argc < 2)
	{
		printf("Current value : %dn", get_current_value());
		exit(0);
	}

	if (argc == 2)
	{
		int value = calculate_new_value(argv[1]);
		outb(0x04 | (value << 4), 0xB3);
		outb(0xBF, 0xB2);
		printf("new value: %dn", value);
	}

	return 0;
}

SMC 2835W V3 and SMC 2802W V2

Yesterday I bought a SMC EzConnect 54g Wireless CardBus adapter for my Linux laptop. There are, at least, two different and incompatible versions of this card. The SMC 2835W V2 uses the Intersil 3890 Prism54 chipset, which is supported natively by Linux Prism54 driver. The SMC 2835W V3, which is the one I bought, isn’t. Fortunately, ndiswrapper fully supports this card by wrapping native Windows NDIS drivers around a linux kernel driver. Also, the same happens with the EzConnect 54g Wireless PCI adapter I bought for my Pentium IV machine, which sports the SMC 2802W V2 chipset. The Prism54 Linux driver doesn’t support this card yet.

The SMC 2802W card I bought looks like this:

SMC2802W V2

The SMC 2835W V3 CardBus NIC I bought looks like this:

SMC2835W V3

In order to use these cards, I downloaded the latest tarball for ndiswrapper from the ndiswrapper site which, at this time, was version 1.1. The tarball comes with a Red Hat .spec file that requires no tweaking, except updating the version tag to 1.1.

rpmbuild -ba ndiswrapper.spec

This will compile the ndiswrapper userspace tools and kernel module against the current kernel source tree which must be available under “/lib/modules/`uname -r`/kernel” (for Fedora Core 4 and later, there exists a kernel-devel package which contains all kernel source files needed to compile out-of-tree kernel modules).

Once compiled, I installed the resulting ndiswrapper and kernel-module-ndiswrapper RPM packages. Next, I copied the Windows XP NDIS driver directly from the CD-ROM supplied with the card itself to the hard disk. The driver is fully contained within the Utility folder inside the CD-ROM:

cd /tmp
mount /media/cdrom
cp -Rdp /media/cdrom/Utility .
cd Utility
ndiswrapper -i SMC2835W.INF

These steps are required to pick the Windows NDIS driver and firmware and make them available to the ndiswrapper kernel module. In fact, the driver files will get copied into “/etc/ndiswrapper/” where equals “2802w” for the SMC 2802W PCI NIC or “2835w” for the SMC 2835W CardBus NIC.

NOTE: The SMC 2835W V3 Windows driver can also be obtained from this link if desired. The SMC 2802W V2 Windows driver can be obtained from this link.

The last step consists in removing the prism54 module that gets autoloaded by udev/hotplug when the card is plugged into any CardBus slot, then modprobing the ndiswrapper kernel module:

modprobe -r prism54
modprobe ndiswrapper

If everything is ok, something like this should get logged to the kernel dmesg ring:

ndiswrapper version 1.1 loaded (preempt=no, smp=no)
ndiswrapper: driver smc2835w (SMC,04/29/2004, 3.0.11.1) loaded
ndiswrapper: using irq 10
wlan0: ndiswrapper ethernet device 00:04:xx:xx:xx:xx using driver smc2835w, configuration file 1260:3890.5.conf
wlan0: encryption modes supported: WEP, WPA with TKIP, WPA with AES/CCMP

Also, it’s possible to use the following command to check the driver loaded:

ndiswrapper -l

Once everything is working, the last steps try to find a wireless network, associate with it and configure the network stack:

iwlist wlan0 scan | grep ESSID

We should choose among the listed ESSIDs or, if the ESSID broadcasting is disabled, an specific, known ESSID.

iwconfig wlan0 essid 
ifconfig wlan0 up
dhclient

To make the changes permanent, the following command should be used:

ndiswrapper -m

This last command usually appends the following line to “/etc/modprobe.conf”:

alias wlan0 ndiswrapper

This tells the initscripts to modprobe for the ndiswrapper module when trying to bring the wlan0 interface up. Usually, the initscripts do a modprobe during boot, where equals the kernel’s interface name, like eth0, eth1 or wlan0. The previous alias line makes “modprobe wlan0” to become “modprobe ndiswrapper”.

It’s also recommended to add prism54 to udev/hotplug’s blacklist to prevent it from loading on systems where both ndiswrapper and prism54 kernel modules are present:

echo prism54 >> /etc/hotplug/blacklist