For quite some time I had been experiencing power-off problems on a HP Proliant server running Solaris (or OpenSolaris). Most of the time, the poweroff or init 5 commands will not cut the power and the machine will hang and stay up with fans spinning at full speed. The only solution was to manually power cycle or use LOM to shut the machine down. This has caused data corruption problems for me several times, specially when the HP SmartArray batteries get discharged.

Turns out there is a HP support document, SHUTDOWN PROCEDURE REQUIRED for ProLiant Server Running Sun Solaris 10 to Properly Flush Cache on Smart Array Controller Prior to Shutdown that precisely describes my situation. It seems it’s an interaction problem between Solaris and the cpqary3 drivers (2.1.0 or older) that causes buffers in the HP SmartArray controller to not be flushed on shut down. If the batteries drain, eventually data loss might occur. And precisely, since I was using cpqary3 driver version 2.1.0, I have just upgraded to cpqary3-2.2.0 to see if this solves my problems for once and all. We’ll see 🙂

After all my failures trying to get this HP computer to work, I decided that perhaps OpenSolaris was the solution. And it turned out I was right. OpenSolaris x86 LiveCD uses a 64-bit kernel by default. The tricky part was to get the driver loaded before starting the OpenSolaris installation. And to my fortune, I read Installing OpenSolaris 2008.11 on a server with a HP Smart Array Controller. The post was extremely useful.

To keep it short, download the cpqary3 driver from the HP Web site. Or, if you trust in me (you should always download drivers from the manufacturer’s Web site), I also keep a copy of the 2.2.0 driver that can be downloaded from here.

Next,

$ cd /tmp
$ tar zxvf CPQary3-2.2.0-solaris10-i386.tar
$ cd CPQary3-2.2.0-solaris10-i386
$ pfexec touch /ADD_DRV_IGNORE_ROOT_BASEDIR
$ pfexec pkgadd -d ././CPQary3.pkg
The following packages are available:
1 CPQary3 HP Smart Array Controller Driver
(i386) 2.2.0, ...

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]:

Processing package instance from

HP Smart Array Controller Driver(i386) 2.2.0,...
Copyright 2009 Hewlett-Packard Development Company, L.P.
## Executing checkinstall script.
Using as the package base directory.
## Processing package information.
## Processing system information.
11 package pathnames are already properly installed.
## Verifying package dependencies.
## Verifying disk space requirements.
WARNING:
The /usr filesystem has 0 free blocks. The current installation
requires 158 blocks, which includes a required 150 block
buffer for open deleted files. 158 more blocks are needed.

WARNING:
The /usr filesystem has 0 free file nodes. The current
installation requires 26 file nodes, which includes a required 25
file node buffer for temporary files. 26 more file nodes are needed.

Do you want to continue with the installation of [y,n,?] y
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with
super-user permission during the process of installing this
package.

Do you want to continue with the installation of [y,n,?] y

Installing HP Smart Array Controller Driver as

## Installing part 1 of 1.
/kernel/drv/amd64/cpqary3
/kernel/drv/cpqary3
/kernel/drv/cpqary3.conf
/usr/share/man/man7d/cpqary3.7d
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
ERROR: attribute verification of failed
pathname does not exist
[ verifying class ]
[ verifying class ]
## Executing postinstall script.

Installation of partially failed.

Even if the installation seems to fail the key consists of checking if the driver itself has been loaded:

$ modinfo
modinfo | grep cpq
 53 fffffffff79a6000  129f8 265   1  cpqary3 (HP Smart Array Driver Ver 2.2.0)

If modinfo shows a line like that it means the driver has been loaded and the OpenSolaris installer should see all the logical volumes.

And now it is the time to try Solaris 10 on this HP machine. The first thing to take into account is that the HP SmartArray P212 controller requires a closed-source binary from HP that is not distributed in the Solaris media. The cpqary3 driver can be downloaded from the HP Web site, although it is not very easy to find it out.

I downloaded the archived file for the cpqary3 driver, extracted its contents and burnt the ISO file to a CD-RW. Then I booted from the Solaris 10 DVD and applied the driver update. However, this didn’t work very well. The Solaris 10 DVD uses a 32-bit kernel which, apparently, does not like my >1.5TB disks. When the cpqary3 driver gets it prints a message to the console complaining that the detected disks have too many blocks for a 32-bit kernel. Very weird. I’m not entirely sure what a 32-bit kernel has to do with 64-bit LBA addressing but, so far, I haven’t been able to get Solaris 10 to recognize and get to like my disks.

So far, so good. VMware ESXi 4.0 didn’t work. Solaris 10 didn’t work either. Perhaps some of you, that are more experienced with Solaris than me, know how to fix this problem. In the mean time, I decided to take OpenSolaris for a spin. Fortunately, the final outcome with OpenSolaris has been a success. More on that later.