FreeNX, usermode authentication and Mac OS X

I’ve always been looking for a way in NX/FreeNX to be able to authenticate using mechanisms other than username and password, like SSH private/public keys or Kerberos. Turns out that it is possible 🙂

Someone pointed me to the FreeNX 0.7.3 announcement that contains the following excerpt:

Usermode and SUID Wrapper

We are now very close to login directly with users and I also heard of a C program, which can be seamlessly put between nxclient and nxssh. So with client support we now have three alternatives:

1. Login as user via ssh and connect to server with ssh command on server again.
2. Login as user and use usermode to save all sessions locally for each user.
3. Use a SUID nx (not root!) wrapper to startup a new "trusted" session.

One is error prone, two is good, but looses the central structure, three is best of both worlds and with being suid nx also has the most advantages, however not the dreaded public key problems.

_Yes_, this means if you use the suid wrapper, you still need the nx user, but you can remove the public keys and it'll still work.

The SUID wrapper is a part from the work of the redesign and thanks goes to Alistair Riddoch from Google here.

By default, NoMachine’s NX nxserver requires nxclient to login via SSH into the remote machine as user nx. As nxserver is defined as the login shell, it is run by the sshd daemon. From there on, there is a dialogue between nxclient and nxserver where nxclient supplies the user credentials (username and password that were specified in the nxclient configuration). There is, in fact, a second authentication that is performed via another SSH session to using nxclient’s supplied credentials. If this second authentication succeeds, the NX session is activated and accessible from the NX client.

This works well for remote servers that are shared by multiple users, as the nx user and its centralized approach makes it very easy to see how many sessions are currently running (or suspended), terminate them, etc. However, for machines that are not shared by multiple users, or in those cases where authentication mechanisms other than username and password are required, this model does not work very well.

This is where FreeNX’s usermode enters the scene. Basically, what it means, is that authentication to nxserver does no longer happen as the nx user but as the end-user himself. Now, the number of SSH sessions is reduced to one that authenticates the user directly by means of SSH’s built-in authentication capabilities, and where nxserver is run under the end-user credentials instead of the nx user. This, obviously, kills the centralized approach originally envisioned by NoMachine, since now all the control and session files can’t be stored easily and securely in a central location but are now stored in the user’s home directory. But I think the upsides of the usermode support outdo the lack of centralized management. At least in my case, I don’t need centralized management since it’s me who manages all my boxes and logs into them.

How to install and configure FreeNX to support usermode

Next I describe what I had to do, both on the remote machine and also on the client, to get a working FreeNX environment that supports usermode. Other modes are also supported, like legacy nx-based, SUID and others.

Download NX4U tarball from BerliOS and extract it

$ wget
$ sudo tar -C /opt -zxf NX4U.tar.gz

NOTE: The NX4U tarball that I used can also be downloaded from this Web site here.

NOTE: The NX4U set and the nxssh wrapper are smart enough so that you can also extract the NX4U tarball in other locations. Looking at the source code for the nxssh wrapper — nxssh-4US.c — nxssh wrapper uses the following PATH to locate the nxserver binary:


Compile the nxssh wrapper

First, download the source code from the SVN repository:

$ svn checkout

NOTE: I saved a copy of the SVN repository that I used. The tarball is available in this Web site here.
Build the nxssh wrapper for Mac OS X. nxssh is a simple C program that currently compiles for me with no problems on Linux and Mac OS X:

$ cd trunk/freenx-utils/nxpublickey/
$ make nxssh

NOTE: The Makefile also has a target named nxssh.exe to compile the wrapper for Windows.

Now, let’s rename NoMachine’s nxssh binary to mxssh (the nxssh wrapper expects NoMachine’s nxssh binary to be renamed to mxssh), then install the nxssh wrapper:

$ sudo bash
# mv /usr/NX/bin/nxssh /usr/NX/bin/mxssh
# install -m755 nxssh /usr/NX/bin/nxssh
# ^D

Configure .ssh/config

What looks like a bug in NoMachine’s nxssh, will cause authentication requests using public key to fail with a "percent_expand: NULL replacement" error unless .ssh/config is modified to explicitly state the location of the public key. For example:

        IdentityFile ~/.ssh/id_dsa

Configure nxclient

In order to use usermode authentication, make sure to prepend the hostname with the @ (at) sign:


Also, make sure the username has the @ (at) sign prepended plus @U (at U) appended. These non-standard forms are parsed by the nxssh wrapper and enable usermode authentication (or other authentications like SUID):

Username: @myself@U

For more information about possible syntaxes, take a look at freenx-utils/nxpublickey/nxssh-wrapper (the shell script implementation of the nxssh wrapper).

Installing FreeNX 0.7.1 on Ubuntu


DISCLAIMER: The contents of this post are mostly based on Manual Installation How-To. Thanks to Brent Davidson and Fabian Franz for writing such a nice HowTo and the beautiful open and free implementation of FreeNX, respectively.

I decided to use FreeNX instead of NoMachine’s own implementation due to the instability of the latter. Most of the times, I could not reconnect to my running sessions, or else NX decided to kill my running session and start a new one. FreeNX is a collection of shell scripts, which makes it easier to debug and troubleshoot problems.

The process described in this post starts with the NoMachine’s binary components, downloadable from the Web, and then overwrites or replaces key components that are binary-only and closed with FreeNX’s open and free shell scripts, which provides much more flexibility. In my own experience, FreeNX is more robust, stable, predictable, easier to customize and to debug than NoMachine’s closed binary components.

Installing the base NoMachine’s NX binary components

Download nxclient, nxnode and nxserver from NoMachine as .tar.gz files. The files can be found in the NoMachine downloads or can be downloaded directly from this site, if you trust me:

For IA-32 systems:

For x86_64 systems:

Extract the files to /usr. Since the .tar.gz packages always contain relative pathnames that start with ./NX, this will create a whole directory tree under /usr/NX.

# tar -C /usr –xzf nxserver-3.0.0-79.i386.tar.gz
# tar -C /usr –xzf nxclient-3.0.0-84.i386.tar.gz
# tar -C /usr –xzf nxnode-3.0.0-93.i386.tar.gz

Compiling the NX compression libraries

Compiling nxcomp

Download the source code for nxcomp from NoMachine’s source code or here from

The ./configure is not very robust and doesn’t check for missing dependencies. This are the packages that are needed to compile nxcomp:

# apt-get install zlib1g-dev libX11-dev libjpeg-dev libpng12-dev 
    x11proto-xext-dev libxdamage-dev libxrandr-dev libxtst-dev 

Configuring, building the library and copying it to its final location is just as easy as running:

# tar -xzf nxcomp-3.0.0-48.tar.gz
# cd nxcomp
# ./configure --prefix=/usr/NX
# make
# cp -P* /usr/NX/lib

Compiling nxcompext

Download the source code for nxcompext and nx-X11 from NoMachine’s source code or here from
nxcompext-3.0.0-18.tar.gz and nx-X11-3.0.0-37.tar.gz, and extract them:

# tar -xzf nxcompext-3.0.0-18.tar.gz
# tar -xzf nx-X11-3.0.0-37.tar.gz

Before compiling nxcompext, apply the NXlib-xgetioerror.patch.

# cd nxcompext
# patch -p0 < NXlib-xgetioerror.patch

This is required or else the resulting shared library will complain about _XGetIOError symbol being undefined. In order to troubleshoot this, I had to enable logging in /usr/NX/etc/node.conf:


Then, looking at /var/log/nxserver.log I found the following error message:

Info: Established X client connection.
Info: Using shared memory parameters 1/1/1/4096K.
Info: Using alpha channel in render extension.
Info: Not using local device configuration changes.
/usr/NX/bin/nxagent: symbol lookup error: /usr/NX/lib/
undefined symbol: _XGetIOError
NX> 1006 Session status: closed

Applying the patch solves the problem:

# ./configure --x-includes="/usr/include/xorg -I/usr/include/X11" --prefix=/usr/NX
# make
# cp -P* /usr/NX/lib

Compiling nxcompshad

Download the source code for nxcompshad from NoMachine’s source code or here from

# tar -xzf nxcompshad-3.0.0-19.tar.gz
# cd nxcompshad
# ./configure --prefix=/usr/NX
# make
# cp -P* /usr/NX/lib

Compiling nxesd

Download the source code for nxesd from NoMachine’s source code or here from

# tar -xzf nxesd-3.0.0-4.tar.gz
# cd nxesd
# ./configure --prefix=/usr/NX
# make
# make install

Installing FreeNX

Download FreeNX from FreeNX downloads, or from this Web site at freenx-0.7.1.tar.gz and extract them and apply the gentoo-machine.diff patch:

# tar -xzf freenx-X.Y.Z.tar.gz
# cd freenx-X.Y.Z
# patch -p0 < gentoo-nomachine.diff

The gentoo-machine.diff patch must be applied if you are using the /usr/NX directory structure that the NoMachine libraries use.

Next, we replace the original NoMachine key binaries (in fact, they are compiled Perl scripts) with the FreeNX shell scripts:

# cp -f nxkeygen /usr/NX/bin/
# cp -f nxloadconfig /usr/NX/bin/
# cp -f nxnode /usr/NX/bin/
# cp -f nxnode-login /usr/NX/bin/
# cp -f nxserver /usr/NX/bin/
# cp -f nxsetup /usr/NX/bin/
# cp -f nxcups-gethost /usr/NX/bin/

Next, we need to compile the nxserver-helper binary, which is used by the slave mode of nxnode. Basically, nxserver-helper runs a command that has both /dev/fd/3 and /dev/fd/4 mapped into both ends of a UNIX SOCKET.

# cd nxserver-helper
# make
# cp -f nxserver-helper /usr/NX/bin/

Before being able to set up the FreeNX, install expect, the OpenSSH server and smbmount and smbumount:

$ sudo apt-get install expect smbfs openssh-server

The next step creates symbolic links in /usr/bin to all FreeNX scripts that live in /usr/NX/bin and additional symbolic links for NX compatibility:

# ln -s /usr/NX/bin/nxserver /usr/bin/nxserver
# ln -s /usr/NX/bin/nxsetup /usr/sbin/nxsetup
# ln -s /usr/NX/bin/nxloadconfig /usr/sbin/nxloadconfig
# ln -s /usr/NX/lib/ /usr/NX/lib/
# ln -s /usr/NX/bin/nxagent /usr/NX/bin/nxdesktop
# ln -s /usr/NX/bin/nxagent /usr/NX/bin/nxviewer
# ln -s /usr/bin/foomatic-ppdfile /usr/lib/cups/driver/foomatic-ppdfile
# ln -s /etc/X11/xinit /etc/X11/xdm
# ln -s /sbin/mount.cifs /sbin/smbmount
# ln -s /sbin/umount.cifs /sbin/smbumount

The final step consists is running the installation stage of FreeNX:

# nxsetup --install

This will create /usr/NX/var directory tree, create the nx user, install the appropiate SSH keys (either the NoMachine’s keys or custom keys).

Before being able to use FreeNX, create the node.conf configuration file that allow changing the behavior of FreeNX, like logging, path names to several scripts used to start GNOME or KDE, and so on:

# cd freenx-X.Y.Z
# cp node.conf.sample /usr/NX/etc/node.conf

Future development and ideas

  • Not having to depend on any single binary file from NoMachine.

    The idea is compiling all the components from source code, instead of starting with a binary distribution and replacing key components with their open and free counterparts.

  • Customizing FreeNX so that I can bypass NoMachine’s nxclient completely.

    Most of my network components are Kerberized and having to keep supplying my password to nxclient seems like a thing of the past to me. The idea is customizing FreeNX in such a way that I can leverage Kerberos authentication and drop password-based authentication completely.

FreeNX on Linux

FreeNX is based on NX compression GPL components to allow a fast, graphical remote desktop terminal session for UNIX-based systems. NX uses SSH tunneling to perform authentication and link parameters negotiation. has NX viewer clients for Mac OS X and Linux. By default, the nxclient software uses a built-in private key to allow connecting to the remote NX server via SSH DSA public key. When using FreeNX, it’s recommeded to generate a new private-public DSA key pair and iinstall them onto the client machines and the remote NX servers.

The NX server software uses the “nx” user account, configured to allow for public key authentication, which is then used to start up the remote agent and proxy components used by the NX protocol. The NX client starts a remote SSH session against the NX server using this “nx” user. Thus, we need to manually generate a DSA pair key. The private DSA key will get installed into the client, while the public key will get installed into the NX server.

FreeNX can be obtained from the following sites:

For Fedora Core:

For Debian:
By adding the following to “/etc/apt/sources.list”:

deb ./

the running

# apt-get install freenx

  1. Generating the DSA private-public key pair.

    We must use the “ssh-keygen” command line tool to create a private-public key pair. For example, by issuing the following command on the client machine:

    # ssh-keygen -t dsa
    Generating public/private dsa key pair.
    Enter file in which to save the key (/Users/falfaro/.ssh/id_dsa):
    Enter passphrase (empty for no passphrase):
    Enter same passphrase again:
    Your identification has been saved in /Users/falfaro/.ssh/id_dsa.
    Your public key has been saved in
    The key fingerprint is:
    76:f1:09:07:f3:ef:4d:0a:a9:b7:ac:48:49:93:67:fe falfaro@mac.local

    The private key should NOT be protected by a passphrase, as it will be directly used by the NX client software before any authentication is performed.

  2. Installing the private key into the NX client software

    The next step is replacing the NX client software built-in private key with the one we have just created. NoMachine’s NX client software stores the DSA private key in “/usr/NX/share/client.id_dsa.key”:

    # ls -l /usr/NX/share/client.id_dsa.key
    -rw-r--r-- 1 root wheel 668 27 Dec 13:59 /usr/NX/share/client.id_dsa.key

    Thus, we should execute the following command:

    # mv /usr/NX/share/client.id_dsa.key /usr/NX/share/client.id_dsa.key.OLD
    # mv /Users/falfaro/.ssh/id_sa /usr/NX/share/client.id_dsa.key
    # chown root:wheel /usr/NX/share/client.id_dsa.key
    # chmod 644 /usr/NX/share/client.id_dsa.key

  3. Installing the public key into the NX server software

    The last step is installing the public key, which corresponds to the “nx” user, into remote server. The public key will be installed as an “authorized_keys2” file inside the home directory for the “nx” user. The OpenSSH service will use this file to store the “nx” user public key the NX client software uses to authenticate against the NX server.

    Depending on the distribution and FreeNX implementation, the home directory for the “nx” user will be located in different places. In Fedora Core, this is usually “/var/lib/nxserver/nxhome”. In Debian, this is usually “/home/.nx”.

    The last step is distributing the “” file to the remote NX server machine and authorize it:

    # scp /Users/falfaro/.ssh/ root@NXSERVER:
    # rm /Users/falfaro/.ssh/
    # ssh root@NXSERVER
    # mv /root/ /home/.nx/.ssh/authorized_keys2
    # chown nx:root /home/.nx/.ssh/authorized_keys2
    # chmod 600 /home/.nx/.ssh/authorized_keys2

  4. Testing public key authentication

    Before using the NX client software to connect to the remote NX server, it’s recommended to check whether we can connect remotely to the NX server using an SSH client using public key authentication for the “nx” user:

    # ssh -i /usr/NX/share/client.id_dsa.key nx@NXSERVER
    Linux NXSERVER 2.6.10 #1 Sat Dec 25 05:20:24 CET 2004 i686 GNU/Linux
    HELLO NXSERVER - Version 1.4.0-02 OS_(GPL)
    NX> 105 quit
    NX> 999 Bye
    Connection to ubuntu closed.

    If this works, we can be pretty sure the NX client will allow us to establish a remote session against the NX server.

  5. Configuring FreeNX server to support resuming of suspended sessions

    In file “/usr/bin/nxserver”:

    Replace the line that reads:




    Replace the line that reads:

    session_list_user_suspended "$USER" 'Suspended' "$(getparam geometry)" "$(getparam type)" | log_tee


    session_list_user_suspended "$USER" 'Suspended$|^status=Running$' "$(getparam geometry)" "$(getparam type)" | log_tee

    This is very important as sometimes, when the NX client is disconnected from the NX server, the session is not marked as suspended.