A Tour of NTL: Using NTL with GMP

GMP is the GNU Multi-Precision library. You can get more information about it, as well as the latest version from here.

Briefly, GMP is a library for long integer arithmetic. It has hand-crafted assembly routines for a wide variety of architectures. For basic operations, like integer multiplication, it can be two to three (and sometimes bit more) times faster than NTL's traditional long integer package. The speedup is most dramatic on x86 machines.

You can choose one of three different ways of implementing long integer arithmetic in NTL:

1. One can use the traditional NTL long integer arithmtic package, and avoid dealing with GMP entirely.

2. One can use traditional NTL long integer arithmtic package, but with GMP as a supplemental long integer package. This gives you many (though not all) of the performance benefits of GMP, but while still maintaining complete backward compatability.

3. One can use GMP as the primary long integer package. This gives you essentially all of the performance benefits of GMP, but there are some minor backward incompatabilities (see below).

The use of GMP as the primary long integer package is the preferred method of using GMP. The use of GMP as a supplemental long integer package is intended primarily for backward compatability.

Building NTL with GMP (either as a supplemental or primary long integer package) takes a few extra minutes work, and you certainly do not need to use NTL with GMP if you don't want to. As far as I know, GMP is only available on Unix systems and on Windows systems using Cygwin tools.

Even if you do not use GMP as the primary long integer package, you should still read the section below on backward compatabilty so that you can write portable code and avoid deprecated constructs.

Downloading and building GMP

Download GMP from here. You will get a file gmp-XXX.tar.gz.

Now do the following:

   % gunzip gmp-XXX.tar.gz
   % tar xf gmp-XXX.tar
   % cd gmp-XXX
   % ./configure --prefix=$HOME/sw
   % make
   % make check
   % make install

You can also supply the option --disable-shared to the configure script, if you only want static libraries. However, if you ultimately want to build NTL as a shared library, then you must also buld GMP as a shared library.

You must ensure that NTL and GMP have the same ABI. Usually, GMP's configure script will automatically choose a 64-bit ABI if available.

Building and using NTL with GMP

When building NTL with GMP, you have to tell NTL that you want to use GMP as the primary long integer package, and where the include files and library are. The easiest way to do this is by passing the argument NTL_GMP_LIP=on to the NTL configuration script when you are installing NTL. Assuming you installed GMP in $HOME/sw as above, and you also want to install NTL in $HOME/sw, you execute:

   % ./configure PREFIX=$HOME/sw NTL_GMP_LIP=on  GMP_PREFIX=$HOME/sw

   % ./configure DEF_PREFIX=$HOME/sw NTL_GMP_LIP=on 

If you installed GMP in /usr/local (or some other standard system directory where your compiler will look by default) then simply

   % ./configure PREFIX=$HOME/sw NTL_GMP_LIP=on

   % ./configure NTL_GMP_LIP=on

If instead you want to use GMP as a supplemental long integer package, you should pass the argument NTL_GMP_HACK=on to the configure script, instead of NTL_GMP_LIP=on. One still has to specify where to find GMP using the GMP_PREFIX variable in the configuration script.

Instead of passing arguments to the configure script, you can also just edit the config.h and makefile by hand. The documentation in these files should be self-explanatory.

When compiling programs that use NTL with GMP, you need to link with the GMP library. If GMP is installed as above in $HOME/sw, rather than in a standard system directory, this just means adding -L$HOME/sw/lib -lgmp to the compilation command. If you installed GMP in a standard system directory, then just -lgmp does the job. Note that -lgmp must come after -lntl on the command line. Finally, if NTL and GF2X are installed as shared libraries, then you don't even need -lgmp.

NTL has been tested and works correctly with GMP versions 3.1, 3.1.1, and 4.1.4 (among others) as the primary long integer package. It is not possible to use versions of GMP prior to 3.1 as the primary long integer package.

NTL has been tested and works correctly with versions 2.0.2, 3.0.1, 3.1, 3.1.1, and 4.1.4 (among otheers) of GMP as a supplemental long integer package. It is not recommended to use versions of GMP prior to 3.1.1, as these are generally more buggy and less efficient.

When using NTL with GMP as either primary or supplemental long integer package, as a user of NTL, you do not need to know or understand anything about the the GMP library. So while there is detailed documentation available about how to use GMP, you do not have to read it. The programming interface to the long integer package completely hides implementation details.

Some implementation details

When using GMP as the primary long integer package, the code used by NTL is essentially a layer of C routines that call the low level mpn routines in the GMP package. These NTL wrapper routines provide essentially the same functionality of the higher level mpz routines in GMP, but while presenting an interface to the rest of NTL that is almost identical to that of the traditional NTL long integer package. There are, however, some very minor backward incompatabilities (see below).

When using GMP as a supplemental long integer package, the code employs a "quick and dirty", yet fairly effective hack. This quick and dirty approach converts "on the fly" between the classic LIP and GMP representations. This makes the use of GMP completely invisible to higher layer software. Of course, there is a penalty: converting between representations takes time. For operations like addition, conversion would take longer than performing the operation, and so it is not done. However, for computationally expensive operations like multiplication, the "overhead" is not so bad, at least for numbers that are not too small. To multiply two 256-bit numbers on a Pentium-II, the extra time required for the data conversions is just 35% of the time to do the multiplication in GMP, i.e., the "overhead" is 35%. For 512-bit numbers, the corresponding opportunity cost is about 14%, and for 1024-bit numbers, it is less than 10%. For smaller numbers, the opportunity cost is greater, but never much worse than about 50%.

Backward compatbility

With version 5.0 of NTL, some aspects of the programming interface are 'deprecated' so as to allow the use of another long integer package, such as GMP, as the primary long integer package.

Prior to version 5.0, the macro NTL_NBITS was defined, along with the macro NTL_RADIX defined to be (1L << NTL_NBITS). While these macros are still available when using NTL's traditional long integer package (i.e., when NTL_GMP_LIP is not set), they are not available when using the GMP as the primary long integer package (i.e., when NTL_GMP_LIP is set). Furthermore, when writing portable programs, one should avoid these macros.

Also, the static function long ZZ::digit(const ZZ &); is defined when using traditional long integer arithmetic, but is not available when using GMP as the primary long integer package, and in any case, its use should be avoided when writing portable programs.

Instead of the above macros, one should use the followng macros:

   NTL_ZZ_NBITS -- number of bits in a zzigit;
                   a ZZ is represented as a sequence of zzigits.

   NTL_SP_NBITS -- max number of bits in a "single-precision" number

   NTL_WSP_NBITS -- max number of bits in a "wide single-precision" number

The following relations hold:

   NTL_SP_NBITS <= NTL_WSP_NBITS <= NTL_ZZ_NBITS
   26 <= NTL_SP_NBITS <= min(NTL_BITS_PER_LONG-2, NTL_DOUBLE_PRECISION-3)
   NTL_WSP_NBITS <= NTL_BITS_PER_LONG-2

Note that NTL_ZZ_NBITS may be less than, equal to, or greater than NTL_BITS_PER_LONG -- no particular relationship should be assumed to hold. In particular, expressions like (1L << NTL_ZZ_BITS) might overflow.

"single-precision" numbers are meant to be used in conjunction with the single-precision modular arithmetic routines.

"wide single-precision" numbers are meant to be used in conjunction with the ZZ arithmetic routines for optimal efficiency.

Note that when using traditional long integer arithmetic, we have

    NTL_ZZ_NBITS = NTL_SP_NBITS = NTL_WSP_NBITS = NTL_NBITS.

The following auxilliary macros are also defined:

NTL_FRADIX -- double-precision value of 2^NTL_ZZ_NBITS
NTL_SP_BOUND -- (1L << NTL_SP_NBITS)
NTL_WSP_BOUND -- (1L << NTL_WSP_NBITS)

Note that for a ZZ n, n.size() returns the number of "zzigits" of n. This is supported with either traditional or GMP integer arithemtic. Note, however, that some old codes might write n.size() <= 1 as a way to test if NumBits(n) <= NTL_NBITS. This is no longer the right thing to do, if one wants portable code that works with either traditional or GMP long integer arithmetic. First, one has to decide whether one wants to test if NumBits(n) is bounded by NTL_ZZ_NBITS, NTL_SP_NBITS, or NTL_WSP_NBITS. In the first case, n.size() <= 1 is still the right way to test this. In the second case, write this as n.SinglePrecision(). In the third case, write this as n.WideSinglePrecision(). The routines SinglePrecision and WideSinglePrecision are new to NTL version 5.0.

Most "high level" applications that use NTL should not be affected by these changes to NTL's programming interface, and if they are, changing the programs should be quite easy.