The naginterfaces Python Package¶

Copyright: The Numerical Algorithms Group Limited, 2017-2024.

Introduction¶

The naginterfaces package is a set of Python interfaces for the Mark 30.3 NAG Library Engine, which is the software implementation of NAG’s collection of several hundred mathematical and statistical routines serving a diverse range of application areas.

Please see also the getting started guide for this product.

NAG’s collection of algorithms is outlined in the Submodule Summary section of the documentation for the naginterfaces.library subpackage.

The package is compatible with 64-bit Python 3.7, 3.8, 3.9 and 3.10. See the Applicability section below for more details.

A selection of example files are included in the package distribution. These serve to demonstrate some important usage points and can also be used as short tests of the package.

The required NAG library and its appropriate runtime dependencies are included and installed with the package. You do not need to perform any installations of other off-the-shelf NAG Libraries to use this product. The included NAG library is customized for the package and cannot be used in any other context.

Two variants of the package are available, differing in the underlying linear algebra routines used by the NAG library: one set uses vendor-optimized routines and the other is ‘self contained’, which uses NAG-supplied versions.

In the front matter of the package’s documentation the following sections are available:

Changelog: The Changelog for the package, listing interesting enhancements and fixes introduced in each naginterfaces release.
Known Issues: An ‘issues’ document, describing known performance issues in the current release of the product.
Supplementary Information: Details of additional applicability or usage information for the product.
Technical Support: Information on how to receive further support from NAG.
Terms and Conditions: By installing this software you agree to NAG’s Terms and Conditions as outlined here.

Dependencies¶

A valid licence for this product. (See the Obtaining a Licence section below for further details.)
64-bit Python.
NumPy 1.15 or newer.

Installation¶

Installation is from a pre-compiled Python wheel archive via pip (Python’s package-management system) run from the command line. The licence-managed package is served from a NAG-hosted repository. Hence by using pip to install the software as documented in the Installing Using pip section below, the latest version of the appropriate package for the system you are running pip on is downloaded and installed in one combined operation.

Please see also the getting started guide for this product.

If you have a special need to interact directly with a source distribution of the package, for example to collaborate with NAG, how to do this is covered in the subsequent Source Distribution section.

By special arrangement a non-licence-managed .zip for the software can be obtained from a secure NAG URL.

The primary (64-bit) Python distribution supported by NAG is the official CPython distribution from

https://www.python.org

Use is also supported through Intel Python 2021

https://software.intel.com/en-us/articles/using-intel-distribution-for-python-with-anaconda

via Anaconda

https://www.continuum.io/

and using a NumPy enabled with Intel MKL version 2021.

pip is included by default in Python 3 from 3.4.

You must have permission to modify the directory tree of the Python interpreter with which you wish to install the package. Installation is often easiest when using a virtual Python environment, but doing so is not obligatory. See the Appendix section below for more details.

If you wish to add the package to an existing Python installation then we strongly advise that this installation should not contain the earlier NAG Python release nag4py, because the runtime systems used by this and naginterfaces are not mutually compatible. There is a high chance of unexpected behaviour if both packages are imported into the same Python session.

Managing the Package¶

Installing Using `pip`¶

Licence-managed Package

The installation command (using pip from the command line) looks like
```
python -m pip install --extra-index-url https://www.nag.com/downloads/py/naginterfaces_mkl naginterfaces
```
for installing the version of the package that relies on Intel MKL for optimized linear algebra routines, or
```
python -m pip install --extra-index-url https://www.nag.com/downloads/py/naginterfaces_nag naginterfaces
```
for the version that uses self-contained NAG-supplied linear algebra routines.

Both URLs supply the package as a pre-compiled Python wheel archive applicable to the system you are running pip on.

In cases where the target machine for the installation has restricted access to external URLs it will suffice for you to visit your desired download URL,

https://www.nag.com/downloads/py/naginterfaces_mkl/naginterfaces

or

https://www.nag.com/downloads/py/naginterfaces_nag/naginterfaces

from a networked machine and download the wheel (.whl) installer relevant to you. Once distributed across your internal network, the wheel can be installed from its file path via, for example,
```
python -m pip install naginterfaces-*.whl
```
Non-licence-managed Package

If you have obtained a non-licence-managed .zip for the software, please refer to the Installing the Non-licence-managed Package section of the Appendix, below.

The prerequisite NumPy will be automatically installed from the Python Package Index if not already present.

If the above pip command executes successfully for you, you may now skip to the Before Use section if you wish.

Upgrading Using `pip`¶

To upgrade an existing installation of naginterfaces, add the --upgrade switch to the pip line

python -m pip install --upgrade --extra-index-url https://www.nag.com/downloads/py/naginterfaces_mkl naginterfaces

for the licence-managed package, for example.

Uninstalling Using `pip`¶

Once installed by pip the package can later be uninstalled using

python -m pip uninstall naginterfaces

Switching Package Variants¶

If at any point you wish to switch between the naginterfaces_nag and naginterfaces_mkl variants, since by design both variants supply a package with the same name (naginterfaces) you must uninstall the existing naginterfaces and then install the alternate. This can be done in one command by instructing pip to forceably reinstall, by using the --upgrade --force-reinstall switches.

Accessing Older Versions¶

The NAG package repository does not support searching using pip search, but the URLs

https://www.nag.com/downloads/py/naginterfaces_mkl/naginterfaces

and

https://www.nag.com/downloads/py/naginterfaces_nag/naginterfaces

both give simple lists of all versions available of the licence-managed distribution files, including the latest version.

Source Distribution¶

As a regular user you are unlikely to need to interact directly with a source distribution of the package (which comprises the pure Python source for the package as well as the underlying compiled libraries).

If you do have a special need to access a source distribution of the package, for example to collaborate with NAG, how to do this is as follows.

Because of the platform-specific libraries included in the package there is no conveniently-sized system-agnostic Python source distribution of naginterfaces that is installable by pip using the same scheme as above. To access a (licence-managed) naginterfaces source distribution you must locate the .zip for your target platform from the list at the URL

https://www.nag.com/downloads/py/naginterfaces_mkl/naginterfaces

or at

https://www.nag.com/downloads/py/naginterfaces_nag/naginterfaces

and then you can use pip to install from the file’s full URL, or download the file and run pip on the result, for example

python -m pip install naginterfaces-{ver}-{platform tag}.zip

The {ver} marker denotes the (pip compliant) version number of the package—for example, 26.2.0.1—and {platform tag} denotes a (pip compliant) platform indicator—for example, win-amd64 for 64-bit Windows.

The package’s embedded setup.py file will raise RuntimeError if the target system for installation is not supported or does not match the binary files distributed in the .zip archive.

Before Use¶

Following a successful deployment of the naginterfaces materials, it is recommended that you review the full documentation for the package before use. To access from a command prompt execute

python -m pydoc naginterfaces

or see the Package Documentation section.

The HTML documentation is the canonical documentation source for the package. It may have minor changes applied to it in place when these do not warrant a full re-release of the package.

Note that the presence of other libraries via settings in your environment might introduce runtime incompatibilities with naginterfaces. Please see the Software-Stack Compatibility Notes section below.

Obtaining a Licence¶

Obtaining a NAG licence may require you to communicate your ‘Kusari hostid’ to NAG. For convenience, the module

python -m naginterfaces.kusari

will determine whether a valid licence could be found, and if not it will display the Kusari hostid for you on Linux and Mac. On Windows it will launch a GUI form.

For more information on how to obtain a valid licence for the underlying NAG library see the kusari Submodule section.

Sanity Check¶

As a basic verification that the package is ready to use, the ‘quick check’ function, callable via

python -c "from naginterfaces import quick_check; quick_check()"

will confirm availability of a licence and go on to perform a small check that the underlying compiled libraries load correctly.

Quick Start¶

Once you have successfully installed the package and obtained a valid licence (if using licence-managed software) we strongly advise that you study the full documentation for the package. These steps are covered in the preceding sections Installation, Obtaining a Licence and Before Use, respectively. You may also wish to visit the getting started guide for this product.

This Quick Start section serves to provide a short worked example of how to get started using the package via solving an optimization problem.

Suppose that we wish to minimize the ‘generalized Rosenbrock’ function using bound constrained optimization.

Interfaces to the NAG Library are provided in the naginterfaces.library subpackage:

>>> import naginterfaces.library as ni

(We use the convention that the >>> marker denotes the Python prompt.)

One can see from the HTML documentation at naginterfaces.library or by examining the output from

>>> help(ni)
Help on package naginterfaces.library in naginterfaces:

NAME
    naginterfaces.library
...
    Submodule Summary
    -----------------
    Python interfaces are available for the following algorithmic submodules.
...
    ``opt`` - Minimizing or Maximizing a Function (e04)

      This module provides functions for solving various mathematical
      optimization problems by solvers based on local stopping criteria.
...

that the relevant algorithmic submodule for (local) optimization is opt. (The opt Chapter is also known by its ‘short’ name e04 in the NAG Library Manual.) The HTML documentation for this submodule is at naginterfaces.library.opt.

In the naginterfaces HTML documentation all submodules provide a Functionality Index. Studying the opt Functionality Index, or alternatively viewing the output at the Python prompt after invoking

>>> from naginterfaces.library import opt as ni_opt; help(ni_opt)
Help on module naginterfaces.library.opt in naginterfaces.library:

...
FUNCTIONS
...
    bounds_quasi_func_easy(ibound, funct1, bl, bu, x, liw=None, lw=None, data=None)
        Bound constrained minimum, quasi-Newton algorithm, using function
        values only (easy-to-use).
...
        Parameters
        ----------
...
        funct1 : callable fc = funct1(xc, data=None)
...
        x : float, array-like, shape (n)
            `x`[j-1] must be set to an estimate of the jth component of the
            position of the minimum, for j = 1,2,...,n.
...

confirms that the relevant optimization solver to call is bounds_quasi_func_easy. The HTML documentation for this solver is at naginterfaces.library.opt.bounds_quasi_func_easy().

(Information can also be discovered by using the search facility in the naginterfaces HTML documentation.)

The optimization solver may be imported directly if desired:

>>> from naginterfaces.library.opt import bounds_quasi_func_easy

Now define the optimization problem: first, the objective function for the generalized Rosenbrock problem. From the signature displayed above we can infer that parameter funct1 in bounds_quasi_func_easy may be specified as a lambda expression in our case, where we do not have any communication data (data) to pass to the function:

>>> rosen = lambda x: (
...     sum(100.0*(x[1:]-x[:-1]**2.0)**2.0 + (1.0-x[:-1])**2.0)
... )

Then define an initial guess for the optimization. In the naginterfaces.library subpackage input array data may be supplied in any ‘array-like’ container, as noted above in the float, array-like, shape (n) specification for argument x. For our one-dimensional x, this means that any sequence of data will be a suitable container, as will a numpy.ndarray. (In functions taking multi-dimensional data, nested sequences and again instances of numpy.ndarray are valid.) Furthermore, the shape (length) of the x we supply determines the (inferred) value of n for the problem.

Our chosen start point is $(1.2, 1.0, 1.2, 1.0)$ and thus any of the following may be used to supply the ‘array-like’ vector x:

as a list

>>> x = [1.2, 1.0, 1.2, 1.0]

as a tuple

>>> x = (1.2, 1.0, 1.2, 1.0)

as an ndarray

>>> import numpy as np; x = np.array([1.2, 1.0, 1.2, 1.0])

Now define box bounds for the problem:

>>> n = len(x)
>>> bl, bu = ([0.0]*n, [2.0]*n)
>>> ibound = 0

Minimize the problem:

>>> opt_soln = bounds_quasi_func_easy(
...     ibound, rosen, bl, bu, x,
... )

Display the results:

>>> print('Function value at lowest point found is {:.5f}.'.format(opt_soln.f))
Function value at lowest point found is 0.00000.

>>> print('The corresponding x is (' +
...     ', '.join(['{:.4f}'] * n).format(*opt_soln.x) +
... ').')
The corresponding x is (1.0000, 1.0000, 1.0000, 1.0000).

Example Files¶

Each example file for the package can be run as a Python module, invocable via for instance

python -m naginterfaces.library.examples.info.impl_details_ex

Running

python -m pydoc naginterfaces.library.examples.info.impl_details_ex

will confirm the installation location of the associated file. All examples can be run as standard Python ‘main’ scripts from the command prompt, by executing for instance

python path/to/naginterfaces/library/examples/info/impl_details_ex.py

Likewise, the package’s source distribution contains all the example files in subdirectories of naginterfaces/library/examples.

Python’s inspect module can be used to access the source code of objects:

>>> import inspect
>>> from naginterfaces.library.examples.opt import handle_solve_dfls_ex
>>> print(''.join(inspect.getsourcelines(handle_solve_dfls_ex)[0]))
#!/usr/bin/env python3
"``naginterfaces.library.opt.handle_solve_dfls`` Python Example."
...

If your Python session is running in an Integrated Development Environment (IDE) such as IDLE, there may be a facility to open the source file for a requested module, for example naginterfaces.library.examples.info.impl_details_ex. Please refer to the documentation for your chosen IDE.

All examples may be executed sequentially by running

python -m naginterfaces.library.examples

Every example is executed using the doctest module, which will print details of any unexpected output.

To display the full list of example source files on disk, but not run them, execute

python -m naginterfaces.library.examples --locate

Run

python -m naginterfaces.library.examples --help

to see any additional usage.

Applicability¶

This package has been tested on the following systems:

64-bit Linux (Fedora 32)
64-bit Intel Mac (macOS 10.15/Catalina, 11/Big Sur and 12/Monterey)
64-bit Windows (Windows 10)

(We expect that Linux OS compatibility will be achieved by any system having GNU C Library version 2.17 or compatible.)

The Python distributions tested were

CPython
- Python 3.10.0 (64-bit; all systems)
with NumPy
- PyPI numpy 1.22.2
Intel Python 2021
- Python 3.9.7 (64-bit; all systems)
with NumPy
- Intel numpy 1.21 enabled with Intel MKL 2021

In particular, 32-bit Python is not supported. Please take care on Windows if installing CPython from https://www.python.org because the Python download suggested by the front page may be 32-bit Python, even on 64-bit Windows. The 64-bit Windows Python download can be found by following the link to the project’s downloads area and choosing the required Windows x86-64 installer.

Software-Stack Compatibility Notes¶

Runtime Libraries¶

The intention is for the package to be able to load the underlying NAG library and its compatible runtimes, which are side-by-side in the distribution, without you needing to make any explicit settings in your environment. However, if the underlying NAG library fails to load (and the package raises NagException) it may be that incompatible runtime libraries have been preloaded via settings in your environment; i.e., LD_LIBRARY_PATH on Linux, DYLD_LIBRARY_PATH on Mac, or PATH on Windows.

(The package’s exception hierarchy, which includes NagException, is documented in the naginterfaces.base.utils submodule.)

If you believe that your environment settings are interfering with the loading of the underlying NAG library please consider running your Python session with the other (e.g. NAG and Intel) settings excised from the environment.

You may also find that it helps to use the MKL-free naginterfaces_nag version of the naginterfaces package.

Alternatively, prepend the installed directory for the runtimes into the environment before beginning the Python session.

The installation directory can be displayed by executing

python -c "from naginterfaces import ENGINE_LIBDIR; print(ENGINE_LIBDIR)"

To add the displayed directory into the environment: for Bourne shell users on Linux

LD_LIBRARY_PATH=path/to/naglib_dir:"${LD_LIBRARY_PATH}"
export LD_LIBRARY_PATH

for example, or for Bourne shell users on Mac

DYLD_LIBRARY_PATH=path/to/naglib_dir:"${DYLD_LIBRARY_PATH}"
export DYLD_LIBRARY_PATH

or from a DOS prompt on Windows

PATH=path\to\naglib_dir;%PATH%

Appendix¶

Creating a New Python Environment¶

Python virtual environments can be created using the venv module

python3 -m venv ni

say.

To use an existing Anaconda installation to create a new Intel Python environment including an MKL-enabled NumPy, with conda on your path invoke

conda create --channel intel --prefix ni python=3 intelpython=2021 mkl=2021 numpy

You can then ‘activate’ the resulting Python environment.

Environments created using venv can be activated as follows: for Bourne shell users on Linux and macOS

. ni/bin/activate

say, or on Windows

ni\Scripts\activate

For Anaconda environments, under the Bash or Zsh shells (Linux and macOS)

source activate ni

or on Windows

activate ni

for example.

Environments can be deactivated in a similar way. For venv environments run

deactivate

For Anaconda environments, on Linux and macOS

source deactivate

or on Windows

deactivate

Obtaining the NAG Library Manual for Offline Access¶

A downloadable archive of the Library Manual to accompany this release can be found at

https://www.nag.com/numeric/nl/nagdoc_30.3/nagdoc_30.3.zip

Installing the Non-licence-managed Package¶

This section is relevant only when NAG has supplied you with a link to download a .zip for a non-licence-managed version of the package. Please read this section in conjunction with the main Installation section above.

The .zip file to which you are given access recreates the structure of the NAG package repository used by the licence-managed software, for each supported platform. Therefore you can emulate the same hosting of the package on your own systems if desired.

The name of the archive you access will look something like naginterfaces-{ver}-{plat}.zip, with {ver} being the package’s version number separated with dashes (for example, 26-2-0-1) and {plat} a platform indicator (for example, win for Windows).

Once the archive is unzipped, the wheel (.whl) installer for the MKL-enabled variant of the package can be found in

naginterfaces-{ver}/naginterfaces_mkl/naginterfaces/

Likewise, the variant of the package that uses NAG-supplied linear algebra can be found in

naginterfaces-{ver}/naginterfaces_nag/naginterfaces/

The wheel you wish to use can be installed directly from the local disk with pip. The name of the wheel file contains pip compliant markers indicating amongst other things the applicable Python versions for the package as well as its target platform. It will be of the form

naginterfaces-{ver}-{python tag}-none-{platform tag}.whl

Call this {ni_whl}. Each naginterfaces subdirectory of the unzipped archive will only contain one such wheel.

Therefore, the command for installing the MKL-enabled package will look like

python -m pip install naginterfaces-{ver}-{plat}/naginterfaces_mkl/naginterfaces/{ni_whl}

and for installing the package with NAG-supplied linear algebra, like:

python -m pip install naginterfaces-{ver}-{plat}/naginterfaces_nag/naginterfaces/{ni_whl}

The unzipped archive also contains .zip files for the package’s source distributions, in the same directories as the wheels. These source distributions have names like naginterfaces-{ver}-{platform tag}.zip.

NAG and Python

Return to Front

The naginterfaces Python Package¶

Introduction¶

Dependencies¶

Installation¶

Managing the Package¶

Installing Using `pip`¶

Upgrading Using `pip`¶

Uninstalling Using `pip`¶

Switching Package Variants¶

Accessing Older Versions¶

Source Distribution¶

Before Use¶

Obtaining a Licence¶

Sanity Check¶

Quick Start¶

Example Files¶

Applicability¶

Software-Stack Compatibility Notes¶

Runtime Libraries¶

Appendix¶

Creating a New Python Environment¶

Obtaining the NAG Library Manual for Offline Access¶

Installing the Non-licence-managed Package¶

The naginterfaces Python Package¶

Introduction¶

Dependencies¶

Installation¶

Managing the Package¶

Installing Using pip¶

Upgrading Using pip¶

Uninstalling Using pip¶

Switching Package Variants¶

Accessing Older Versions¶

Source Distribution¶

Before Use¶

Obtaining a Licence¶

Sanity Check¶

Quick Start¶

Example Files¶

Applicability¶

Software-Stack Compatibility Notes¶

Runtime Libraries¶

Appendix¶

Creating a New Python Environment¶

Obtaining the NAG Library Manual for Offline Access¶

Installing the Non-licence-managed Package¶

Installing Using `pip`¶

Upgrading Using `pip`¶

Uninstalling Using `pip`¶