Starlink Project

Starlink logo

Starlink logo

The Starlink Project, referred to by users as Starlink and by developers as simply The Project, was a UK astronomical computing project which supplied general-purpose data reduction software. Until the late nineties, it also supplied computing hardware and system administration personnel to UK astronomical institutes. In the former respect at least, it was analogous to the US IRAF project, with which it long maintained a friendly rivalry.

The Project was formally started in 1980, though the funding had been agreed, and some work begun, a year earlier. It was closed down when its funding was withdrawn by PPARC in 2005. In 2006 the Joint Astronomy Centre released its own updated version of Starlink and took over maintenance; the latest version was released on 27 July 2009.

History

From its beginning, the project aimed to cope with the ever-increasing data volumes which astronomers had to handle. A 1982 paper exclaimed that astronomers were returning from observing runs (a week or so of observations at a remote telescope) with more than 10 Gigabits of data on tape; at the end of its life the project was rolling out libraries to handle data of more than 4 Gigabytes per single image.

File:Starlink sites map test1.svg

Starlink circa 1999

The Project provided centrally-purchased (and thus discounted) hardware, professional system administrators, and the developers to write astronomical data-reduction applications for the UK astronomy community and beyond. At its peak size in the late eighties and early nineties, the Project had a presence at around 30 sites, located at most of the UK universities with an astronomy department, plus facilities at the Joint Astronomy Centre, the home of UKIRT and the JCMT in Hawai'i. The number of active developers fluctuated between five and more than a dozen.

By 1982, the project had a staff of 17, serving about 400 users at six sites, using seven VAXen (six VAX-11/780s and one VAX-11/750, representing a total of about 6.5 GB of disk space). They were networked from the outset, first with DECNET and later with X.25.

Between 1992 and 1995 the project switched to UNIX (and switched the networking to TCP/IP), supporting Digital UNIX on Alpha-based systems, and Solaris on systems from Sun Microsystems. By the late 1990s it was additionally supporting Linux, and by 2005 it was supporting Red Hat Linux, Solaris, and Tru64 UNIX. It was about this time that the Project open-sourced its software (using the GPL; it had previously had an `academic use only' licence), and reworked its build system so that the software could be built on a much broader range of POSIX-like systems, including Mac OS X and Cygwin.

Though it was not explicitly funded to do so, the Project was an early participant in the Virtual Observatory movement, and contributed to the IVOA. Its best-known VO application was TOPCAT, development of which continues, with AstroGrid funding.

Applications, libraries, and other facilities

The Project produced a number of well-known applications and libraries. Of these, some of the highlights are:

GAIA

The main GUI application, which acts as a general astronomical image viewer, as well as a front end to many of the other applications.

ORAC-DR

The ORAC-DR data reduction system, developed at JAC Hawai'i, is a data processing pipeline for incoming data. It is in use for online data reduction at UKIRT and JCMT for a variety of instruments. This is not a Starlink application as such, but it is tightly integrated with the Starlink suite, and by default uses Starlink software as its application engines. See the ORAC-DR home page for further details.

KAPPA

A suite of general-purpose data-analysis and visualisation tools, usable both from the command-line and graphically. It provides general-purpose applications that have wide applicability, concentrating on image processing, data visualisation, and manipulating NDF components. It integrates with other Starlink packages. In a wider context, KAPPA offers facilities not in IRAF, for instance handling of data errors, quality masking, a graphics database, availability from the shell, as well as more n-dimensional applications, widespread use of data axes, and a different style. It integrates with instrument packages developed at UK observatories. With the automatic data conversion and the availability of KAPPA and other Starlink packages from within the IRAF command language, it's possible to pick the best of the relevant tools from both systems to get the job done.

CCDPACK

A package of programs for reducing CCD-like data. They allow you to debias, remove dark current, pre-flash, flatfield, register, resample, normalize and combine your data.

AST

A flexible and powerful library for handling World Coordinate Systems, partly based on the SLALIB library. If you are writing software for astronomy and need to use celestial coordinates (e.g. RA and Dec), spectral coordinates (e.g. wavelength, frequency, etc.), or other coordinate system information, then this library should be of interest. It provides solutions for most of the problems you will meet and allows you to write robust and flexible software. It is able to read and write WCS information in a variety of formats, including FITS-WCS. It has both Fortran and C bindings.

SLALIB

A library of routines intended to make accurate and reliable positional-astronomy applications easier to write. Most SLALIB routines are concerned with astronomical position and time, but a number have wider trigonometrical, numerical or general applications. As well as this GPL version, there is also a commercial version of SLALIB available from its original author.

HDS

A Hierarchical Data System -- is a portable, flexible system for storing and retrieving data, and takes over from a computer's filing system at the level of an individual file. A conventional file effectively contains an 1-dimensional sequence of data elements, whereas an HDS file can contain a more complex structure. It predates the Hierarchical Data Format by several years.

NDF

NDF is the Project's principal data format. Built upon HDS the N-dimensional Data Format -- is for storing bulk data in the form of n-dimensional arrays of numbers: mostly spectra, images, and cubes. It supports concepts such as quality, data errors, world coordinate systems, and Metadata. It is also extensible to handle user-defined information.

ADAM

The ADAM environment was a standardised software environment developed initially by the RGO, and then adopted and developed by Starlink between 1985 and 1990. It was initially designed as a telescope control system, installed at the AAT at Siding Spring, the WHT at the ING on La Palma, and at the JCMT on Mauna Kea (where it is still working in legacy systems), but its role expanded to cover graphics, data access, interprocess communication, and the full range of functionality required to support a diverse range of interoperable applications. Although it is no longer seriously used for telescope control, other layers of it live on in the current versions of the Starlink applications and libraries.

For fuller details of any of these utilities, see the list of `SUN's on the Starlink Project's documentation page. The Project also produced a number of cookbooks on various astronomical topics.

By the end, the Project's codebase consisted of around 100 components, totalling around 2100kSLOC written by the project or curated by it, in various languages including Fortran, C, C++, Java, Perl and Tcl/Tk, plus another 700kSLOC of customised third-party code. Just to put this in context, David A Wheeler's SLOCCount program would have us believe that that's worth over $100M and, according to his analysis of the RH7.1 distribution , it appears that 2.1MSLOC is larger than anything in that distribution except the kernel, Mozilla and XFree86. The Project was busy.

Obtaining the software

At present, though funding for the Project has ceased, the software is still available, either as prebuilt distributions, or from a Git repository: see the Starlink developers' pages.

The Joint Astronomy Centre took over the maintenance of the Starlink codebase (with support from STFC), and has made the following releases since:

• Keoe (Vega) release on 2006 September 7
• Hokulei (Capella) release in Spring 2007 March 1
• Puana (Procyon) release on 2007 August 22
• Humu (Altair) release on 2008 February 8.
• Lehuakona (Antares) release on 2008 November 12.
• Nanahope (Pollux) release on 2009 July 27.
• Hawaiki (Deneb) release on 2010 January 20.

External links

• JAC Starlink page

References

1. ^  1982 article discussing the early history of the project (Disney, M. J.; Wallace, P. T., QJ Royal Astron. Soc. vol.23, p.485 (1982), courtesy of ADS)
2. ^  David A Wheeler, More Than a Gigabuck: Estimating GNU/Linux's Size (2002). Available online (cited September 2005).