stilts-taplint - Tests TAP services

Author

       Mark Taylor (Bristol University)

                                                    Mar 2017                                   STILTS-TAPLINT(1)

Description

taplint runs a series of tests on a Table Access Protocol (TAP) service and reports the results. Unlike
most of the other tools in this package it is not likely to be of use to normal users; its intended use
is for people developing or operating TAP services to assess their services, perhaps with a view to im‐
proving compliance.

Testing takes place in a number of stages; it is possible to choose which stages are run in by using the
stages parameter. The default output (format=text) is line-based text to standard output, and each report
line is of the (fairly greppable) form: T-SSS-MMMMxN aaaaa... where the parts have the following mean‐
ings:

* T: Report type, one of E(rror), W(arning), I(nfo), S(ummary), F(ailure). See the documentation of the
report parameter for further description of what these mean. The report parameter can be used to sup‐
press some of these; only E indicates actual service compliance errors, but including the others may
make it easier to see what's going on.

* SSS: Stage abbreviation, as used in the stages parameter. The stages parameter can be used to select
which stages are run.

* MMMM: Message label, which is always the same for messages generated by the same test, is usually
different for messages generated by different tests, and may be somewhat mnemonic.

* x: Continuation indicator, either "-" or "+". In most cases it is "-", indicating the first line of a
message, but multi-line messages (rare) use "-" for the first line and "+" for any continuation
lines.

* N: Sequence number, which is 1 for the first time message T-SSS-MMMM is reported, and increases by
one for each subsequent appearance. After a certain maximum (determined by the maxrepeat parameter)
additional reports with the same code are no longer output individually, but a summary of the number
of reports so discarded is written at the end of the section with the character "x" instead of the
sequence number. This behaviour prevents the output being swamped by multiple reports of the same is‐
sue. If the maxrepeat parameter is increased above 9, more than one digit will be used here (so e.g.
for maxrepeat=999, the format would be NNN not N).

* aaaaa...: Message text, a free text description of what is being reported.

If you don't like that format, others may be selected using the format parameter, which currently also
supports JSON. For more flexible interaction with the output you can invoke taplint programmatically and
supply your own instance.

TAP is a complicated beast, referencing many standards (including TAP, UWS, VODataService, ADQL, VORe‐
source, VOSI, TAPRegExt, DALI, ObsCore, ObsLocTAP, EPN-TAP, VOTable, UCD, VOUnits, SSO, SoftID, HTTP, RD‐
Fa Lite), and it is hard to write a validator which is comprehensive, especially one which can provide
useful output for services with a range of compliance levels. This tool tries to make a wide range of
tests, but does not claim to be comprehensive. An idea of what tests it does perform can be gained from
the stages listed in the description of the stages parameter. It does make a fairly good job of checking
that declared metadata is consistent and matches the data actually returned from queries, it tests job
submission in most of the various ways permitted by the TAP standard, and it checks all returned VOTables
by effectively running them through votlint. Things it does not test much include complex ADQL queries,
coordinate/STC-related data types, queries in non-ADQL languages, and service registration.

HTTP connections made by this validator are flagged in the User-Agent field with the token "(IVOA-test)".

Name

       stilts-taplint - Tests TAP services

Options

tapurl=<url-value>
The base URL of a Table Access Protocol service. This is the bare URL without a trailing
"/[a]sync".

In the usual case, the default values of the various endpoints (sync and async query submission,
tables metadata, service-provided examples etc) use this URL as a parent and append standard sub-
paths.

In some cases however, determination of the endpoints is more complicated, as determined by the
interface parameter which may cause endpoints to be read from the capabilities document at
tapurl/capabilities, and by other endpoint-specific parameters (syncurl, asyncurl, tablesurl, ca‐pabilitiesurl, availabilityurl, examplesurl) for fine tuning.

stages=[+/-]XXX...
Determines the validation stages which the validator will peform. Each stage is represented by a
short code, as follows:

* TMV: Validate table metadata against XML schema

* TME: Check content of tables metadata from /tables

* TMS: Check content of tables metadata from TAP_SCHEMA

* TMC: Compare table metadata from /tables and TAP_SCHEMA

* CPV: Validate capabilities against XML schema

* CAP: Check TAP and TAPRegExt content of capabilities document

* AVV: Validate availability against XML schema

* QGE: Make ADQL queries in sync GET mode

* QPO: Make ADQL queries in sync POST mode

* QAS: Make ADQL queries in async mode

* UWS: Test asynchronous UWS/TAP behaviour

* MDQ: Check table query result columns against declared metadata

* OBS: Test implementation of ObsCore Data Model

* LOC: Test implementation of ObsLocTAP Data Model

* UPL: Make queries with table uploads

* EXA: Check content of examples document

This parameter can specify what stages to run in the following ways:

* if left blank, the default list of stages (which is most or all of them) will be run

* if the value is a space-separated list of three-letter codes, it lists the stages that will be
run

* if the value is a space separated list of three-letter codes preceded by a "+" or "-" charac‐
ter, the named stages will be removed or added to the default list
So "TMECAP" will run only Table Metadata and Capability stages, while "-EXA-UPL" will run all
the default stages apart from Examples and Upload. The order in which stages are listed is not
significant.

Note that removing some stages may affect the operation of others; for instance table metadata is
acquired from the metadata stages, and avoiding those will mean that later stages which use the
table metadata to pose queries will not be able to do so with knowledge of the database schema.

maxtable=<int-value>
Limits the number of tables from the service that will be tested. Currently, this only affects
stage MDQ. If the value is left blank (the default), or if it is larger than the number of tables
actually present in the service, it will have no effect.

tables=<name-list>
If supplied, this specifies a list of tables to test. It may be set to a space- or comma-separated
list of table names for consideration; any tables not covered by this list are mostly ignored for
the purposes of reporting. Matching against table names is case-insensitive, and the asterisk
character "*" may be used as a wildcard to match any sequence of characters. Note that matching is
against the declared table name which may or may not include a schema name prefix depending on
service behaviour.

By default this parameter is not set, which means that all tables are considered.

format=text|json
Determines the format of the output. Possible values are text, json.

Note not all of the other parameters may be applicable to all output formats.

report=[EWISF]+
Letters indicating which message types should be listed. Each character of the string is one of
the letters E, W, I, S, F with the following meanings:

* E: Error in operation or standard compliance of the service.

* W: Warning that service behaviour is questionable, or contravenes a standard recommendation,
but is not in actual violation of the standard.

* I: Information about progress, for instance details of queries made.

* S: Summary of previous successful/unsuccessful reports.

* F: Failure of the validator to perform some testing. The cause is either some error internal
to the validator, or some error or missing functionality in the service which has already been
reported.

maxrepeat=<int-value>
Puts a limit on the number of times that a single message will be repeated. By setting this to
some reasonably small number, you can ensure that the output does not get cluttered up by millions
of repetitions of essentially the same error.

truncate=<int-value>
Limits the line length written to the output.

debug=true|false
If true, debugging output including stack traces will be output along with the normal validation
messages.

interface=tap1.0|tap1.1|cap
Defines how the service endpoints and the version of the TAP protocol to use for queries is deter‐
mined. This may take one of the following (case-insensitive) values:

* TAP1.0: The standard TAP endpoints are used, without examining the service's capabilities doc‐
ument. The service is queried using version 1.0 of the TAP protocol.

* TAP1.1: The standard TAP endpoints are used, without examining the service's capabilities doc‐
ument. The service is queried using version 1.1 of the TAP protocol.

* cap: The service's capabilities document is examined, and the endpoints listed there are used.

The capabilities document, if used, is read from tapurl/capabilities unless the capabilitiesurl
parameter is defined, in which case that is used.

The baseline value of all the TAP service endpoints (sync, async, tables, capabilities, examples)
are determined by this parameter, but each of those endpoint values may be overridden individually
by other endpoint-specific parameters (syncurl, asyncurl, tablesurl, capabilitiesurl, availabili‐tyurl, examplesurl)

For default (unauthenticated) access, the default value is usually suitable.

auth=true|false
If true, then an attempt will be made to authenticate with the TAP service even if anonymous oper‐
ation is permitted. If the service offers authentication, you will be asked for credentials.

To use this option in non-interactive contexts, you may want to use the auth.username and
auth.password system properties.

syncurl=<url-value>
Sets the URL to use for the sync endpoint of the TAP service. The default value would be
<tapurl>/sync, but it may be influenced by the chosen interface value, and it can be further over‐
ridden by setting this value.

asyncurl=<url-value>
Sets the URL to use for the async endpoint of the TAP service. The default value would be
<tapurl>/async, but it may be influenced by the chosen interface value, and it can be further
overridden by setting this value.

tablesurl=<url-value>
Sets the URL to use for the tables endpoint of the TAP service. The default value would be
<tapurl>/tables, but it may be influenced by the chosen interface value, and it can be further
overridden by setting this value.

capabilitiesurl=<url-value>
Sets the URL to use for the capabilities endpoint of the TAP service. The default value would be
<tapurl>/capabilities, but it may be influenced by the chosen interface value, and it can be fur‐
ther overridden by setting this value.

availabilityurl=<url-value>
Sets the URL to use for the availability endpoint of the TAP service. The default value would be
<tapurl>/availability, but it may be influenced by the chosen interface value, and it can be fur‐
ther overridden by setting this value.

examplesurl=<url-value>
Sets the URL to use for the examples endpoint of the TAP service. The default value would be
<tapurl>/examples, but it may be influenced by the chosen interface value, and it can be further
overridden by setting this value.

Synopsis

stiltstaplint [tapurl=<url-value>] [stages=[+/-]XXX...] [maxtable=<int-value>] [tables=<name-list>]
                      [format=text|json] [report=[EWISF]+] [maxrepeat=<int-value>] [truncate=<int-value>]
                      [debug=true|false] [interface=tap1.0|tap1.1|cap] [auth=true|false] [syncurl=<url-value>]
                      [asyncurl=<url-value>] [tablesurl=<url-value>] [capabilitiesurl=<url-value>]
                      [availabilityurl=<url-value>] [examplesurl=<url-value>]

Version

       STILTS version 3.5.1-debian

       This  is the Debian version of Stilts, which lack the support of some file formats and network protocols.
       For differences see
       file:///usr/share/doc/stilts/README.Debian