*ClassAds*
==========

The ClassAd language consists of two parts: structured data (called
"ClassAds"), and expressions.

HTCondor uses ClassAds to describe various things, primarily machines and
jobs; it uses expressions as constraints for querying ClassAds,
and for defining what it means for two ClassAds to match each other.

.. _`Data Syntax`:

Data Syntax
-----------

A ClassAd is a list of attribute-value pairs, separated by newlines.
Values may be booleans, integers, reals, strings, dictionaries, lists,
or the special values UNDEFINED and ERROR.
Dictionaries are marked by square brackets and lists by braces;
dictionaries separate elements with semicolons,
and lists separate elements with commas.

.. code-block:: condor-classad

    attribute_name  = "attribute-value"
    pi              = 3.141
    count           = 3
    list            = { "red", "green", "blue" }
    dictionary      = [ type = "complex"; real = 7.75; imaginary = -3 ]
    structured_attr = [ hostnames = { "submit-1", "submit", "submit1" };
                        ip = "127.0.0.1"; port = "9618" ]

For the list of ClassAd attributes generated by HTCondor, see
https://htcondor.readthedocs.io/en/latest/classad-attributes/index.html.

.. _`Expression Syntax`:

Expression Syntax
-----------------

An expression consists of literals (from the data syntax) and attribute
references composed with operators and functions.  The value of a ClassAd
attribute may be an expression.

.. code-block:: condor-classad-expr

    MY.count < 10 && regexp( ".*example.*", attribute_name )

Attribute References
''''''''''''''''''''

An attribute reference always includes an attribute name.  In HTCondor,
when determining if two ClassAds match, an expression may specify which
ad's value is used by prefixing it with ``MY.`` or ``TARGET.``.  Attribute
references are case-insensitive.

.. code-block:: condor-classad-expr

    MY.count
    TARGET.machine

An element of a dictionary is referenced by using the subscript operator
(``[]``) with an expression that evaluates to a string *or* with a dot
(``.``), as follows:

.. code-block:: condor-classad-expr

    MY.structured_attr.hostnames
    MY.structured_attr["hostnames"]

Note that the following references the attribute named by the attribute
``hostnames``, not the attribute named hostnames:

.. code-block:: condor-classad-expr

    MY.structured_attr[hostnames]

List elements are referenced by an expression that evaluates to an
integer, where the first element in the list is numbered 0.  For
example, if ``colors = { [ x = "1" ], [ x = "2", y = "3" ] }``, then
``MY.structure_attr.colors[0]`` results in ``[ x = "1" ]``.

UNDEFINED and ERROR
'''''''''''''''''''

The ClassAd language includes two special values, UNDEFINED and ERROR.
An attribute may be set to either explicitly, but these values typically
result from referring to an attribute that doesn't exist, or asking
for something impossible:

.. code-block:: condor-classad

    undefined_reference = MY.undefined_attribute
    explicitly_undefined = UNDEFINED
    one_error_value = "three" * 7
    another_error_value = 1.3 / 0

Most expressions that refer to values that are UNDEFINED will evaluate
to UNDEFINED.  The same applies to ERROR.

Operators
'''''''''

The operators ``*``, ``/``, ``+`` and ``-`` operate arithmetically, on
integers and reals.

The comparison operators ``==``, ``!=``, ``<=``, ``<``, ``>=`` and ``>``
operate on booleans, integers, reals and strings.  String comparison is
case-insensitive.  Comparing a string and a non-string value results in
ERROR.

The special comparison operator ``=?=`` is like ``==`` except in the
following two ways: it is case-sensitive when comparing strings; and,
when comparing values to UNDEFINED, results in FALSE instead of UNDEFINED.
(If comparing UNDEFINED to itself, the operator ``=?=`` results in TRUE).

The special comparison operator ``=!=`` is the negation of ``=?=``.

The logical operators ``&&`` and ``||`` operate on integers and reals;
non-zero is true, and zero is false.

The ternary operator ``x ? y : z`` operates on expressions.

The default operator ``x ?: z`` returns ``x`` if ``x`` is defined
and ``z`` otherwise.

The ``IS`` and ``ISNT`` operators are synonyms for ``=?=`` and ``=!=``.

Functions
'''''''''

Function name are case-insensitive.  Unless otherwise noted, if any of a
function's arguments are UNDEFINED or ERROR, so is the result.  If an
argument's type is noted, the function will return ERROR unless the argument
has that type.

.. glossary::
    integer INT( *expr* )
        If *expr* is numeric, return the closest integer.  If *expr*
        evaluates to a string, attempt to convert the string to an
        integer.  Return ERROR if the string is not an integer, or
        if *expr* is neither numeric nor a string.

.. glossary::
    boolean MEMBER( *expr*, list *l* )
        Returns TRUE if *expr* is equal, as defined by the operator ``==``,
        to any member of the list *l*, or FALSE if it isn't.

.. glossary::
    boolean REGEXP( string *pattern*, string *target*\ [, string *options*] )
        Return TRUE if the PCRE regular expression *pattern* matches *target*,
        or FALSE if it doesn't.
        Return ERROR if *pattern* is a not a valid regular expression.
        If specified, *options* is a PCRE option string (one or more
        of ``f``, ``i``, ``s``, ``m``, and ``g``).  See the :ref:`Specification`
        section for details.

.. glossary::
    list SPLIT( string *s*\ [, string *tokens* ] )
        Separate *s* by whitespace or comma, or instead by any of the
        characters in *tokens*, if specified, and return the result as
        a list of strings.

.. glossary::
    boolean STRINGLISTIMEMBER( string *s*, string *list*\ [, string *tokens*] )
        Equivalent to ``MEMBER( *s*, SPLIT( *list*, *tokens* ))``.

.. glossary::
    string SUBSTR( string *s*, integer *offset*\ [, integer *length*] )
        Returns the substring of *s* from *offset* to the end of the string,
        or instead for *length* characters, if specified.  The first
        character in *s* is at position 0.  If *offset* is negative,
        the substring begins *offset* characters before the end of the
        string.  If *length* is negative, the substring ends that many
        characters before the end of the string.  If the substring contains
        no characters, return the empty string.  Thus, the following two
        calls both return the string "78":

        .. code-block:: text

            substr( "0123456789", 7, 2 )
            substr( "0123456789", -3, -1 )

All ClassAd functions are defined in the references below.

Reserved Words
''''''''''''''

The words ``UNDEFINED``, ``ERROR``, ``IS``, ``ISNT``, ``TRUE``, ``FALSE``,
``MY``, ``TARGET``, and ``PARENT`` may not be used as attribute names.

Testing ClassAd Expressions
---------------------------

Use `classad_eval` to test ClassAd expressions.  For
instance, if you want to test to see if a regular expression matches
some fixed string, you could check in the following way (on Linux or Mac;
the quoting rules are different on Windows):

.. code-block:: console

    $ classad_eval 'regexp( ".*tr.*", "string" )'
    [ ]
    true

This prints out the ClassAd used as context in the evaluation (in this case,
there wasn't one, so it's the empty ad) and the result.



Examples
--------

These examples assume a Linux shell environment and a working HTCondor pool.

Selecting a Slot based on Job ID
''''''''''''''''''''''''''''''''

If job 288.7 is running:

.. code-block:: console

    $ condor_status -const 'JobId == "288.7"'

Selecting Jobs based on Execute Machine
'''''''''''''''''''''''''''''''''''''''

If jobs are running on the machine ``example-execute-node``:

.. code-block:: console

    $ condor_q -all -const 'regexp( "@example-execute-node$", RemoteHost )'

String Manipulation
'''''''''''''''''''

In this example, an administrator has just added twelve new hosts
to the pool -- ``compute-296`` to ``compute-307`` -- and wants to see if
they've started running jobs yet.

.. code-block:: console

    $ condor_status -const '296 <= int(substr( Machine, 8 )) && int(substr( Machine, 8 )) <= 307'

You could also write this as follows:

.. code-block:: console

    $ condor_status -const '296 <= int(split(Machine, "-")[1]) && int(split(Machine, "-")[1]) <= 307'

Selecting Machines with a Particular File-Transfer Plugin
'''''''''''''''''''''''''''''''''''''''''''''''''''''''''

If you're considering using the gdrive file-transfer plugin, and you'd like
to see which machines have it, select from the slot ads based on the
corresponding attribute, but only print out the machine name, and then
throw away the duplicates:

.. code-block:: console

    $ condor_status -af Machine \
        -const 'StringListIMember( "gdrive", HasFileTransferPluginMethods )' \
        | uniq

You could instead use a constraint to ignore dynamic slots for a report
on the resources available to run jobs which require the gdrive plugin.
Note that you can also use expressions when formatting the output.  In
this case, it's just to make the output prettier.

.. code-block:: console

    $ condor_status -af Machine CPUs Memory Disk \
        '(GPUs =!= undefined && GPUs >= 1) ? CUDACapability : "[no GPUs]"' \
        -const 'SlotType =!= "Dynamic" && StringListIMember( "gdrive", HasFileTransferPluginMethods )'

.. _`Specification`:

Specification
-------------

For use in HTCondor, including a complete list of functions, see
https://htcondor.readthedocs.io/en/latest/misc-concepts/classad-mechanism.html.

For the language specification,
see https://research.cs.wisc.edu/htcondor/classad/refman/.

Author
------

Center for High Throughput Computing, University of Wisconsin-Madison

Copyright
---------

Copyright © 1990-2019 Center for High Throughput Computing, Computer
Sciences Department, University of Wisconsin-Madison, Madison, WI. All
Rights Reserved. Licensed under the Apache License, Version 2.0.
