(parameterized-packages:-an-update . (blog . (lispy . tech)))

I started work on adding Parameterized Packages to GNU Guix 3 weeks ago, and this post is a short status update on the things I've done so far.

1. Summary of additions

I have written

Record types for parameters
Processors for boolean, non-negative parameters
Macros for using parameters inside package definitions

I started my work by writing draft parsers that acted on S-expressions, before moving on to records. The parameterization process follows 3 phases:

Reading values from the parameter-spec record
Resolving the values against user-input, and returning a final parameter list
Applying transforms and macros in accordance with the final parameter list

The parsers can currently resolve boolean and non-negative parameters. This means that parameters can either be on or off, and the parser does not automatically assume that parameter! implies (not parameter). I am now working on adding support for enumerable parameters (i.e. parameters with multiple values which are not just on and off) and adding support for negation of boolean parameters. I next wrote Guix-style record types for parameters, parameter types and parameter specification. I also wrote sanitizers and macros to make it convenient to input values in these records, as the code otherwise gets a bit boilerplate-y. I then ported the previously written parsers to these newly made records, and also wrote a few macros that let you use parameters inside special if and cond-style expressions inside the package definition to have conditional options for fields depending on if one or all of the required parameters are switched on. Next I wrote a macro package-with-parameters that returns a package definition with the default parameter transforms applied. This means that if by default my package uses two parameters a and b, the macro will return the package with both their parameter transforms applied to it. Lastly, I wrote a modified modify-inputs macro that accepts parameters as conditions for the traditional modify inputs actions.

2. Illustrative example

EDIT 9/18/2023: The example has been updated to match the latest syntax. The following code builds Guile from the git source. Please keep in mind that the syntax is still heavily subject to change, however the underlying mechanisms will remain consistent.

(define-public guile-parameterized
  (package-with-parameters
   (parameter-spec
    (local
        (list
         (package-parameter
          (name "git")
          (variant 
           (parameter-variant-match
            (_ (with-git-url #:package-name
                             "https://git.savannah.gnu.org/git/guile.git")))))))
    (defaults '((git off))))
   (inherit guile-3.0)
   (name "guile-parameterized")
   (inputs (parameter-modify-inputs (package-inputs guile-3.0)
                                    (((git on)) (append autoconf)
                                     (append automake)
                                     (append libtool)
                                     (append gnu-gettext)
                                     (append flex)
                                     (append gperf)
                                     (append texinfo))))
   (arguments (append
               (parameter-if ((git on))
                             (list #:make-flags #~'("VERBOSE=1"))
                             '())
               (package-arguments guile-3.0)))))

The package-with-parameters macro takes in a parameter-spec and a package definition and gets the default parameter alist from it, and applies transformations according to it. Here we're defining a version of guile that inherits guile code and adds a parameter-spec property. Inside the parameter-spec definition, we start by defining local parameters. These parameters are only available to the package, and override any global parameters with the same name. This overriding feature will make parameter specifications resilient to globalization of previously local parameters, and also make it possible to globally declare what local parameters you'd like to access across packages. The name field of the parameter record accepts both strings and symbols, but when referring to a parameter in other fields one must use a symbol. The variants field is explained in further detail in the next blog post. The transforms follow the usual cons cell syntax used when defining package variants. Next we have the defaults field, which takes a list of parameters which are expected to be switched on by default. Some fields of the parameter-spec record omitted from this example are:

optional, for declaring global parameters that can optionally be used
one-of, which is a list of lists of parameters of which only one can be used per list
required, which is a list of parameters that are absolutely required. It exists mostly for global parameters
canonical, which contains canonical combinations, a proposed feature for solving the substitute problem
parameter-alist, (not meant to be modified by the user) contains the final list of active parameters and their values, on or off.

These all come together to make it possible to define an arbitrary combination of parameters in arbitrary states and test them against the parameter-spec to see if they work and apply them if they do. The package-with-parameters macro is proof of this working, it calculates transforms pertaining to default values and applies them to the package record defined inside it based on the contents of the parameter-spec.

3. Parametric Conditionals

I have written a number of conditional macros that check if a given parameter is set to on in the parameter-alist and update the package record appropriately. parameter-if and parameter-modify-inputs have been used in the example above, and below is an explanation of how they work:

3.1. parameter-if

parameter-if takes a parameter or a list of parameters and checks if any of them are on. If they are, it returns the first expression, but if all of them are off, it returns either nothing or the second expression. It behaves similarly to Guile's if macro. It is being used in this snippet from the guile-parameterized example:

;; inside package definition
(arguments (append
            (parameter-if ((git on))
                          (list #:make-flags #~'("VERBOSE=1"))
                          '())
            (package-arguments guile-3.0)))

Here, the arguments field is given a list formed by appending #:make-flags with the value "VERBOSE=1" if the parameter git is set to on, or appending an empty list '() otherwise.

3.2. parameter-match

parameter-match is somewhat similar to Guile's cond, but also very different. It takes in a set of lists of the form ((parameters ...) clauses ...), wherein if any in the list of parameters is set to on, the clauses are executed. This behavior is not short-circuiting, and the other lists are checked once one is evaluated regardless of the result. A list may be prefixed with #:all if all parameters are required to be switched on. Also, a _ can be used to always match. For example, the parameter-if example above can be rewritten with parameter-match like this:

(arguments (append
            (parameter-match
             (((git on)) (list #:make-flags #~'("VERBOSE=1")))
             (_ '()))
            (package-arguments guile-3.0)))

parameter-match has a variant called parameter-match-case which is the same as parameter/match, but it short-circuits when a matching list is found.

3.3. parameter/modify-inputs

The modify-inputs macro is used very frequently when defining package variants, but due to it being a macro we cannot use parameter-match inside it to pick arguments. To this end, I have defined a new macro called parameter-modify-inputs that takes in a list of parameters and a corresponding list of arguments to modify-inputs that can be used instead of it. _ can be used to always execute the clauses, and #:all may be used to require all parameters to be positive. In the example package above, it has been used like this:

;; inside the package definition
(parameter-modify-inputs (package-inputs guile-3.0)
                                    (((git on)) (append autoconf)
                                     (append automake)
                                     (append libtool)
                                     (append gnu-gettext)
                                     (append flex)
                                     (append gperf)
                                     (append texinfo)))

Here, if the parameter git is switched on, autoconf, automake, libtool, gnu-gettext, flex, gperf and texinfo are added to the package's inputs. This is quite useful as these inputs are required for building guile from its git source.

4. Global Parameters

The handling of global parameters is an important topic that needs more discussion. Right now, the idea is to require all global parameters to be defined in one file and to access them through a hash-table called %global-parameters. This is to ensure that global parameter symbols are both short and unique. To make the process of adding values to this hash-table easier, I've written a macro called define-global-parameter that takes a parameter definition and makes it global. For example, if I wanted to define a global parameter that disables tests for guile-3.0, I can do it like this:

(define-global-parameter
  (package-parameter
    (name "guile-3.0-tests!")
    (description "Disables tests for Guile 3.0")
    (variants
     (parameter-variant-match
      (_ (without-tests . "guile-3.0"))))))

Now any package that uses this global transform will have guile-3.0's tests disabled.

5. Results

It is now possible to define a package with parameters and change the parameter-alist to use the parameters. Next, I'll be working on parsing negated and enumerated parameters, along with adding support for modify-inputs and package-rewriting in the parameter record itself.

EDIT: This work has been done, and can be seen in the next blog post.

Stay tuned for updates, and happy hacking!

Parameterized Packages: An Update