Bang

Bang is a Ninja file generator scriptable in LuaX.

If you like Bang (or LuaX) and are willing to support its development, please consider donating via Github or Liberapay.

Installation

Bang is written in LuaX. It can be compiled with Ninja and LuaX.

LuaX

$ git clone https://github.com/CDSoft/luax
$ cd luax
$ ./bootstrap.sh
$ ninja install  # install LuaX to ~/.local/bin

Bang

$ git clone https://github.com/CDSoft/bang
$ cd bang
$ ./boot.lua
$ ninja install  # build bang with Ninja and install it to ~/.local/bin

or set $PREFIX to install bang to a custom directory ($PREFIX/bin):

$ PREFIX=/path ninja install # install bang to /path/bin

Pure Lua implementation

Bang also comes with a pure Lua implementation for environments where LuaX can not be executed. In this case $PREFIX/bin/bang.lua can be executed with any standard Lua 5.4 interpreter.

Precompiled binaries

In case precompiled binaries are needed (GNU/Linux, MacOS, Windows), some can be found at cdelord.fr/pub. These archives contain bang as well as some other softwares more or less related to LuaX.

Usage

$ bang -h
Usage: bang [-h] [-v] [-q] [-g cmd] [-o output] [<input>]

Ninja file generator

Arguments after "--" are given to the input script

Arguments:
   input                 Lua script (default: build.lua)

Options:
   -h, --help            Show this help message and exit.
   -v                    Print Bang version
   -q                    Quiet mode (no output on stdout)
   -g cmd                Set a custom command for the generator rule
   -o output             Output file (default: build.ninja)

For more information, see https://github.com/CDSoft/bang

• bang reads build.lua and produces build.ninja.
• bang input.lua -o output.ninja reads input.lua and produces output.ninja.

Ninja functions

Comments

bang can add comments to the Ninja file:

comment "This is a comment added to the Ninja file"

section adds comments separated by horizontal lines:

section [[
A large title
that can run on several lines
]]

Variables

var adds a new variable definition:

var "varname" "string value"
var "varname" (number)
var "varname" {"word_1", "word_2", ...} -- will produce `varname = word_1 word_2 ...`

var returns the name of the variable (prefixed with "$").

The global variable vars is a table containing a copy of all the Ninja variables defined by the var function. vars has a function vars.expand that takes a string and expands all Ninja variables (i.e. prefixed with "$").

var "foo" "xyz"
...
vars.foo    -- "xyz"
vars["foo"] -- same as vars.foo
vars.expand "$foo/bar" -- "xyz/bar"

Ninja required version

The special variable ninja_required_version shall be set by the ninja_required_version function. ninja_required_version will change the default required version only if the script requires a higher version.

ninja_required_version "1.42"

Rules

rule adds a new rule definition:

rule "rule_name" {
    description = "...",
    command = "...",
    -- ...
}

Variable values can be strings or lists of strings. Lists of strings are flattened and concatenated (separated with spaces).

Rules can defined variables (see Rule variables).

Bang allows some build statement variables to be defined at the rule level:

• implicit_in: list of implicit inputs common to all build statements
• implicit_out: list of implicit outputs common to all build statements
• order_only_deps: list of order-only dependencies common to all build statements

These variables are added at the beginning of the corresponding variables in the build statements that use this rule.

The rule function returns the name of the rule ("rule_name").

Build statements

build adds a new build statement:

build "outputs" { "rule_name", "inputs" }

generates the build statement build outputs: rule_name inputs. The first word of the input list (rule_name) shall be the rule name applied by the build statement.

The build statement can be added some variable definitions in the inputs table:

build "outputs" { "rule_name", "inputs",
    varname = "value",
    -- ...
}

There are reserved variable names for bang to specify implicit inputs and outputs, dependency orders and validation statements:

build "outputs" { "rule_name", "inputs",
    implicit_out = "implicit outputs",
    implicit_in = "implicit inputs",
    order_only_deps = "order-only dependencies",
    validations = "build statements used as validations",
    -- ...
}

The build function returns the outputs ("outputs"), as a string if outputs contains a single output or a list of string otherwise.

Rules embedded in build statements

Some rules are specific to a single output and are used once. This leads to write pairs of rules and build statements.

Bang can merge rules and build statements into a single build statement containing the definition of the associated rule.

A build statement with a command variable is split into two parts:

1. a rule with all rule variables found in the build statement definition
2. a build statement with the remaining variables

In this case, the build statement definition does not contain any rule name.

E.g.:

build "output" { "inputs",
    command = "...",
}

is internally translated into:

rule "output" {
    command = "...",
}

build "output" { "output", "inputs" }

Pools

pool adds a pool definition:

pool "name" {
    depth = pool_depth
}

The pool function returns the name of the pool ("pool_name").

Default targets

default adds targets to the default target:

default "target1"
default {"target2", "target3"}

Phony targets

phony is a shortcut to build that uses the phony rule:

phony "all" {"target1", "target2"}
-- same as
build "all" {"phony", "target1", "target2"}

Bang variables

Bang arguments

The command line arguments of bang are stored in a global table named bang. This table contains:

• bang.input: name of the Lua input script
• bang.output: name of the output Ninja file

Build script arguments

Arguments after "--" are given to the input script in the global arg table.

Bang functions

Accumulations

Bang can accumulate names (rules, targets, ...) in a list that can later be used to define other rules or build statements.

A standard way to do this in Lua would use a Lua table and table.concat or the list[#list+1] pattern. Bang provides a simple function to simplify this usage:

my_list = {}
-- ...
acc(my_list) "item1"
acc(my_list) {"item2", "item3"}
--...
my_list -- contains {"item1", "item2", "item3"}

Case expressions

The case function provides a switch-like structure to simplify conditional expressions. case is a curried function that takes a value (generally a string) and a table. It searches for the value in the keys of the table and returns the associated value. If the key is not found it returns the value associated to the Nil key or nil.

E.g.:

local cflags = {
    case(mode) {
        debug   = "-g -Og",
        release = "-s -O3",
        [Nil]   = Nil,  -- equivalent to [Nil] = {}
    }
}

Note: Nil is a special value that can be used to represent nothing (no value) in a list. Nil is ignored by bang.

File listing

The ls function lists files in a directory. It returns a list of filenames, with the metatable of LuaX F lists.

• ls "path": list of file names in path
• ls "path/*.c": list of file names matching the "*.c" pattern in path
• ls "path/**": recursive list of file names in path
• ls "path/**.c": recursive list of file names matching the "*.c" pattern in path

E.g.:

ls "doc/*.md"
: foreach(function(doc)
    build (doc:chext".pdf") { "md_to_pdf", doc }
end)
-- where md_to_pdf is a rule to convert Markdown files to PDF

Dynamic file creation

The file function creates new files. It returns a callable object to add text to a file (note that the write method is deprecated). The file is actually written when bang exits successfully.

file "name" "content"

The file can be generated incrementally by calling the file object several times:

f = file "name"
-- ...
f "Line 1\n"
-- ...
f "Line 2\n"
-- ...

Pipes

It is common in Makefiles to write commands with pipes. But pipes can be error prone since only the failure of the last process is captured by default. A simple solution (for Makefiles or Ninja files) is to chain several rules.

The pipe function takes a list of rules and returns a function that applies all the rules, in the order of the list. This function takes two parameters: the output and the inputs of the pipe.

Intermediate outputs are stored in $builddir/tmp (this directory can be changed by adding a builddir attribute to the rule list). If a rule name contains a dot, its « extension » is used to name intermediate outputs.

E.g.:

rule "ypp.md"     { command = "ypp $in -o $out" }
rule "panda.html" { command = "panda $in -o $out", implicit_in = "foo.css" }

local ypp_then_panda = pipe { "ypp.md", "panda.md" }

ypp_then_panda "$builddir/doc/mydoc.html" "doc/mydoc.md"

is equivalent to:

build "$builddir/tmp/doc/mydoc.md" { "ypp.md", "doc/mydoc.md" }
build "$builddir/doc/mydoc.html"   { "panda.html", "$builddir/tmp/doc/mydoc.md" }

Since rule returns the name of the rule, this can also be written as:

local ypp_then_panda = pipe {
    rule "ypp.md"     { command = "ypp $in -o $out" },
    rule "panda.html" { command = "panda $in -o $out", implicit_in = "foo.css" },
}

The input list can contain variable definitions. These variables are added to all build statements, except for implicit_in and implicit_out that are added respectively to the first and last build statements only.

e.g.:

ypp_then_panda "out.html" { "in.md",
    implicit_in = "foo.in",
    implicit_out = "foo.out",
    other_var = "42",
}

is equivalent to:

build "$builddir/tmp/doc/out-1.md" { "ypp.md", "doc/in.md",
    implicit_in = "foo.in",
    other_var = "42",
}
build "out.html" { "panda.html", "$builddir/tmp/doc/out-1.md",
    implicit_out = "foo.out",
    other_var = "42",
}

Clean

Bang can generate targets to clean the generated files. The clean function takes a directory name that shall be deleted by ninja clean.

clean "$builddir"   -- `ninja clean` cleans $builddir
clean "tmp/foo"     -- `ninja clean` cleans tmp/foo

clean defines the target clean (run by ninja clean) and a line in the help message (see ninja help).

In the same vein, clean.mrproper takes directories to clean with ninja mrproper.

Install

Bang can generate targets to install files outside the build directories. The install function adds targets to be installed with ninja install

The default installation prefix can be set by install.prefix:

install.prefix "$$HOME/foo/bar"     -- `ninja install` installs to ~/foo/bar

The default prefix in ~/.local.

It can be overridden by the PREFIX environment variable when calling Ninja. E.g.:

$ PREFIX=~/bar/foo ninja install

Artifacts are added to the list of files to be installed by the function install. This function takes the name of the destination directory, relative to the prefix and the file to be installed.

install "bin" "$builddir/bang" -- installs bang to $PREFIX/bin/

install defines the target install (run by ninja install) and a line in the help message (see ninja help).

Help

Bang can generate an help message (stored in a file next to the Ninja file) displayed by ninja help.

The help message is composed of three parts:

• a description of the Ninja file
• a list of targets with their descriptions
• an epilogue

The description and epilogue are defined by the help.description and help.epilog (or help.epilogue) functions. Targets can be added by the help function. It takes the name of a target and its description.

help.description "A super useful Ninja file"
help.epilog "See https://cdelord.fr/bang"
-- ...
help "compile" "Compile every thing"
-- ...

Generator

Bang generates a generator rule to update the Ninja file when the build description changes. This behaviour can be customized or disabled with the generator function:

• generator(true): bang adds a generator rule at the end of the ninja file (default behaviour)
• generator(false): bang does not add a generator rule
• generator(t): if t is a table, bang adds a generator rule with the additional variables defined in t

The generator rule runs bang with the same options than the initial bang command.

E.g.:

generator {
    implicit_in = { "foo", "bar" },
}

In this example, the generator statement will be executed if the Lua script has changed as well as foo and bar.

Target

The target function is a simple helper function to select a luaxc compilation target from the command line. It takes a target name or a list of parameters potentially containing target names and returns the target description (an entry in sys.targets) and the remaining arguments.

E.g.:

local target, args = target(arg)
if #args > 0 then
    error(args:unwords()..": unexpected arguments")
end

--[[
if arg contains "linux-x86_64"
then target is {
     name="linux-x86_64",
     os="linux",
     arch="x86_64",
     libc="gnu",
     exe="",     -- executable extension (.exe on Windows)
     so=".so",   -- shared library extension (.dylib on MacOS, .dll on Windows)
}
--]]

-- e.g. to build a LuaX application in a specific directory:

var "builddir" (".build"/(target and target.name))

Higher level build modules

C compilers

The module C creates C compilation objects. This module is available as a build function metamethod.

The module itself is a default compiler that should works on most Linux and MacOS systems and uses cc.

A new compiler can be created by calling the new method of an existing compiler with a new name and changing some options. E.g.:

local gcc = build.C:new "gcc"   -- creates a new C compiler named "gcc"
    : set "cc" "gcc"                        -- compiler command
    : add "cflags" { "-O2", "-Iinclude" }   -- compilation flags

A compiler has two methods to modify options:

• set changes the value of an option
• add adds values to the current value of an option
• insert adds values before the current value of an option

A compiler can compile a single C source as well as complete libraries and executables. Inputs of libraries or executables can be C sources (which will be compiled in a subdirectory of builddir) or other static libraries.

Examples:

-- Compilation of a single source file
local obj_file = gcc:compile "$builddir/file.o" "file.c"

-- Creation of a static library
local lib_a = gcc:static_lib "$builddir/lib.a" {
    obj_file,       -- already compiled
    ls "lib/*.c",   -- compile and archive all sources in the lib directory
}

-- Creation of a dynamic library
local lib_so = gcc:dynamic_lib "$builddir/lib.so" {
    lib_a,
    ls "dynlib/*.c",
}

-- Creation of an executable file
local exe = gcc:executable "$builddir/file.exe" {
    lib_a,
    "src/main.c",
}

-- Same with an all-in-one statement
local exe = gcc:executable "$builddir/file.exe" {
    "file.c",
    ls "lib/*.c",
    "src/main.c",
}

-- The `__call` metamethod is a shortcut to the `executable` method
local exe = gcc "$builddir/file.exe" {
    "file.c",
    ls "lib/*.c",
    "src/main.c",
}

LuaX compilers

The module luax creates LuaX compilation objects. This module is available as a build function metamethod.

The module itself is a default compiler that generates a Lua script executable by luax.

A new compiler can be created by calling the new method of an existing compiler with a new name and changing some options. E.g.:

local luaxq = build.luax:new "luax-q"   -- creates a new LuaX compiler named "luax-q"
    : add "flags" "-q"                  -- add some compilation flags

A compiler has two methods to modify options:

• set changes the value of an option
• add adds values to the current value of an option
• insert adds values before the current value of an option

The module also provides the methods set_global, add_global and insert_global to add flags to all builtin LuaX compilers.

Examples:

-- make all LuaX compiler silent
build.luax.add_global "flags" "-q"

-- Compile hello.lua and some libs to a linux-x86_64-musl executable
build.luax["linux-x86_64-musl"] "hello" { "hello.lua", "lib1.lua", "lib2.lua" }

Builders

The build function has methods to create new builder objects (note that these methods are also available in the "builders" module).

The build metamethods contain some predefined builders:

A new builder can be created by calling the new method of an existing builder with a new name and changing some options. E.g.:

local tac = build.cat:new "tac"   -- new builder "tac" based on "cat"
    : set "cmd" "tac"

The new method of the build function can also be used to create a new builder from scratch:

local tac = build.new "tac"   -- new builder "tac"
    : set "cmd" "tac"
    : set "args" "$in > $out"

A builder has two methods to modify options:

• set changes the value of an option
• add adds values to the current value of an option
• insert adds values before the current value of an option

Other options are added to the rule definition (note that name can not be used as a rule variable).

Examples:

-- preprocess Markdown files with ypp
-- and convert them to a single HTML file with pandoc
build.pandoc "$builddir/output.html" {
    build.ypp "$builddir/output.md" { "input1.md", "input2.md" }
}

Examples

The Ninja file of bang (build.ninja) is generated by bang from build.lua.

The example directory contains a larger example:

• source files discovering
• multi-target compilation
• multiple libraries
• multiple executables

License

This file is part of bang.

bang is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

bang is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with bang.  If not, see <https://www.gnu.org/licenses/>.

For further information about bang you can visit
https://cdelord.fr/bang