Lua Sandbox Extensions

Overview

Package management for Lua Sandbox
modules and sandboxes. The goal is to simplify the lua_sandbox core by
decoupling the module and businesss logic maintenance and deployment.

Full Documentation

Installation

Prerequisites

• C compiler (GCC 4.7+, Visual Studio 2013)
• CMake (3.6+) - http://cmake.org/cmake/resources/software.html
• Git http://git-scm.com/download
• luasandbox (1.2+) https://github.com/mozilla-services/lua_sandbox
• Module specific (i.e. if buiding the ssl module openssl will be required)

Optional (used for documentation)

• gitbook (2.3) - https://www.gitbook.com/
• lua (5.1) - https://www.lua.org/download.html

CMake Build Instructions

git clone https://github.com/mozilla-services/lua_sandbox_extensions.git
cd lua_sandbox_extensions
mkdir release
cd release

# UNIX
cmake -DCMAKE_BUILD_TYPE=release -DENABLE_ALL_EXT=true -DCPACK_GENERATOR=TGZ ..
# or cherry pick using -DEXT_xxx=on i.e. -DEXT_lpeg=on (specifying no
# extension will provide a list of all available extensions)
make
ctest
make packages

# Windows Visual Studio 2013
cmake -DCMAKE_BUILD_TYPE=release -G "NMake Makefiles" -DEXT_lpeg=on ..
nmake
ctest
nmake packages

Docker Images

Docker images are built from
the dev and master branches. These images contain hindsight,
lua_sandbox and have all of the extensions
from this repository installed.

# Get master branch docker image
docker pull mozilla/lua_sandbox_extensions:master

# Get dev branch docker image
docker pull mozilla/lua_sandbox_extensions:dev

Releases

• The master branch is the current release and is considered stable at all
  times.
• New versions can be released as frequently as every two weeks (our sprint
  cycle). The only exception would be for a high priority patch.
• All active work is flagged with the sprint milestone and tracked in the
  project dashboard.
• New releases occur the day after the sprint finishes.
  • The version in the dev branch is updated
  • The changes are merged into master
  • A new tag is created

Contributions

• All pull requests must be made against the dev branch, direct commits to
  master are not permitted.
• All non trivial contributions should start with an issue being filed (if it is
  a new feature please propose your design/approach before doing any work as not
  all feature requests are accepted).

Heka Sandbox

Decoder API Convention

Each decoder module should implement a decode function according to the
specification below. Also, if the decoder requires configuration options it
should look for a table, in the cfg, with a variable name matching the module
name (periods replaced by underscores "decoders.foo" -> "decoders_foo"). This
naming convention only allows for a single instance of each decoder per sandbox
i.e., if you need to parse more than one type of Nginx access log format you
should use multiple input sandboxes (one for each).

decode

The decode function should parse, decode, and/or transform the original data and
inject one or more Heka messages into the system.

Arguments

• data (string) - Raw data from the input sandbox that needs
  parsing/decoding/transforming
• default_headers (table/nil/none) - Heka message table containing the default
  header values to use, if they are not populated by the decoder. If 'Fields'
  is specified it should be in the hashed based format see:
  http://mozilla-services.github.io/lua_sandbox/heka/message.html. In the case
  of multiple decoders this may be the message from the previous input/decoding
  step.
• mutable (bool/nil/none) - Flag indicating if the decoder can modify the
  default_headers/msg structure in place or if it has to be copied first.

Return

• err (nil, string) or throws an error on invalid data or an inject message
  failure
  • nil - if the decode was successful
  • string - error message if the decode failed (e.g. no match)

Encoder API Convention

Each encoder module should implement an encode function according to the
specification below. Also, if the encoder requires configuration options it
should look for a table, in the cfg, with a variable name matching the module
name (periods replaced by underscores "encoders.foo" -> "encoders_foo").

encode

The encode function should concatenate, encode, and/or transform the Heka
message into a byte array.

Arguments

• none

Return

• data (string, userdata, nil)
  • string - raw data ready to be output
  • userdata - a userdata object that supports the lua_sandbox zero copy API
  • nil - the output sandbox should skip the message and return -2
  • error - throws an error on an invalid transformation or incompatible
    userdata