# Sanitizer support

[Sanitizers](https://github.com/google/sanitizers) can be used in custom Julia builds to make it
easier to detect certain kinds of errors in Julia's internal C/C++ code.

## Address Sanitizer: easy build

From a source-checkout of Julia, you should be able to build a version
supporting address sanitization in Julia and LLVM as follows:

```sh
$ mkdir /tmp/julia
$ contrib/asan/build.sh /tmp/julia/
```

Here we've chosen `/tmp/julia` as a build directory, but you can
choose whatever you wish. Once built, run the workload you wish to
test with `/tmp/julia/julia`. Memory bugs will result in errors.

If you require customization or further detail, see the documentation below.

## General considerations

Using Clang's sanitizers obviously requires you to use Clang (`USECLANG=1`), but there's another
catch: most sanitizers require a run-time library, provided by the host compiler, while the instrumented
code generated by Julia's JIT relies on functionality from that library. This implies that the
LLVM version of your host compiler must match that of the LLVM library used within Julia.

An easy solution is to have a dedicated build folder for providing a matching toolchain, by building
with `BUILD_LLVM_CLANG=1`. You can then refer to this toolchain from another build
folder by specifying `USECLANG=1` while overriding the `CC` and `CXX` variables.

The sanitizers error out when they detect a shared library being opened using `RTLD_DEEPBIND`
(ref: [google/sanitizers#611](https://github.com/google/sanitizers/issues/611)).
Since [libblastrampoline](https://github.com/staticfloat/libblastrampoline) by default
uses `RTLD_DEEPBIND`, we need to set the environment variable `LBT_USE_RTLD_DEEPBIND=0`
when using a sanitizer.

To use one of of the sanitizers set `SANITIZE=1` and then the appropriate flag for the sanitizer you
want to use.

On macOS, this might need some extra flags also to work. Altogether, it might
look like this, plus one or more of the `SANITIZE_*` flags listed below:

    make -C deps USE_BINARYBUILDER_LLVM=0 LLVM_VER=svn stage-llvm

    make -C src SANITIZE=1 USECLANG=1 \
        CC=~+/deps/scratch/llvm-svn/build_Release/bin/clang \
        CXX=~+/deps/scratch/llvm-svn/build_Release/bin/clang++ \
        CPPFLAGS="-isysroot $(xcode-select -p)/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk" \
        CXXFLAGS="-isystem $(xcode-select -p)/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1"

(or put these into your `Make.user`, so you don't need to remember them every time).

## Address Sanitizer (ASAN)

For detecting or debugging memory bugs, you can use Clang's [address sanitizer (ASAN)](https://clang.llvm.org/docs/AddressSanitizer.html).
By compiling with `SANITIZE_ADDRESS=1` you enable ASAN for the Julia compiler and its generated code.
In addition, you can specify `LLVM_SANITIZE=1` to sanitize the LLVM library as well. Note that
these options incur a high performance and memory cost. For example, using ASAN for Julia and
LLVM makes `testall1` take 8-10 times as long while using 20 times as much memory (this can be
reduced to respectively a factor of 3 and 4 by using the options described below).

By default, Julia sets the `allow_user_segv_handler=1` ASAN flag, which is required for signal
delivery to work properly. You can define other options using the `ASAN_OPTIONS` environment flag,
in which case you'll need to repeat the default option mentioned before. For example, memory usage
can be reduced by specifying `fast_unwind_on_malloc=0` and `malloc_context_size=2`, at the cost
of backtrace accuracy. For now, Julia also sets `detect_leaks=0`, but this should be removed in
the future.

### Example setup

#### Step 1: Install toolchain

Checkout a Git worktree (or create out-of-tree build directory) at
`$TOOLCHAIN_WORKTREE` and create a config file `$TOOLCHAIN_WORKTREE/Make.user`
with

```
USE_BINARYBUILDER_LLVM=1
BUILD_LLVM_CLANG=1
```

Run:

```sh
cd $TOOLCHAIN_WORKTREE
make -C deps install-llvm install-clang install-llvm-tools
```

to install toolchain binaries in `$TOOLCHAIN_WORKTREE/usr/tools`

#### Step 2: Build Julia with ASAN

Checkout a Git worktree (or create out-of-tree build directory) at
`$BUILD_WORKTREE` and create a config file `$BUILD_WORKTREE/Make.user` with

```
TOOLCHAIN=$(TOOLCHAIN_WORKTREE)/usr/tools

# use our new toolchain
USECLANG=1
override CC=$(TOOLCHAIN)/clang
override CXX=$(TOOLCHAIN)/clang++
export ASAN_SYMBOLIZER_PATH=$(TOOLCHAIN)/llvm-symbolizer

USE_BINARYBUILDER_LLVM=1

override SANITIZE=1
override SANITIZE_ADDRESS=1

# make the GC use regular malloc/frees, which are hooked by ASAN
override WITH_GC_DEBUG_ENV=1

# default to a debug build for better line number reporting
override JULIA_BUILD_MODE=debug

# make ASAN consume less memory
export ASAN_OPTIONS=detect_leaks=0:fast_unwind_on_malloc=0:allow_user_segv_handler=1:malloc_context_size=2

JULIA_PRECOMPILE=1

# tell libblastrampoline to not use RTLD_DEEPBIND
export LBT_USE_RTLD_DEEPBIND=0
```

Run:

```sh
cd $BUILD_WORKTREE
make debug
```

to build `julia-debug` with ASAN.

## Memory Sanitizer (MSAN)

For detecting use of uninitialized memory, you can use Clang's [memory sanitizer (MSAN)](https://clang.llvm.org/docs/MemorySanitizer.html)
by compiling with `SANITIZE_MEMORY=1`.

## Thread Sanitizer (TSAN)

For debugging data-races and other threading related issues you can use Clang's [thread sanitizer (TSAN)](https://clang.llvm.org/docs/ThreadSanitizer.html)
by compiling with `SANITIZE_THREAD=1`.
