Profiling in ELPS

Note The profilers in this directory are experimental code and may not always works as anticipated.

Hooks are provided on the lisp.Runtime to contain a profiler implementing the interface defined in lisp.Profiler.

Profilers available

Callgrind profiler

This profiler produces output in callgrind/cachegrind format which can be read using KCachegrind, QCachegrind and similar programs. It captures source, call, time and memory information from every LISP call made.

This profiler is probably the most useful for locating problems whilst developing code in ELPS at present.

To enable this profiler, it needs to be linked to the ELPS runtime and it must be instructed to complete the profile and close the file before the profile can be read.

package a
import (
    "github.com/luthersystems/elps/lisp"
    "github.com/luthersystems/elps/lisp/x/profiler"
)

func runLispStuff() {
    // Get a LISP env
    env := lisp.NewEnv(nil)
    // Attach it to a profiler
    cgp := profiler.NewCallgrindProfiler(env.Runtime)
    // Tell the profiler where to put the output
    if err := profiler.SetFile("./callgrind.test_prof"); err != nil {
        panic("Could not open file")
    }
    // Init the env as usual (you can enable the profiler before you do this
    // if you want)
    lisp.InitializeUserEnv(env)
    cgp.Enable()
    // use your now profiling ELPS env
    // ...
    // ...
    // and then tell the profiler to wrap up and close the file when you're done
    cgp.Complete()
}

You can then open the resulting profile in KCachegrind or similar.

Opencensus annotator

This profiler appends OpenCensus tracing spans to an input context for each LISP call made. It may be useful for adding production metrics to non-performant code, especially if using one of the many supported exporters.

Starting this profiler requires a context to be supplied when initialising the profiler. The profiler will not do this for you as it is likely that you will wish to tie the resulting spans to a web request or similar.

package a
import (
    "context"
    "github.com/luthersystems/elps/lisp"
    "github.com/luthersystems/elps/lisp/x/profiler"
)

func runLispStuff(ctx context.Context) {
    // Get a LISP env
    env := lisp.NewEnv(nil)
    // Attach it to a profiler
    ocp := profiler.NewOpenCensusAnnotator(env.Runtime, ctx)
    // Init the env as usual (you can enable the profiler before you do this
    // if you want)
    lisp.InitializeUserEnv(env)
    ocp.Enable()
    // use your now profiling ELPS env
    // ...
    // ...
    // and then tell the profiler to wrap up and close the file when you're done
    ocp.Complete()
}

pprof annotator

This profiler appends tags for called functions to a CPU profile generated by the standard golang profiler pprof. These can be useful in identifying slow code whilst working on ELPS itself or custom builtins, more than production code or debugging LISP.

This profiler is initalised much like the OpenCensus profiler, though providing a context is not obligatory if you do not have labels from an upstream context to provide.

package a
import (
    "context"
    "os"
    "github.com/luthersystems/elps/lisp"
    "github.com/luthersystems/elps/lisp/x/profiler"
    "runtime/pprof"
)

func runLispStuff(ctx context.Context) {
    // Get a LISP env
    env := lisp.NewEnv(nil)
    // Attach it to a profiler
    ppa := profiler.NewPprofAnnotator(env.Runtime, ctx)
    // Get a file handle for pprof to write to
    file, err := os.Create("./pprof.out")
    // And start the profile
	if err != nil {
        pprof.StartCPUProfile(file)
        defer pprof.StopCPUProfile()
    }
    // Init the env as usual (you can enable the profiler before you do this
    // if you want)
    lisp.InitializeUserEnv(env)
    ppa.Enable()
    // use your now profiling ELPS env
    // ...
    // ...
    // and then tell the profiler to wrap up and close the file when you're done
    ppa.Complete()
}

It would also be possible to use this within a process profiled using the pprof http interface. To do so, you could simply omit the lines starting profiling above as long as your imports in main were set up correctly.

##Overhead of profiling The overhead of running profiling is small but not insignificant. This overhead is principally caused by the overhead of the systems profiled to rather than the profiler itself, but should be taken into account when deciding when to use these interfaces. However, the overhead of running the OpenCensus and pprof annotators is not markedly greater than running those systems without the ELPS annotations.