Is it possible to update the log level of a zap logger at runtime?

Question

I created a logger with kubebuilder, it is based on zap logger:

import (
    "flag"
    "github.com/gin-gonic/gin"
    "net/http"
    "os"
    "go.uber.org/zap/zapcore"
    uzap "go.uber.org/zap"
    // Import all Kubernetes client auth plugins (e.g. Azure, GCP, OIDC, etc.)
    // to ensure that exec-entrypoint and run can make use of them.
    _ "k8s.io/client-go/plugin/pkg/client/auth"

    "k8s.io/apimachinery/pkg/runtime"
    utilruntime "k8s.io/apimachinery/pkg/util/runtime"
    clientgoscheme "k8s.io/client-go/kubernetes/scheme"
    ctrl "sigs.k8s.io/controller-runtime"
    "sigs.k8s.io/controller-runtime/pkg/healthz"
    "sigs.k8s.io/controller-runtime/pkg/log/zap"

)

var (
    scheme   = runtime.NewScheme()
    setupLog = ctrl.Log.WithName("setup")
)

var zapOpts []uzap.Option
    zapOpts = append(zapOpts, uzap.AddCaller())
    zapOpts = append(zapOpts, uzap.AddCallerSkip(1))
    zapOpts = append(zapOpts, uzap.AddStacktrace(uzap.DebugLevel))

    opts := zap.Options{
        Development:     developmentFlag,
        StacktraceLevel: stacktraceLevel,
        Level:           level,
        Encoder:         encoder,
        ZapOpts:  zapOpts,
    }

    opts.BindFlags(flag.CommandLine)
    flag.Parse()

    ctrl.SetLogger(zap.New(zap.UseFlagOptions(&opts)))

Now I want to change the log level to zapcore.InfoLevel at run time. I didn't find any SetLogLevel or similar API.

Do I need to create new opts and then set the new level?

Also I need to set the logger with sigs.k8s.io/controller-runtime/pkg/log/zap library. The interface of the logger is from go-logr and it implements logr.Logger interface. If I tried to change it to zapcore.NewCore than I can't set the logger with ctrl.SetLogger anymore.

I want to keep the options to update all the options of zap.Options and also to change the log level, and still to use the zap from sigs.k8s.io/controller-runtime/pkg/log/zap.

Is it possible to do it with sigs.k8s.io/controller-runtime/pkg/log/zap and sigs.k8s.io/controller-runtime?

Stampede

Answer 1

Better answer: as suggested by @Oliver Dain, use zap.AtomicLevel. See their answer for details.

Another option is to create a core with a custom LevelEnabler function. You can use zap.LevelEnablerFunc to convert a closure to a zapcore.LevelEnabler.

The relevant docs:

LevelEnabler decides whether a given logging level is enabled when logging a message.

LevelEnablerFunc is a convenient way to implement zapcore.LevelEnabler with an anonymous function.

That function may then return true or false based on some other variable that changes at runtime:

    // will be actually initialized and changed at run time 
    // based on your business logic
    var infoEnabled bool 

    errorUnlessEnabled := zap.LevelEnablerFunc(func(level zapcore.Level) bool {
        // true: log message at this level
        // false: skip message at this level
        return level >= zapcore.ErrorLevel || infoEnabled
    })

    core := zapcore.NewCore(
        zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
        os.Stdout,
        errorUnlessEnabled,
    )
    logger := zap.New(core)

    logger.Info("foo") // not logged
    
    infoEnabled = true

    logger.Info("foo again") // logged

PS: this code is contrived. Your program will have to implement initialization, value change at run-time and proper synchronization (if needed) over the infoEnabled variable.

You can run this example in the playground: https://play.golang.org/p/oT3nvnP1Bwc

Comrade

thanks for the help , I am little bit confused how I can pass the zap.Options{ Development: developmentFlag, StacktraceLevel: stacktraceLevel, Level: level, Encoder: logfmtEncoder, ZapOpts: zapOpts, } , – Stampede

@Stampede zap in your case is sigs.k8s.io/controller-runtime/pkg/log/zap, isn't it? Then you can set errorUnlessEnabled into zap.Options.Level field. The type is the same as in my example, zapcore.LevelEnabler – Comrade

@ blackgreen - The issue that I need to use the library "sigs.k8s.io/controller-runtime/pkg/log/zap" and ctrl "sigs.k8s.io/controller-runtime" when I set the logger and not direct with "go.uber.org/zap" – Stampede

@Stampede to use ctrl.SetLogger with a custom zap logger, it seems you can use the github.com/go-logr/zapr package. zapr.NewLogger or zapr.NewLoggerWithOptions. Please let me know if this may work, and I'll update the answer – Comrade

Answer 2

Yes it is possible using AtomicLevel. From the docs:

atom := zap.NewAtomicLevel()

// To keep the example deterministic, disable timestamps in the output.
encoderCfg := zap.NewProductionEncoderConfig()
encoderCfg.TimeKey = ""

logger := zap.New(zapcore.NewCore(
    zapcore.NewJSONEncoder(encoderCfg),
    zapcore.Lock(os.Stdout),
    atom,
))
defer logger.Sync()

logger.Info("info logging enabled")
atom.SetLevel(zap.ErrorLevel)
logger.Info("info logging disabled")

Sohn

Do I have to keep variable atom as global to be able to change log level somewhere else in other packages ? I could not find a way to retrieve the created atom variable and I know it is possible via rest api which ı do not want to use – Wolves

You have to keep it somewhere but that need not be global. You have some code somewhere that needs to be able to access the level so that code needs access to atom but you could pass it to that code, or pass it to a thing that code has access to, etc. – Sohn

It would be nice to get it form zap. Thanks – Wolves

Answer 3

I thought I would mention in new versions of zap you don't need to do anything special, you can just retrieve it from the logger itself with Level(). https://pkg.go.dev/go.uber.org/zap#AtomicLevel.Level

package main
import (
    "fmt"
    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
)
func main() {
    logCfg := zap.NewProductionConfig()
    logCfg.Level = zap.NewAtomicLevelAt(zapcore.DebugLevel)
    logger, err := logCfg.Build()
    if err != nil {
        fmt.Println("failed to build")
        return
    }
    fmt.Println(logCfg.Level)
    fmt.Println(logger.Level())

    logger.Debug("debug")
    logger.Info("info")
}

Hubby