Create a new key in hydra DictConfig from python file

Question

I'd like to add a key + value after my Hydra Config is loaded. Essentially I want to run my code and check if a gpu is available or not. If it is, log the device as gpu, else keep the cpu.

Essentially saving the output of :

torch.cuda.is_available()

When I try to add a key with "setdefault" as such:

@hydra.main(config_path="conf", config_name="config")
def my_app(cfg: DictConfig) -> None:
    cfg.setdefault("new_key", "new_value")

Same error if I do:

  cfg.new_key = "new_value"
  print(cfg.new_key)

I get the error:

omegaconf.errors.ConfigKeyError: Key 'new_key' is not in struct
        full_key: new_key
        reference_type=Optional[Dict[Union[str, Enum], Any]]
        object_type=dict

My current workaround is to just use OmegaConfig as such:

  cfg = OmegaConf.structured(OmegaConf.to_yaml(cfg))
  cfg.new_key = "new_value"
  print(cfg.new_key)
  >>>> new_value

Surely there must be a better way to do this?

Perseus

Answer 1

Hydra sets the struct flag on the root of the OmegaConf config object it produces. See this for more info about the struct flag.

You can use open_dict() to temporarily disable this and to allow the addition of new keys:

>>> from omegaconf import OmegaConf,open_dict
>>> conf = OmegaConf.create({"a": {"aa": 10, "bb": 20}})
>>> OmegaConf.set_struct(conf, True)
>>> with open_dict(conf):
...   conf.a.cc = 30
>>> conf.a.cc
30

Noggin

Thanks that worked! I think you should add this line to your code to make it usable: from omegaconf.omegaconf import open_dict – Perseus

Just curious, what does OmegaConf.set_struct(conf, True) do? Since by default, hydra doesn't allow to add new keys anyway. – Awoke

Hydra calls it on the config object by default once it's done composing it. See the docs of OmegaConf to see what it does. – Noggin