xmodaler.config

class xmodaler.config.CfgNode(init_dict=None, key_list=None, new_allowed=False)

Bases: CfgNode

The same as fvcore.common.config.CfgNode, but different in:

1. Use unsafe yaml loading by default. Note that this may lead to arbitrary code execution: you must not load a config file from untrusted sources before manually inspecting the content of the file.

2. Support config versioning. When attempting to merge an old config, it will convert the old config automatically.

DEPRECATED_KEYS = '__deprecated_keys__'

IMMUTABLE = '__immutable__'

NEW_ALLOWED = '__new_allowed__'

RENAMED_KEYS = '__renamed_keys__'

__init__(init_dict=None, key_list=None, new_allowed=False)

Parameters:

• init_dict (dict) – the possibly-nested dictionary to initailize the CfgNode.

• key_list (list[str]) – a list of names which index this CfgNode from the root. Currently only used for logging purposes.

• new_allowed (bool) – whether adding new key is allowed when merging with other configs.

classmethod _create_config_tree_from_dict(dic, key_list)

Create a configuration tree using the given dict. Any dict-like objects inside dict will be treated as a new CfgNode.

Parameters:

• dic (dict) –

• key_list (list[str]) – a list of names which index this CfgNode from the root. Currently only used for logging purposes.

classmethod _decode_cfg_value(value)

Decodes a raw config value (e.g., from a yaml config files or command line argument) into a Python object.

If the value is a dict, it will be interpreted as a new CfgNode. If the value is a str, it will be evaluated as literals. Otherwise it is returned as-is.

_immutable(is_immutable)

Set immutability to is_immutable and recursively apply the setting to all nested CfgNodes.

classmethod _load_cfg_from_file(file_obj)

Load a config from a YAML file or a Python source file.

classmethod _load_cfg_from_yaml_str(str_obj)

Load a config from a YAML string encoding.

classmethod _load_cfg_py_source(filename)

Load a config from a Python source file.

classmethod _open_cfg(filename)

Defines how a config file is opened. May be overridden to support different file schemas.

clear() → None.  Remove all items from D.

clone()

Recursively copy this CfgNode.

copy() → a shallow copy of D

defrost()

Make this CfgNode and all of its children mutable.

dump(*args, **kwargs)

Returns:

a yaml string representation of the config

Return type:

str

freeze()

Make this CfgNode and all of its children immutable.

fromkeys(value=None, /)

Create a new dictionary with keys from iterable and values set to value.

get(key, default=None, /)

Return the value for key if key is in the dictionary, else default.

is_frozen()

Return mutability.

is_new_allowed()

items() → a set-like object providing a view on D's items

key_is_deprecated(full_key)

Test if a key is deprecated.

key_is_renamed(full_key)

Test if a key is renamed.

keys() → a set-like object providing a view on D's keys

classmethod load_cfg(cfg_file_obj_or_str)

Load a cfg. :param cfg_file_obj_or_str: Supports loading from:

• A file object backed by a YAML file

• A file object backed by a Python source file that exports an attribute “cfg” that is either a dict or a CfgNode

• A string that can be parsed as valid YAML

load_from_file_tmp(cfg_filename: str, allow_unsafe: bool = True) → None

classmethod load_yaml_with_base(filename: str, allow_unsafe: bool = False) → Dict[str, Any]

Just like yaml.load(open(filename)), but inherit attributes from its

_BASE_.

Parameters:

• filename (str or file-like object) – the file name or file of the current config. Will be used to find the base config file.

• allow_unsafe (bool) – whether to allow loading the config file with yaml.unsafe_load.

Returns:

the loaded yaml

Return type:

(dict)

merge_from_file(cfg_filename: str, allow_unsafe: bool = True) → None

Merge configs from a given yaml file.

Parameters:

• cfg_filename – the file name of the yaml config.

• allow_unsafe – whether to allow loading the config file with yaml.unsafe_load.

merge_from_list(cfg_list: List[str]) → Callable[[], None]

Parameters:

cfg_list (list) – list of configs to merge from.

merge_from_other_cfg(cfg_other: CfgNode) → Callable[[], None]

Parameters:

cfg_other (CfgNode) – configs to merge from.

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised

popitem() → (k, v), remove and return some (key, value) pair as a

2-tuple; but raise KeyError if D is empty.

raise_key_rename_error(full_key)

register_deprecated_key(key)

Register key (e.g. FOO.BAR) a deprecated option. When merging deprecated keys a warning is generated and the key is ignored.

register_renamed_key(old_name, new_name, message=None)

Register a key as having been renamed from old_name to new_name. When merging a renamed key, an exception is thrown alerting to user to the fact that the key has been renamed.

set_new_allowed(is_new_allowed)

Set this config (and recursively its subconfigs) to allow merging new keys from other configs.

setdefault(key, default=None, /)

Insert key with a value of default if key is not in the dictionary.

Return the value for key if key is in the dictionary, else default.

update([E, ]**F) → None.  Update D from dict/iterable E and F.

If E is present and has a .keys() method, then does: for k in E: D[k] = E[k] If E is present and lacks a .keys() method, then does: for k, v in E: D[k] = v In either case, this is followed by: for k in F: D[k] = F[k]

values() → an object providing a view on D's values

xmodaler.config.get_cfg() → CfgNode

Get a copy of the default config.

Returns:

a X-modaler CfgNode instance.

xmodaler.config.set_global_cfg(cfg: CfgNode) → None

Let the global config point to the given cfg.

Assume that the given “cfg” has the key “KEY”, after calling set_global_cfg(cfg), the key can be accessed by:

from xmodaler.config import global_cfg
print(global_cfg.KEY)

By using a hacky global config, you can access these configs anywhere, without having to pass the config object or the values deep into the code. This is a hacky feature introduced for quick prototyping / research exploration.

xmodaler.config.downgrade_config(cfg: CfgNode, to_version: int) → CfgNode

Downgrade a config from its current version to an older version.

Parameters:

• cfg (CfgNode) –

• to_version (int) –

Note

A general downgrade of arbitrary configs is not always possible due to the different functionalities in different versions. The purpose of downgrade is only to recover the defaults in old versions, allowing it to load an old partial yaml config. Therefore, the implementation only needs to fill in the default values in the old version when a general downgrade is not possible.

xmodaler.config.upgrade_config(cfg: CfgNode, to_version: Optional[int] = None) → CfgNode

Upgrade a config from its current version to a newer version.

Parameters:

• cfg (CfgNode) –

• to_version (int) – defaults to the latest version.

xmodaler.config.configurable(init_func=None, *, from_config=None)

Decorate a function or a class’s __init__ method so that it can be called with a CfgNode object using a from_config() function that translates CfgNode to arguments.

Examples:

# Usage 1: Decorator on __init__:
class A:
    @configurable
    def __init__(self, a, b=2, c=3):
        pass

    @classmethod
    def from_config(cls, cfg):   # 'cfg' must be the first argument
        # Returns kwargs to be passed to __init__
        return {"a": cfg.A, "b": cfg.B}

a1 = A(a=1, b=2)  # regular construction
a2 = A(cfg)       # construct with a cfg
a3 = A(cfg, b=3, c=4)  # construct with extra overwrite

# Usage 2: Decorator on any function. Needs an extra from_config argument:
@configurable(from_config=lambda cfg: {"a: cfg.A, "b": cfg.B})
def a_func(a, b=2, c=3):
    pass

a1 = a_func(a=1, b=2)  # regular call
a2 = a_func(cfg)       # call with a cfg
a3 = a_func(cfg, b=3, c=4)  # call with extra overwrite

Parameters:

• init_func (callable) – a class’s __init__ method in usage 1. The class must have a from_config classmethod which takes cfg as the first argument.

• from_config (callable) – the from_config function in usage 2. It must take cfg as its first argument.

xmodaler.config.global_cfg

The same as fvcore.common.config.CfgNode, but different in:

1. Use unsafe yaml loading by default. Note that this may lead to arbitrary code execution: you must not load a config file from untrusted sources before manually inspecting the content of the file.

2. Support config versioning. When attempting to merge an old config, it will convert the old config automatically.

xmodaler.config.kfg

The same as fvcore.common.config.CfgNode, but different in:

1. Use unsafe yaml loading by default. Note that this may lead to arbitrary code execution: you must not load a config file from untrusted sources before manually inspecting the content of the file.

2. Support config versioning. When attempting to merge an old config, it will convert the old config automatically.