geowatch.utils.util_dotdict module

Utilities for dictionaries where dots in keys represent nestings

class geowatch.utils.util_dotdict.DotDict(*args, **kwargs)

Bases: UDict

I’m sure this data structure exists on pypi. This should be replaced with that if we find it.

SeeAlso:

DotDictDataFrame

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'proc1.param1': 1,
>>>     'proc1.param2': 2,
>>>     'proc2.param1': 3,
>>>     'proc2.param2': 4,
>>>     'proc3.param1': 5,
>>>     'proc3.param2': 6,
>>>     'proc4.part1.param1': 7,
>>>     'proc4.part1.param2': 8,
>>>     'proc4.part2.param2': 9,
>>>     'proc4.part2.param2': 10,
>>> })
>>> self.get('proc1')
>>> self.prefix_get('proc4')
>>> 'proc1' in self

>>> nested = self.to_nested()
>>> recon = DotDict.from_nested(nested)
>>> assert nested != self
>>> assert recon == self

classmethod from_nested(data)

Parameters:

data (Dict) – nested data

to_nested()

Converts this flat DotDict into a nested representation. I.e. keys are broken using the “.” separtor, with each separator becoming a new nesting level.

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict(**{
>>>     'foo.bar.baz': 1,
>>>     'foo.bar.biz': 1,
>>>     'foo.spam': 1,
>>>     'eggs.spam': 1,
>>> })
>>> nested = self.to_nested()
>>> print(f'nested = {ub.urepr(nested, nl=2)}')
nested = {
    'foo': {
        'bar': {'baz': 1, 'biz': 1},
        'spam': 1,
    },
    'eggs': {
        'spam': 1,
    },
}

to_nested_keys()

Converts this flat DotDict into a nested key representation. The difference between this and to_nested is that the leafs are sets of keys whereas the leafs in DotDict are dicts

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict(**{
>>>     'foo.bar.baz': 1,
>>>     'foo.bar.biz': 1,
>>>     'foo.spam': 1,
>>>     'eggs.spam': 1,
>>> })
>>> nested = self.to_nested_keys()
>>> print(f'nested = {ub.urepr(nested, nl=2)}')
nested = {
    'foo': {
        'bar': {'baz': 'foo.bar.baz', 'biz': 'foo.bar.biz'},
        'spam': 'foo.spam',
    },
    'eggs': {
        'spam': 'eggs.spam',
    },
}

suffix_get(suffix, default=NoParam, backend='trie')

Retrieve all key-value pairs whose keys end with a given dot-suffix.

Parameters:

• suffix (str) – dot-separated suffix string

• default – fallback if no matches found

• backend (str) – ‘trie’ or ‘loop’

Returns:

DotDict

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'a.b.c': 1,
>>>     'x.b.c': 2,
>>>     'z.y': 3,
>>> })
>>> self.suffix_get('b.c')
{'a.b.c': 1, 'x.b.c': 2}

prefix_get(key, default=NoParam)

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict(**{
>>>     'foo.bar.baz': 1,
>>>     'foo.bar.biz': 1,
>>>     'foo.spam': 1,
>>>     'eggs.spam': 1,
>>> })
>>> self.prefix_get('foo')
{'bar.baz': 1, 'bar.biz': 1, 'spam': 1}

suffix_subdict(suffixes, backend='trie')

Filter DotDict to only contain keys ending with any given suffixes.

Parameters:

• suffixes (List[str]) – list of dot-suffixes

• backend (str) – ‘trie’ or ‘loop’

Returns:

DotDict

References

https://chatgpt.com/c/6841a161-2cd4-8002-ad9b-5593f5a2d70c

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'proc1.param1': 1,
>>>     'proc2.param1': 2,
>>>     'proc3.param2': 3,
>>>     'proc4.part1.param1': 4,
>>>     'proc4.part2.param2': 5,
>>> })
>>> new = self.suffix_subdict(['param1', 'part2.param2'])
>>> print(f'new = {ub.urepr(new, nl=1, sort=1)}')
new = {
    'proc1.param1': 1,
    'proc2.param1': 2,
    'proc4.part1.param1': 4,
    'proc4.part2.param2': 5,
}

prefix_subdict(prefixes, backend='trie')

Filter DotDict to only contain keys starting with any given prefixes.

Parameters:

• prefixes (List[str]) – list of dot-prefixes

• backend (str) – ‘trie’ or ‘loop’

Returns:

DotDict

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'proc1.param1': 1,
>>>     'proc1.param2': 2,
>>>     'proc2.param1': 3,
>>>     'proc3.param2': 4,
>>>     'proc4.part1.param1': 5,
>>>     'proc4.part2.param2': 6,
>>> })
>>> new = self.prefix_subdict(['proc1', 'proc4.part1'])
>>> print(f'new = {ub.urepr(new, nl=1, sort=1)}')
new = {
    'proc1.param1': 1,
    'proc1.param2': 2,
    'proc4.part1.param1': 5,
}

add_prefix(prefix)

Adds a prefix to all items

insert_prefix(prefix, index)

Adds a prefix to all items

Parameters:

• prefix (str) – prefix to insert

• index (int) – the depth to insert the new param

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'proc1.param1': 1,
>>>     'proc1.param2': 2,
>>>     'proc2.param1': 3,
>>>     'proc4.part1.param2': 8,
>>>     'proc4.part2.param2': 9,
>>>     'proc4.part2.param2': 10,
>>> })
>>> new = self.insert_prefix('foo', index=1)
>>> print('self = {}'.format(ub.urepr(self, nl=1)))
>>> print('new = {}'.format(ub.urepr(new, nl=1)))

query_keys(col)

Finds columns where one level has this key

Example

>>> from geowatch.utils.util_dotdict import *  # NOQA
>>> self = DotDict({
>>>     'proc1.param1': 1,
>>>     'proc1.param2': 2,
>>>     'proc2.param1': 3,
>>>     'proc4.part1.param2': 8,
>>>     'proc4.part2.param2': 9,
>>>     'proc4.part2.param2': 10,
>>> })
>>> list(self.query_keys('param1'))

print_graph()

geowatch.utils.util_dotdict.dotdict_to_nested(d)

geowatch.utils.util_dotdict.dotkeys_to_nested(keys)

Parameters:

keys (List[str]) – a list of dotted key names

geowatch.utils.util_dotdict.indexable_to_graph(data)

geowatch.utils.util_dotdict.explore_nested_dict(data)

TODO: some sort of textual interface