geowatch.mlops.aggregate_loader module¶
Logic for loading raw results from the MLops DAG root dir.
Used by ./aggregate.py
- geowatch.mlops.aggregate_loader.build_tables(root_dpath, pipeline, io_workers, eval_nodes, cache_resolved_results)[source]¶
- geowatch.mlops.aggregate_loader.load_result_worker(fpath, node_name, out_node_key, use_cache=True)[source]¶
Main driver for loading results
- geowatch.mlops.aggregate_loader.new_process_context_parser(proc_item)[source]¶
Load parameters out of data saved by a ProcessContext object
- geowatch.mlops.aggregate_loader.load_result_resolved(node_dpath)[source]¶
Recurse through the DAG filesytem structure and load resolved configurations from each step.
- Parameters:
node_dpath (str | PathLike) – the path to the evaluation node directory. The specific type of evaluation node must have a known (currently hard-coded) condition in this function that knows how to parse it.
- Returns:
Dict - flat_resolved - a flat dot-dictionary with resolved params
Todo
Some mechanism to let the user specify how to parse an evaluation node of a given type.