geowatch.utils package

Subpackages

• geowatch.utils.lightning_ext package
  • Subpackages
    • geowatch.utils.lightning_ext.callbacks package
      • Submodules
      • Module contents
  • Submodules
    • geowatch.utils.lightning_ext._jsonargparse_ext_ge_4_21_and_lt_4_22 module
    • geowatch.utils.lightning_ext._jsonargparse_ext_ge_4_22_and_lt_4_24 module
    • geowatch.utils.lightning_ext._jsonargparse_ext_ge_4_24_and_lt_4_xx module
    • geowatch.utils.lightning_ext._jsonargparse_ext_ge_4_xx_and_lt_4_21 module
      • extract_scriptconfig_params()
      • extract_scriptconfig_param_args()
      • ArgumentParserPatches
      • _find_action_and_subcommand()
      • _action_aliases()
    • geowatch.utils.lightning_ext.argparse_ext module
      • get_init_arguments_and_types()
      • str_to_bool_or_str()
      • str_to_bool_or_int()
      • str_to_bool()
      • _gpus_allowed_type()
      • _int_or_float_type()
      • parse_docstring_args()
      • add_arginfos_to_parser()
      • add_argparse_args()
    • geowatch.utils.lightning_ext.demo module
      • LightningToyNet2d
      • demo_trainer()
    • geowatch.utils.lightning_ext.lightning_cli_ext module
      • LightningArgumentParser_Extension
      • LightningCLI_Extension
    • geowatch.utils.lightning_ext.monkeypatches module
    • geowatch.utils.lightning_ext.old_parser_devices module
      • parse_gpu_ids()
      • _sanitize_gpu_ids()
      • _normalize_parse_gpu_string_input()
      • _normalize_parse_gpu_input_to_list()
      • _get_all_available_gpus()
      • _check_unique()
      • _check_data_type()
      • _parse_gpu_ids()
    • geowatch.utils.lightning_ext.util_device module
      • coerce_devices()
      • _test_lightning_is_sane()
      • _test_coerce_is_sane()
    • geowatch.utils.lightning_ext.util_globals module
    • geowatch.utils.lightning_ext.util_model module
      • model_hparams()
  • Module contents

Submodules

• geowatch.utils.configargparse_ext module
  • RawDescriptionDefaultsHelpFormatter
  • ArgumentParser
    • ArgumentParser.parse_known_args()
• geowatch.utils.ext_monai module
  • FocalLoss
    • FocalLoss.forward()
  • DiceLoss
    • DiceLoss.forward()
  • MaskedDiceLoss
    • MaskedDiceLoss.forward()
  • _ohem_mask()
  • GeneralizedWassersteinDiceLoss
    • GeneralizedWassersteinDiceLoss.forward()
    • GeneralizedWassersteinDiceLoss.wasserstein_distance_map()
    • GeneralizedWassersteinDiceLoss._compute_generalized_true_positive()
    • GeneralizedWassersteinDiceLoss._compute_denominator()
    • GeneralizedWassersteinDiceLoss._compute_alpha_generalized_true_positives()
  • DiceCELoss
    • DiceCELoss.ce()
    • DiceCELoss.forward()
  • DiceFocalLoss
    • DiceFocalLoss.forward()
  • Dice
  • dice_ce
  • dice_focal
  • generalized_wasserstein_dice
• geowatch.utils.ijson_ext module
  • UnexpectedSymbol
  • utf8_encoder()
  • Lexer()
  • parse_value()
  • parse_string()
  • basic_parse_basecoro()
• geowatch.utils.kwcoco_extensions module
  • filter_image_ids()
  • populate_watch_fields()
  • coco_populate_geo_heuristics()
  • coco_populate_geo_img_heuristics2()
  • _populate_valid_region()
  • _populate_canvas_obj()
  • _is_writeable()
  • _coerce_overwrite()
  • coco_populate_geo_video_stats()
  • check_kwcoco_spatial_transforms()
  • check_geo_transform_consistency()
  • check_unique_channel_names()
  • _check_unique_channel_names_in_image()
  • coco_list_asset_infos()
  • check_geotiff_formats()
  • rewrite_geotiffs()
  • geotiff_format_info()
  • ensure_transfered_geo_data()
  • transfer_geo_metadata()
  • _search_video_for_other_geo_assets()
  • _find_geotiffs_without_metadata()
  • transfer_geo_metadata2()
  • _execute_transfer_task()
  • _make_coco_img_from_geotiff()
  • _sensor_channel_hueristic()
  • _introspect_num_bands()
  • _num_band_hueristic()
  • _recompute_auxiliary_transforms()
  • coco_channel_stats()
  • TrackidGenerator
    • TrackidGenerator.update_generator()
    • TrackidGenerator.exclude_trackids()
  • coco_img_wld_info()
  • warp_annot_segmentations_from_geos()
  • warp_annot_segmentations_to_geos()
  • visualize_rois()
  • covered_image_geo_regions()
  • covered_video_geo_regions()
  • covered_annot_geo_regions()
  • category_category_colors()
  • associate_images()
  • reorder_video_frames()
  • pick_channels()
• geowatch.utils.process_context module
  • ProcessContext
    • ProcessContext.write_invocation()
    • ProcessContext._timestamp()
    • ProcessContext._hostinfo()
    • ProcessContext._osinfo()
    • ProcessContext._pyinfo()
    • ProcessContext._meminfo()
    • ProcessContext._cpuinfo()
    • ProcessContext._machine()
    • ProcessContext.start()
    • ProcessContext.flush()
    • ProcessContext.stop()
    • ProcessContext._start_emissions_tracker()
    • ProcessContext._flush_emissions_tracker()
    • ProcessContext._stop_emissions_tracker()
    • ProcessContext.add_device_info()
    • ProcessContext.add_disk_info()
  • jsonify_config()
  • Reconstruction
  • main()
• geowatch.utils.result_analysis module
  • Result
    • Result.to_dict()
    • Result.demo()
  • ResultTable
    • ResultTable.table
    • ResultTable.result_list
    • ResultTable.demo()
    • ResultTable.coerce()
    • ResultTable.varied
  • ResultAnalysis
    • ResultAnalysis.demo()
    • ResultAnalysis.run()
    • ResultAnalysis.analysis()
    • ResultAnalysis.table
    • ResultAnalysis.metric_table()
    • ResultAnalysis.varied
    • ResultAnalysis.abaltion_groups()
    • ResultAnalysis._objective_is_ascending()
    • ResultAnalysis.tune()
    • ResultAnalysis.ablate()
    • ResultAnalysis.abalation_groups()
    • ResultAnalysis.abalate()
    • ResultAnalysis.test_group()
    • ResultAnalysis.build()
    • ResultAnalysis.report()
    • ResultAnalysis._report_one()
    • ResultAnalysis.conclusions()
    • ResultAnalysis.plot()
  • SkillTracker
    • SkillTracker.predict_win()
    • SkillTracker.observe()
  • varied_values()
  • varied_value_counts()
  • GroupbyFutureWrapper
  • fix_groupby()
• geowatch.utils.reverse_hashid module
  • ReverseHashTable
    • ReverseHashTable.load()
    • ReverseHashTable.register()
    • ReverseHashTable.query()
  • condense_config()
• geowatch.utils.simple_dvc module
  • __test_simple_dvc()
  • SimpleDVC
    • SimpleDVC.init()
    • SimpleDVC.cache_dir
    • SimpleDVC.demo_dpath()
    • SimpleDVC.coerce()
    • SimpleDVC.find_root()
    • SimpleDVC._ensure_root()
    • SimpleDVC._ensure_remote()
    • SimpleDVC._resolve_root_and_relative_paths()
    • SimpleDVC.add()
    • SimpleDVC.pathsremove()
    • SimpleDVC._dvc_path_op()
    • SimpleDVC.check_ignore()
    • SimpleDVC.git_pull()
    • SimpleDVC.git_push()
    • SimpleDVC.git_commit()
    • SimpleDVC.git_commitpush()
    • SimpleDVC._verbose_extra_args()
    • SimpleDVC._remote_extra_args()
    • SimpleDVC.push()
    • SimpleDVC.pull()
    • SimpleDVC.request()
    • SimpleDVC.unprotect()
    • SimpleDVC.is_tracked()
    • SimpleDVC.find_file_tracker()
    • SimpleDVC.find_dir_tracker()
    • SimpleDVC.read_dvc_sidecar()
    • SimpleDVC.resolve_cache_paths()
    • SimpleDVC.find_sidecar_paths()
    • SimpleDVC.resolve_sidecar()
  • _ensure_iterable()
  • SimpleDVC_CLI
    • SimpleDVC_CLI.Add
      • SimpleDVC_CLI.Add.main()
      • SimpleDVC_CLI.Add.default
    • SimpleDVC_CLI.Request
      • SimpleDVC_CLI.Request.main()
      • SimpleDVC_CLI.Request.default
    • SimpleDVC_CLI.CacheDir
      • SimpleDVC_CLI.CacheDir.main()
      • SimpleDVC_CLI.CacheDir.default
  • _import_dvc_main()
• geowatch.utils.util_bands module
  • Sentinal 2 Band Table
  • Landsat 8 Band Table
  • Worldview 3 MUL Band Table
  • Worldview 3 PAN Band Table
  • dicts_contain()
  • SENTINEL2
  • LANDSAT8
  • LANDSAT7
  • PLANETSCOPE_8BAND
  • ALL_BANDS
  • EO_COMMONNAMES
  • GROUND
  • specialized_index_bands()
  • specialized_index_bands2()
• geowatch.utils.util_codes module
  • parse_delimited_argstr()
• geowatch.utils.util_data module
  • DataRegistry
    • DataRegistry.pandas()
    • DataRegistry.list()
    • DataRegistry._open()
    • DataRegistry.add()
    • DataRegistry.set()
    • DataRegistry.remove()
    • DataRegistry.read()
    • DataRegistry.query()
    • DataRegistry.find()
  • find_dvc_dpath()
  • find_smart_dvc_dpath()
• geowatch.utils.util_dotdict module
  • dotdict_to_nested()
  • dotkeys_to_nested()
  • DotDict
    • DotDict.from_nested()
    • DotDict.to_nested()
    • DotDict.to_nested_keys()
    • DotDict._prefix_trie
    • DotDict.prefix_get()
    • DotDict.add_prefix()
    • DotDict.insert_prefix()
    • DotDict.print_graph()
    • DotDict.query_keys()
  • indexable_to_graph()
  • explore_nested_dict()
• geowatch.utils.util_framework module
  • egress_item()
  • ingress_item()
  • download_mtd_msil1c()
  • download_file()
  • download_http_file()
  • CacheItemOutputS3Wrapper
  • _default_item_selector()
  • _default_asset_selector()
  • IngressProcessEgressWrapper
  • download_region()
  • determine_region_id()
  • AWS_S3_Command
    • AWS_S3_Command.cmd_known_flags
    • AWS_S3_Command.cmd_known_keyvals
    • AWS_S3_Command.update()
    • AWS_S3_Command.finalize()
    • AWS_S3_Command.run()
  • ta2_collate_output()
  • fixup_and_validate_site_and_region_models()
  • NodeStateDebugger
    • NodeStateDebugger.print_environment()
    • NodeStateDebugger.print_current_state()
  • PrintLogger
    • PrintLogger.info()
    • PrintLogger.debug()
    • PrintLogger.error()
    • PrintLogger.warning()
    • PrintLogger.critical()
  • _devcheck_retry()
  • _test_s3_hack()
• geowatch.utils.util_fsspec module
  • FSPath
    • FSPath._new_fs()
    • FSPath._current_fs()
    • FSPath.fs
    • FSPath.coerce()
    • FSPath.relative_to()
    • FSPath.is_remote()
    • FSPath.is_local()
    • FSPath.open()
    • FSPath.ls()
    • FSPath.touch()
    • FSPath.move()
    • FSPath.delete()
    • FSPath.rm()
    • FSPath.mkdir()
    • FSPath.stat()
    • FSPath.is_dir()
    • FSPath.is_file()
    • FSPath.is_link()
    • FSPath.exists()
    • FSPath.write_text()
    • FSPath.read_text()
    • FSPath.walk()
    • FSPath.parent
    • FSPath.name
    • FSPath.stem
    • FSPath.suffix
    • FSPath.suffixes
    • FSPath.parts
    • FSPath.copy()
    • FSPath.joinpath()
    • FSPath.tree()
  • LocalPath
    • LocalPath.ensuredir()
    • LocalPath.absolute()
    • LocalPath.appdir()
  • RemotePath
  • S3Path
    • S3Path._as_gdal_vsi()
    • S3Path.ensuredir()
  • SSHPath
    • SSHPath.host
  • MemoryPath
• geowatch.utils.util_gdal module
  • import_gdal()
  • import_osr()
  • DummyLogger
    • DummyLogger.warning()
  • _demo_geoimg_with_nodata()
  • GDalCommandBuilder
    • GDalCommandBuilder.config_options
    • GDalCommandBuilder.common_options
    • GDalCommandBuilder.options
    • GDalCommandBuilder.flags
    • GDalCommandBuilder.append()
    • GDalCommandBuilder.extend()
    • GDalCommandBuilder.finalize()
    • GDalCommandBuilder.set_cog_options()
  • gdal_single_translate()
  • _ShrinkingTimeout
    • _ShrinkingTimeout.remaining()
    • _ShrinkingTimeout.coerce()
  • gdal_single_warp()
  • gdal_multi_warp()
  • _execute_gdal_command_with_checks()
  • list_gdal_drivers()
  • GdalOpen()
  • GdalSupressWarnings
    • GdalSupressWarnings.handler()
  • GdalDataset
    • GdalDataset.open()
    • GdalDataset.coerce()
    • GdalDataset.closed
    • GdalDataset.mode
    • GdalDataset.shape
    • GdalDataset.close()
    • GdalDataset.info()
    • GdalDataset.get_overview_info()
    • GdalDataset.get_all_metadata()
• geowatch.utils.util_girder module
  • grabdata_girder()
• geowatch.utils.util_gis module
  • plot_geo_background()
  • geopandas_pairwise_overlaps()
  • latlon_text()
  • demo_regions_geojson_text()
  • load_geojson()
  • read_geojson()
  • get_crs84()
  • _get_crs84()
  • _flip()
  • shapely_flip_xy()
  • project_gdf_to_local_utm()
  • utm_epsg_from_latlon()
  • UTM_TransformContext
    • UTM_TransformContext._coerce_geo_series()
    • UTM_TransformContext.finalize()
  • _demo_convert_latlon_to_utm()
  • find_local_meter_epsg_crs()
  • check_latlons()
  • coerce_geojson_datas()
  • coerce_geojson_paths()
  • _coerce_raw_geojson()
  • _load_json_from_path()
  • load_geojson_datas()
  • crs_geojson_to_gdf()
  • coerce_crs()
• geowatch.utils.util_globals module
  • configure_global_attributes()
• geowatch.utils.util_hardware module
  • get_cpu_mem_info()
  • get_cpu_info()
  • get_mem_info()
  • disk_info_of_path()
  • _device_is_hdd()
  • _zfs_status()
• geowatch.utils.util_iter module
  • consume()
  • chunks()
• geowatch.utils.util_kwarray module
  • cartesian_product()
  • tukey_biweight_loss()
  • asymptotic()
  • robust_limits()
  • unique_rows()
  • find_robust_normalizers()
  • _custom_quantile_extreme_estimator()
  • _tukey_quantile_extreme_estimator()
  • apply_normalizer()
  • normalize()
  • balanced_number_partitioning()
  • torch_array_equal()
  • combine_mean_stds()
  • _no_keepdim_indexer()
  • _postprocess_keepdims()
  • apply_robust_normalizer()
  • biased_1d_weights()
  • Remedian
    • Remedian._calc_k_arrs()
    • Remedian._calc_k_arr_sizes()
    • Remedian.add_obs()
  • argsort_threshold()
• geowatch.utils.util_kwimage module
  • _auto_kernel_sigma()
  • upweight_center_mask()
  • perchannel_colorize()
  • ensure_false_color()
  • colorize_label_image()
  • local_variance()
  • find_lowvariance_regions()
  • find_samecolor_regions()
  • find_high_frequency_values()
  • polygon_distance_transform()
  • devcheck_frame_poly_weights()
  • find_low_overlap_covering_boxes()
  • find_low_overlap_covering_boxes_optimize()
  • exactly_1channel()
• geowatch.utils.util_kwplot module
  • phantom_legend()
  • cropwhite_ondisk()
  • dataframe_table()
  • humanize_dataframe()
  • scatterplot_highlight()
  • humanize_labels()
  • relabel_xticks()
  • LabelModifier
    • LabelModifier.copy()
    • LabelModifier.add_mapping()
    • LabelModifier.update()
    • LabelModifier._modify_text()
    • LabelModifier._modify_labels()
    • LabelModifier._modify_legend()
    • LabelModifier.relabel_yticks()
    • LabelModifier.relabel_xticks()
    • LabelModifier.relabel_axes_labels()
    • LabelModifier.relabel_legend()
    • LabelModifier.relabel()
  • FigureFinalizer
    • FigureFinalizer.copy()
    • FigureFinalizer.update()
    • FigureFinalizer.finalize()
  • fix_matplotlib_dates()
  • fix_matplotlib_timedeltas()
  • extract_legend()
  • ArtistManager
    • ArtistManager._normalize_attrs()
    • ArtistManager.plot()
    • ArtistManager.add_linestring()
    • ArtistManager.add_ellipse()
    • ArtistManager.add_circle()
    • ArtistManager.add_ellipse_marker()
    • ArtistManager.add_circle_marker()
    • ArtistManager.build_collections()
    • ArtistManager.add_to_axes()
    • ArtistManager.bounds()
    • ArtistManager.setlims()
  • time_sample_arcplot()
  • Palette
    • Palette.coerce()
    • Palette.update()
    • Palette.add_labels()
    • Palette.make_legend_img()
    • Palette.sorted_keys()
    • Palette.reorder()
  • PaletteManager
  • color_new_labels()
  • autompl2()
• geowatch.utils.util_logging module
  • setup_logging()
  • get_logger()
  • PrintLogger
    • PrintLogger.setLevel()
    • PrintLogger.debug()
    • PrintLogger.info()
    • PrintLogger.warning()
    • PrintLogger.error()
    • PrintLogger.exception()
    • PrintLogger.critical()
    • PrintLogger.log()
    • PrintLogger.findCaller()
    • PrintLogger.makeRecord()
    • PrintLogger.filter()
    • PrintLogger._log()
    • PrintLogger.handle()
    • PrintLogger.addHandler()
    • PrintLogger.removeHandler()
    • PrintLogger.hasHandlers()
    • PrintLogger.callHandlers()
    • PrintLogger.getEffectiveLevel()
    • PrintLogger.isEnabledFor()
    • PrintLogger.getChild()
• geowatch.utils.util_nesting module
  • debug_shapes()
  • shape_summary()
• geowatch.utils.util_netharn module
  • _update_defaults()
  • Initializer
    • Initializer.forward()
    • Initializer.history()
    • Initializer.get_initkw()
    • Initializer.coerce()
  • NoOp
    • NoOp.forward()
  • Orthogonal
    • Orthogonal.forward()
  • KaimingUniform
    • KaimingUniform.forward()
  • KaimingNormal
    • KaimingNormal.forward()
  • apply_initializer()
  • trainable_layers()
  • number_of_parameters()
• geowatch.utils.util_nvidia module
  • NvidiaSMIError
  • nvidia_smi()
  • _query_nvidia_smi()
• geowatch.utils.util_pandas module
  • DataFrame
    • DataFrame._constructor
    • DataFrame.random()
    • DataFrame.safe_drop()
    • DataFrame.reorder()
    • DataFrame._orig_groupby()
    • DataFrame.groupby()
    • DataFrame.match_columns()
    • DataFrame.search_columns()
    • DataFrame.varied_values()
  • pandas_reorder_columns()
  • pandas_argmaxima()
  • pandas_suffix_columns()
  • pandas_nan_eq()
  • pandas_shorten_columns()
  • pandas_condense_paths()
  • pandas_truncate_items()
  • DotDictDataFrame
    • DotDictDataFrame._clear_column_caches()
    • DotDictDataFrame._column_prefix_trie
    • DotDictDataFrame._column_suffix_trie
    • DotDictDataFrame._column_node_groups
    • DotDictDataFrame.nested_columns
    • DotDictDataFrame.find_column()
    • DotDictDataFrame.query_column()
    • DotDictDataFrame.lookup_suffix_columns()
    • DotDictDataFrame.lookup_prefix_columns()
    • DotDictDataFrame.find_columns()
    • DotDictDataFrame.match_columns()
    • DotDictDataFrame.search_columns()
    • DotDictDataFrame.subframe()
  • pandas_add_prefix()
  • aggregate_columns()
  • _handle_const()
  • SpecialAggregators
    • SpecialAggregators.hash()
    • SpecialAggregators.hash12()
    • SpecialAggregators.unique()
    • SpecialAggregators.min_max()
    • SpecialAggregators.normalize_special_key()
    • SpecialAggregators.special_lut
  • NonConstantError
  • nan_eq()
  • GroupbyFutureWrapper
  • _fix_groupby()
  • pandas_fixed_groupby()
• geowatch.utils.util_param_grid module
  • handle_yaml_grid()
  • coerce_list_of_action_matrices()
  • prevalidate_param_grid()
  • expand_param_grid()
  • github_action_matrix()
  • extended_github_action_matrix()
• geowatch.utils.util_raster module
  • _ensure_open()
  • _swapxy()
  • mask()
  • ResampledRaster
    • ResampledRaster.raster
    • ResampledRaster.scale
    • ResampledRaster.read
    • ResampledRaster.resampling
    • ResampledRaster._abc_impl
• geowatch.utils.util_regex module
  • _AbstractRegexBuilder
    • _AbstractRegexBuilder.lookahead()
    • _AbstractRegexBuilder.lookbehind()
    • _AbstractRegexBuilder.named_field()
    • _AbstractRegexBuilder.bref_field()
    • _AbstractRegexBuilder.escape()
    • _AbstractRegexBuilder.optional()
    • _AbstractRegexBuilder.group()
    • _AbstractRegexBuilder.oneof()
  • VimRegexBuilder
  • PythonRegexBuilder
• geowatch.utils.util_resolution module
  • _int_or_float()
  • ExtendedTransformer
    • ExtendedTransformer.parser()
    • ExtendedTransformer.parse()
    • ExtendedTransformer._abc_impl
  • ResolvedTransformer
    • ResolvedTransformer.magnitude()
    • ResolvedTransformer.unit()
    • ResolvedTransformer.resolved_unit()
    • ResolvedTransformer._abc_impl
  • ResolvedUnitTransformer
    • ResolvedUnitTransformer._abc_impl
  • ResolvedScalarTransformer
    • ResolvedScalarTransformer.resolved_scalar()
    • ResolvedScalarTransformer._abc_impl
  • ResolvedWindowTransformer
    • ResolvedWindowTransformer.window_1d_dim()
    • ResolvedWindowTransformer.window_2d_dim()
    • ResolvedWindowTransformer.window()
    • ResolvedWindowTransformer.resolved_window()
    • ResolvedWindowTransformer._abc_impl
  • Resolved
    • Resolved.parse()
    • Resolved.coerce()
  • ResolvedUnit
    • ResolvedUnit.coerce()
  • ResolvedScalar
    • ResolvedScalar.at_resolution()
  • ResolvedWindow
    • ResolvedWindow.at_resolution()
• geowatch.utils.util_rgdc module
  • group_tiles()
  • bands_landsat()
  • bands_sentinel2()
• geowatch.utils.util_s3 module
  • get_file_from_s3()
  • send_file_to_s3()
• geowatch.utils.util_stringalgo module
  • shortest_unique_prefixes()
  • _trie_iternodes()
  • _trie_iteritems()
  • shortest_unique_suffixes()
• geowatch.utils.util_torchmetrics module
  • BinaryF1Score
    • BinaryF1Score.compute()
    • BinaryF1Score._abc_impl
  • BinaryOverallAccuracy
    • BinaryOverallAccuracy.compute()
    • BinaryOverallAccuracy._abc_impl
  • BinaryBalancedAccuracy
    • BinaryBalancedAccuracy.compute()
    • BinaryBalancedAccuracy._abc_impl
  • BinaryTruePositiveRate
    • BinaryTruePositiveRate.compute()
    • BinaryTruePositiveRate._abc_impl

Module contents

geowatch.utils.find_dvc_dpath(name=NoParam, on_error='raise', **kwargs)

Return the location of the GEOWATCH DVC Data path if it exists and is in a “standard” location.

NOTE: other team members can add their “standard” locations if they want.

SeeAlso:

WATCH_DATA_DPATH=$(geowatch_dvc)

python -m geowatch.cli.find_dvc –hardware=hdd python -m geowatch.cli.find_dvc –hardware=ssd

geowatch.utils.find_smart_dvc_dpath(*args, **kw)