geowatch.tasks.fusion.datamodules.temporal_sampling package

Submodules

Module contents

Temporal sampling submodule

mkinit ~/code/watch/geowatch/tasks/fusion/datamodules/temporal_sampling/__init__.py –lazy_loader -w

class geowatch.tasks.fusion.datamodules.temporal_sampling.CommonSamplerMixin[source]

Bases: object

classmethod from_coco_video(dset, vidid, gids=None, **kwargs)[source]
classmethod from_datetimes(datetimes, time_span='full', affinity_type='soft2', **kwargs)[source]
class geowatch.tasks.fusion.datamodules.temporal_sampling.MultiTimeWindowSampler(unixtimes, sensors, time_window=None, affinity_type='hard', update_rule='distribute', deterministic=False, gamma=1, time_span=None, time_kernel=None, name='?', allow_fewer=True)[source]

Bases: CommonSamplerMixin

A wrapper that contains multiple time window samplers with different affinity matrices to increase the diversity of temporal sampling.

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import datetime as datetime_mod
>>> from datetime import datetime as datetime_cls
>>> low = datetime_cls.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> unixtimes = np.array(sorted(rng.randint(low, high, 32)), dtype=float)
>>> sensors = ['a' for _ in range(len(unixtimes))]
>>> time_window = 5
>>> self = MultiTimeWindowSampler(
>>>     unixtimes=unixtimes, sensors=sensors, time_window=time_window, update_rule='pairwise+distribute',
>>>     #time_span=['2y', '1y', '5m'])
>>>     time_span='7d-1m',
>>>     affinity_type='soft2')
>>> self.sample()
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autosns()
>>> self.show_summary(10)

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import datetime as datetime_mod
>>> from datetime import datetime as datetime_cls
>>> low = datetime_cls.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> unixtimes = np.array(sorted(rng.randint(low, high, 32)), dtype=float)
>>> sensors = ['a' for _ in range(len(unixtimes))]
>>> time_window = 5
>>> self = MultiTimeWindowSampler(
>>>     unixtimes=unixtimes, sensors=sensors, time_window=time_window, update_rule='distribute',
>>>     time_kernel=['-1y,-3m,0,3m,+1y', '-1m,-1d,0,1d,1m'],
>>>     affinity_type='soft2')
>>> self.sample()
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autosns()
>>> self.show_summary(10, show_indexes=1, fnum=1)
>>> list(self.sub_samplers.values())[0].show_summary(10, show_indexes=1, fnum=2)
>>> list(self.sub_samplers.values())[1].show_summary(10, show_indexes=1, fnum=3)

self.subplots_adjust

Parameters:

  • time_span (List[List[str]]) – a list of time spans. e.g. [‘2y’, ‘1y’, ‘5m’]

  • time_kernel (List[str]) – a list of time kernels.

property affinity

Approximate combined affinity, for this multi-sampler

sample(main_frame_idx=None, include=None, exclude=None, return_info=False, error_level=0, rng=None)[source]

Chooses a sub-sampler and samples from it.

Parameters:

  • main_frame_idx (int) – “main” sample index.

  • include (List[int]) – other indexes forced to be included

  • exclude (List[int]) – other indexes forced to be excluded

  • return_info (bool) – for debugging / introspection

  • error_level (int) – See affinity_sample().

Returns:

ndarray | Tuple[ndarray, Dict]

show_summary(samples_per_frame=1, show_indexes=0, fnum=1)[source]

Similar to TimeWindowSampler.show_summary()

exception geowatch.tasks.fusion.datamodules.temporal_sampling.TimeSampleError[source]

Bases: IndexError

class geowatch.tasks.fusion.datamodules.temporal_sampling.TimeWindowSampler(unixtimes, sensors, time_window=None, affinity_type='hard', update_rule='distribute', deterministic=False, gamma=1, time_span=None, time_kernel=None, affkw=None, name='?', allow_fewer=True)[source]

Bases: CommonSamplerMixin

Object oriented API to produce random temporal samples given a set of keyframes with metadata.

This works by computing a pairwise “affinity” NxN matrix for each of the N keyframes. The details of the affinity matrix depend on parameters passed to this object. Intuitively, the value at Affinity[i, j] represents how much frame-i “wants” to be in the same sample as frame-j.

Parameters:

  • unixtimes (List[int]) – list of unix timestamps for each frame

  • sensors (List[str]) – list of attributes for each frame

  • time_window (int) – number of frames to sample

  • affinity_type (str) – Method for computing the affinity matrix for the underlying sampling algorithm. Can be:

    “soft” - The old generalized random affinity matrix. “soft2” - The new generalized random affinity matrix. “soft3” - The newer generalized random affinity matrix. “hard” - A simplified affinity algorithm. “hardish” - Like hard, but with a blur. “contiguous” - Neighboring frames get high affinity.

  • update_rule (str) – “+” separated string that can contain {“distribute”, “pairwise”}. See affinity_sample() for details.

  • gamma (float) – Modulates sampling probability. Higher values See affinity_sample() for details.

  • time_span (Coercible[datetime.timedelta]) – The ideal distince in time that frames should be separated in. This is typically a string code. E.g. “1y” is one year.

  • name (str) – A name for this object. For developer convinience, has no influence on the algorithm.

  • deterministic (bool) – if True, on each step we choose the next timestamp with maximum probability. Otherwise, we randomly choose a timestep, but with probability according to the current distribution. This is an attribute, which can be modified to change behavior (not thread safe).

Variables:

main_indexes

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import kwcoco
>>> import geowatch
>>> data_dvc_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> coco_fpath = data_dvc_dpath / 'Drop6/data_vali_split1.kwcoco.zip'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_window=5,
>>>     affinity_type='hardish3', time_span='1y',
>>>     update_rule='distribute')
>>> self.deterministic = False
>>> self.show_summary(samples_per_frame=1, fnum=1)
>>> self.deterministic = True
>>> self.show_summary(samples_per_frame=3, fnum=2)
compute_affinity(affinity_type=None, update_rule=None)

Construct the affinity matrix given the current affinity_type.

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import geowatch
>>> data_dvc_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> coco_fpath = dvc_dpath / 'Drop6/data_vali_split1.kwcoco.zip'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_window=5,
>>>     affinity_type='contiguous',
>>>     update_rule='pairwise')
>>> self.deterministic = True
>>> self.show_procedure(fnum=1)
property main_indexes
sample(main_frame_idx=None, include=None, exclude=None, return_info=False, error_level=0, rng=None)[source]
Parameters:

  • main_frame_idx (int) – “main” sample index.

  • include (List[int]) – other indexes forced to be included

  • exclude (List[int]) – other indexes forced to be excluded

  • return_info (bool) – for debugging / introspection

  • error_level (int) – See affinity_sample().

Returns:

ndarray | Tuple[ndarray, Dict]

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> import kwcoco
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> from geowatch.utils.util_data import find_dvc_dpath
>>> dvc_dpath = find_dvc_dpath()
>>> coco_fpath = dvc_dpath / 'Drop2-Aligned-TA1-2022-02-15/data.kwcoco.json'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_span='1y',
>>>     time_window=3,
>>>     affinity_type='soft2',
>>>     update_rule='distribute+pairwise')
>>> self.deterministic = False
>>> self.show_summary(samples_per_frame=1 if self.deterministic else 10, fnum=1)
>>> self.show_procedure(fnum=2)

Example

>>> import os
>>> import kwcoco
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import geowatch
>>> dset = geowatch.coerce_kwcoco('geowatch-msi', geodata=True, dates=True, num_frames=32, image_size=(32, 32), num_videos=1)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_span='1y',
>>>     time_window=3,
>>>     affinity_type='soft2',
>>>     update_rule='distribute+pairwise')
>>> self.deterministic = True
>>> # xdoctest: +REQUIRES(--show)
>>> self.show_summary(samples_per_frame=1 if self.deterministic else 10, fnum=1)
>>> self.show_procedure(fnum=2)
show_affinity(fnum=3)[source]

Simple drawing of the affinity matrix.

show_procedure(idx=None, exclude=None, fnum=2, rng=None)[source]

Draw a figure that shows the process of performing on call to TimeWindowSampler.sample(). Each row illustrates an iteration of the algorithm. The left column draws the current indicies included in the sample and the right column draws how that sample (corresponding to the current row) influences the probability distribution for the next row.

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> from geowatch.utils.util_data import find_dvc_dpath
>>> dvc_dpath = find_dvc_dpath()
>>> coco_fpath = dvc_dpath / 'Drop1-Aligned-L1-2022-01/data.kwcoco.json'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_window=5,
>>>     affinity_type='soft2',
>>>     update_rule='distribute+pairwise')
>>> self.deterministic = False
>>> self.show_procedure(idx=0, fnum=10)
>>> self.show_affinity(fnum=100)
for idx in xdev.InteractiveIter(list(range(self.num_frames))):

self.show_procedure(idx=idx, fnum=1) xdev.InteractiveIter.draw()

self = TimeWindowSampler.from_coco_video(dset, vidid, time_window=5, affinity_type=’soft2’, update_rule=’distribute+pairwise’) self.deterministic = True self.show_summary(samples_per_frame=20, fnum=1) self.deterministic = False self.show_summary(samples_per_frame=20, fnum=2)

self = TimeWindowSampler.from_coco_video(dset, vidid, time_window=5, affinity_type=’hard’, update_rule=’distribute’) self.deterministic = True self.show_summary(samples_per_frame=20, fnum=3) self.deterministic = False self.show_summary(samples_per_frame=20, fnum=4)

self = TimeWindowSampler.from_coco_video(dset, vidid, time_window=5, affinity_type=’hardish’, update_rule=’distribute’) self.deterministic = True self.show_summary(samples_per_frame=20, fnum=5) self.deterministic = False self.show_summary(samples_per_frame=20, fnum=6)

>>> self.show_procedure(fnum=1)
>>> self.deterministic = True
>>> self.show_procedure(fnum=2)
>>> self.show_procedure(fnum=3)
>>> self.show_procedure(fnum=4)
>>> self.deterministic = False
>>> self.show_summary(samples_per_frame=3, fnum=10)
show_summary(samples_per_frame=1, fnum=1, show_indexes=False, with_temporal=True, compare_determ=True, title_suffix='')[source]

Visualize the affinity matrix and two views of a selected sample.

Plots a figure with three subfigures.

  1. The affinity matrix.

(2) A visualization of a random sampled over “index-space”. A matrix M, where each row is a sample index, each column is a timestep, M[i,j] = 1 (the cell is colored white) to indicate that a sample-i includes timestep-j.

(3) A visualization of the same random sample over “time-space”. A plot where x is the time-axis is drawn, and vertical lines indicate the selectable time indexes. For each sample, a horizontal line indicates the timespan of the sample and an “x” denotes exactly which timesteps are included in that sample.

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> from geowatch.utils.util_data import find_dvc_dpath
>>> import kwcoco
>>> # xdoctest: +REQUIRES(--show)
>>> dvc_dpath = find_dvc_dpath()
>>> coco_fpath = dvc_dpath / 'Drop6/data_vali_split1.kwcoco.zip'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> video_ids = list(ub.sorted_vals(dset.index.vidid_to_gids, key=len).keys())
>>> vidid = video_ids[2]
>>> # Demo behavior over a grid of parameters
>>> grid = list(ub.named_product({
>>>     'affinity_type': ['hard', 'soft2', 'hardish3', 'hardish2'],
>>>     'update_rule': ['distribute', 'pairwise+distribute'][0:1],
>>>     #'deterministic': [False, True],
>>>     'deterministic': [False],
>>>     'time_window': [5],
>>> }))
>>> import kwplot
>>> kwplot.autompl()
>>> for idx, kwargs in enumerate(grid):
>>>     print('kwargs = {!r}'.format(kwargs))
>>>     self = TimeWindowSampler.from_coco_video(dset, vidid, **kwargs)
>>>     self.show_summary(samples_per_frame=30, fnum=idx, show_indexes=False, deterministic=True)
update_affinity(affinity_type=None, update_rule=None)[source]

Construct the affinity matrix given the current affinity_type.

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> import os
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.sampler import *  # NOQA
>>> import geowatch
>>> data_dvc_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> coco_fpath = dvc_dpath / 'Drop6/data_vali_split1.kwcoco.zip'
>>> dset = kwcoco.CocoDataset(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_window=5,
>>>     affinity_type='contiguous',
>>>     update_rule='pairwise')
>>> self.deterministic = True
>>> self.show_procedure(fnum=1)
geowatch.tasks.fusion.datamodules.temporal_sampling.affinity_sample(affinity, size, include_indices=None, exclude_indices=None, allow_fewer=False, update_rule='pairwise', gamma=1, deterministic=False, time_kernel=None, unixtimes=None, error_level=2, rng=None, return_info=False, jit=False)[source]

Randomly select size timesteps from a larger pool based on affinity.

Given an NxN affinity matrix between frames and an initial set of indices to include, chooses a sample of other frames to complete the sample. Each row and column in the affinity matrix represent a “selectable” timestamp. Given an initial set of include_indices that indicate which timesteps must be included in the sample. An iterative process is used to select remaining indices such that size timesteps are returned. In each iteration we choose the “next” timestep based on a probability distribution derived from (1) the affinity matrix (2) the currently included set of indexes and (3) the update rule.

Parameters:

  • affinity (ndarray) – pairwise affinity matrix

  • size (int) – Number of sample indices to return

  • include_indices (List[int]) – Indices that must be included in the sample

  • exclude_indices (List[int]) – Indices that cannot be included in the sample

  • allow_fewer (bool) – if True, we will allow fewer than the requested “size” samples to be returned.

  • update_rule (str) – Modifies how the affinity matrix is used to create the probability distribution for the “next” frame that will be selected. a “+” separated string of codes which can contain:

    • pairwise - if included, each newly chosen sample will

      modulate the initial “main” affinity with it’s own affinity. Otherwise, only the affinity of the initially included rows are considered.

    • distribute - if included, every step of weight updates will

      downweight samples temporally close to the most recently selected sample.

  • gamma (float, default=1.0) – Exponent that modulates the probability distribution. Lower gamma will “flatten” the probability curve. At gamma=0, all frames will be equally likely regardless of affinity. As gamma -> inf, the rule becomes more likely to sample the maximum probability at each timestep. In the limit this becomes equivalent to deterministic=True.

  • deterministic (bool) – if True, on each step we choose the next timestamp with maximum probability. Otherwise, we randomly choose a timestep, but with probability according to the current distribution.

  • error_level (int) – Error and fallback behavior if perfect sampling is not possible. error level 0:

    might return excluded, duplicate indexes, or 0-affinity indexes if everything else is exhausted.

    error level 1:

    duplicate indexes will raise an error

    error level 2:

    duplicate and excluded indexes will raise an error

    error level 3:

    duplicate, excluded, and 0-affinity indexes will raise an error

  • rng (Coercible[RandomState]) – random state for reproducible sampling

  • return_info (bool) – If True, includes a dictionary of information that details the internal steps the algorithm took.

  • jit (bool) – NotImplemented - do not use

  • time_kernel (ndarray) – if specified, the sample will attempt to conform to this time kernel.

Returns:

The chosen indexes for the sample, or if return_info is True, then returns a tuple of chosen and the info dictionary.

Return type:

ndarray | Tuple[ndarray, Dict]

Raises:

TimeSampleError – if sampling is impossible

Possible Related Work:

  • Random Stratified Sampling Affinity Matrix

  • A quasi-random sampling approach to image retrieval

CommandLine

xdoctest -m geowatch.tasks.fusion.datamodules.temporal_sampling.affinity affinity_sample:0 --show
xdoctest -m geowatch.tasks.fusion.datamodules.temporal_sampling.affinity affinity_sample:1 --show

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.affinity import *  # NOQA
>>> low = datetime_cls.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> unixtimes = np.array(sorted(rng.randint(low, high, 113)), dtype=float)
>>> #
>>> affinity = soft_frame_affinity(unixtimes, version=2, time_span='1d')['final']
>>> include_indices = [5]
>>> size = 5
>>> chosen, info = affinity_sample(affinity, size, include_indices, update_rule='pairwise',
>>>                                return_info=True, deterministic=True)
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.plots import show_affinity_sample_process
>>> sns = kwplot.autosns()
>>> plt = kwplot.autoplt()
>>> show_affinity_sample_process(chosen, info)
>>> kwplot.show_if_requested()

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> low = datetime_cls.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> unixtimes = np.array(sorted(rng.randint(low, high, 5)), dtype=float)
>>> self = TimeWindowSampler(unixtimes, sensors=None, time_window=4,
>>>     affinity_type='soft2', time_span='0.3y',
>>>     update_rule='distribute+pairwise', allow_fewer=False)
>>> self.deterministic = False
>>> import pytest
>>> with pytest.raises(IndexError):
>>>     self.sample(0, exclude=[1, 2, 4], error_level=3)
>>> with pytest.raises(IndexError):
>>>     self.sample(0, exclude=[1, 2, 4], error_level=2)
>>> self.sample(0, exclude=[1, 2, 4], error_level=1)
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autompl()
>>> chosen, info = self.show_procedure(idx=0, fnum=10, exclude=[1, 2, 4])
>>> print('info = {}'.format(ub.urepr(info, nl=4)))
>>> kwplot.show_if_requested()

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.utils import coerce_time_kernel
>>> import kwarray
>>> import geowatch
>>> data_dvc_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> coco_fpath = data_dvc_dpath / 'Drop6/imgonly-KR_R001.kwcoco.json'
>>> dset = geowatch.coerce_kwcoco(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> time_kernel_code = '-3m,-1w,0,3m,1y'
>>> self = TimeWindowSampler.from_coco_video(
>>>     dset, vidid,
>>>     time_window=5,
>>>     time_kernel=time_kernel_code,
>>>     affinity_type='soft3',
>>>     update_rule='')
>>> self.deterministic = False
>>> self.show_affinity()
>>> include_indices = [len(self.unixtimes) // 2]
>>> exclude_indices = []
>>> affinity = self.affinity
>>> size = self.time_window
>>> deterministic = self.deterministic
>>> update_rule = self.update_rule
>>> unixtimes = self.unixtimes
>>> gamma = self.gamma
>>> time_kernel = self.time_kernel
>>> rng = kwarray.ensure_rng(None)
>>> deterministic = True
>>> return_info = True
>>> error_level = 2
>>> chosen, info = affinity_sample(
>>>     affinity=affinity,
>>>     size=size,
>>>     include_indices=include_indices,
>>>     exclude_indices=exclude_indices,
>>>     update_rule=update_rule,
>>>     gamma=gamma,
>>>     deterministic=deterministic,
>>>     error_level=error_level,
>>>     rng=rng,
>>>     return_info=return_info,
>>>     time_kernel=time_kernel,
>>>     unixtimes=unixtimes,
>>> )
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autompl()
>>> info['title_suffix'] = chr(10) + time_kernel_code
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.plots import show_affinity_sample_process
>>> show_affinity_sample_process(chosen, info, fnum=1)
>>> kwplot.show_if_requested()
geowatch.tasks.fusion.datamodules.temporal_sampling.cython_aff_samp_mod()[source]

Old JIT code, no longer works

geowatch.tasks.fusion.datamodules.temporal_sampling.guess_missing_unixtimes(unixtimes, assume_delta=86400)[source]

Hueristic solution to fill in missing time values via interpolation / extrapolation.

To succesfully interpolate nan values must be between two non-nan values. In all other cases we have to make an assumption about the timedelta between frames, which can be specified and is one day by default.

Parameters:

  • unixtimes (ndarray) – numpy array of numeric unix timestamps that may contain nan values.

  • assume_delta (float) – The fallback delta between timesteps when surrounding context is unavailable. Defaults to 86400 seconds - i.e. 1 day.

Returns:

The same array, but nan values are filled with interpolated or extrapolated values.

Return type:

ndarray

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.utils import *  # NOQA
>>> import ubelt as ub
>>> cases = [
>>>     np.array([np.nan, np.nan, np.nan, np.nan, np.nan]),
>>>     np.array([np.nan, 20, 30, np.nan, np.nan]),
>>>     np.array([0, np.nan, np.nan, np.nan, 10]),
>>>     np.array([np.nan, np.nan, 9001, np.nan, np.nan]),
>>>     np.array([1, 2, 3, 4, 5]),
>>>     np.array([1, 2, np.nan, 4, 5]),
>>> ]
>>> for case_ in cases:
>>>     unixtimes = case_
>>>     print('case_ = {}'.format(ub.urepr(case_, nl=1)))
>>>     guess = guess_missing_unixtimes(unixtimes)
>>>     print('guess = {}'.format(ub.urepr(guess, nl=1)))
geowatch.tasks.fusion.datamodules.temporal_sampling.hard_frame_affinity(unixtimes, sensors, time_window, time_kernel=None, time_span=None, blur=False)[source]
geowatch.tasks.fusion.datamodules.temporal_sampling.hard_time_sample_pattern(unixtimes, time_window, time_kernel=None, time_span=None)[source]

Finds hard time sampling indexes

Parameters:

  • unixtimes (ndarray) – list of unix timestamps indicating available temporal samples

  • time_window (int) – number of frames per sample

References

https://docs.google.com/presentation/d/1GSOaY31cKNERQObl_L3vk0rGu6zU7YM_ZFLrdksHSC0/edit#slide=id.p

Example

>>> low = datetime_cls.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> base_unixtimes = np.array(sorted(rng.randint(low, high, 20)), dtype=float)
>>> unixtimes = base_unixtimes.copy()
>>> #unixtimes[rng.rand(*unixtimes.shape) < 0.1] = np.nan
>>> time_window = 5
>>> sample_idxs = hard_time_sample_pattern(unixtimes, time_window, time_span='2y')
>>> name = 'demo-data'

>>> #unixtimes[:] = np.nan
>>> time_window = 5
>>> sample_idxs = hard_time_sample_pattern(unixtimes, time_window, time_span='2y')
>>> name = 'demo-data'
geowatch.tasks.fusion.datamodules.temporal_sampling.plot_dense_sample_indices(sample_idxs, unixtimes, title_suffix='', linewidths=0)[source]

Visualization helper

Parameters:

  • sample_idxs (List[List[int]] | ArrayLike[ndim=2]) – A list of frame indexes that index into unixtimes. I.e. multiple samples of frame index groups.

  • unixtimes (List | ArrayLike[ndim=1] | None) – [ An array of unix timestamps corresonding to frame indexes. If unspecified, then frame indexes are shown directly.

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> unixtimes = None
>>> sample_idxs = [
>>>     [0, 1, 2],
>>>     [3, 5, 6],
>>>     [2, 3, 6],
>>> ]
>>> plot_dense_sample_indices(sample_idxs, unixtimes)
geowatch.tasks.fusion.datamodules.temporal_sampling.plot_temporal_sample(affinity, sample_idxs, unixtimes, sensors=None, fnum=1)[source]

Visualization helper

geowatch.tasks.fusion.datamodules.temporal_sampling.plot_temporal_sample_indices(sample_idxs, unixtimes=None, sensors=None, title_suffix='')[source]

Visualization helper

Parameters:

  • sample_idxs (List[List[int]]) – A list of frame indexes that index into unixtimes. I.e. multiple samples of frame index groups.

  • unixtimes (List | None) – An array of unix timestamps corresonding to frame indexes. If unspecified, then frame indexes are shown directly.

Example

>>> # xdoctest: +REQUIRES(module:kwplot)
>>> unixtimes = None
>>> sample_idxs = [
>>>     [0, 1, 2],
>>>     [3, 5, 6],
>>>     [2, 3, 6],
>>> ]
>>> plot_temporal_sample_indices(sample_idxs, unixtimes)
geowatch.tasks.fusion.datamodules.temporal_sampling.show_affinity_sample_process(chosen, info, fnum=1)[source]

Debugging / demo visualization of the iterative sample algorithm. For details see TimeWindowSampler.show_procedure().

geowatch.tasks.fusion.datamodules.temporal_sampling.soft_frame_affinity(unixtimes, sensors=None, time_kernel=None, time_span=None, version=1, heuristics='default')[source]

Produce a pairwise affinity weights between frames based on a dilated time heuristic.

Example

>>> from geowatch.tasks.fusion.datamodules.temporal_sampling.affinity import *  # NOQA
>>> low = datetime_mod.datetime.now().timestamp()
>>> high = low + datetime_mod.timedelta(days=365 * 5).total_seconds()
>>> rng = kwarray.ensure_rng(0)
>>> base_unixtimes = np.array(sorted(rng.randint(low, high, 113)), dtype=float)

>>> # Test no missing data case
>>> unixtimes = base_unixtimes.copy()
>>> allhave_weights = soft_frame_affinity(unixtimes, version=2)
>>> #
>>> # Test all missing data case
>>> unixtimes = np.full_like(unixtimes, fill_value=np.nan)
>>> allmiss_weights = soft_frame_affinity(unixtimes, version=2)
>>> #
>>> # Test partial missing data case
>>> unixtimes = base_unixtimes.copy()
>>> unixtimes[rng.rand(*unixtimes.shape) < 0.1] = np.nan
>>> anymiss_weights_1 = soft_frame_affinity(unixtimes, version=2)
>>> unixtimes = base_unixtimes.copy()
>>> unixtimes[rng.rand(*unixtimes.shape) < 0.5] = np.nan
>>> anymiss_weights_2 = soft_frame_affinity(unixtimes, version=2)
>>> unixtimes = base_unixtimes.copy()
>>> unixtimes[rng.rand(*unixtimes.shape) < 0.9] = np.nan
>>> anymiss_weights_3 = soft_frame_affinity(unixtimes, version=2)

>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autoplt()
>>> pnum_ = kwplot.PlotNums(nCols=5)
>>> kwplot.figure(fnum=1, doclf=True)
>>> # kwplot.imshow(kwarray.normalize(daylight_weights))
>>> kwplot.imshow(kwarray.normalize(allhave_weights['final']), pnum=pnum_(), title='no missing dates')
>>> kwplot.imshow(kwarray.normalize(anymiss_weights_1['final']), pnum=pnum_(), title='any missing dates (0.1)')
>>> kwplot.imshow(kwarray.normalize(anymiss_weights_2['final']), pnum=pnum_(), title='any missing dates (0.5)')
>>> kwplot.imshow(kwarray.normalize(anymiss_weights_3['final']), pnum=pnum_(), title='any missing dates (0.9)')
>>> kwplot.imshow(kwarray.normalize(allmiss_weights['final']), pnum=pnum_(), title='all missing dates')

>>> import pandas as pd
>>> sns = kwplot.autosns()
>>> fig = kwplot.figure(fnum=2, doclf=True)
>>> kwplot.imshow(kwarray.normalize(allhave_weights['final']), pnum=(1, 3, 1), title='pairwise affinity')
>>> row_idx = 5
>>> df = pd.DataFrame({k: v[row_idx] for k, v in allhave_weights.items()})
>>> df['index'] = np.arange(df.shape[0])
>>> data = df.drop(['final'], axis=1).melt(['index'])
>>> kwplot.figure(fnum=2, pnum=(1, 3, 2))
>>> sns.lineplot(data=data, x='index', y='value', hue='variable')
>>> fig.gca().set_title('Affinity components for row={}'.format(row_idx))
>>> kwplot.figure(fnum=2, pnum=(1, 3, 3))
>>> sns.lineplot(data=df, x='index', y='final')
>>> fig.gca().set_title('Affinity components for row={}'.format(row_idx))

Example

>>> # xdoctest: +REQUIRES(env:SMART_DATA_DVC_DPATH)
>>> from geowatch.tasks.fusion.datamodules.temporal_sampling import *  # NOQA
>>> import geowatch
>>> import kwimage
>>> data_dvc_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> coco_fpath = data_dvc_dpath / 'Drop6/imgonly-KR_R001.kwcoco.json'
>>> dset = geowatch.coerce_kwcoco(coco_fpath)
>>> vidid = dset.dataset['videos'][0]['id']
>>> self = TimeWindowSampler.from_coco_video(dset, vidid, time_window=5, time_kernel='-1y,-3m,0,3m,1y', affinity_type='soft3')
>>> unixtimes = self.unixtimes
>>> sensors = self.sensors
>>> time_kernel = self.time_kernel
>>> time_span = None
>>> version = 4
>>> heuristics = 'default'
>>> weights = soft_frame_affinity(unixtimes, sensors, time_kernel, time_span, version, heuristics)
>>> # xdoctest: +REQUIRES(--show)
>>> import kwplot
>>> kwplot.autoplt()
>>> pnum_ = kwplot.PlotNums(nCols=5)
>>> kwplot.figure(fnum=1, doclf=True)
>>> kwplot.imshow(kwarray.normalize(weights['final']), pnum=pnum_(), title='all missing dates')

>>> import pandas as pd
>>> sns = kwplot.autosns()
>>> fig = kwplot.figure(fnum=2, doclf=True)
>>> kwplot.imshow(weights['final'], pnum=(1, 3, 1), title='pairwise affinity', cmap='viridis')
>>> row_idx = 200
>>> df = pd.DataFrame({k: v[row_idx] for k, v in weights.items()})
>>> df['index'] = np.arange(df.shape[0])
>>> data = df.drop(['final'], axis=1).melt(['index'])
>>> kwplot.figure(fnum=2, pnum=(1, 3, 2))
>>> sns.lineplot(data=data, x='index', y='value', hue='variable')
>>> fig.gca().set_title('Affinity components for row={}'.format(row_idx))
>>> kwplot.figure(fnum=2, pnum=(1, 3, 3))
>>> sns.lineplot(data=df, x='index', y='final')
>>> fig.gca().set_title('Affinity components for row={}'.format(row_idx))