geowatch.geoannots.geomodels module

Geojson object oriented interface for region and site models.

This defines two classes SiteModel and RegionModel, both of which inherit from geojson.FeatureCollection, so all geojson operations are valid, but these classes contain extra convenience methods for loading, dumping, manipulating, validating, and inspecting the data.

A non exhaustive list of convenience methods / properties of note are shared by both site and region models are:

• dumps - convert to a geojson string

• pandas - convert to a geopandas data frame

• coerce_multiple - read multiple geojson files at once.

• header - a quick way to access the singular header row (region for region models and site for site models).

• body_features - any row that is not a header is a body feature (site_summaries for region models and observations for site models).

• validate - checks the site/region model against the schema.

• random - classmethod to make a random instance of the site / region model for testing

Official T&E Terminology:

A Region Model gives an overview of entire region and summarizes all sites it contains. It consists of:

• A single header feature with type=”region” that defines the region spacetime bounds

• Multiple body features with type=”site_summary” that correspond to the bounds of an entire site. (i.e. there is one for each site in the region). A site summary has a “status” that applies to the entire temporal range of the site. (i.e. positive, negative, ignore)

A Site Model gives a detailed account of a single site within a region. It consists of:

• A single header feature with type=”site” that roughly corresponds to one of the “site_summary” features in the region model. It also contains the holistic “status” field.

• Multiple body features with type=”observation”. This represents a single keyframe at a single point in time within the site’s activity sequence. It contains a “current_phase” label that describes the specific phase of an activity at that current point in time.

Note: A site summary may exist on its own (i.e. without a corresponding site model) that gives a rough overview with holistic status, rough spatial bounds and a start / end date.

New region model specific convenience methods / properties are:

• site_summaries

• region_id

• pandas_summaries

• pandas_region

New site model specific convenience methods / properties are:

• observations

• pandas_observations

• as_summary

• region_id

• site_id

• status

SeeAlso:

../rc/registry.py

The following example illustrates how to read region / site models efficiently

Example

>>> # xdoctest: +REQUIRES(env:HAS_DVC)
>>> import geowatch
>>> dvc_data_dpath = geowatch.find_dvc_dpath(tags='phase2_data', hardware='auto')
>>> region_models_dpath = dvc_data_dpath / 'annotations/drop6/region_models'
>>> site_models_dpath = dvc_data_dpath / 'annotations/drop6/site_models'
>>> from geowatch.geoannots import geomodels
>>> region_models = list(geomodels.RegionModel.coerce_multiple(region_models_dpath))
>>> site_models = list(geomodels.SiteModel.coerce_multiple(site_models_dpath, workers=8))
>>> print(f'Number of region models: {len(region_models)}')
>>> print(f'Number of site models: {len(site_models)}')
>>> # Quick demo of associating sites to regions
>>> region_id_to_sites = ub.group_items(site_models, key=lambda s: s.header['properties']['region_id'])
>>> region_id_to_num_sites = ub.udict(region_id_to_sites).map_values(len)
>>> print('region_id_to_num_sites = {}'.format(ub.urepr(region_id_to_num_sites, nl=1)))
>>> # It is also easy to convert these models to geopandas
>>> region_model = region_models[0]
>>> gdf = region_model.pandas()
>>> print(gdf)

XDEV_PROFILE=1 xdoctest ~/code/watch/geowatch/geoannots/geomodels.py

For testing the following example shows how to generate and inspect a random site / region model.

Example

>>> from geowatch.geoannots.geomodels import *
>>> # Generate a region model and also return its sites
>>> region, sites = RegionModel.random(with_sites=True, rng=0)
>>> # A region model consists of a region header
>>> region_header = region.header
>>> # And multiple site summaries. (We take the first one here)
>>> site_summary = list(region.site_summaries())[0]
>>> print('region_header.properties = {}'.format(ub.urepr(region_header['properties'], nl=1)))
region_header.properties = {
    'type': 'region',
    'region_id': 'DR_R684',
    'version': '2.4.3',
    'mgrs': '51PXM',
    'start_date': '2011-05-28',
    'end_date': '2018-09-13',
    'originator': 'demo-truth',
    'model_content': 'annotation',
    'comments': 'demo-data',
}
>>> print('site_summary.properties = {}'.format(ub.urepr(site_summary['properties'], nl=1)))
site_summary.properties = {
    'type': 'site_summary',
    'status': 'positive_annotated',
    'version': '2.0.1',
    'site_id': 'DR_R684_0000',
    'mgrs': '51PXM',
    'start_date': '2011-05-28',
    'end_date': '2018-09-13',
    'score': 1,
    'originator': 'demo',
    'model_content': 'annotation',
    'validated': 'True',
    'cache': {'color': [0.5511393746687864, 1.0, 0.0]},
}
>>> # A site model consists of a site header that roughly corresponds to a
>>> # site summary in the region file
>>> site = sites[0]
>>> site_header = site.header
>>> # It also contains one or more observations
>>> site_obs = list(site.observations())[0]
>>> print('site_header.properties = {}'.format(ub.urepr(site_header['properties'], nl=1)))
site_header.properties = {
    'type': 'site',
    'status': 'positive_annotated',
    'version': '2.0.1',
    'site_id': 'DR_R684_0000',
    'mgrs': '51PXM',
    'start_date': '2011-05-28',
    'end_date': '2018-09-13',
    'score': 1,
    'originator': 'demo',
    'model_content': 'annotation',
    'validated': 'True',
    'cache': {'color': [0.5511393746687864, 1.0, 0.0]},
    'region_id': 'DR_R684',
}
>>> print('site_obs.properties = {}'.format(ub.urepr(site_obs['properties'], nl=1)))
site_obs.properties = {
    'type': 'observation',
    'observation_date': '2011-05-28',
    'source': 'demosat-220110528T132754',
    'sensor_name': ...'demosat-2'...,
    'current_phase': 'No Activity',
    'is_occluded': 'False',
    'is_site_boundary': 'True',
    'score': 1.0,
}

class geowatch.geoannots.geomodels.RegionModel(features, **extra)

Bases: _Model

Wrapper around a geojson region model FeatureCollection

Todo

Rename to Region?

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = RegionModel.random()
>>> print(self)
>>> self.validate(strict=False)

Initialises a FeatureCollection object from the :param features: List of features to constitute the FeatureCollection. :type features: list :return: FeatureCollection object :rtype: FeatureCollection

info()

classmethod load_schema(strict=True)

site_summaries()

classmethod coerce(data, parse_float=None)

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> import ubelt as ub
>>> dpath = ub.Path.appdir('geowatch/tests/geoannots/coerce').ensuredir()
>>> region = RegionModel.random(with_sites=False, rng=0)
>>> data = fpath = (dpath/ 'region.geojson')
>>> fpath.write_text(region.dumps())
>>> region_models = list(RegionModel.coerce_multiple(fpath))
>>> region_model = RegionModel.coerce(fpath)

pandas_summaries()

Returns:

the site summaries as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = RegionModel.random()
>>> gdf = self.pandas_summaries()
>>> print(gdf)
>>> # Test empty pandas summary
>>> self = RegionModel.random(num_sites=0)
>>> gdf = self.pandas_summaries()
>>> print(gdf)
>>> assert len(gdf) == 0

pandas_region()

Returns:

the region header as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = RegionModel.random()
>>> print(self.pandas_region())

pandas_header()

Returns:

the region header as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = RegionModel.random()
>>> print(self.pandas_region())

pandas_body()

Returns:

the site summaries as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = RegionModel.random()
>>> gdf = self.pandas_summaries()
>>> print(gdf)
>>> # Test empty pandas summary
>>> self = RegionModel.random(num_sites=0)
>>> gdf = self.pandas_summaries()
>>> print(gdf)
>>> assert len(gdf) == 0

classmethod random(with_sites=False, **kwargs)

Creates a random region model optionally with random sites for use in testing / demos.

Parameters:

• with_sites (bool) – also returns site models if True

• **kwargs – passed to geowatch.demo.metrics_demo.demo_truth.random_region_model(). Some of these args are:

  num_sites num_observations start_time end_time region_poly rng

Returns:

RegionModel | Tuple[RegionModel, SiteModelCollection]

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> region1 = RegionModel.random(with_sites=False, rng=0)
>>> region2, sites2 = RegionModel.random(with_sites=True, rng=0)
>>> assert region1 == region2, 'rngs should be the same'

add_site_summary(summary)

Add a site summary to the region.

Parameters:

summary (SiteSummary | SiteModel) – a site summary or site model. If given as a site model it is converted to a site summary and then added.

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> region = RegionModel.random(num_sites=False)
>>> site1 = SiteModel.random(region=region)
>>> site2 = SiteModel.random(region=region)
>>> site3 = SiteModel.random(region=region)
>>> summary = site2.as_summary()
>>> region.add_site_summary(site1)
>>> region.add_site_summary(summary)
>>> region.add_site_summary(dict(site3.as_summary()))
>>> import pytest
>>> with pytest.raises(TypeError):
...     region.add_site_summary(dict(site3))
>>> assert len(list(region.site_summaries())) == 3

property region_id

Get the region_id from the geojson header

fixup()

Fix common issues with this region model

Returns:

RegionModel

fix_multipolygons()

remove_invalid_properties()

Remove invalid properties from this region model that have caused issues in the past.

ensure_comments()

infer_header(region_header=None)

Infer any missing header information from site summaries.

If this region model does not have a header, but it contains site summaries, then use that information to infer a header value. Useful when converting site summaries to full region models.

Parameters:

region_header (RegionHeader) – if specified, use this information when possible and then infer the rest.

SeeAlso:

• SiteModelCollection.as_region_model()

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> # Make a region without a header
>>> self = RegionModel.random()
>>> self.features.remove(self.header)
>>> assert self.header is None
>>> # Infer the header using site summaries
>>> self.infer_header()
>>> assert self.header is not None

class geowatch.geoannots.geomodels.SiteModel(features, **extra)

Bases: _Model

Wrapper around a geojson site model FeatureCollection

Todo

Rename to Site?

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = SiteModel.random()
>>> print(self)
>>> self.validate(strict=False)

Initialises a FeatureCollection object from the :param features: List of features to constitute the FeatureCollection. :type features: list :return: FeatureCollection object :rtype: FeatureCollection

info()

classmethod load_schema(strict=True)

property header

observations()

Features containing specific observations with phase labels

pandas_observations()

Returns:

the site summaries as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = SiteModel.random()
>>> gdf = self.pandas_observations()
>>> print(gdf)
>>> # Test empty pandas summary
>>> del self.features[1:]
>>> gdf = self.pandas_observations()
>>> print(gdf)
>>> assert len(gdf) == 0

pandas_site()

Returns:

the region header as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = SiteModel.random()
>>> print(self.pandas_site())

pandas_header()

Returns:

the region header as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = SiteModel.random()
>>> print(self.pandas_site())

pandas_body()

Returns:

the site summaries as a data frame

Return type:

geopandas.GeoDataFrame

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = SiteModel.random()
>>> gdf = self.pandas_observations()
>>> print(gdf)
>>> # Test empty pandas summary
>>> del self.features[1:]
>>> gdf = self.pandas_observations()
>>> print(gdf)
>>> assert len(gdf) == 0

classmethod random(rng=None, region=None, site_poly=None, **kwargs)

Parameters:

• rng (int | str | RandomState | None) – seed or random number generator

• region (RegionModel | None) – if specified generate a new site in this region model. (This will overwrite some of the kwargs).

• site_poly (kwimage.Polygon | shapely.geometry.Polygon | None) – if specified, this polygon is used as the geometry for new site models. Note: all site models will get this geometry, so typically this is only used when num_sites=1.

• **kwargs – passed to geowatch.demo.metrics_demo.demo_truth.random_region_model().

Returns:

SiteModel

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> region1 = RegionModel.random(with_sites=False, rng=0)
>>> region2, sites2 = RegionModel.random(with_sites=True, rng=0)
>>> assert region1 == region2, 'rngs should be the same'

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> region = RegionModel.random(with_sites=False, rng=0)
>>> site = SiteModel.random(region=region)
>>> assert region.region_id == site.region_id

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> import kwimage
>>> region = RegionModel.random(with_sites=False, rng=0)
>>> # Test specification of the site geometry.
>>> site_poly = kwimage.Polygon.coerce(region.geometry)
>>> site = SiteModel.random(region=region, site_poly=site_poly)
>>> assert abs(region.geometry.area - site.geometry.area) < 1e-7
>>> site = SiteModel.random(region=region, site_poly=site_poly.scale(10))
>>> assert abs(region.geometry.area - site.geometry.area) > 1e-7

as_summary()

Modify and return this site header feature as a site-summary body feature for a region model.

Returns:

SiteSummary

property region_id

property site_id

property status

fix_geom()

fix_sensor_names()

fix_current_phase_salient()

fix_multipolygons()

fixup()

Fix common issues with this site model

Returns:

SiteModel

fix_aliased_properties()

Some models are written with aliased properties (e.g. stop_date instead of end_date). This fixes them.

fix_old_schema_properties()

If an old schema property exists and is not null, move it to the cache.

ensure_isodates()

Ensure that dates are provided as dates and not datetimes

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> site = SiteModel.random()
>>> # Set props as datetimes
>>> site.header['properties']['start_date'] = '1970-01-01T000000'
>>> site.features[1]['properties']['observation_date'] = '1970-01-01T000000'
>>> site.ensure_isodates()
>>> # The fixup ensure dates
>>> assert site.features[1]['properties']['observation_date'] == '1970-01-01'
>>> assert site.header['properties']['start_date'] == '1970-01-01'

clamp_scores()

remove_invalid_properties()

Remove invalid properties from this site model

class geowatch.geoannots.geomodels.Point(id=None, geometry=None, properties=None, **extra)

Bases: _Feature

Initialises a Feature object with the given parameters.

Parameters:

• id (str, int) – Feature identifier, such as a sequential number.

• geometry – Geometry corresponding to the feature.

• properties (dict) – Dict containing properties of the feature.

Returns:

Feature object

Return type:

Feature

class geowatch.geoannots.geomodels.RegionHeader(id=None, geometry=None, properties=None, **extra)

Bases: _Feature

The region header feature of a region model.

Initialises a Feature object with the given parameters.

Parameters:

• id (str, int) – Feature identifier, such as a sequential number.

• geometry – Geometry corresponding to the feature.

• properties (dict) – Dict containing properties of the feature.

Returns:

Feature object

Return type:

Feature

classmethod empty()

Create an empty region header

classmethod coerce(data)

Example

>>> data = RegionModel.random()
>>> h1 = RegionHeader.coerce(data)
>>> h2 = RegionHeader.coerce(data.header)
>>> assert h1 == h2

RegionHeader.coerce(orig_region_model)

ensure_isodates()

class geowatch.geoannots.geomodels.SiteSummary(id=None, geometry=None, properties=None, **extra)

Bases: _Feature, _SiteOrSummaryMixin

The site-summary body feature of a region model.

Initialises a Feature object with the given parameters.

Parameters:

• id (str, int) – Feature identifier, such as a sequential number.

• geometry – Geometry corresponding to the feature.

• properties (dict) – Dict containing properties of the feature.

Returns:

Feature object

Return type:

Feature

classmethod from_geopandas_frame(df, drop_id=True)

as_site()

Modify and return this site summary feature as a site header feature for a site model.

Returns:

SiteHeader

Example

>>> # Convert a RegionModel to a collection of SiteModels
>>> from geowatch.geoannots import geomodels
>>> region = geomodels.RegionModel.random()
>>> sites = []
>>> for sitesum in region.site_summaries():
>>>     # Current hacky way to pass along region ids
>>>     sitesum['properties']['cache']['region_id'] = region.region_id
>>>     # This only produces a site header, we may need to add
>>>     # observations to the site model itself as well
>>>     site_header = sitesum.as_site()
>>>     site = SiteModel(features=[site_header])
>>>     sites.append(site)

fixup()

Fixup the site summary

classmethod coerce(data)

classmethod random(rng=None, region=None, site_poly=None, **kwargs)

Parameters:

• rng (int | str | RandomState | None) – seed or random number generator

• region (RegionModel | None) – if specified generate a new site in this region model. (This will overwrite some of the kwargs).

• site_poly (kwimage.Polygon | shapely.geometry.Polygon | None) – if specified, this polygon is used as the geometry for new site models. Note: all site models will get this geometry, so typically this is only used when num_sites=1.

• **kwargs – passed to geowatch.demo.metrics_demo.demo_truth.random_region_model().

Returns:

SiteSummary

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> sitesum = SiteSummary.random(rng=0)
>>> print('sitesum = {}'.format(ub.urepr(sitesum, nl=2)))

class geowatch.geoannots.geomodels.SiteHeader(id=None, geometry=None, properties=None, **extra)

Bases: _Feature, _SiteOrSummaryMixin

The site header feature of a site model.

Initialises a Feature object with the given parameters.

Parameters:

• id (str, int) – Feature identifier, such as a sequential number.

• geometry – Geometry corresponding to the feature.

• properties (dict) – Dict containing properties of the feature.

Returns:

Feature object

Return type:

Feature

classmethod empty()

Create an empty region header

Example

from geowatch.geoannots.geomodels import * # NOQA self = SiteHeader.empty() …

as_summary()

Modify and return this site header feature as a site-summary body feature for a region model.

Returns:

SiteSummary

classmethod coerce(data)

classmethod random(rng=None, region=None, site_poly=None, **kwargs)

Parameters:

• rng (int | str | RandomState | None) – seed or random number generator

• region (RegionModel | None) – if specified generate a new site header in this region model.

• site_poly (kwimage.Polygon | shapely.geometry.Polygon | None) – if specified, this polygon is used as the geometry

• **kwargs – passed to geowatch.demo.metrics_demo.demo_truth.random_region_model().

Returns:

SiteHeader

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> site_header = SiteHeader.random(rng=0)
>>> print('site_header = {}'.format(ub.urepr(site_header, nl=2)))

class geowatch.geoannots.geomodels.Observation(id=None, geometry=None, properties=None, **extra)

Bases: _Feature

The observation body feature of a site model.

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> Observation()

Initialises a Feature object with the given parameters.

Parameters:

• id (str, int) – Feature identifier, such as a sequential number.

• geometry – Geometry corresponding to the feature.

• properties (dict) – Dict containing properties of the feature.

Returns:

Feature object

Return type:

Feature

classmethod coerce(data)

property observation_date

classmethod random(rng=None, region=None, site_poly=None, **kwargs)

Parameters:

• rng (int | str | RandomState | None) – seed or random number generator

• region (RegionModel | None) – if specified generate a new observation in this region model.

• site_poly (kwimage.Polygon | shapely.geometry.Polygon | None) – if specified, this polygon is used as the geometry for new observation

• **kwargs – passed to geowatch.demo.metrics_demo.demo_truth.random_region_model().

Returns:

Observation

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> obs = Observation.random(rng=0)
>>> print(f'obs={obs}')

classmethod empty()

Create an empty observation

Example

>>> from geowatch.geoannots.geomodels import *  # NOQA
>>> self = Observation.empty()
>>> print(f'self = {ub.urepr(self, nl=2)}')

class geowatch.geoannots.geomodels.ModelCollection(iterable=(), /)

Bases: list

A storage container for multiple site / region models

fixup()

validate(strict=False, stop_on_failure=True, verbose=1, mode='process', workers=0)

Validate multiple models in parallel

class geowatch.geoannots.geomodels.PointModelCollection(iterable=(), /)

Bases: ModelCollection

class geowatch.geoannots.geomodels.SiteModelCollection(iterable=(), /)

Bases: ModelCollection

as_region_model(region_header=None, region_id=None, strict=True)

Convert a set of site models to a region model

Parameters:

• region (RegonModel | RegionHeader | None) – If specified, use this information to generate the new region header. If unspecified, we attempt to infer this from the site models.

• region_id (str | None) – if specified, use this as the region id

• strict (bool) – if False, ignore missing uninferable information.

Returns:

a new region model where each site in this collection

appears as a site summary.

Return type:

RegonModel

Example

>>> from geowatch.geoannots.geomodels import RegionModel
>>> region, sites = RegionModel.random(with_sites=True, rng=0)
>>> self = SiteModelCollection(sites)
>>> self.as_region_model()

to_point_model()

classmethod coerce(data)

Create a collection of site models from input - usually a directory containing site model geojson files.

SeeAlso:

SiteModel.coerce_multiple().

class geowatch.geoannots.geomodels.PointModel(features, **extra)

Bases: _Model

Initialises a FeatureCollection object from the :param features: List of features to constitute the FeatureCollection. :type features: list :return: FeatureCollection object :rtype: FeatureCollection

geowatch.geoannots.geomodels.handle_error(msg, extype=<class 'ValueError'>, strict=True)

geowatch.geoannots.geomodels.coerce_site_or_region_model(model_data)

Parameters:

model_data (dict) – A geojson FeatureCollection that should correspond to a SiteModel or RegionModel.

Returns:

SiteModel | RegionModel - return type depends on the input data