@@ -16,7 +16,22 @@ From Downloading L1C to computing radiometric indices
Downloading L1C Product
-----------------------
A function allows tiles to be downloaded at L1C level according to a set of parameters, which the most important are :
- **provider** (Default to peps): EO products provider.
- **download** (Default to True): Download condition. When False only search and returns L1C products available.
- **dl_mode** (Default to "multit" for multithreading.): Download mode among sequential, multiprocessing or multithreading.
- **start** (Default to "2015-01-01"): Period of Interest first date.
- **end** (Default to "9999-12-31"): Period of Interest last date.
- **min_cloudcover** (Default to 0): Cloudcover minimum threshold value to filter searched products.
- **max_cloudcover** (Default to 0): Cloudcover maximmum threshold value to filter searched products.
The python reference guide gives all the possible parameters for this function.
In the following case, the line of code will launch the download of the 40KCB tile from 2022-01-01 on the PEPS provider, selecting images that are less than 80% cloudy.
@@ -25,19 +40,18 @@ Three functions are used to compute the different products from downloaded raw L
For all 3, the process is the same: entire database is scanned to identify a list of missing products for which the computing will be done.
In case of no missing products, nothing is done. Each function has its own set of specific parameters (see details bellow), with these in common:
- _date_min_ and _date_max_: the date after, date before, or time range between which the missing products will be computed. Default values _None_ results in the entire database being scanned.
- _reprocess (boolean)_: possibility to reprocess already computed products. Default value _False_.
- _nb_proc_: All products in the process list are computed using multiprocessing to reduce processing time. Default value is 4, to increase according to your hardware. Limited by the number of cores of your processor.
- **date_min and _date_max**: the date after, date before, or time range between which the missing products will be computed. Default values _None_ results in the entire database being scanned.
- **reprocess (boolean)**: possibility to reprocess already computed products. Default value _False_.
- **nb_proc**: All products in the process list are computed using multiprocessing to reduce processing time. Default value is 4, to increase according to your hardware. Limited by the number of cores of your processor.
### Computing L2A
This function is used to compute missing L2A for L1C products present in the local database. Possibility to specify dates or reprocessing, see parameters above.
@@ -64,20 +78,19 @@ This function computes all missing indices for l2a products. Indices are given a
- **nodata_clouds**: Default value _True_, to produce a second set of cloudmasked index images (cloudmasks have to be be present). If not specified CM001 will be used. For CM003 & CM004, other CM specific parameters can be adjusted.
You can compute quicklooks for multiple provided tiles for L1C and/or L2A products. If no tile provided, whole L1C + L2A product database is used. Possibility to set specific output QL resolution (default 750m/px). Possibility to set output format JPEG (default) or TIFF.
The _size_ function retrieves the used disk space for each kind of product. Execution can take time, depending on database size, disk or network speed. Local and archived disk spaces are analysed.
Different functions are called to retrieve a list all database products :
```python
>>> t.l1c # list downloaded l1c indice products
>>> t.l2a # list l2a indice products
>>> t.cloudmasks # list cloudmask products
>>> t.ndvi # list ndvi indice products
>>> t.ndwigao # list ndwi Gao indice products
>>> t.mndwi # list mndwi indice products
```
.. code-block:: python
t.l1c # list downloaded l1c indice products
t.l2a # list l2a indice products
t.cloudmasks # list cloudmask products
t.ndvi # list ndvi indice products
t.ndwigao # list ndwi Gao indice products
t.mndwi # list mndwi indice products
As a ProductList class this retrieved list of products can be reduced using multiples functions :
```python
>>> t.l2a.products # list of product names
>>> t.l2a.dates # list of product dates
>>> t.l2a.first # the first (oldest) product in database
>>> t.l2a.last # the last (newest) product in database
.. code-block:: python
>>> t.l2a.filter_dates(date_min = "yyyy-mm-dd", date_max = "yyyy-mm-dd") # list of products between date_min and date_max
>>> t.l2a.filter_dates(date_min = "yyyy-mm-dd") # list of products after date_min
>>> t.l2a.filter_dates(date_max = "yyyy-mm-dd") # list of products before date_max
t.l2a.products # list of product names
t.l2a.dates # list of product dates
t.l2a.first # the first (oldest) product in database
t.l2a.last # the last (newest) product in database
.. code-block:: python
t.l2a.filter_dates(date_min = "yyyy-mm-dd", date_max = "yyyy-mm-dd") # list of products between date_min and date_max
t.l2a.filter_dates(date_min = "yyyy-mm-dd") # list of products after date_min
t.l2a.filter_dates(date_max = "yyyy-mm-dd") # list of products before date_max
.. code-block:: python
>>> t.l2a.filter_clouds(cover_min = 0, cover_max = 100) # list of products with a cloud cover between cover_min and cover_max
>>> t.l2a.filter_clouds(cover_min = 20) # list of products with more than 20% cloud cover
>>> t.l2a.filter_clouds(cover_max = 50) # list of products with less than 50% cloud cover
```
t.l2a.filter_clouds(cover_min = 0, cover_max = 100) # list of products with a cloud cover between cover_min and cover_max
t.l2a.filter_clouds(cover_min = 20) # list of products with more than 20% cloud cover
t.l2a.filter_clouds(cover_max = 50) # list of products with less than 50% cloud cover
Cloudmask and indice lists as CloudMaskList class objects can be reduced using 4 different functions, one for each type of cloud mask :
```python
>>> t.cloudmasks.cm001 # list cloudmask products of type CM001
>>> t.cloudmasks.cm002 # list cloudmask products of type CM002
>>> t.cloudmasks.cm003 # list cloudmask products of type CM003
>>> t.cloudmasks.cm004 # list cloudmask products of type CM004
```
.. code-block:: python
t.cloudmasks.cm001 # list cloudmask products of type CM001
t.cloudmasks.cm002 # list cloudmask products of type CM002
t.cloudmasks.cm003 # list cloudmask products of type CM003
t.cloudmasks.cm004 # list cloudmask products of type CM004
For cloudmasks of types CM003 and CM004 the _params_ function is used to select cloudmasks with specific parameters. See specific wiki page for cloudmasks implementation within Sen2Chain (**_to do_**). Default values are :
...
...
@@ -168,31 +184,31 @@ For cloudmasks of types CM003 and CM004 the _params_ function is used to select
- **cld_hi_prob**: _True_, used with CM004. Boolean value for masking clouds with high probability from 20m Sen2Chain Scene Classification.
- **thin_cir**: _True_, used with CM004. Boolean value for masking thin cirrus clouds from 20m Sen2Chain Scene Classification.
```python
>>> t.cloudmasks.cm003.params() # list cloudmask products of type CM003 with default values
>>> t.cloudmasks.cm003.params(probability = 25) # list cloudmask products of type CM003 with pixel cloud probability of 25%
>>> t.cloudmasks.cm004.params() # list cloudmask products of type CM004 with default values
>>> t.cloudmasks.cm004.params(cld_shad = False) # list cloudmask products of type CM004, without considering cloud shadows
```
.. code-block:: python
t.cloudmasks.cm003.params() # list cloudmask products of type CM003 with default values
t.cloudmasks.cm003.params(probability = 25) # list cloudmask products of type CM003 with pixel cloud probability of 25%
t.cloudmasks.cm004.params() # list cloudmask products of type CM004 with default values
t.cloudmasks.cm004.params(cld_shad = False) # list cloudmask products of type CM004, without considering cloud shadows
Indice lists can be filtered using one of the 3 defined functions: _raws_, _masks_, and _quicklooks_
```python
>>> t.ndvi.raws #list only non masked ndvi indices
>>> t.ndwigao.masks #list only masked ndwi Gao indices (by any mask)
>>> t.ndwigao.masks.cm001 #list only ndwi Gao indices masked by CM001 cloudmasks
>>> t.mndwi.quicklooks #list only mndwi indice quicklooks
```
.. code-block:: python
t.ndvi.raws #list only non masked ndvi indices
t.ndwigao.masks #list only masked ndwi Gao indices (by any mask)
t.ndwigao.masks.cm001 #list only ndwi Gao indices masked by CM001 cloudmasks
t.mndwi.quicklooks #list only mndwi indice quicklooks
## Cleaning corrupted products
Data in image database can sometimes be corrupted during their production, while downloading and unzipping l1c products, computing l2a with Sen2Cor (frequent corruptions with old 2.5.5 Sen2Cor version), cloudmasks or indices. The _clean_lib_ function has been introduced to check for data integrity, with the possibility to remove corrupted products (whole SAFE folder). Removed products have to be downloaded / computed again.
>>> t.clean_lib(remove = True) # Check and remove corrupted products
```
t.clean_lib(remove = True) # Check and remove corrupted products
## Archiving products
...
...
@@ -204,11 +220,11 @@ The _size_only_ option offers the possibility to get the info, whithout moving a
Every moved SAFE folder is replaced by its symlink in the original product folder. L2A SAFE products can be moved only if there is no file error (_clean_lib_ function automatically performed first).
