Docs update and modifications.

docs/build/html/_sources/preprocessing/weather_cube.rst.txt

@@ -48,7 +48,7 @@ Downloaded data from the ERA-5 Land server is projected on a WGS 84 lattitude/lo
 The output dataset is the one used by the processing chain to run the models. This dataset can be create using this function:
 
 
-.. autofunction:: modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_nc
+.. autofunction:: modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel
 
 :math:`ET_0` calculation is done with this function:
 

docs/build/html/genindex.html

@@ -118,6 +118,10 @@
 
 <h2 id="E">E</h2>
 <table style="width: 100%" class="indextable genindextable"><tr>
+  <td style="width: 33%; vertical-align: top;"><ul>
+      <li><a href="preprocessing/weather_cube.html#modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel">era5Land_daily_to_yearly_pixel() (in module modspa_pixel.preprocessing.lib_era5_land_pixel)</a>
+</li>
+  </ul></td>
   <td style="width: 33%; vertical-align: top;"><ul>
       <li><a href="preprocessing/ndvi_cube.html#modspa_pixel.preprocessing.download_S2.extract_zip_archives">extract_zip_archives() (in module modspa_pixel.preprocessing.download_S2)</a>
 </li>

docs/build/html/preprocessing/weather_cube.html

@@ -57,6 +57,7 @@
 </ul>
 </li>
 <li class="toctree-l3"><a class="reference internal" href="#formatting-and-preprocessing-the-raw-data">Formatting and preprocessing the raw data</a><ul>
+<li class="toctree-l4"><a class="reference internal" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel"><code class="docutils literal notranslate"><span class="pre">era5Land_daily_to_yearly_pixel()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.calculate_ET0_pixel"><code class="docutils literal notranslate"><span class="pre">calculate_ET0_pixel()</span></code></a></li>
 <li class="toctree-l4"><a class="reference internal" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.combine_weather2netcdf"><code class="docutils literal notranslate"><span class="pre">combine_weather2netcdf()</span></code></a></li>
 </ul>
@@ -243,13 +244,75 @@ download operations have a low CPU demand.</p>
 <p>The raw dataset now has a precipitation variable and a <span class="math notranslate nohighlight">\(ET_0\)</span> variable.</p>
 <p>Downloaded data from the ERA-5 Land server is projected on a WGS 84 lattitude/longitude projection, at 0.1° resolution. This data then needs to be reprojected on the same grid as the NDVI data, which is usually at a much higher resolution (of the order of 10 meters). The reprojection and clipping is done by the <code class="docutils literal notranslate"><span class="pre">OTB</span></code> <a class="reference external" href="https://www.orfeo-toolbox.org/CookBook/Applications/app_Superimpose.html">**Superimpose**</a> application, which has the advantage of being very efficient for large datasets. However, it only handles <strong>Geotiff</strong> images and not <strong>netCDF4</strong> files. The low resolution dataset first has to be saved as Geotiff files (one per variable), reprojected and clipped with <code class="docutils literal notranslate"><span class="pre">otbcli_Superimpose</span></code> and then opened and converted to a netCDF4 file. This can take some time for large datasets.</p>
 <p>The output dataset is the one used by the processing chain to run the models. This dataset can be create using this function:</p>
+<dl class="py function">
+<dt class="sig sig-object py" id="modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel">
+<span class="sig-prename descclassname"><span class="pre">modspa_pixel.preprocessing.lib_era5_land_pixel.</span></span><span class="sig-name descname"><span class="pre">era5Land_daily_to_yearly_pixel</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">list_era5land_files</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">output_file</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">raw_S2_image_ref</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ndvi_path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">10</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">max_ram</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">8</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">remove</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">bool</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">True</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">str</span></span></span><a class="reference internal" href="../_modules/modspa_pixel/preprocessing/lib_era5_land_pixel.html#era5Land_daily_to_yearly_pixel"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel" title="Link to this definition"></a></dt>
+<dd><p>Calculate ET0 values from the ERA5 netcdf weather variables.
+Output netcdf contains the ET0 and precipitation values for
+each day in the selected time period and reprojected
+(reprojection run on two processors) on the same grid as the
+NDVI values.</p>
+<section id="id3">
+<h3>Arguments<a class="headerlink" href="#id3" title="Link to this heading"></a></h3>
+<ol class="arabic simple">
+<li><dl class="simple">
+<dt>list_era5land_files: <code class="docutils literal notranslate"><span class="pre">List[str]</span></code></dt><dd><p>list of netcdf files containing the necessary variables</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>output_file: <code class="docutils literal notranslate"><span class="pre">str</span></code></dt><dd><p>output file name without extension</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>raw_S2_image_ref: <code class="docutils literal notranslate"><span class="pre">str</span></code></dt><dd><p>raw Sentinel 2 image at right resolution for reprojection</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>ndvi_path: <code class="docutils literal notranslate"><span class="pre">str</span></code></dt><dd><p>path to ndvi dataset, used for attributes and coordinates</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>h: <code class="docutils literal notranslate"><span class="pre">float</span></code> <code class="docutils literal notranslate"><span class="pre">default</span> <span class="pre">=</span> <span class="pre">10</span></code></dt><dd><p>height of ERA5 wind measurements in meters</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>max_ram: <code class="docutils literal notranslate"><span class="pre">int</span></code> <code class="docutils literal notranslate"><span class="pre">default</span> <span class="pre">=</span> <span class="pre">8</span></code></dt><dd><p>max ram (in GiB) for reprojection and conversion. Two
+subprocesses are spawned for OTB, each receiviving 
+half of requested memory.</p>
+</dd>
+</dl>
+</li>
+<li><dl class="simple">
+<dt>remove: <code class="docutils literal notranslate"><span class="pre">bool</span></code> <code class="docutils literal notranslate"><span class="pre">default</span> <span class="pre">=</span> <span class="pre">True</span></code></dt><dd><p>weather to remove temporary files</p>
+</dd>
+</dl>
+</li>
+</ol>
+</section>
+<section id="id4">
+<h3>Returns<a class="headerlink" href="#id4" title="Link to this heading"></a></h3>
+<ol class="arabic simple">
+<li><dl class="simple">
+<dt>output_file_final: <code class="docutils literal notranslate"><span class="pre">str</span></code></dt><dd><p>path to <code class="docutils literal notranslate"><span class="pre">netCDF4</span></code> file containing precipitation and ET0 data</p>
+</dd>
+</dl>
+</li>
+</ol>
+</section>
+</dd></dl>
+
 <p><span class="math notranslate nohighlight">\(ET_0\)</span> calculation is done with this function:</p>
 <dl class="py function">
 <dt class="sig sig-object py" id="modspa_pixel.preprocessing.lib_era5_land_pixel.calculate_ET0_pixel">
 <span class="sig-prename descclassname"><span class="pre">modspa_pixel.preprocessing.lib_era5_land_pixel.</span></span><span class="sig-name descname"><span class="pre">calculate_ET0_pixel</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">pixel_dataset</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">Dataset</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lat</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">lon</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">h</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">float</span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">10</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">ndarray</span></span></span><a class="reference internal" href="../_modules/modspa_pixel/preprocessing/lib_era5_land_pixel.html#calculate_ET0_pixel"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.calculate_ET0_pixel" title="Link to this definition"></a></dt>
 <dd><p>Calculate ET0 over the year for a single pixel of the ERA5 weather dataset.</p>
-<section id="id3">
-<h3>Arguments<a class="headerlink" href="#id3" title="Link to this heading"></a></h3>
+<section id="id5">
+<h3>Arguments<a class="headerlink" href="#id5" title="Link to this heading"></a></h3>
 <ol class="arabic simple">
 <li><dl class="simple">
 <dt>pixel_dataset: <code class="docutils literal notranslate"><span class="pre">xr.Dataset</span></code></dt><dd><p>extracted dataset that contains all information for a single pixel</p>
@@ -273,8 +336,8 @@ download operations have a low CPU demand.</p>
 </li>
 </ol>
 </section>
-<section id="id4">
-<h3>Returns<a class="headerlink" href="#id4" title="Link to this heading"></a></h3>
+<section id="id6">
+<h3>Returns<a class="headerlink" href="#id6" title="Link to this heading"></a></h3>
 <ol class="arabic simple">
 <li><dl class="simple">
 <dt>ET0_values: <code class="docutils literal notranslate"><span class="pre">np.ndarray</span></code></dt><dd><p>numpy array containing the ET0 values for each day</p>
@@ -290,8 +353,8 @@ download operations have a low CPU demand.</p>
 <dt class="sig sig-object py" id="modspa_pixel.preprocessing.lib_era5_land_pixel.combine_weather2netcdf">
 <span class="sig-prename descclassname"><span class="pre">modspa_pixel.preprocessing.lib_era5_land_pixel.</span></span><span class="sig-name descname"><span class="pre">combine_weather2netcdf</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">rain_file</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ET0_tile</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">ndvi_path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">save_path</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">str</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">available_ram</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><span class="pre">int</span></span></em><span class="sig-paren">)</span> <span class="sig-return"><span class="sig-return-icon">&#x2192;</span> <span class="sig-return-typehint"><span class="pre">None</span></span></span><a class="reference internal" href="../_modules/modspa_pixel/preprocessing/lib_era5_land_pixel.html#combine_weather2netcdf"><span class="viewcode-link"><span class="pre">[source]</span></span></a><a class="headerlink" href="#modspa_pixel.preprocessing.lib_era5_land_pixel.combine_weather2netcdf" title="Link to this definition"></a></dt>
 <dd><p>Convert the Rain and ET0 geotiffs into a single weather netcdf dataset.</p>
-<section id="id5">
-<h3>Arguments<a class="headerlink" href="#id5" title="Link to this heading"></a></h3>
+<section id="id7">
+<h3>Arguments<a class="headerlink" href="#id7" title="Link to this heading"></a></h3>
 <ol class="arabic simple">
 <li><dl class="simple">
 <dt>rain_file: <code class="docutils literal notranslate"><span class="pre">str</span></code></dt><dd><p>path to Rain tif</p>
@@ -320,8 +383,8 @@ download operations have a low CPU demand.</p>
 </li>
 </ol>
 </section>
-<section id="id6">
-<h3>Returns<a class="headerlink" href="#id6" title="Link to this heading"></a></h3>
+<section id="id8">
+<h3>Returns<a class="headerlink" href="#id8" title="Link to this heading"></a></h3>
 <p><code class="docutils literal notranslate"><span class="pre">None</span></code></p>
 </section>
 </dd></dl>

docs/source/preprocessing/weather_cube.rst

@@ -48,7 +48,7 @@ Downloaded data from the ERA-5 Land server is projected on a WGS 84 lattitude/lo
 The output dataset is the one used by the processing chain to run the models. This dataset can be create using this function:
 
 
-.. autofunction:: modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_nc
+.. autofunction:: modspa_pixel.preprocessing.lib_era5_land_pixel.era5Land_daily_to_yearly_pixel
 
 :math:`ET_0` calculation is done with this function: