Skip to content

Changes

Page history
new repos2 authored by gmorvan's avatar gmorvan
Show whitespace changes
Inline Side-by-side
Introduction.md 0 → 100644
View page @ 00bc9b05
* [Introduction](#Introduction)
* [Usage](#introduction-usage)
* [How does it works](#introduction-usage-how-does-it-works)
* [Getting Started](#introduction-usage-getting-started)
* [Example](#introduction-usage-example)
* [Contents](#introduction-contents)
* [Models Version](#introduction-contents-models-version)
* [Listing of Global inputs stored](#introduction-contents-listing-of-global-inputs-stored)
* [Repository description](#introduction-contents-repository-description)
* [Configurations](#introduction-configurations)
* [Domains](#introduction-configurations-domains)
* [Listing](#introduction-configurations-listing)
<a name="introduction-usage"></a>
===============================
## Usage
<a name="introduction-usage-how-does-it-works"></a>
========================
#### How does it works
First, We get here mirror repository using a Git clone as an intermediary. As follow, Maintainer get pull from the source repository (official CROCO gitlab or NEMO svn repository) and then git push to the GitLab Legos repository.
Each user can retrieve (via the git clone command) a project from the repos (NEMO_CFGS/CROCO_CFGS/TOOLS) on his working environment. The environment here is partitioned, i.e. the user cannot write on the LEGOS repos, nor modify the official repository of the model, as he doesn't have developper right access.
> **Note**: It is important to remember that these configurations have been tested and validated on a given cluster, the user can of course test them on another environment, but the documentation is not planned.
![](images/mermaid-diagram.png)
<a name="introduction-usage-getting-started"></a>
========================
#### Getting started
page d accueil ==> https://git.outils-is.ird.fr/legos_atnum/repos
![](images/tree2.png)
###### Connexion ssh
First of all, to retrieve the repository make sure that your public keys have been copied and pasted into the box (see picture below)
If necessary, you will have to go to your local directory **~/.ssh/** and retrieve the *.pub file
If not present type on your pc:
```console
ssh-keygen -t rsa -C "key for you"
```
![](images/test_ssh.png)
see on https://git.outils-is.ird.fr/profile/keys
###### Connexion https (if you are working under proxy)
add this (due to legos proxy):
```console
git config --global http.https://git.outils-is.ird.fr.proxy http://proxy.legos.obs­-mip.fr:3128
```
###### get repos from your PC (example)
Cloning a repository is nothing more than initializing an empty repository at home locally (_git init_) and bringing a remote branch repository back to it (_git fetch_) and merge it on your master branch. We will work on these new sources on the branch by default (master). It is possible to create branches and work on them. At any time, you can know where you are and if any changes have been made with _git status_
For user2 (Robert) :
First, if the first time on git, please modify your ~/.gitconfig file or create it:
```
git config --global user.name "Robert"
git config --global user.email "Robert@domain.fr"
```
Prepare your environment, for each configurations go to see tab and instructions in details [Configurations](https://git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/setup/wikis/Configurations):
```
WRKDIR=$(pwd)
mkdir -p ${WRKDIR}/NEMO_CFGS/models/NEMO
mkdir -p ${WRKDIR}/NEMO_CFGS/setup
```
Then clone repos...:
```
>cd ${WRKDIR}/NEMO_CFGS/models/NEMO
>git clone https://username@git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/models/nemosrc.git dev_XXXXX_config_legos
Cloning into 'dev_XXXXX_config_legos'...
>cd ${WRKDIR}/NEMO_CFGS/setup
>git clone https://username@git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/setup.git CONFIG_CASE
Cloning into 'CONFIG_CASE'...
```
You will have 2 directories now
* NEMO_CFGS/models/NEMO/dev_XXXXX_config_legos which have all the code of Official NEMO and our contributions
* NEMO_CFGS/setup/CONFIG_CASE Which give to user tools or specific files for running the configuration
###### Go to NEMO repos and create a new branch and work on it
```
>cd ${WRKDIR}/NEMO_CFGS/models/NEMO/dev_XXXXX_config_legos
>git checkout -b Myconfig
Switched to a new branch 'Myconfig'
>git branch
* Myconfig
master
>git status
>git checkout Myconfig-CASE
```
###### Only for developers with access permissions on gitlab repos!
```
>cd ${WRKDIR}/NEMO_CFGS/models/NEMO
>git remote -v
origin https://git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/models/nemosrc.git (fetch)
origin https://git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/models/nemosrc.git (push)
>git status
# On branch Myconfig
nothing to commit, working directory clean
>git push -u origin Myconfig
```
<a name="introduction-usage-example"></a>
========================
#### Example
If you are not friendly with the repos, the user can proceed in this way to search for a configuration on the gitlab server:
* Search config **TATL025BIO** by keywords in the project wiki
* In the Wiki, click on Configurations/TATL025BIO (context + description + command line to do) :
* Preliminary information (tests already performed on which cluster? information architecture/makefile files associated)
* Help to retrieve the official ocean model repository + config. Legos: it give associated revision number and/or tag
* Help to retrieve auxiliary models (XIOS/OASIS/WRF ...)
* Which pre-processing tools are associated with configuration?
* Module to be loaded on computer, where are the input data (large file)?
<a name="introduction-contents"></a>
===============================
## Contents
<a name="introduction-contents-models-version"></a>
========================
#### Models version
* _NEMO_ (“Nucleus for European Modelling of the Ocean”) is a framework of ocean-related engines. Within NEMO, the ocean engine is interfaced with a sea-ice model (SI3), passive tracers and biogeochemical models (TOP) and, via the OASIS coupler, with several atmospheric general circulation models. It also supports two-way grid embedding by means of the AGRIF software. For more information, please refer to the [nemo website](#https://www.nemo-ocean.eu/). Since January 2019, The version of Nemo passed to 4.0.
* _XIOS_ a été développé par LSCE- IPSL, il permet de prendre en charge les écritures des entrées/sorties des simulations (bibliotheques xml). Son utilisation permet donc de libérer des processus initialement alloué au modèle Nemo pour realiser cette tache d'écriture. Si cette outil est utilisé comme serveur (processus distincts alloué pour écriture (detached mode)), il permet de travailler plus facilement sur les paramètres de configuration car il n'y a pas besoin de recompiler à chaque fois. Il permet aussi de créér de nouveaux fichiers de sortie, de nouvelles variables issus de celles déjà référencés ainsi que d'y appliquer différents filtres temporelle.
* _OASIS_ est un outil développé par le Cerfacs permettant l'échange de paramètre online entre 2 modèles, très utilisé notamment dans le couplage Océan/Atmosphère ...
* _WRF_ modèle atmosphérique à aire limitée (développé par le NCAR)
<a name="introduction-contents-listing-of-global-inputs-stored"></a>
========================
#### Listing of global inputs stored
* MERCATOR GLORYS2V4 (1993 to 2017)
* DFS5.2 (Drakkar Forcing Set) : Reanalysis product (Mix ERA5/ ERAinterim and correction from 1958 to 2017)
<a name="introduction-contents-repository-description"></a>
========================
#### Repos description (after downloads instructions done)
###### NEMO_CFGS directory
All these files are versioned for a configuration given here _CONFIG_CASE_
There are 2 repos **NEMO/** and **setup/**:
1. NEMO : Fichiers Sources, contributions LEGOS+depot officiel sous
NEMO, ainsi que tools contenant les outils NEMO de preprocessing
* NEMO_CFGS/models/NEMO/dev_rXXXX_config_legos/cfgs/CONFIG_CASE/MY_SRC/(*.F90)
EXPREF/(namelist*/xml files)
cpp_CONFIG_CASE.fcm (cle cpp)
* NEMO_CFGS/models/NEMO/dev_rXXXX_config_legos/tools/DOMAINcfg/
REBUILD_NEMO/
WEIGHTS/
> **Note** : On garde cette arborescence , on pourrait imaginer des
modifications de sources sur d'autres modèles tel WRF (pas les sources officiels) pour certaine
config couplé avec NEMO par exemple. On aura:
NEMO_CFGS/models/WRFSRC/... voir NEMO_CFGS/models/SDAP en mirroir...
2. setup : fichier de conf includefile.ini contenant l'ensemble des
variables d'environnement correspondant a la config CONFIG_CASE ainsi
que les scripts de lancements ayant servi pour l'execution du modele sur
le cluster.
* NEMO_CFGS/setup/CONFIG_CASE/conf/includefile.ini
scripts/run_nemo_tpl.ksh
save_nemo_tpl.ksh
modif_bathy.py
Si l'utilisateur le souhaite il peut faire appel à des outils de preprocessing disponible dans le projet TOOLS/
Les scripts présents sur ce dépôt doivent etre generique et normalement s'appliquer sur les differentes configs à venir, ils sont initialisés avec les variables contenus dans le fichier de configuration **includefile.ini**
###### TOOLS repos
Les scripts et outils
* TOOLS/preprocessing/NEMO/build_NEMO/build_nemo_main.ksh
to_compile_nemo.sh
prepa_rundir_nemo.ksh
build_tools_nemo.ksh
* TOOLS/preprocessing/NEMO/build_input/(jobs|python scripts)
* TOOLS/preprocessing/XIOS/build_XIOS/to_compile_xios.sh
<a name="introduction-configurations"></a>
===============================
## Configurations
<a name="introduction-configurations-domains"></a>
========================
#### Domains
<img src="images/BlankMap-World-plus_configs.png" alt="Planets" usemap="#planetmap">
<a name="introduction-configurations-listing"></a>
========================
#### Listing
[Configurations](https://git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/setup/wikis/Configurations) are stored since December 2019. The table below shows the different characteristics of these. In the dedicated section, the configurations are explained, including the inputs used and the procedure to follow for the recovery of the repository and tools with their proper revision.
| Configurations | Name | Model | Res. | Lat<br>Lon</br> | ix<br>iy</br><br>iz</br> | Type | Tests on | Cluster | Cpus<br>/model</br> | Time elapse<br>/yr</br> |
|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|
| <span style='color:red'>[<span style='color:red'>TATL025BIO</span>](https://git.outils-is.ird.fr/legos_atnum/repos/nemo_cfgs/setup/wikis/Configurations/TATL025BIO)</span> | Tropical Atlantic | NEMO4.0.1 (rev.11837)<br>PISCESv2</br><br>XIOS2.5 (rev.1646)</br> | 0.25° | [-34,05;34,46N]<br>[-99;20E]</br> | 477<br>293</br><br>75</br> | Forced | 1993 <br>to 1997</br> | Irene (TGCC-cea) | 138<br>6</br> | 4h00
| To be continued ... | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓
===============================
## References
.. bibliography:: configurations.bib
:all:
:style: unsrt
:labelprefix: C
.. Links and substitutions
===============================
## About
BLABLA