Skip to content

Changes

Page history
Miscelleaneous changement .. authored by gmorvan's avatar gmorvan
Show whitespace changes
Inline Side-by-side
Introduction.md
View page @ c2f449fc
...@@ -4,7 +4,6 @@ ...@@ -4,7 +4,6 @@
* [Usage](#introduction-usage) * [Usage](#introduction-usage)
* [How does it works](#introduction-usage-how-does-it-works) * [How does it works](#introduction-usage-how-does-it-works)
* [Getting Started](#introduction-usage-getting-started) * [Getting Started](#introduction-usage-getting-started)
* [Example](#introduction-usage-example)
* [Contents](#introduction-contents) * [Contents](#introduction-contents)
* [Models Version](#introduction-contents-models-version) * [Models Version](#introduction-contents-models-version)
* [Listing of Global inputs stored](#introduction-contents-listing-of-global-inputs-stored) * [Listing of Global inputs stored](#introduction-contents-listing-of-global-inputs-stored)
...@@ -20,23 +19,35 @@ ...@@ -20,23 +19,35 @@
#### How does it works #### How does it works
page d accueil ==> https://git.outils-is.ird.fr/legos_atnum/repos2 Home page ==> https://git.outils-is.ird.fr/legos_atnum/repos
![](images/tree3.png) ![](images/tree_repos.png)
First, each user have to download reference repository (NEMOREF) with the good tag associated to the configurations. Then, user type **git clone** command to download legos repo (NEMO_CFGS) and do synchronization ( **rsync** command) with reference repo. First, each user have to download reference repository (NEMOREF) with the good tag associated to the configurations. Then, user type **git clone** command to download legos repo (NEMO_CONFIGS) and do synchronization ( **rsync** command) with reference repo.
![](images/mermaid_repos3.png) ![](images/Mermaid_tree_bon.png)
Cloning a repository is nothing more than initializing an empty repository at home locally (_git init_) and bringing a remote branch repository back to it (_git fetch_) and merge it on your master branch. We will work on these new sources on the branch by default (master). It is possible to create branches and work on them. At any time, you can know where you are and if any changes have been made with typing your best friend _git status_
For user1 (Robert) :
First, if the first time on git, please modify your ~/.gitconfig file or create it:
```
git config --global user.name "Robert"
git config --global user.email "Robert@domain.fr"
```
<a name="introduction-usage-getting-started"></a> <a name="introduction-usage-getting-started"></a>
======================== ========================
#### Getting started #### Getting started
---
###### Connexion ssh ###### Connexion ssh
First of all, to retrieve the repository make sure that your public keys have been copied and pasted into the box (see picture below) First of all, to retrieve the repository make sure that your public keys have been copied and pasted into the box (see picture below)
If necessary, you will have to go to your local directory **~/.ssh/** and retrieve the *.pub file If necessary, you will have to go to your local directory **~/.ssh/** and retrieve the *.pub file
...@@ -48,95 +59,24 @@ ssh-keygen -t rsa -C "key for you" ...@@ -48,95 +59,24 @@ ssh-keygen -t rsa -C "key for you"
![](images/test_ssh.png) ![](images/test_ssh.png)
see on https://git.outils-is.ird.fr/profile/keys ===> see on https://git.outils-is.ird.fr/profile/keys
---
###### Connexion https (if you are working under proxy) ###### Connexion https (if you are working under proxy)
add this (due to legos proxy): add this (due to legos proxy):
```console ```console
git config --global http.https://git.outils-is.ird.fr.proxy http://proxy.legos.obs­-mip.fr:3128 git config --global http.https://git.outils-is.ird.fr.proxy http://proxy.legos.obs­-mip.fr:3128
``` ```
---
###### Instruction in details
###### get repos from your PC (example) For each configurations go to see tab and instructions in details [Configurations](https://git.outils-is.ird.fr/legos_atnum/repos/nemo_configs/wikis/Configurations):
Cloning a repository is nothing more than initializing an empty repository at home locally (_git init_) and bringing a remote branch repository back to it (_git fetch_) and merge it on your master branch. We will work on these new sources on the branch by default (master). It is possible to create branches and work on them. At any time, you can know where you are and if any changes have been made with _git status_
For user1 (Robert) :
First, if the first time on git, please modify your ~/.gitconfig file or create it:
```
git config --global user.name "Robert"
git config --global user.email "Robert@domain.fr"
```
Prepare your environment, for each configurations go to see tab and instructions in details [Configurations](https://git.outils-is.ird.fr/legos_atnum/repos2/nemo_cfgs/wikis/Configurations):
```
WRKDIR=$(pwd)
mkdir -p ${WRKDIR}/NEMOREF
```
###### Download official repo (with appropriate revision rXXXX):
```
cd ${WRKDIR}/NEMOREF
git clone https://username@git.outils-is.ird.fr/legos_atnum/repos2/nemoref.git dev_r11837_config_legos
cd $WRKDIR/NEMOREF/dev_r11837_config_legos
git checkout dev_r11837_config_nemo_legos
```
###### Download legos repos... :
```
cd ${WRKDIR}
git clone https://username@git.outils-is.ird.fr/legos_atnum/repos2/nemo_cfgs.git NEMO_CFGS
```
###### Choose configuration experiment :
User have to go to NEMO repos and create a new branch and work on it
```
>cd ${WRKDIR}/NEMO_CFGS/
>git checkout -b Myconfig
Switched to a new branch 'Myconfig'
>git branch
* Myconfig
master
>git status
>git checkout Myconfig-CASE
```
Then create new nemo directory...:
```
rsync -av ${WRKDIR}/NEMO_CFGS/models/NEMO/* ${WRKDIR}/NEMOREF/dev_rXXXX_config_legos/.
mv ${WRKDIR}/NEMOREF/dev_rXXXX_config_legos/ ${WRKDIR}/NEMO_CFGS/models/NEMO/.
```
You will have 2 directories now
* NEMO_CFGS/models/NEMO/ Which have all our contributions and official code
* NEMO_CFGS/setup/ Which give to user tools or specific files for running the configuration
###### Download tools repos... :
```
cd ${WRKDIR}
git clone https://username@git.outils-is.ird.fr/legos_atnum/repos2/tools.git TOOLS
git checkout TAG
```
<a name="introduction-usage-example"></a>
========================
#### Example
If you are not friendly with the repos, the user can proceed in this way to search for a configuration on the gitlab server: If you are not friendly with the repos, the user can proceed in this way to search for a configuration on the gitlab server:
* Search config **TATL025BIO** by keywords in the project wiki * Search config **TATL025BIO** on sidebar (Configurations) in the project wiki
* In the Wiki, click on Configurations/TATL025BIO (context + description + command line to do) : * In the Wiki, click on Configurations/TATL025BIO (context + description + command line to do) :
* Preliminary information (tests already performed on which cluster? information architecture/makefile files associated) * Preliminary information (tests already performed on which cluster? information architecture/makefile files associated)
* Help to retrieve the official ocean model repository + config. Legos: it give associated revision number and/or tag * Help to retrieve the official ocean model repository + config. Legos: it give associated revision number and/or tag
...@@ -145,6 +85,7 @@ If you are not friendly with the repos, the user can proceed in this way to sear ...@@ -145,6 +85,7 @@ If you are not friendly with the repos, the user can proceed in this way to sear
* Module to be loaded on computer, where are the input data (large file)? * Module to be loaded on computer, where are the input data (large file)?
<a name="introduction-contents"></a> <a name="introduction-contents"></a>
=============================== ===============================
## Contents ## Contents
...@@ -156,11 +97,11 @@ If you are not friendly with the repos, the user can proceed in this way to sear ...@@ -156,11 +97,11 @@ If you are not friendly with the repos, the user can proceed in this way to sear
* _NEMO_ (“Nucleus for European Modelling of the Ocean”) is a framework of ocean-related engines. Within NEMO, the ocean engine is interfaced with a sea-ice model (SI3), passive tracers and biogeochemical models (TOP) and, via the OASIS coupler, with several atmospheric general circulation models. It also supports two-way grid embedding by means of the AGRIF software. For more information, please refer to the [nemo website](#https://www.nemo-ocean.eu/). Since January 2019, The version of Nemo passed to 4.0. * _NEMO_ (“Nucleus for European Modelling of the Ocean”) is a framework of ocean-related engines. Within NEMO, the ocean engine is interfaced with a sea-ice model (SI3), passive tracers and biogeochemical models (TOP) and, via the OASIS coupler, with several atmospheric general circulation models. It also supports two-way grid embedding by means of the AGRIF software. For more information, please refer to the [nemo website](#https://www.nemo-ocean.eu/). Since January 2019, The version of Nemo passed to 4.0.
* _XIOS_ a été développé par LSCE- IPSL, il permet de prendre en charge les écritures des entrées/sorties des simulations (bibliotheques xml). Son utilisation permet donc de libérer des processus initialement alloué au modèle Nemo pour realiser cette tache d'écriture. Si cette outil est utilisé comme serveur (processus distincts alloué pour écriture (detached mode)), il permet de travailler plus facilement sur les paramètres de configuration car il n'y a pas besoin de recompiler à chaque fois. Il permet aussi de créér de nouveaux fichiers de sortie, de nouvelles variables issus de celles déjà référencés ainsi que d'y appliquer différents filtres temporelle. * _XIOS_ has been developed by LSCE-IPSL, it supports I/O writing of the simulations (use of xml libraries), the simulation of the model is simulated apart on different processes. Its use thus makes it possible to better manage the tuning of different number of I/O processes. If this tool is used as a server (separate processes allocated for writing (detached mode)), it makes it easier to work on the configuration parameters because there is no need to recompile each time. It also allows you to create new output files, new variables from those already referenced and to apply different time filters to them.
* _OASIS_ est un outil développé par le Cerfacs permettant l'échange de paramètre online entre 2 modèles, très utilisé notamment dans le couplage Océan/Atmosphère ... * _OASIS_ is a tool developed by Cerfacs allow the online exchange of parameters between 2 models (or even more), it is widely used in particular in the Ocean/Atmosphere coupling configurations...
* _WRF_ modèle atmosphérique à aire limitée (développé par le NCAR) * _WRF_ is a LAM (limited area model) developed by the NCAR, the atmospheric research american center)
<a name="introduction-contents-listing-of-global-inputs-stored"></a> <a name="introduction-contents-listing-of-global-inputs-stored"></a>
...@@ -173,66 +114,62 @@ If you are not friendly with the repos, the user can proceed in this way to sear ...@@ -173,66 +114,62 @@ If you are not friendly with the repos, the user can proceed in this way to sear
<a name="introduction-contents-repository-description"></a> <a name="introduction-contents-repository-description"></a>
======================== ========================
#### Repos description (after downloads instructions done)
#### Repos description (after downloads instructions done)
###### NEMO_CFGS directory ---
###### NEMO_CONFIGS directory
All these files are versioned for a configuration given here _CONFIG_CASE_: All these files are versioned for a configuration given here _CONFIG_CASE_:
1. models/NEMO : Fichiers Sources, contributions LEGOS+depot officiel sous **1**. models/NEMO : Source files, LEGOS contributions and official NEMO repository, as well as tools containing NEMO preprocessing tools
NEMO, ainsi que tools contenant les outils NEMO de preprocessing Fichiers Sources, contributions LEGOS+depot officiel sous
* NEMO_CFGS/models/NEMO/dev_rXXXX_config_legos/cfgs/CONFIG_CASE/MY_SRC/(*.F90) * NEMO_CONFIGS/models/NEMO/dev_rXXXX_config_legos/cfgs/CONFIG_CASE/MY_SRC/(*.F90)
EXPREF/(namelist*/xml files) EXPREF/(namelist*/xml files)
cpp_CONFIG_CASE.fcm (cle cpp) cpp_CONFIG_CASE.fcm (cpp key)
* NEMO_CFGS/models/NEMO/dev_rXXXX_config_legos/tools/DOMAINcfg/ * NEMO_CONFIGS/models/NEMO/dev_rXXXX_config_legos/tools/DOMAINcfg/
REBUILD_NEMO/ REBUILD_NEMO/
WEIGHTS/ WEIGHTS/
> **Note** : On garde cette arborescence , on pourrait imaginer des > **Note** : We keep this tree structure, we could imagine source modified from other models such as WRF, for some coupled configuration with NEMO for example. In this case, We will have: NEMO_CONFIGS/models/WRFSRC/... see NEMO_CONFIGS/models/SDAP mirrored... see new repo SDAP ?
modifications de sources sur d'autres modèles tel WRF (pas les sources officiels) pour certaine
config couplé avec NEMO par exemple. On aura:
NEMO_CFGS/models/WRFSRC/... voir NEMO_CFGS/models/SDAP en mirroir... voire new repo SDAP ?
2. setup : fichier de conf includefile.ini contenant l'ensemble des
variables d'environnement correspondant a la config CONFIG_CASE ainsi
que les scripts de lancements ayant servi pour l'execution du modele sur
le cluster.
* NEMO_CFGS/setup/CONFIG_CASE/conf/includefile.ini **2**. setup : configuration file __includefile.ini__ contains set of environment variables corresponding to the CONFIG_CASE config as well as the jobs scripts used to execute the model on a given cluster
* NEMO_CONFIGS/setup/CONFIG_CASE/conf/includefile.ini
scripts/run_nemo_tpl.ksh scripts/run_nemo_tpl.ksh
save_nemo_tpl.ksh save_nemo_tpl.ksh
modif_bathy.py modif_bathy.py
Si l'utilisateur le souhaite il peut faire appel à des outils de preprocessing disponible dans le projet TOOLS/ User can use preprocessing tools available in the NEMO_TOOLS/ project.
Les scripts présents sur ce dépôt doivent etre generique et normalement s'appliquer sur les differentes configs à venir, ils sont initialisés avec les variables contenus dans le fichier de configuration **includefile.ini** The scripts present on this repository must be generic and normally apply to the different future configs, thee scripts are initialized with the variables contained in the configuration file **includefile.ini**.
###### TOOLS repos ---
Les scripts et outils ###### NEMO_TOOLS repos
Scripts and tools for NEMO:
* TOOLS/preprocessing/NEMO/build_NEMO/build_nemo_main.ksh * NEMO_TOOLS/preprocessing/build_NEMO/build_nemo_main.ksh
to_compile_nemo.sh to_compile_nemo.sh
prepa_rundir_nemo.ksh prepa_rundir_nemo.ksh
build_tools_nemo.ksh build_tools_nemo.ksh
* TOOLS/preprocessing/NEMO/build_input/(jobs|python scripts) * NEMO_TOOLS/preprocessing/setup_inputs_NEMO/(jobs|python scripts)
* TOOLS/preprocessing/XIOS/build_XIOS/to_compile_xios.sh * NEMO_TOOLS/preprocessing/build_XIOS/to_compile_xios.sh
* NEMO_TOOLS/preprocessing/setup_environment/setup_directories.sh
---
===============================
## References ## References
---
===============================
## About ## About