NWPSAF 1D-Var v1.2 ---------------------------------------------------- The full documentation for this code can be found in docs/nwpsaf-mo-ud-032_NWPSAF_1DVar_Manual.html. Contents of this file: - Prerequisites - Updates for this release - Installing the 1D-Var - Other considerations for compiling - Considerations specific to the use of the Portland Fortran Compiler - Using an emissivity atlas - A note on the 1D-Var Sample Output - Installing the Observation Simulation Code - Installing the code to convert from BT to PC and/or vice versa - Retrieving surface emissivity Prerequisites ------------- This code requires RTTOV 11 or 12 to be downloaded and compiled. It is incorporated into the 1D-Var as a library. RTTOV is an NWPSAF deliverable and may be obtained from http://nwpsaf.eu/site/ You will also require RTTOV coefficient files for any instruments you wish to simulate. These can also be obtained at the above URL. As with RTTOV itself, you may require HDF or NetCDF libraries to use certain coefficient files or atlases. You should only need a NetCDF library if you are running RTTOV11 with the Emissivity Atlas and cannot use HDF5 for some reason. RTTOV12 does not support NetCDF. The 1D-Var does not require anything in addition to what is required for RTTOV itself. Updates for this release ------------------------ With this release the capability to retrieve surface emissivity has been added. The last paragraph in this file explains the technical details how to use this capability while further scientific aspects can be found in the user manual docs/nwpsaf-mo-ud-032_NWPSAF_1DVar_Manual.html. The docs/nwpsaf-mo-ud-031-NWPSAF_1DVar_ReleaseNote.pdf describes the changes to the 1D-Var package at this and previous releases. Installing the 1D-Var --------------------- 1) Compile the code - Go to the build directory - Edit the top section of the makefile to point to your installation of RTTOV, and change to your compiler of choice. Check the options for HDF etc are set correctly, and add the location of these libraries if you need to. - type make - if you need to run a clean build, you can type "make clean" first - compiled object files and the executable NWPSAF_1DVar should appear in the build directory - with many compilers you will get an unimportant warning message about "bad preprocessor" directive: #include "throw.h" - If you have compiled with EmissAtlas=0, you will get a warning about an unused variable in NWPSAF_RTTOVxx.Interface.f90 TYPE(rttov_emis_atlas_data) :: emis_atlas This is not an error and can be ignored. 2) Set up a directory from which to run the 1D-Var code: - cd into WorkDir (or make a copy of this directory if you prefer - the name of this directory is not important). - Link or copy in to this directory any RTTOV coefficients required. An example: ln -s /my_coef_path/rtcoef*.dat . Note that, unlike some other applications, the 1DVar constructs the RTTOV coefficient filename itself from the instrument triplet (series, platform number, instrument). This means that the file names of the coefficient files must be simply rtcoef_series_platform_instrument.dat (or .H5). When you link, make sure that you remove any non-standard parts of filenames. E.g. ln -s /my_coef_path/rtcoef_metop_2_iasi_pc_compat.H5 rtcoef_metop_2_iasi.H5 - Edit Run_1DVar_test.ksh with the location of HDF/NetCDF libraries, and the path to the top directory of your 1D-Var installation. - Make any other changes you want to make to the test script (such as to $OUTPUT_DIR). - Note that with ifort, you will need to use the command 'ulimit -s unlimited' in your running script before running the 1D-Var to avoid a memory fault. 3) Run test scripts to check the code works - The script is set up to run 54L profiles; you can change line 33 to LEVELS=43 if you want to check the use of profile interpolation as well. - See NWPSAF_vn1.1_Release_Info.txt for more information 4) Optionally: compare your results with the sample results provided - You can run a differencing tool to compare your results with those in directories Sample_Output_RTTOV11 and Sample_Output_RTTOV12 - See NWPSAF_vn1.1_Release_Info.txt for more information 5) Optionally: a test script and sample input and output files are provided for PC score runs as well (with sample output in the directories above) and example files for running in Radiance units rather than brightness temperatures. Other considerations for compiling ---------------------------------- If you change the code yourself, in particular the interfaces between subroutines, you will often find you need to run 'make clean' before recompiling. This is not new behaviour - the reason is that the makefile is basic, and includes only the bare minimum of dependencies to allow the code to compile. Considerations specific to the use of the Portland Fortran Compiler ------------------------------------------------------------------- An issue has been found with the Portlsnd Fortran Compiler, versions 16.10 and 18.7. The code compiles without problems, but at runtime the arrays profiles and profiles_k_pc in the routines NWPSAF_RTTOV11_Interface.f90 and NWPSAF_RTTOV12_Interface.f90 are not dimensioned correctly. This reason for this does not seem to be erroneous source code. To support users of the Portland Fortran Compiler, modified versions of the two interface routines have been added to the package which work fine with the Portland Fortran Compiler. These routines are NWPSAF_RTTOV11_Interface_change_for_pgf90.f90 and NWPSAF_RTTOV11_Interface_change_for_pgf90.f90, respectively Using an emissivity atlas: -------------------------- - In order to compile the default code to use the emissivity atlas, you will need to inform the compiler that you wish to do so by setting EmissAtlas = 1 This will switch on compilation of the code that reads in the emissivity atlas and passes it to the RTTOV call. - A library must also be linked in the makefile to enable the atlas to be read in. It is recommended that you use HDF where possible, but NetCDF can be used for RTTOV11 (not RTTOV12) if necessary. You will need to modify the top of the makefile to account for this. - The HDF/NetCDF options at the top of the makefile control the linking in of the required library. Different linker flags are required depending on whether RTTOV has been compiled with the NetCDF or HDF libraries. Note that NetCDF 4.2+ is supported by the supplied makefile. To use NetCDF 4.1 or older, you must modify the linker flags in the makefile. See the RTTOV/build/Makefile.local to see what you need to do. - When reading emissivities from an atlas an extra line needs to be added to each observation's entry in the observation file. This line goes between the line beginning "Obs ID:" and the one beginning "Latitude:". The information required on this line is the date of the observation in the form Year: ???? Month: ?? Day: ?? The code is capable of reading observation files with or without this extra line but the month is definitely required when using an emissivity atlas. This line has been added to some of the sample observation files. A note on 1D-Var Sample Output ------------------------------ - The output is marginally sensitive to compiler, but very sensitive to the choice of RTTOV coefficient files. This is not new behaviour. For this reason, the test script does not do a diff automatically. - Sample output is provided for gfortran 4.4.7 and for ifort 12.0, for both RTTOV11 and RTTOV12. - Note that the same test observations, background profiles (and, for the most part, namelist settings) have been used for this release as for previous releases, but the output will not be the same because of the sensitivity to RTTOV coefficient files. In particular, the test observations were simulated with an older version of RTTOV, and are becoming increasinly incompatible with the latest science. This means that with RTTOV12, with the latest interpolation option (interp_mode=5), the Sample Output for CrIS shows non-convergence of the 1D-Var. This behaviour in the sample output is normal, but the user should be cautious when using simulated observations with the 1D-Var that the simulations are done with settings that are consistent with the RT simulation in the 1D-Var itself (or that the differences are accounted for by increasing the observation error term). This is not new behaviour - it is a result of improvements in radiative transfer science. Installing the observation simulation code ------------------------------------------ 1) Set up the code to meet your needs - Edit the code src/sim_spec/sim_spec_rttov11.f90 or sim_spec_rttov12.f90 to include paths to coefficient files, profile files, error covariance files and output files. - Modify the options in the files to match your requirements (e.g. simulate radiances, add errors etc) - you may also want to modify the RTTOV options (though take care that your simluation settings match those that you use in the 1D-Var - see the note above on Sample Output) - Note that these routines were written to simulate IASI observations, so you may need to change some array sizes etc depending on what you are simulating 2) Compile - Go to build directory - type "make sim_spec_rttov11" or "make sim_spec_rttov12" - if you need to run a clean build, you can type "make clean_simspec" first 3) Run test scripts - in WorkDir, edit Run_SimSpec.ksh for your HDF library and executable name. - run Run_SimSpec.ksh 4) Example IASI output is provided in Sample_Obs/SimSpec_RTTOV11 and Sample_Obs/SimSpec_RTTOV12 (the PC output is used in the PC test script mentioned above) Installing the code to convert from BT to PC score and/or vice versa -------------------------------------------------------------------- This code is independent of RTTOV, though still requires RTTOV pccoef files to be read in. The format of these files has not changed between RTTOV-11 and RTTOV-12 so only one routine is provided. 1) Set up the code to meet your needs - Edit the code src/sim_spec/spec_to_pc.f90 or pc_to_spec.f90 to include paths to coefficient files, input observation files and output files. - Modify the options in the file pc_to_spec to match your requirements: you can choose whether to convert to radiance or brightness temperatures. You may also want to modify the PC-RTTOV options. - spec_to_pc reads the observation file header to determine the units of the input file and converts BTs to radiances first if necessary. - Note that these routines were written to convert IASI observations, so you may need to change some array sizes etc depending on what you are converting 2) Compile - Go to build directory - type "make spec_to_pc" or "make pc_to_spec" - if you need to run a clean build, you can type "make clean_simspec" first 3) Run test scripts - in WorkDir, edit Run_Convert.ksh for your HDF library and executable name. - run Run_Convert.ksh - this converts from BT/Rad to PC and back again (depending on input files and settings). 4) Example IASI output is provided in Sample_Obs/SimSpec_RTTOV11 and Sample_Obs/SimSpec_RTTOV12 Retrieving surface emissivity ----------------------------- Version 1.2 of the NWPSAF 1D-Var package adds the new capability to retrieve surface emissivity for the first time. The surface emissivity is represented by principal components in the retrieval state vector. This allows a compact representation of the surface emissivity and keeps the number of additional state vector elements low. The new capability is applied to IASI for up to 314 channels. See also the user manual for details. For the surface emissivity retrieval, two additional auxiliary files are required which can be found in the Sample_SurfEmiss/ directory One of these files, "EmisEigenVec" contains the principal components and the other an atlas specifying the weights (or scores) of these "EmisPCAtlas". The latter are a function of latitude, longitude and month, which are specified by the user in the "ObsFile.dat". All the user has to do to trigger a surface emissivity retrieval is to add a line in "Retrievals.NL". An example script to demonstrate the surface emissivity retrieval capability has been constructed: WorkDir/Run_1DVar_SurfEmiss.ksh. The simulated observation is contained in Sample_ObsFiles/ObsFile_IASI_SURFEMISS.dat and the quantities in the retrieval state vector to be retrieved (which include the surface emissivity principal components) are controlled by Sample_Namelists/Retrieval_IASI_SURFEMISS_54L.NL. The files specific to the test case in the IASI_COEFFS_DIR directory are ChannelChoice_surf_emiss.dat and Rmatrix_surf_emiss which are specifically designed for the 314 IASI channels involved. See also docs/nwpsaf-mo-ud-031_release_note.pdf for the test case. --------------------------- Met Office 25th March 2020