README_LOLA_RDR_SOFTWARE Software in this directory consists of several command-line executable programs that read Reduced Data Record (RDR) files from the Lunar Orbiter Laser Altimeter (LOLA). LOLA RDR files are binary table files generated from Experiment Data Record (EDR) files, to which calibrations and geometric location information have been applied. Each RDR has a detached human-readable .LBL file with the same root name. For more details regarding LOLA and data products, see the files in the DOCUMENTS directory. The programs take one input file, and generate output in various formats for use in data analysis. The ASCII comma delimited output format (rdr2csv) is provided for interoperability with spreadsheets and troubleshooting. The space-delimited output (rdr2table) is suitable for plotting individual range points, locations and other values vs time. The binary output (rdr2xyz) consists of selected longitude, latitude, and dependent-variable triples. Since the RDR tables have more columns than are generally used in any one instance, the "xyz" files, or point clouds can be restricted to a single output variable over a small region. The binary output format is for use with other packages (such as Generic Mapping Tools, http://gmt.soest.hawaii.edu) that can read raw floating-point numbers without the overhead of format conversion. To jumpstart users, we provide executables for several common OS platforms and architectures. *************** * Usage and I/O *************** Each program will give a usage message and exit if run with no arguments. The input RDR filename (complete path) is specified as the first argument on the command line, i.e., "rdr2tab LRO_NO_01/LOLARDR_092582152.DAT a f h u". If binary output is needed (rdr2xyz), the filepath is the second command line argument. Following these are alphanumeric options specified by the first character, i.e. "2 W" and "2much weird" will cause spot 2 returns to be output, including those flagged as weird. ASCII output goes to standard output and/or standard error, the default FORTRAN units 6 and 0. Binary output files have extensions filepath.[0-5].xyz. The BROWSE directory should be checked when a problem is encountered. For instance, if no data are returned, first check the topography and/or energy in the BROWSE directory. In some RDRs, the data may be limited to spacecraft position if lunar returns were not recorded, such as when the laser is off or the spacecraft is pointing away from the Moon. Lack of output may also result from limitations specified when executing the programs with option "L". ************** * Installation * Modification ************** Programs are run from a command line (shell) which must find the executable program file in its path (for example, the current directory). Optional supporting files, the JPL Navigation and Ancillary Information SPICE leap-second file "naif0009.tls", and a limits file if desired, defaulting to "r2x.prm", should be placed in the working directory. There are no other installation requirements. Modifications to the programs are encouraged, and source code for the software is provided in FORTRAN-77 syntax with some f90 extensions. (Some simple functions are coded in C for efficiency and to remedy the lack of 'UNSIGNED' in FORTRAN.) A 2009 or later version of the GNU gfortran/gcc compiler is recommended, since 'stream' input was only implemented recently in gfortran and results in faster handling of RDRs (see usage notes). To compile any program in this suite and link them to create the executable, see the make_rdr_software.sh script. The library "spicelib.a" must be in the path of the script when it executes for rdr2csv and rdr2tab. http://naif.jpl.nasa.gov/naif/toolkit.html will supply the correct version of "spicelib.a" for the target system. The spice libraries are 32 bit, which should be accounted for when compiling programs on a 64 bit system. ************* NOTE FOR Sun and PowerPC users especially *********** BIGENDIAN machines, like Sun, PowerPC: substitute kzext2_bigendian.c for kzext2.c in the compilation, and run programs using the 'Swap' option, or modify the source code to initialize the variable "swap" to .TRUE. ******************************************************************* For instance, to compile rdr2xyz: % gcc -O -c kzext2.c iswap4.c unsigned.c [creates kzext2.o, unsigned.o, iswap4.o] % gfortran -ffixed-line-length-none -O -o rdr2xyz rdr2xyz.f unsigned.o iswap4.o kzext2.o -static-libgfortran The executables have been created using the -O (optimize) flag. The -ffixed-line-length-none flag allows for lines longer than 72 characters. The -static-libgfortran flag creates a standalone executable, which allows it to run on machines that lack the run-time fortran library. ************* * RDR2CSV * ************* RDR2CSV writes the RDRs to lengthy Comma-Separated Values tables that may be imported to spreadsheets. To execute: To get a usage message, run the executable 'rdr2csv' with no other arguments. % rdr2csv rdr2csv 1.2 - Converting reduced data record to a comma-separated-values file usage: rdr2csv [[H|h](eader)] [[S|s](wap_bytes)] You will need the leap second file, e.g., naif0009.tls, in the current directory. The CSV table is written to standard output in engineering and scientific units. Since each table record has 68 columns and exceeds 720 characters, it is advisable to pipe the output to a unix filter such as 'more', or redirect the output to a file, e.g., % rdr2csv LOLARDR_100392325.DAT > RDR2CSVOUTPUTFILE.CSV. The detached PDS 3 format label (RDRCSV.LBL) references this output file and should be edited to reflect your filename - change the pointer in the header. Few spreadsheet programs can make use of ASCII csv files with 28 lines for each second of data, so the file may need to be split into quarters. Flags ----- H|h - writes a header line, which is particularly useful if you are not using the detached label file. Consult the label file for a more detailed description of the columns S|s - This swaps the byte order of the data so that data written as small-endian integers can be processed on a big endian processor, and vice versa. If compiled on bigendian machine, the function kzext2 must be changed to handle the swap that converts two-byte unsigned integers to int. (This is provided for legacy reasons; the swaps have already been done for PC-type architectures.) To compile: using gfortran -------------- % gfortran -ffixed-line-length-none -O -o rdr2csv rdr2csv.f unsigned.o kzext2.o iswap4.o spicelib.a using ifort ----------- ifort -132 -assume byterecl -O -o rdr2csv rdr2csv.f unsigned.o kzext2.o iswap4.o spicelib_ifort.a NOTE: To compile correctly, you will need the appropriate NAIF SPICE toolkit library (http://naif.jpl.nasa.gov/naif/toolkit.html), version N0063 recommended. This is only called in order to format UTC time strings using the "et2utc" subroutine. ************* * RDR2TAB * ************* Output a table line for each detector that triggers, according to options. The program rdr2tab implements limits as in rdr2xyz. (rdr2table uses stream I/O and has fewer options) rdr2tab 2.46 - LOLA reduced data record to table file NOT using stream i/o usage: rdr2tab [A 0 1 2 3 4 5][F|flag][G|geoid][H|header][L|limit] [O|origin][C|ckernel][D|dateline][M|manual][N|n][O|origin][Q|quarrel][U|utc] [S|swap_bytes][V|verbose][W|weird] 0 Spacecraft position, altitude, attitude, and laser pulse information. 1-5 Any combination of number(s) between 1 and 5 to select detector spots. A,a Include all detector spots, equivalent to 1 2 3 4 5 C,c check for C kernel attitude data missing F,f Ground return flag is checked, so noise is excluded G,g Geoid (selenoid) reference instead of IAU radius (1737.4 km) datum H,h Include a header row L,lfilename - use Limits in r2x.prm or filename containing lonmin,lonmax,latmin,latmax,tmin, tmax, offndr, zmax M,m Manual flag bit only test. Default is test M and N N,n Noise editor flag bit only test - a density, slope test O,o Origin is time in seconds from first shot in RDR file, else SC_MET Q,q Quarrels between manual and automatic noise editor output S,s Swap bytes from bigendian to little endian, and vice versa (legacy) U,u UTC DOY and time output V,v diagnostic output to unit 0 W,w Points flagged as 'weird' are output regardless of other flags Example: (pipe the output to a unix filter such as 'head', or redirect the output to a file) % rdr2tab LOLARDR_100392325.DAT o a f h| head -2 SCLK_LOLA alt_km id longitudeE latitudeN range_km energy noise thrs gain flg reflect pulsewd 20334315.010 -0.2004 1 289.937882 0.232839 42.610 0.1918 4 58.8052 50.2106 0 0.04276 21.60 The output table provides a time series starting nominally at the ascending equator crossing, for topography with calibrated values for instrument settings, pulse energy, and bidirectional reflectivity. Caution - the size of the table is typically large, up to a million lines. Typical use is to pipe columns into unix 'more' or 'awk' or a plotting program. Example: (create a plot of altitude vs. time colored by detector using Generic Mapping Tools psxy) % rdr2table LOLARDR_100392325.DAT a f | psxy -JX7/5 -R0/7000/-9/11 -Sc.01 -Clola.hsv -Bf60a600:seconds:/f.1a1:km:SWen >plot.ps produces a huge plot, with symbols colored as: #COLOR_MODEL = +HSV 1 0 1 0 2 210 1 1 2 240 1 1 3 240 1 1 3 120 1 1 4 120 1 1 4 30 1 1 5 30 1 1 5 0 1 1 6 0 1 1 To compile using gfortran: gfortran -ffixed-line-length-none -O rdr2table.f unsigned.o iswap4.o kzext2.o -static-libgfortran -o rdr2table ************* * RDR2XYZ * ************* RDR2XYZ provides a template for RDR binary output of LOLA channnels 1-5. After reading in an RDR data record, it can output spacecraft position, height, geoid, transmitted and received pulse width, energy, longitude, latitude, height, and energy noise. Most users will find this program useful for generating maps, or handling large quantities of data. It can select based on user-specified limits, is fast and has compact output. However, the binary output files are not "human-readable". You may wish to examine a particular RDR using RDR2CSV or RDR2TABLE to check results obtained by RDR2XYZ. To execute: rdr2xyz LOLARDR_100392325.DAT mydir/LOLARDR_100392325.DAT 3 4 Lmydir/r2x.prm creates mydir/LOLARDR_100392325.3.xyz and mydir/LOLARDR_100392325.4.xyz using the limits specified in file 'r2x.prm'. To get the argument list, run the executable 'rdr2xyz' with no input file. % rdr2xyz rdr2xyz 3.142-LOLA reduced data record to binary files usage: rdr2xyz [1-5] selected spots Invalid ranges are excluded by default (unless F) [A|a]ll 5 spots [B|b]i-directional reflectance [D|d]ateline (plus/minus 180 instead of 0-360) [E|e]xtend - pads with extra data beyond 0, 360 according to lonmin, lonmax [F|f]lag for ground is NOT checked [G|g]eoid reference instead of IAU radius (1737.4 km) datum [L|l]filename - default is r2x.prm, can be any file containing lonmin, lonmax, latmin, latmax, tmin, tmax, offndr, zmax [M,m]anual flag bit only test. Default is test M and N [N,n]oise editor flag bit only test - a density, slope test [P|p]ulsewidth in nanoseconds (full width) [R|r]ange to surface(slant range) [S|s]wap bytes if bigendian data on PC or vice versa [T|t]riggers are output if any flag bits are SET [X|x]transmit energy (mJ) on 0 [W|w]eird flags are considered ground Option Flags - some combination of 0, 1-5, or A is required to generate output files. ------------ 1-5 Use number(s) between 1 and 5 to select specific channels. A,a Include all spots B,b gives the target bidirectional reflectance at zero phase D,d dateline, output longitude as +/- 180 E,e extend - pads with extra data beyond 0, 360 according to lonmin, lonmax F,f The ground return flag is not checked, so noise is not excluded G,g To specify a reference geoid datum from the RDR, rather than the IAU radius of 1737.4 km L,l[filename] - input or use file containing geographical and observation limits (lonmin, lonmax, latmin, latmax, start time, stop time, min/max off-nadir angle, max solar incidence). (Default 'r2x.prm') P,p Output pulsewidth in nanoseconds at threshold R,r Output range to surface S,s Change from bigendian to little endian, and vice versa (legacy) T,t test for bad data - not yet implemented X,x Output transmit energy of laser, mJ W,w Weird (questionable) points are included outfileroot is the root of output file names, which will be name outfileroot.a.xyz, where a is 1-5, depending on which detector channel the file is written out for. The program uses the rootfile up to the first "." and adds the extension ".a.xyz", so the accompanying sample script (sample_script_xyz) provided passes in the entire RDR file name as the root, and the .DAT extension is removed by rdr2xyz. Once the xyz files are generated (output), they can be used to create map products, for instance, using the software of your choice, such as the Generic Mapping Tools (gmt.soest.hawaii.edu) or the MATLAB/IDL environments. ******* BUGS * ******* The programs do very little error checking. The input file must be a valid RDR data table, e.g., ending in '.DAT' or '.dat'. For example 'rdr2table LOLARDR_100392325.LBL a f' is not likely to return useful results. The message "SPICE(FILEOPENFAILED) Could not open naif0009.tls." will result if this file, or at least a link to it, is not in the working directory. The outfileroot passed to "rdr2xyz" must not begin with a dotted directory path like "../" Please report errors to Gregory Neumann (gregory.a.neumann@nasa.gov) and/or Kopal Jha (kopal.jha@sigmaspace.com)