$Header: /cvsroot/nco/nco/doc/TODO,v 1.123 2002/02/05 07:34:36 zender Exp $ 

******************************************************************************
NCO Wish list:
Numbers were assigned in chronological order
About 15 high priority items are shown first, then another ~100 items
******************************************************************************

ncap-specific items:
4. Fix memory leak in ncap_var_conform_dim() or delete routine completely after replacing it with ncap_var_stretch()	
7. Combine binary operations into binary_op_var_var and binary_op_var_att types to reduce number of functions
8. Cure final few bothersome compilation warnings, mainly in bison.simple.
Note that fixing these via hardcoded prototypes should allow for pure_parser
to be turned off, since this apparently changes the prototypes.
ncap.tab.c (warning with Linux):
/usr/lib/bison.simple:432: warning: implicit declaration of function `yylex'
This must be something like
extern int yylex(YYSTYPE *,prs_arg); /* Prototype for lexer */
but where?
ncap.tab.c (IRIX warnings caused by using IRIX cc rather than GNU gcc, so 
function prototypes are dumbed-down in bison.simple, and size should be
cast to (unsigned char)):
cc-1177 cc: WARNING File = /usr/freeware/share/bison.simple, Line = 334
  The indicated argument is incompatible with the corresponding formal parameter.
        __yy_memcpy ((char *)yyss, (char *)yyss1, size * sizeof (*yyssp));
cc-1177 cc: WARNING File = /usr/freeware/share/bison.simple, Line = 336
  The indicated argument is incompatible with the corresponding formal parameter.
        __yy_memcpy ((char *)yyvs, (char *)yyvs1, size * sizeof (*yyvsp));
9. Add outer loop over files so script may apply to each file
12. Ensure lexer does not perform un-necessary work during initial_scan (e.g., LHS casting...)
13. Test that LHS casting works regardless of value of LHS template. If true, remove value initialization block
14. If tally is superfluous in ncap then do not ever allocate it
15. Link to GSL functions, especially gsl_sf_* (special functions)
16. Get ncap build working with pure_parser turned off so lex/yacc builds work
17. Start in console mode if no script and no output file specified, e.g., nco> 
18. Do not presume math float functions (e.g., tanf) exist, use double functions when they do not

Highest priority NCO-wide items that You can help with:
41. Use getopt_long() from FSF (has alloca()-dependency problems) or Sittler (BSD-licensed version with no problems)
59. Add "averaged over dimension x from y to z" attribute to ncwa-processed variables
83. Always build operators to link dynamically to libnco.so (autoconf/autotool?)
95. Handle CCM "time since..." conversions in ncrcat (use Unidata udunits)?
97. Add x_op=average... attributes to averagers for ncar_csm
124. Use `install' program and mkdir -p
128. Ensure target 'dir' is made before *.d dependency files 
150. Fully automate RPM production, e.g., 'make rpm' should build rpms and upload to redhat.com
152. Change make or nco_dst.pl to add docs to /usr/doc in RPM
164. Spiff up NCO homepage web page (Logo?)
165. Create Makefile dependency to do 'make dir' if lib, obj, bin directories do not exist
168. Implement autoconf/automake build environment?
177. Make RPM production allowable by non-root, since building RPMs as 
sudo root screws up permissions in obj/bin directories of user
178. Make _FillValue behave same as missing_value
179. add netCDF to required packages list in RPM build
191. Create short man pages for each operator which point user to info pages or online manual

Remaining items of lesser priority:

28. Use var.nc_id as file ID from now on and rewrite old routines to do so
32. Implement packing with scale_factor and add_offset
50. Allow user-specified scalar weights to ncwa
51. Make getopt() a shared subroutine
53. Add -t wall clock timer option? (then again, users can always use, e.g., timex())
55. var_conform_dim() should be able to expand a (x,64,128) array into an (nx,64,128) array (maybe)
58. Add -o fl_out switch to supercede positional argument (GNU style)
60. Make something that can mask a field without having to average over a dimension
66. Make history attribute get appended, rather than overwritten, when in append mode
69. ncks option for file/field summary: var name list,var min,max,avg,mss_val,# mss_val,
70. ncxx with no options should do usg_prn() without error msg (like ncwa does)
75. Add warning to ncwa triggered by averaging over lat without weighting by gw with NCAR_CSM_FORMAT
79. Allow ncrename .$var_nm syntax to work on all operators (so variables not present cause warning, not error)
82. Make libnco namespace-safe by prefixing all library subroutines in nc_utl.c, pck.c with nco_, and those in csz.c with csz_
86. Extend stride/wraparound capability to all operators possible
88. ncks is slow on large files on Crays: ncks -H -v time,date,datesec /tmp/eaton/pcw101/h0009.nc will this be fixed when move beyond netcdf2.4.2 cray libs?
92. ncrcat -O -v base_time,time_offset arm.cdf arm.cdf foo.nc;ncks -H -C -h foo.nc | m gives bad time_offset on LINUX not SUNMP (problem with LINUX adding record coordinate to open file?)
94. ~/bin/sh/texi2html -monolithic nco.texi has started dumping core. adding -verbose switch fixes problem, though I've no idea why.
99. Allow ncatted to dump file in ncatted format
103. Implement trapezoidal rule option in ncra, ncea
104. Investigate/Document reliability of -A operator when appending from large rank variables to small rank files. Add error message when variables are same size but one has a degenerate dimension.
In general, adding large rank variable to small rank file does work, e.g.,
ncks -C -O -v one_dmn_rec_var in.nc foo.nc
ncks -C -A -v three_dmn_rec_var in.nc foo.nc
ncdump foo.nc
105. Add easy-to-read tabular display option for file contents for ncks
106. Add switch to keep degenerate dimensions with ncwa
109. Change copyright printing so order is NCO vrs, netCDF vrs, URL, Copyright
111. Investigate whether using ncwa to average files where variables of type 
NC_CHAR have a record dimension causes core dumps or other problems.
i.e., LSM variable timecom caused subtle ncra problems for a while.
113. Rethink normalization switch options in ncwa
114. Fix var_conform_dim() so that returned weight always has same size tally array as the template variable
115. When sum of denominator in ncwa (i.e., sum of weight) is zero, SIGFPE results 
Users should not be doing weighted averages if denominator sums to zero, but...
Desired fix here is a graceful exit instead of a core dump
121. Problem reported when mss_val = 0.0 in averagers. C and Fortran 
versions of var_add_real() etc. do not handle missing_value attributes in 
exactly the same way. C versions test both operands agains mss_val,
Fortran tests only op1. Since running averages are initialized to 0.0 before
entry, problems arise in C version when missing_value = 0.0. Only fix seems
to be checking whether tally == 0 before checking whether op2 == mss_val
123. Add HTTP protocol to fl_get() (in addition to DODS)
127. Make mss: signal retrieval from NCAR mass store
134. Problem on CRAY with ncra test #2, averaging float whose
missing value is double:
ncra -O -C -v rec_var_flt_mss_val_dbl in.nc foo.nc;ncks -H foo.nc
ncra -O -C -d time,0,1 -v rec_var_flt_mss_val_dbl in.nc foo.nc ; ncks -H foo.nc
Not too worried about this since it is Cray specific, 
Maybe 1.0e36d does not convert to 1.0e36f on Crays?
138. If fastest varying dimension of NC_CHAR variable is of size one,
then should not expect it to be NUL-terminated.
139. ncks does not correctly print coordinate indices in hyperslabs with wrapping
ncks -H -C -v one_dmn_rec_var -d time,9,6 ~/nco/data/in.nc
Must compute dmn_ss using WRP information for each dimension
140. ncks prints dimensional indices relative to disk values, but
variable index relative to hyperslab origin for all data types except
NC_CHAR. For NC_CHAR the variable index is also relative to the disk
value, not the hyperslab origin. 
141. NC_CHAR printing should include final dimension coordinates and
character ranges relative to disk. Change variable index-ranges to be
relative to hyperslab to be consistent with all other data types.
142. Implement some way of discriminating between NUL-terminated
strings and non-NUL-terminated characters arrays in ncks printouts.
Perhaps double quotes around former and single quotes around latter
143. Add -W or -q switch to turn off squelch most warning messages in ncra, ncrcat (especially warnings about non-monotonicity)
144. Add option(s) to select all N-D variables where N=0,1,2...
145. Talk to Brian and implement latest netCDF conventions, e.g., time_op
146. Allow proxies for coordinate variables so that, e.g., hyperslabs
may be specified for "date" even though "time" is the coordinate. 
Proxies must be one dimensional and monotonic, of course.
147. Use builtin mkstemp() so file is built in fast
directory rather than, say, NFS-mounted directory like /fs/cgd
148. Put in more useful error diagnostics for Schweitzer's data holes
These only appear in ncra/ncrcat because ncks treats same hyperslab as a wrap!
Must define a consistent convention here
cd ~/nco/data
ncks -O -C -d lon,10.0,80.0 -v time_lon in.nc foo.nc;ncks -H foo.nc
ncrcat -O -C -d lon,10.0,80.0 -v time_lon in.nc foo.nc;ncks -H foo.nc
149. Add ncks hyperslab tests to nco_tst to catch inconsistencies like #148
151. Figure out how to dynamically modify web pages so files with
changing version numbers can be directly linked
154. Add mirror capability to web pages
158. Add policy for hyperslabbing single level values to User's Guide. 
Multi-file operators with record coordinate
treat single point hyperslabs differently than single file operators 
or than any operators with single point cuts on non-record coordinates.
See explanatory notes in lmt_evl().
159. Add capability to handle superfluous intermediate files to ncra,
ncrcat when lmt_typ = dim_idx.
This would allow these multifile operators to handle hyperslabs where,
for example, every other file was not needed. This capability is already
implemented for lmt_typ = crd_val. This is an issue only when stride
is non-unity.
160. Add documentation of ncra, ncrcat superfluous file capabilities to nco.texi
162. Add GiNaC (Computer Algebra System) support to ncap www.ginac.de (maybe?)
163. Figure out way to make doc/TAG and doc/VERSION change
automagically. Currently must be updated manually unless distribute
with nco_dst.pl
166. Document 'make test' so people know it exists and use it
167. Improve/redesign test suite so it is more comprehensive and easier to add to
169. Merge TODO into sourceforge Task manager and bug manager as appropriate
170. rgn.sh appears to cause core dump over some ocean basins?
173. Add switch to ncra which allows user to remove degenerate time dimension from result
174. Setup ncra,ncea,ncwa to do standard deviations sdn with one call
176. Replace Bison, Flex components of ncap with smaller C++ parser
181. Split nc_utl.c into smaller files (wait until 1.3 becomes stable branch)
182. Add -L switch to print license terms
187. Are OpenMP critical regions sufficient for netCDF or must all other threads stop completely?
188. Make binary RPMS build with DODS option
189. Verify ncrcat multithreading works, produced "not a netCDF file error" once
190. Make typ_pck et al. use nc_type enum or something sane for default
192. ncdiff will sometimes subtract files of incommensurate sizes without complaint
ncks -O -v one_dmn_rec_var -d time,0,1 in.nc foo1.nc
ncks -O -v one_dmn_rec_var -d time,0,2 in.nc foo2.nc
ncdiff -O foo1.nc foo2.nc foo3.nc
This may make sense, e.g., for 1-D record variables of different lengths, but not for multidimensional variables
Former case should print warning, latter case should exit with error
193. Convert dimension sizes from type "long int" to type "size_t"?
194. ncwa core dump under very specific conditions: 
ncks -O -v DSTODXC,gw,date dstmch58_1998.nc bug.nc
ncwa -O -a time -w gw bug.nc foo.nc
Works fine without -w option
Works fine with -D 5 option!!!
Works fine when compiled without OPTS=D
Perhaps this is a gcc-2.96 bug?
195. Itanium builds
196. ncwa errors in nco_tst.sh tests 15--29 when built with HDF4 libraries
No ncwa problems when built with NETCDF2_ONLY with Unidata libraries
197. Construct HDF format in.hdf from in.cdl for testing HDF-enabled NCO:
ncks -O in.nc in.hdf 
Command executes but in.hdf is full of screwy values
198. HDF-enabled ncgen does not parse in.cdl
199. Internationalize (i18n) NCO with gettext()
201. Adopt new nc_type=undefined enum for nctype instantiation 
202. Make nco_tst.sh somehow summarize its own errors, e.g., "Total errors reported = 12" so
that regression testing is easier. Perhaps nco_tst.sh should be rewritten in Perl or Python 
to accomplish this.
203. Perform I/O timing tests with NCO 1.3.x vs NCO 1.2.x to see if netCDF3 interface really speeds up I/O
204. Update user's guide to reflect new netCDF 3.x API
206. Add example NCO script, such as dst.sh, to doc directory
207. Bug on Linux. Core dump converting datesec,date from NC_INT to NC_DOUBLE:
ncwa -D 5 -O -a time,lat,lon -w gw $DATA/dstccm34/dstccm34_1990.nc $DATA/dstccm34/dstccm34_1990_xy.nc
This is a subtle memory bug which disappears when -w gw removed,
or variable list is made more specific, or MALLOC_CHECK_=1 is used
209. Add rules to make MANIFEST and automaticlally update Sourceforge download area
210. Break nco.h into system-specific headers, csz.h, pck.h...
211. Replace -1 by NC_GLOBAL where appropriate, especially in ncatted.c
212. Ensure structures are always passed by reference (default is pass by value)
213. Implement "const" types in all applicable headers (may improve speed)
214. Disable threads > 1 on SGI temporarily because thread #0 is doing all the work, why?
215. var_conform_dim() does not abort when variables do not conform because dimension list of one is subset of other but ORDER of dimensions differs, e.g., a(lat,lev,lon) !~ b(lon,lev). Attempting this will return incorrect answers!
216. Use %z for printing size_t rather than casting to long (is %z portable?)
218. Fix all leaks etc. enumerated in insure.txt
219. Sourceforge FTP nco.tar.gz link appears broken
ftp://nco.sourceforge.net/pub/nco/nco.tar.gz
220. Modify network code in csz.c so gcc -ansi works (fix resolv.h structures) 
221. Add option to strip executables
************************************************************************
End NCO Wish list
************************************************************************

