Gui interface for ADCP autoedit package ==================================================== "gautoedit", pronounced "gee! autoedit" ==================================================== See "autoedit.html" for a better description of the editing process. aREADME_details.txt : documentation for ADCP editing package, "gautoedit" (see also aREADME_basics.txt for a more basic introduction) (see also autoedit.html for a web-based introduction to gautoedit) --------- Jules Hummon; Dec 2000 (called aREADME.verbose) May 2002, consolidating and clarifying, written for refsm ref layer; tailored for "demo" gautoedit requires Matlab 5.3 or higher quick_adcp.prl requires perl 4.0 or higher aREADME_basics.txt contents: (I) Introduction (II) Directions (easy start) aREADME_details.txt contents: (I) description of "asetup.m" and "aflagit_setup.m" parameters (II) description of "agetmat.m" and "aplotit.m" (III) description of all autoedit programs ======================================================================== ======= (I) details of "asetup" "aflagit_setup" parameters =========== ======================================================================== The structure "config" contains information used by agetmat (to get the data) and aplotit (to plot the data). Each function starts with defaults for "config" which can be overridden by the structure passed in, and those values can be overridden by varargin: any field of "config" can be overridden by varargin {name,value} pairs. asetup.m: %set these parameters: they get stored in the structure called %"config", which gets passed to most engines % --> usually run all this from edit/ subdirectory % %start_dd = 240; % start decimal day %end_dd = 275; % beginning of last decimal day %ddrange = 1.5 % extract and display this time range %yearbase = 2000; % year base %comment = 'ka0007'; % comment %dbpath = '../adcpdb/'; % path to database (from here: INCLUDE "/") %dbname = 'a0007'; % dbname not including "???.blk" %reflayer_range = [4 12]; % should match what you did with smoothr %numbins = 40; % number of bins to extract from database %longitude_split = []; % specify a longitude not crossed % (eg. 0 if in the Pacific, 180 if % in the Atlantic). should not be % necessary to use this option %prefix = 'a_' % getmat prefixes %navfile = ''; % read a *_xy.mat file from adcpsect for % navigation - this overrides the navigation % created with getmat (which is the original % user buffer navigation). NOTE: if you use aflagit_setup: %This is a setup script to set the structure values used by aflagit.m % "pfc" is "profile flags config" % substructures are: % flag (the value of the bad data) % cfg (configuration information) % % You either have good data and want to automate the old-fashioned editing, or % you have lousy data and want to cavalierly eliminate anything of dubious % character. Here is a table with suggested settings for a "good" or "bad" % dataset. The defaults for this file will start out for "good" data so you % don't lose too much right at the start. % % %% field of structure "pfc" good dataset bad dataset %% (eg. Kaimimoana) (eg. NBPalmer) %% ----------------------- ------------ ----------- %% pfc.refl_startbin config.rl_range(1) config.rl_range(1) %% pfc.refl_endbin config.rl_range(2) config.rl_range(2) %% pfc.yearbase config.yearbase config.yearbase %% %% pfc.cfg.bigtarget_amp 25 25 %% pfc.cfg.bigtarget_offset 5 5 %% pfc.cfg.bigtarget_avg 2 2 %% pfc.cfg.pgcutoff 30 50 (data dependent) %% pfc.cfg.maxpgbin_fudge 0 3 (data dependent) %% pfc.cfg.badpgrefl 30 50 %% pfc.cfg.badpgrefl_nbins (maybe 70% of the number of bins in the %% reference layer? easier to code it: %% round(.7*(pfc.refl_endbin - pfc.refl_startbin)) %% pfc.cfg.trimtopbins 0 1 (data dependent) %% pfc.cfg.amp_cutoff_atbin1 0 130 (data dependent) %% pfc.cfg.ampdecrperbin 10 10 (data dependent) %% pfc.cfg.ecutoff 100 100 (data dependent) %% pfc.cfg.wire_ecutoff 70 70 (data dependent) %% pfc.cfg.wire_onstation 2 2 (data dependent) %% pfc.cfg.wire_binrange [1 20] [1 20](data dependent) %% pfc.cfg.jitter_cutoff 5 5 (data dependent) %% pfc.cfg.orphan_numbins 0 pfc.cfg.badpgrefl_nbins %% pfc.cfg.orphan_slide 0 3 (data dependent) % % ---------- initialize some useful information ---------- % pfc.refl_startbin = ... % ref layer should match what was % config.rl_range(1); % used in smoothr (for consistency) % pfc.refl_endbin = ... % (get it from ../nav/as_nav.tmp or % config.rl_range(2); % ( ../nav/adcpsect.cnt) % pfc.yearbase = config.yearbase; % for labelling % % % ---------- for the bottom ---------- % looking for a high amplitude signal which SHOULD be the bottom. % these values might have to be changed if we end up flagging % scattering layers. These values are good for the bottom being deeper than % about bin 5. If the bottom gets really shallow, try no smoothing % (pfc.cfg.bigtarget_avg = 0) and a small window (pfc.cfg.bigtarget_offset = 2) % and decrease the threshold (pfc.cfg.bigtarget_amp = 10) % %pfc.cfg.bigtarget_avg = 2; % A value of 1, 2, or 3 is for smoothing % % the vertical amplitude profile with a % % a running average of 3, 5, or 7 points % % respectively. A value of zero is no % % smoothing. %pfc.cfg.bigtarget_amp = 25; % an amplitude bump bigger than this is % % identified as the bottom (like % % AMP_THRESHOLD in the zap editing) %pfc.cfg.bigtarget_offset = 5; % calculate the value of the amplitude % % bump by comparing the value at a given % % bin to the value shallower and deeper % % by this number of bins. % % ---------- percent good ---------- % % percent good is used in three ways: % % % % (1) low percent good: eliminate all % % bins with pg less than % % pfc.cgf.pgcutoff % % % (2) ringing: identified by a % % near-surface maximum (caused by a % % decrease in PG near the surface, % % then PG returns to its normal value % % deeper down). We locate the % % shallowest bin where PG attains its % % maximum and chop of everything % % shallower than that. Because PG % % may hover near its maximum for % % several bins, we back off the % % strict requirement of "the % % shallowest bin with the maximum pg" % % and instead look for "the % % shallowest bin within % % pfc.cfg.maxpgbin_fudge of the the % % maximum" % % % % (3) reference layer: eliminate % % profiles if there are too few % % reference layer bins with % % sufficiently good PG. % % % % pfc.cfg.pgcutoff = 30; % (1) low percent good (LEAVE THIS) % pfc.cfg.maxpgbin_fudge = 3; % (2) ringing (allow this much slop % % in identifying the "maximum" PG % % (set to 0 to disable) % pfc.cfg.badpgrefl = 30; % (3) reference layer PG cutoff % pfc.cfg.badpgrefl_nbins = 6; % (nbins required to accept profile) % % (set both to 0 to disable) % % NOTE: should be a large fraction % % of the number of bins in the % % reference layer, say 6 for a % % reference layer with 9 bins % % ---------- trim the top ---------- % % There are two editing criteria in this % % section eliminate bad data at the top. % % % % (1) remove top N bins from all profiles % % % % (2) remove bins where low amplitude % % may be artificially inflating % % signal/noise ratio. At the % % moment this is a linear amplitude % % cutoff with bin. % % % % pfc.cfg.trimtopbins = 0; % (1) exclude top N bins anyway % %%%% DISABLE THE NEXT PARAMETERS %%%% % % disable both by setting % % pfc.cfg.amp_cutoff_atbin1 = 0 % pfc.cfg.amp_cutoff_atbin1 = 0; % (2) amplitude cutoff at bin 1. % pfc.cfg.ampdecrperbin = 10; % Decrease amplitude cutoff by % % this many amplitude counts ber bin % % ---------- error velocity ---------- % % error velocity is used in two ways: % % (1) plain old error velocity cutoff % % % % (2) wire interference: different % % criterion (?), only on station, % % remove an additional bin left, % % right, up, and down from flagged % % bins (disable by setting cutoff to % % a large value (eg 10^6)) % % pfc.cfg.ecutoff = 100; % (1) plain old error velocity cutoff % pfc.cfg.wire_ecutoff = 70; % (2) wire interference cutoff % pfc.cfg.onstation = 2; % (only look when speed is less) % pfc.cfg.binrange = [1 20]; % (only look in this bin range) % % % % ---------- jitter flag ---------- % flag data where the measured data jitter % % is too high: "jitter" is a running % % 5-point std of the difference between % % measured velocity using the original % % navigation and smoothed navigation. % % % pfc.cfg.jitter_cutoff = 5; % flag profiles as bad if jitter % % exceeds this value in cm/s % % % ---------- flag isolated profiles ---------- % % % If the data are mostly bad, we want % % to eliminate isolated profiles as % % suspect. We do this by requiring a % % profile to have a few good close % % friends. % % % NOTE: to disable, must set both of these % % to zero % % % pfc.cfg.orphan_numbins = 0; % number of unflagged bins in ref layer % % required to accept the profile % % (before looking for neighbors) % % (to disable, set to 0) % % (for good data, try 70% of the number % % of bins in the reference layer % % pfc.cfg.orphan_slide = 0 ; % half-width of isolation profile % % i.e. slide left and right by this, % % and accept the center of a run of 3 % % or middle 3 if a run of 5 % % (for really bad data, set to 2) % % for moderately bad data, try 1 % % % the values of the flags are currently: % pfc.flag.pgcutoff = 2^0; % (written to abadbin.asc) % pfc.flag.trimtopbins = 2^1; % (written to abadbin.asc) % pfc.flag.badpgrefl = 2^2; % (written to badprf.asc) % pfc.flag.maxampbin = 2^3; % (written to bottom.asc) % pfc.flag.bottom = 2^4; % (only used in graphics as a mask) % pfc.flag.ecutoff = 2^5; % (written to abadbin.asc) % pfc.flag.orphan = 2^6; % (written to abadprf.asc) % pfc.flag.wire = 2^7; % (written to abadbin.asc) Notes on nav steps for the Palmer: % 1) should probably use something like 15-25 for ref layer % 2) smoothr: % filter_hwidth= .0104 /* 15 min: less smoothing than 0.0208 */ % min_filter_fraction= .75 /* more stringent than 0.5 */ ======================================================================== ======= (II) details of "agetmat" "aplotit" parameters =========== ======================================================================== The program which extracts data from the codas database is "agetmat". It can also be used to simply load the data which has been previously extracted by agetmat (For example, on the command line, during a gautoedit session, you can use the following to load the values into the structure "data": [data,config] = agetmat('a_') The program which makes the panel plots is "aplotit.m" and can be used to make your own panel plots if desired (see below) agetmat: % [data, cfg] = agetmat(config, varargin) % two modes exist: % (1) set up configuration to call getmat and do so % usage: [data, cfg] = agetmat(config, name1, val1, name2, val2) % (2) just read the output of a previous getmat call (eg during % shipboard adcp editing). % usage: [data, cfg] = agetmat; % % % --- mode: run getmat ---- % config has lots of stuff in it. % varargin is name,value pairs which can override the values in config % data is getmat data; cfg is titles, clips, fields % % required fields in config are: % start_dd % ddrange % yearbase % comment % dbpath % dbname % % usage: % [data, cfg] = agetmat(config, name1, val1, name2, val2,...) % % --- mode: just load it --- % suitable for loading getmat data which were generated by shipboard editing % data comes out the same; cfg gets the following: % % usage: % [data, config] = agetmat; % % NOTE: % units of diagnostics are all directly from getmat % abs water speeds are cm/s, ship and relative water speeds are m/s % e and w are X1000 % % --------------------- agetmat: additional details -------------------- % if only loading, % comment = ''; % rungetmat = 0; %generally want to run getmat % os_cutoff = 2; %on-station is under 2m/s % rl_range = [5 12]; %reference layer range % start_dd %from DAYS (loaded) % ddrange %from DAYS (loaded) % yearbase = -9999; comment = ''; dbname = ''; dbpath = ''; % % % % optional fields in config default to: % % nbins = 128; %number of bins to extract using getmat % cleanerparams = [200,1]; %start with [2,3] for more cleaner % comment = ''; % no additional comment % rungetmat = 1; %generally want to run getmat % os_cutoff = 2; %on-station is under 2m/s (to show fore/port ) % rl_range = [5 12]; %reference layer range (should match smoothr) % refstd_window = 5; %running std window length (for jitter calc) % % % substructures containing plotting information defaults, accessed by % field name in structure "data" % % title %structure of titles % minclipval %structure of minimum value to show % maxclipval %structure of maximum value to show default values in substructure of data (from agetmat) % % NBINSxNPRFS loaded variables and their titles % title.pg = 'PERCENT GOOD'; % title.depth = 'DEPTH'; % title.pflag = 'PROFILE_FLAGS'; % title.amp = 'AMPLITUDE'; % title.sw = 'SPECTRAL WIDTH'; % title.umeas = 'MEASURED U'; % title.umeas_refl = 'MEASURED REFL (U)'; % title.uship_sm = 'SHIP U (smoothr)'; % title.vmeas = 'MEASURED V'; % title.vmeas_refl = 'MEASURED REFL (V)'; % title.vship_sm = 'SHIP V (smoothr)'; % title.w = 'W'; % title.wmean = 'REFL W'; % title.e = 'ERROR VELOCITY'; % % NBINSxNPRFS calculated variables and their titles % title.uabs_sm = 'UABS (smoothr)'; % title.uabs_g = 'UABS (gps nav)'; % title.uabs_refl_sm = 'UABS (REFL, smoothr)'; % title.uabs_refl_g = 'UABS (REFL, gps nav)'; % % title.vabs_sm = 'VABS (smoothr)'; % title.vabs_g = 'VABS (gps nav)'; % title.vabs_refl_sm = 'VABS (REFL, smoothr)'; % title.vabs_refl_g = 'VABS (REFL, gps nav)'; % % title.fabs_sm = 'FORE ABS (smoothr)'; % title.fabs_g = 'FORE ABS (gps nav)'; % title.fabs_refl_sm = 'FORE ABS (REFL, smoothr)'; % title.fabs_refl_g = 'FORE ABS (REFL, gps nav)'; % % title.pabs_sm = 'PORT ABS (smoothr)'; % title.pabs_g = 'PORT ABS (gps nav)'; % title.pabs_refl_sm = 'PORT ABS (REFL, smoothr)'; % title.pabs_refl_g = 'PORT ABS (REFL, gps nav)'; % % % NPRFSx1 loaded or calculated variables and their titles % title.smmps = 'SHIP SPEED (smoothr)'; % title.uship_g = 'SHIP SPEED (U: gps)'; % title.vship_g = 'SHIP SPEED (U: gps)'; % title.lon = 'LONGITUDE'; % title.iblkprf = 'BLOCK AND PROFILE'; % title.lat = 'LATITUDE'; % title.dday = 'DECIMAL DAY'; % title.heading = 'SHIP HEADING'; % title.clmps = 'SHIP SPEED (gps)'; % title.bins = 'bins'; % % % default minimum and maximum clip values % % minclipval.pg = 30; maxclipval.pg = 100; % minclipval.amp = 10; maxclipval.amp = 230; % minclipval.sw = 30; maxclipval.sw = 260; % minclipval.e = -70; maxclipval.e = 70; % minclipval.uv = -1; maxclipval.uv = 1; % minclipval.mps = 0; maxclipval.mps = 7; % % aplotit: % cfg = aplotit(data, config, varargin); % plots data vs/ dday as a line or pcolor (w/ colorbar) in current axes. % if just one argument, it assumes abcissa, ordinate, and field are x,y,z. % optional: 'config' holds plotting nformation, varargin will override % %% ========================================================================== % In this context, structure "data" comes from agetmat: % % % ---------- structure "data": % % dday %1xN or Nx1 % depth %NBINSxNPRFS % [fieldname] %pg, sw, amp, u_meas, (see below for a detailed list) % % ---------- structure "config": % fn % plot this field name (defaults to '') % abc_name % fieldname to use as abcissa % % (if one argument, defaults to 'x') % % (if two arguments, defaults to 'dday') % % can also set 'abcissa_title' for xlabel (default: empty) % ord_name % fieldname to use as ordinate (defaults to data.bins) % % can also set 'ordinate_title' for ylabel % % (line plot (1xN), or pcolor (nbinsxN)) % yearbase % should have been in config structure used in agetmat % cur_ax % handle to figure axes (defaults to new axes in gca) % reverse_ydir % reverse direction of ordinate % % (defaults to 1 for bins or depth); otherwise 0 % index % how to subsample abcissa : % % - empty (use it all) % % - scalar (subsample by this many) % % - length 2 vector (range in dday) % % - length > 2 vector (indices of abcissa to use) % xlim % limits of abcissa (default is autoscale) % ylim % limits of ordinate (default is autoscale) % var_range % variable range to use for field: [minval maxval] % % defaults to substructures data.minclipval and % % data.maxclipval (values from fieldname) % cmap % colormap to match var_range (sets the number of colors) % % defaults to jet(20) % flagged % matrix of flags for excluding data in plots. same scheme % % as PFLAGS, i.e. 0 is OK (not flagged), otherwise % % type of flagging is indicated by value in bin % % (default is all zeros, i.e. none flagged) % % (see aflagit_setup or below for parameters to set) % fcolor % color of figure label (defaults to black) % comment2 % additional comment; follows config.comment, specifically % % final_comment is [comment, (comment)] % % % examples: % aplotit(data, config, 'flagged', flagged); % % % dd.x = data.dday; dd.y = data.bins; dd.z = data.pg; aplotit(dd); % % % data.z = something;aplotit(data, config, 'fn', 'z'); % % ANNOYING FACT: latest cmap will override all cmaps % can get around this using 'direct' colormaps instead of 'scaled' % surface(x,y,scaled_z,cmap_for_scaled_z,... % 'CDataMapping, 'direct',... % 'edgecolor','none'); ======================================================================== ==================== (III) - program descriptions ====================== ======================================================================== PROGRAMS YOU CAN EDIT: aflagit_setup.m script: set up editing (flagging) parameters (see below) asetup.m script: set up info for getmat: (database path, name, decimal day (start and end), yearbase PROGRAMS THAT RESIDE IN /home/noio/programs/matlab/autoedit %%% engines -------------------- agetmat.m function: uses structure "config" and varargin to set up information to call getmat, call getmat, load the data and calculate additional values, return everything in structures ("data" and updated "config") aplotit.m function: make a plot in the given axes (uses "data" and "config" structures to make the plot, can override "config" with varargin) aflagit.m function: use "data" and "config" structures to return "flagged" (profile flags at bad points) and "pfc1" (profile flags configuration, masks, flag values) alookit.m script: (called by run_alookit): look at the effects of the flagging without saving to disk. aedit.m script: (called by run_aedit): run through the whole cruise saving matlab files with "flagged" and dday to disk; (reminder to apply editing to database; reminder to rerun nav steps) alistit.m function: lists to ascii (like "list" in standard matlab editing of shipboard adcp data) - makes three files (so far): abottom.asc, abadprf.asc, abadbin.asc %%% obsolete drivers (for running manually) -------------------- run_aplottopog.m (this one is still useful) script: plot a cruisetrack over topography (helps with interpretation of currents and bottom locations) (it's kind of nice but isn't necessary) run_aplotscat.m script: makes a plot of shear vs/ AGC scatter (a diagnostic to help with along-track bias) (development diagnostic - unnecessary for gen user) run_alookit.m script: show the effect of editing graphically but do not save. (calls aflagit_setup) run_aedit.m script: use the flagging parameters set in aflagit_setup to find bad data and build big a big profile_flags matrix: do not show grahically; write matlab files with flag info to disk, list those to standard ascii files (abottom.asc, abadbin.asc, abadprf.asc) (calls aflagit_setup) %%% others (subsidiary tools) %%% apuv.m function: plots navigation info (called by run_alookit.m) %%% nifty tools to use while doing regular shipboard ADCP processing: app.m script: in the context of old-fashioned waterfall-plot shipboard adcp editing, load the currently existing getmat output and plot it similarly to alookit. cl_blkprf.m function: choose a field, click on block and profile and it will output two arrays: [block profile] and dday. (NOTE this exists in gautoedit as a button) % this option, the jitter criteria will % be nearly useless