From 3671f4e2c92f28319e9c8e44071604e98d8d783f Mon Sep 17 00:00:00 2001
From: Blaise Thompson <blaise@untzag.com>
Date: Mon, 16 Apr 2018 08:45:36 -0500
Subject: 2018-04-16 08:45

---
 processing/chapter.tex | 204 ++++++++++++++++++++++++++++---------------------
 1 file changed, 119 insertions(+), 85 deletions(-)

(limited to 'processing/chapter.tex')

diff --git a/processing/chapter.tex b/processing/chapter.tex
index 0d89055..1d11788 100644
--- a/processing/chapter.tex
+++ b/processing/chapter.tex
@@ -507,8 +507,8 @@ WrightTools.  %
     \texttt{wt.data.from\_shimamdzu} & Shimadzu brand spectrometers. \\ \hline
     \texttt{wt.data.from\_Tensor27} & Tensor 27 FT-IR. \\ \hline
   \end{tabular}
-  \caption[CAPTION TODO]{
-    CAPTION TODO
+  \caption{
+    WrightTools data import functions.
   }
   \label{pro:tab:from_functions}
 \end{table}
@@ -586,10 +586,11 @@ After importing and manipulating data, one typically wants to create a plot.  %
 The artists sub-package contains everything users need to plot their data objects.  %
 This includes both ``quick'' artists, which generate simple plots as quickly as possible, and a
 full figure layout toolkit that allows users to generate full publication quality figures.  %
-It also includes ``specialty'' artists which are made to perform certain popular plotting
-operations, as I will describe below.  %
+It also includes ``specialty'' artists which are made to perform certain common plotting
+operations.  %
 
-Currently the artists sub-package is built on-top of the wonderful matplotlib library.  %
+Currently the artists sub-package is built on-top of the wonderful matplotlib library
+\cite{HunterJohnD2007a}.  %
 In the future, other libraries (e.g. Mayavi \cite{Mayavi}), may be incorporated.  %
 
 \subsection{Strategies for 2D visualization}  % ---------------------------------------------------
@@ -598,19 +599,36 @@ Representing two-dimensional data is an important capability for WrightTools, so
 discussion about how such representations work is warranted.  %
 WrightTools data is typically very structured, with values recorded at a grid of positions.  %
 To represent two-dimensional data, then, WrightTools needs to map the values onto a color axis.  %
-There are better and worse choices of colormap... % TODO: elaborate
+When doing this mapping, choosing an appropriate colormap and interpolation type is crucial.  %
 
 \subsubsection{Colormap}
 
+For the purposes of this discussion, there are two types of colormaps: 1. qualitative 2.
+perceptual.  %
+Qualitative colormaps have random orderings of color.  %
+They are best used to represent unordered things, and they typically have high dynamic range.  %
+Perceptual colormaps are monotonic in lightness, and are best at representing ordered information
+(like signal levels in MR-CMDS).  %
+
+Historically the Wright Group has used a qualitative colormap for all plotting.  %
 \autoref{pro:fig:cmaps} shows the red, green, and blue components of four different colormaps.  %
-The black line is the net brightness of each color (larger value means lighter color).  %
+The black line is the net lightness of each color (larger value means lighter color).  %
 Below each figure is a gray-scale representation of the corresponding colormap.  %
 The r, g, and b components are scaled according to human perception.  % TODO: values, from where
-The traditional Wright Group colormap (derived from jet) is shown first.  %
+The traditional Wright Group colormap (derived from jet) is shown first.  i%
 Following are two perceptual colormaps, cubehelix from \texcite{GreenDA2009a}, and viridis, the new
 matplotlib default.
 WrightTools uses the algorithm from \textcite{GreenDA2009a} to define a custom cubehelix colormap
 with good perceptual properties and familiar Wright Group coloration.  %
+The lightness is not perfectly straight as in viridis and cubehelix, but it is monotonic at
+least.  %
+
+\autoref{pro:fig:cmap_comparison} compares the new (top row) and old (bottom row) colormaps in full
+color and greyscale.  %
+While the old colormap shows a greater dynamic range, the new colormap is more perceptually
+uniform.  %
+In my opinion, users should use logarithmic scaling of an appropriate base if more dynamic range is
+required, rather than using a non-perceptual colormap.  %
 
 \subsubsection{Interpolation type}
 
@@ -635,22 +653,27 @@ plots.  %
   \includegraphics[scale=0.5]{"processing/cubehelix_cmap"}
   \includegraphics[scale=0.5]{"processing/viridis_cmap"}
   \includegraphics[scale=0.5]{"processing/default_cmap"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption[Colormap lightness.]{
+    Comparison of four different colormaps.
+    R, G, and B components are each shown.
+    Black line is human-perceived lightness.
+  }
   \label{pro:fig:cmaps}
 \end{figure}
 
 \begin{figure}
   \includegraphics[width=\textwidth]{"processing/cmap_comparison"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption{
+    Colormap comparison.
+  }
   \label{pro:fig:cmap_comparison}
 \end{figure}
 
 \begin{figure}
   \includegraphics[width=\textwidth]{"processing/fill_types"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption{
+    Interpolation types in 2D plotting.
+  }
   \label{pro:fig:fill_types}
 \end{figure}
 
@@ -684,39 +707,40 @@ These functionalities are derived from \python{wt.Data.chop}, discussed further
 \begin{figure}
   \includegraphics[width=0.5\textwidth]{"processing/quick1D 000"}
   \includepython{"processing/quick1D.py"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption[quick1D example.]{
+    Minimal example of the \python{quick1D} function.
+  }
   \label{pro:fig:quick1D}
 \end{figure}
 
 \begin{figure}
   \includegraphics[width=0.5\textwidth]{"processing/quick2D 000"}
   \includepython{"processing/quick2D.py"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption[quick2D example.]{
+    Minimal example of the \python{quick2D} function.
+  }
   \label{pro:fig:quick2D}
 \end{figure}
 
 % TODO: signed data (with and without dynamic_range=True)
 
-\subsection{API}  % -------------------------------------------------------------------------------
+% \subsection{API}  % -------------------------------------------------------------------------------
 
-The artists sub-package offers a thin wrapper on the default matplotlib object-oriented figure
-creation API.  %
-The wrapper allows WrightTools to add the following capabilities on top of matplotlib:
-\begin{ditemize}
-  \item More consistent multi-axes figure layout.
-  \item Ability to plot data objects directly.
-\end{ditemize}
-Each of these is meant to lower the barrier to plotting data.  %
-Without going into every detail of matplotlib figure generation capabilities, this section
-introduces the unique strategy that the WrightTools wrapper takes.  %
+% The artists sub-package offers a thin wrapper on the default matplotlib object-oriented figure
+% creation API.  %
+% The wrapper allows WrightTools to add the following capabilities on top of matplotlib:
+% \begin{ditemize}
+%   \item More consistent multi-axes figure layout.
+%   \item Ability to plot data objects directly.
+% \end{ditemize}
+% Each of these is meant to lower the barrier to plotting data.  %
+% Without going into every detail of matplotlib figure generation capabilities, this section
+% introduces the unique strategy that the WrightTools wrapper takes.  %
 
-% \subsection{Gotchas}  % -------------------------------------------------------------------------
+% % \subsection{Gotchas}  % -------------------------------------------------------------------------
 
 % TODO: mention gotcha of apparently narrowing linewidths with wigners (how to READ colormaps)
 
-\clearpage
 \section{Variables and channels}  % ===============================================================
 
 Data objects are made up of many component channels and variables, each array having the same
@@ -728,7 +752,7 @@ may not have even moved across some (or any) dimensions.  %
 To avoid unnecessarily large arrays, WrightTools allows Channels and Variables to have different
 sizes than the parent data.  %
 As an example, consider the following object.
-\begin{codefragment}{bash}  % TODO: need to use bash here because of box charachters :-(
+\begin{codefragment}{bash}
 >>> import WrightTools as wt; from WrightTools import datasets
 >>> data = wt.data.from_COLORS(datasets.COLORS.v2p1_MoS2_TrEE_movie)
 >>> data.print_tree()
@@ -763,18 +787,34 @@ propagated.  %
 
 \section{Axes} \label{pro:sec:axes}  % ============================================================
 
-The axes have the joint shape of their component variables.  %
-Although not shown in this example, channels also may have axes with length 1.
+Axes are the primary interface to coordinate positions in WrightTools.  %
+Axes are \emph{not} arrays, although they do behave like arrays.  %
+They are merely \emph{interfaces} into the information stored in one or more variables.  %
+
+Each axis has an expression, like \python{'w1'}, \python{'d1=d2'}, \python{2*w3} or
+\python{'w1+w2-wm'}.  %
+These expressions describe an unambiguous mathematical operation involving one or more
+variables.  %
+The axis computes these expressions on the fly when needed, giving users real flexability over how
+they would like to represent and process their results.  %
 
 Axes, variables, and channels are array-likes, so they support slicing operations.  %
+The axes have the joint shape of their component variables.  %
 In addition, all three classes have \python{points} and \python{full} attributes that return the
 squeezed and broadcasted array, respectively.  %
 
+One of the coolest features enabled by this approach to axes is \emph{transformation}.  %
+\autoref{pro:fig:transform} shows how easy it is to transform data from one coordinate system into
+another.  %
+Transforming is essentially free, since axes compute coordinates on the fly.  %
+
 \begin{figure}
-  \includegraphics[width=\textwidth]{"processing/fringes_transform"}
+  \includegraphics[width=0.8\textwidth]{"processing/fringes_transform"}
   \includepython{"processing/fringes_transform.py"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption{
+    Example using transform.
+  }
+  \label{pro:fig:transform}
 \end{figure}
 
 \section{Math}  % =================================================================================
@@ -783,8 +823,6 @@ Now that we know the basics of how the WrightTools \python{Data} class stores da
 some data manipulation.  %
 Let's start with some elementary algebra.  %
 
-% TODO: mention chunkwise strategy
-
 \subsection{In-place operators} \label{pro:sec:in_place}  % ---------------------------------------
 
 In Python, operators are symbols that carry out some computation.  %
@@ -847,10 +885,11 @@ Variables also support in-place operators.  %
 \subsection{Clip}  % ------------------------------------------------------------------------------
 
 Clip allows users to exclude values outside of a certain range.  %
+Excluded values can be replaced with a user-specified value or with not-a-number.  %
 This can be particularly useful in cases like fitting.  %
-See section ... for an example.  % TODO: link to section
+See \autoref{pro:sec:fit} for an example.  %
 
-It's also useful for when noise in a certain region of a spectrum obscures useful data...
+It's also useful for when noise in a certain region of a spectrum obscures useful data.  %
 Particularly true for normalized and signed data.  %
 
 \subsection{Symmetric root}  % --------------------------------------------------------------------
@@ -885,7 +924,7 @@ keyword arguments.  %
 These may be slightly faster than \python{channel.log(base=10)} and
 \python{channel.log(base=2)}.  %
 
-\subsection{Level}  % -----------------------------------------------------------------------------
+% \subsection{Level}  % -----------------------------------------------------------------------------
 
 % TODO: figure from wright.tools
 
@@ -894,6 +933,7 @@ These may be slightly faster than \python{channel.log(base=10)} and
 Trim uses statistical treatment to find and remove outliers from a dataset.  %
 It is useful in cases where the naive strategy employed by \python{wt.Channel.clip} is not
 sufficient, and when preparing for fitting.  %
+See \autoref{pro:fig:outlier} for an example of outlier detection.  %
 
 Currently \python{trim} only supports one statistical treatment: the z-test.  %
 Z-testing compares each pixel to its multidimensional neighborhood of pixels.  %
@@ -902,8 +942,6 @@ neighborhood standard deviation) it is either masked, replaced with \python{np.n
 with the neighborhood mean.  %
 All outliers are found before any outliers are modified, so the algorithm is not directional.  %
 
-% TODO: z-test citation
-
 \python{wt.Channel.trim} can easily be enhanced with other statistical methods as needed.  %
 
 \subsection{Smooth}  % ----------------------------------------------------------------------------
@@ -922,13 +960,20 @@ can confidently smooth collected data in post to achieve clean results.  %
 This strategy is similar to that accomplished in time domain CMDS where a low-pass filter is
 applied on the very high resolution raw data.  %
 
+\begin{figure}
+  \includegraphics[width=0.8\textwidth]{"processing/outlier"}
+  \caption{
+    Outlier detection using a z-test.
+  }
+  \label{pro:fig:outlier}
+\end{figure}
+
 % TODO: figure: example of smoothed data
 
-\clearpage
 \section{Dimensionality manipulation}  % ==========================================================
 
 WrightTools offers several strategies for reducing the dimensionality of a data object.  %
-Also consider using the fit sub-package.  % TODO: more info, link to section
+Also consider using the fit sub-package described in \autoref{pro:sec:fit}.  %
 
 \subsection{Chop} \label{pro:sec:chop}  % ---------------------------------------------------------
 
@@ -941,7 +986,7 @@ This dataset can be chopped to it's component 2D \python{('wm'', 'w1')} spectra.
 >>> import WrightTools as wt; from WrightTools import datasets
 >>> data = wt.data.from_PyCMDS(datasets.PyCMDS.wm_w2_w1_000)
 data created at /tmp/lzyjg4au.wt5::/
-  axes ('wm', 'w2', 'w1')
+  axes ('wm'', 'w2', 'w1')
   shape (35, 11, 11)
 >>> chopped = data.chop('wm', 'w1')  
 chopped data into 11 piece(s) in ('wm', 'w1'')
@@ -977,8 +1022,8 @@ chopped data into 1 piece(s) in ('wm', 'w1')
 >>> chopped.w2.points
 array([1580.0])
 \end{codefragment}
-Note the [0]...  % TODO
-This same syntax used in artists...  % TODO
+% Note the [0]...  % TODO
+% This same syntax used in artists...  % TODO
 
 \subsection{Collapse}  % --------------------------------------------------------------------------
 
@@ -1016,8 +1061,9 @@ To do this, the variable and channel names must agree.  %
 
 \begin{figure}
   \includegraphics[width=\textwidth]{"processing/join_example"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO}
+  \caption{
+    Join example.
+  }
   \label{pro:fig:join_example}
 \end{figure}
 
@@ -1032,7 +1078,7 @@ Rather than attempt to visualize \python{w1, w2} at some specific value of \pyth
 powerful to instead consider the parameters (amplitude, offset, and time constant) of an
 exponential fit at each \python{w1, w2} coordinate.  %
 On a more practical note, this kind of slice-by-slice dimensionality reduction via fitting can
-greatly simplify automated instrumental calibration (see ...)  % TODO: link to opa chapter
+greatly simplify automated instrumental calibration (see \autoref{cha:opa}).  %
 WrightTools employs some simple tricks to enable these kind of fit operations, described here.  %
 
 % TODO: consider inserting figures that demonstrate this story (need to use wt2?)
@@ -1043,22 +1089,16 @@ One challenge of slice-by-slice fitting is making a good intial guess to optimiz
 It is not tractable to ask the user to provide a guess for each slice, so some kind of reasonable
 automated guessing must be used.  %
 WrightTools ``function'' objects are self contained describers of a particular function.  %
-As an example, consider the \python{wt.fit.Expontial} class...
-It has parameters...
-Fit...
-Evaluate...
-Guess...
-
-Can be used directly...
-
-[USERS CAN WRITE THEIR OWN FUNCTION OBJECTS]
+As an example, consider the \python{wt.fit.Expontial} class.
+It can be used directly, as shown in \autoref{pro:fig:fit_function}
 
 \begin{figure}
   \includegraphics[width=0.5\textwidth]{"processing/fit_function"}
   \includepython{"processing/fit_function.py"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO
+  \caption{
+    Fitting a Gaussian.
   }
+  \label{pro:fig:fit_function}
 \end{figure}
 
 \subsection{Fitter}  % ----------------------------------------------------------------------------
@@ -1117,13 +1157,11 @@ the fit parameters, in this case amplitude, tau, and offset. Accordingly, outs i
 than the original data object. model contains the fit evaluated at each coordinate of the original
 dataset—it’s really useful for inspecting the quality of your fit procedure.
 
-[ALSO GOOD FOR WORKUP OF TUNING DATA: SEE SECTION ...]
-
 \begin{figure}
   \includegraphics[width=0.4\textwidth]{"processing/fit_amplitude"}
   \includegraphics[width=0.4\textwidth]{"processing/fit_tau"}
-  \caption[CAPTION TODO]{
-    CAPTION TODO
+  \caption{
+    Fitting as dimensionality reduction. 
   }
   \label{pro:fig:fitted_movie}
 \end{figure}
@@ -1197,12 +1235,12 @@ returned to again and again.  %
 Developers can collaborate together to edit the software by making incremental changes that are
 easy to review.  %
 
-WrightTools uses git \cite{git} for source control, and the package is hosted on GitHub
-\cite{GitHub, WrightToolsGitHub}.  %
+WrightTools uses git for source control, and the package is hosted on GitHub
+\cite{GitHub}.  %
 As of 2018-04-08, WrightTools has 1,018 commits from seven developers, as shown in
 \autoref{pro:tab:commits}.  %
-In addition to simply hosting the git repo, GitHub gives us issue tracking and automated
-integration with zenodo.  %
+In addition to simply hosting the git repository, GitHub gives us issue tracking, continuous
+integration, and Zenodo support.  %
 
 The WrightTools package has a developer controlled version as well, following the semantic
 versioning convention \cite{SemanticVersioning}.  %
@@ -1223,9 +1261,8 @@ These attributes can be accessed through python: \python{wt.__version__} and
     Daniel Kohler & 3 & 113 & 29 \\ \hline
     Rachel Swedin & 1 & 5,197 & 0 \\ \hline
   \end{tabular}
-  \caption[CAPTION TODO]{
+  \caption[Commits to WrightTools.]{
     Commits to WrightTools by individual, ordered by number of commits.
-    Wright Stuff is...
     Note that datasets...
   }
   \label{pro:tab:commits}
@@ -1260,7 +1297,8 @@ of the tests and receive a report on what failed and why.  %
 If a future programmer unintentionally breaks \python{convert}, the above test will fail and alert
 her to the unexpected side effect of her modification.  %
 
-[PARAGRAPH ABOUT CONTINUOUS INTEGRATION]
+WrightTools uses continuous integration services to run unit tests every time a pull request is
+made on GitHub.  %
 
 \subsection{Distribution} \label{pro:sec:distribution}  % ------------------------------------------
 
@@ -1272,8 +1310,7 @@ Luckily for us, distribution is fairly simple within the Python ecosystem.  %
 The same tools that are used to distribute hugely popular packages like numpy are also available
 for anyone else, including WrightTools.  %
 
-The Python Package Index (PyPI), affectionately known as the cheese shop, is the official third
-party software repository for Python.  %
+The Python Package Index (PyPI) is the official third party software repository for Python.  %
 It is community maintained, and supported by the Python Software Foundation and The Python
 Packaging Group.  %
 As of 2018-04-08 PyPI hosts 134,758 Python packages, all for free.  %
@@ -1291,10 +1328,9 @@ Since the scientific Python ecosystem has so many non-Python binary dependencies
 popular choice---especially on Windows where the necessary compilers are not typically
 pre-installed.  %
 Unlike pip, conda is not tied to a single repository.  %
-There is the official repository, maintained by Anaconda, the company that develops conda.
-[CITE]  %
+There is the official repository, maintained by Anaconda, the company that develops conda.  %
 There is also the popular conda-forge repository, which is maintained by the community via
-GitHub. [CITE]  %
+GitHub.  %
 WrightTools is on conda-forge:  %
 \begin{codefragment}{bash}
 conda config --add channels conda-forge
@@ -1308,13 +1344,11 @@ using such a thing immediately.  %
 To this end, it is important to have easy to use, searchable documentation with end-users in
 mind.  %
 
-wright.tools
-
-Built and hosted by readthedocs
-
-Both master and development versions...
+I have built a website for WrightTools documentation at \href{http://wright.tools/}.  %
+The documentation is built and hosted by Read the Docs. \cite{ReadTheDocs}.  %
+Both master and development versions are built, so users of both can get documentation that is up
+to date for them.  %
 
-% \clearpage
 % \section{Future directions}  % ====================================================================
 
 % Single variable decomposition.  %
-- 
cgit v1.2.3

