diff --git a/src/error.rs b/src/error.rs index 61655756..ca644e27 100644 --- a/src/error.rs +++ b/src/error.rs @@ -1,52 +1,70 @@ //! Error types for RustyML //! -//! Every fallible operation in the crate returns [`RustymlResult`](crate::error::RustymlResult), -//! an alias for `std::result::Result`. [`Error`](crate::error::Error) is the single, +//! Every fallible operation in the crate returns [`RustymlResult`], +//! an alias for `std::result::Result`. [`Error`] is the single, //! unified error type. //! //! `Error` aggregates 2 domain-specific error enums: `NnError` (from `neural_network`) and //! `TreeError` (from `machine_learning`). It also wraps the shared -//! [`IoError`](crate::error::IoError). Callers can `match` precisely, since a shared variant never +//! [`IoError`]. Callers can `match` precisely, since a shared variant never //! carries a concern specific to one part of the crate //! //! # Categories //! -//! - **Input validation**: [`Error::EmptyInput`](crate::error::Error::EmptyInput), -//! [`Error::DimensionMismatch`](crate::error::Error::DimensionMismatch), -//! [`Error::ShapeMismatch`](crate::error::Error::ShapeMismatch), -//! [`Error::NonFinite`](crate::error::Error::NonFinite), -//! [`Error::InvalidParameter`](crate::error::Error::InvalidParameter), -//! [`Error::InvalidInput`](crate::error::Error::InvalidInput) -//! - **Model state**: [`Error::NotFitted`](crate::error::Error::NotFitted), and the neural-network +//! - **Input validation**: [`Error::EmptyInput`], +//! [`Error::DimensionMismatch`], +//! [`Error::ShapeMismatch`], +//! [`Error::NonFinite`], +//! [`Error::InvalidParameter`], +//! [`Error::InvalidInput`] +//! - **Model state**: [`Error::NotFitted`], and the neural-network //! states in `NnError` -//! - **Numerics / computation**: [`Error::NotConverged`](crate::error::Error::NotConverged), -//! [`Error::Computation`](crate::error::Error::Computation) +//! - **Numerics / computation**: [`Error::NotConverged`], +//! [`Error::Computation`] //! - **Domain-specific**: `Error::NeuralNetwork` (wrapping `neural_network`'s `NnError`) and //! `Error::Tree` (wrapping `machine_learning`'s `TreeError`) -//! - **I/O and serialization**: [`Error::Io`](crate::error::Error::Io) +//! - **I/O and serialization**: [`Error::Io`] //! //! # Conventions //! //! - A non-finite **hyperparameter** from the user is an -//! [`Error::InvalidParameter`](crate::error::Error::InvalidParameter) (the reason names +//! [`Error::InvalidParameter`] (the reason names //! finiteness). A non-finite value from the **data or a computation** is an -//! [`Error::NonFinite`](crate::error::Error::NonFinite) -//! - [`Error::DimensionMismatch`](crate::error::Error::DimensionMismatch) compares scalar counts -//! (e.g. number of features). Use [`Error::ShapeMismatch`](crate::error::Error::ShapeMismatch) +//! [`Error::NonFinite`] +//! - [`Error::DimensionMismatch`] compares scalar counts +//! (e.g. number of features). Use [`Error::ShapeMismatch`] //! when whole tensor shapes differ //! //! # Constructing errors //! //! Prefer the smart constructors -//! ([`Error::dimension_mismatch`](crate::error::Error::dimension_mismatch), -//! [`Error::invalid_parameter`](crate::error::Error::invalid_parameter), and so on) over building +//! ([`Error::dimension_mismatch`], +//! [`Error::invalid_parameter`], and so on) over building //! variants by hand. They accept flexible argument types and keep the wording consistent across the //! crate. To attach context to a foreign error while preserving its source chain, use -//! [`Context::context`](crate::error::Context::context) / -//! [`Context::with_context`](crate::error::Context::with_context) +//! [`Context::context`] / +//! [`Context::with_context`] //! -//! [`Error`](crate::error::Error) is `#[non_exhaustive]`. Match with a trailing `_` arm to stay +//! [`Error`] is `#[non_exhaustive]`. Match with a trailing `_` arm to stay //! forward-compatible +//! +//! [`RustymlResult`]: crate::error::RustymlResult +//! [`Error`]: crate::error::Error +//! [`IoError`]: crate::error::IoError +//! [`Error::EmptyInput`]: crate::error::Error::EmptyInput +//! [`Error::DimensionMismatch`]: crate::error::Error::DimensionMismatch +//! [`Error::ShapeMismatch`]: crate::error::Error::ShapeMismatch +//! [`Error::NonFinite`]: crate::error::Error::NonFinite +//! [`Error::InvalidParameter`]: crate::error::Error::InvalidParameter +//! [`Error::InvalidInput`]: crate::error::Error::InvalidInput +//! [`Error::NotFitted`]: crate::error::Error::NotFitted +//! [`Error::NotConverged`]: crate::error::Error::NotConverged +//! [`Error::Computation`]: crate::error::Error::Computation +//! [`Error::Io`]: crate::error::Error::Io +//! [`Error::dimension_mismatch`]: crate::error::Error::dimension_mismatch +//! [`Error::invalid_parameter`]: crate::error::Error::invalid_parameter +//! [`Context::context`]: crate::error::Context::context +//! [`Context::with_context`]: crate::error::Context::with_context /// The unified error type for all fallible RustyML operations /// diff --git a/src/machine_learning.rs b/src/machine_learning.rs index c3639755..6f46f10e 100644 --- a/src/machine_learning.rs +++ b/src/machine_learning.rs @@ -2,18 +2,18 @@ //! reduction, and anomaly detection //! //! This module groups models by algorithm family into submodules: -//! [`clustering`](crate::machine_learning::clustering), -//! [`decomposition`](crate::machine_learning::decomposition), -//! [`linear_model`](crate::machine_learning::linear_model), -//! [`manifold`](crate::machine_learning::manifold), [`svm`](crate::machine_learning::svm), -//! [`tree`](crate::machine_learning::tree), [`neighbors`](crate::machine_learning::neighbors), -//! [`discriminant_analysis`](crate::machine_learning::discriminant_analysis), and -//! [`ensemble`](crate::machine_learning::ensemble). This module also re-exports every estimator, +//! [`clustering`], +//! [`decomposition`], +//! [`linear_model`], +//! [`manifold`], [`svm`], +//! [`tree`], [`neighbors`], +//! [`discriminant_analysis`], and +//! [`ensemble`]. This module also re-exports every estimator, //! so it is reachable directly as `machine_learning::`. Supervised and unsupervised -//! estimators implement the shared [`Fit`](crate::traits::Fit) / -//! [`Predict`](crate::traits::Predict) traits. The dimensionality-reduction -//! transformers implement [`Transform`](crate::traits::Transform) / -//! [`FitTransform`](crate::traits::FitTransform). +//! estimators implement the shared [`Fit`] / +//! [`Predict`] traits. The dimensionality-reduction +//! transformers implement [`Transform`] / +//! [`FitTransform`]. //! //! # Supervised learning //! @@ -47,12 +47,12 @@ //! //! # Shared types //! -//! - [`DistanceCalculationMetric`](crate::math::DistanceCalculationMetric): +//! - [`DistanceCalculationMetric`]: //! Euclidean/Manhattan/Minkowski dispatcher, re-exported from [`crate::math`] -//! - [`RegularizationType`](crate::machine_learning::types::RegularizationType): L1 / L2 +//! - [`RegularizationType`]: L1 / L2 //! regularization -//! - [`KernelType`](crate::machine_learning::types::KernelType) / -//! [`Gamma`](crate::machine_learning::types::Gamma): kernel selection and coefficient +//! - [`KernelType`] / +//! [`Gamma`]: kernel selection and coefficient //! //! # Examples //! @@ -65,6 +65,24 @@ //! let y = array![6.0, 9.0, 12.0]; //! model.fit(&x, &y).unwrap(); //! ``` +//! +//! [`clustering`]: crate::machine_learning::clustering +//! [`decomposition`]: crate::machine_learning::decomposition +//! [`linear_model`]: crate::machine_learning::linear_model +//! [`manifold`]: crate::machine_learning::manifold +//! [`svm`]: crate::machine_learning::svm +//! [`tree`]: crate::machine_learning::tree +//! [`neighbors`]: crate::machine_learning::neighbors +//! [`discriminant_analysis`]: crate::machine_learning::discriminant_analysis +//! [`ensemble`]: crate::machine_learning::ensemble +//! [`Fit`]: crate::traits::Fit +//! [`Predict`]: crate::traits::Predict +//! [`Transform`]: crate::traits::Transform +//! [`FitTransform`]: crate::traits::FitTransform +//! [`DistanceCalculationMetric`]: crate::math::DistanceCalculationMetric +//! [`RegularizationType`]: crate::machine_learning::types::RegularizationType +//! [`KernelType`]: crate::machine_learning::types::KernelType +//! [`Gamma`]: crate::machine_learning::types::Gamma pub use crate::math::DistanceCalculationMetric; /// The crate-wide estimator traits, re-exported here for convenience. Their canonical diff --git a/src/machine_learning/decomposition.rs b/src/machine_learning/decomposition.rs index 1304b3f0..95aac8a7 100644 --- a/src/machine_learning/decomposition.rs +++ b/src/machine_learning/decomposition.rs @@ -4,9 +4,11 @@ //! `sklearn.decomposition`: [`PCA`] for linear dimensionality reduction and //! [`KernelPCA`] for its nonlinear, kernelized counterpart. Both learn their //! components from a feature matrix at `fit` time and project new data through -//! `transform`. Both implement the shared [`Fit`](crate::traits::Fit), -//! [`Transform`](crate::traits::Transform), and -//! [`FitTransform`](crate::traits::FitTransform) traits +//! `transform`. Both implement the shared [`Fit`], [`Transform`], and [`FitTransform`] traits +//! +//! [`Fit`]: crate::traits::Fit +//! [`Transform`]: crate::traits::Transform +//! [`FitTransform`]: crate::traits::FitTransform pub use crate::machine_learning::{Gamma, KernelType}; diff --git a/src/machine_learning/error.rs b/src/machine_learning/error.rs index 4b7a2deb..28470347 100644 --- a/src/machine_learning/error.rs +++ b/src/machine_learning/error.rs @@ -1,9 +1,12 @@ //! Decision-tree error type //! //! [`TreeError`] enumerates the failures specific to decision-tree models. Callers receive -//! it through the crate-wide [`Error::Tree`](crate::error::Error::Tree) variant, into which +//! it through the crate-wide [`Error::Tree`] variant, into which //! it converts via `?` (a `#[from]` bridge). See [`crate::error`] for the unified -//! [`Error`](crate::error::Error) that aggregates the per-domain error enums +//! [`Error`] that aggregates the per-domain error enums +//! +//! [`Error::Tree`]: crate::error::Error::Tree +//! [`Error`]: crate::error::Error /// Decision-tree-specific errors, surfaced through [`Error::Tree`](crate::error::Error::Tree) #[non_exhaustive] diff --git a/src/machine_learning/linear_model/logistic_regression.rs b/src/machine_learning/linear_model/logistic_regression.rs index 52ccb039..3bc179fe 100644 --- a/src/machine_learning/linear_model/logistic_regression.rs +++ b/src/machine_learning/linear_model/logistic_regression.rs @@ -2,12 +2,14 @@ //! //! Provides the [`LogisticRegression`] model, trained by full-batch gradient descent on the //! logistic loss, with optional L1 or L2 regularization. Fitting never panics: every precondition -//! failure and numerical issue during training surfaces as an [`Error`](crate::error::Error) from +//! failure and numerical issue during training surfaces as an [`Error`] from //! [`LogisticRegression::fit`]. //! //! Also provides the [`generate_polynomial_features`] helper, which expands a feature //! matrix into its polynomial combinations up to a chosen degree. Use it to fit a //! non-linear decision boundary with the same linear model. +//! +//! [`Error`]: crate::error::Error use crate::error::Error; pub use crate::machine_learning::RegularizationType; diff --git a/src/machine_learning/manifold.rs b/src/machine_learning/manifold.rs index adb5783f..01cf1368 100644 --- a/src/machine_learning/manifold.rs +++ b/src/machine_learning/manifold.rs @@ -1,10 +1,13 @@ //! Manifold-learning estimators for nonlinear dimensionality reduction //! //! [`TSNE`] implements t-distributed Stochastic Neighbor Embedding (t-SNE) for visualizing -//! high-dimensional data. Unlike the [`decomposition`](crate::machine_learning::decomposition) -//! estimators, t-SNE learns no out-of-sample projection. It only embeds the data it is given. -//! It implements [`FitTransform`](crate::traits::FitTransform) but not -//! [`Transform`](crate::traits::Transform). +//! high-dimensional data. Unlike the [`decomposition`] estimators, t-SNE learns no out-of-sample +//! projection. It only embeds the data it is given. It implements [`FitTransform`] but not +//! [`Transform`]. +//! +//! [`decomposition`]: crate::machine_learning::decomposition +//! [`FitTransform`]: crate::traits::FitTransform +//! [`Transform`]: crate::traits::Transform /// t-SNE algorithm for dimensionality reduction pub mod t_sne; diff --git a/src/machine_learning/svm/linear_svc.rs b/src/machine_learning/svm/linear_svc.rs index 79306a0c..63e971bd 100644 --- a/src/machine_learning/svm/linear_svc.rs +++ b/src/machine_learning/svm/linear_svc.rs @@ -2,8 +2,10 @@ //! //! Provides [`LinearSVC`], a hinge-loss classifier trained with stochastic gradient //! descent and L1 or L2 regularization, along with the -//! [`RegularizationType`](crate::machine_learning::RegularizationType) enum that +//! [`RegularizationType`] enum that //! selects the penalty +//! +//! [`RegularizationType`]: crate::machine_learning::RegularizationType use crate::error::Error; pub use crate::machine_learning::RegularizationType; diff --git a/src/machine_learning/svm/svc.rs b/src/machine_learning/svm/svc.rs index a950e3c2..0bccc589 100644 --- a/src/machine_learning/svm/svc.rs +++ b/src/machine_learning/svm/svc.rs @@ -2,10 +2,12 @@ //! //! Provides the [`SVC`] binary classifier, trained with the Sequential Minimal //! Optimization (SMO) algorithm, and the kernel types re-exported as -//! [`KernelType`](crate::machine_learning::KernelType) +//! [`KernelType`] //! //! Labels are `{0.0, 1.0}` on the public API, matching the crate's other binary //! classifiers. Only `fit` uses the internal +/-1 encoding that the SMO dual needs +//! +//! [`KernelType`]: crate::machine_learning::KernelType use crate::error::Error; use crate::machine_learning::parallel::map_collect; diff --git a/src/machine_learning/types.rs b/src/machine_learning/types.rs index f93c9465..3ddd0e47 100644 --- a/src/machine_learning/types.rs +++ b/src/machine_learning/types.rs @@ -1,13 +1,12 @@ //! Configuration enums shared across the machine-learning estimators //! //! - The linear models and the linear SVM use [`RegularizationType`] -//! - [`SVC`](crate::machine_learning::svm::svc::SVC) and -//! [`KernelPCA`](crate::machine_learning::decomposition::kernel_pca::KernelPCA) both use -//! [`Gamma`] and [`KernelType`] +//! - [`SVC`] and [`KernelPCA`] both use [`Gamma`] and [`KernelType`] +//! Each kernel carries its own evaluation logic (`compute` / `compute_matrix`) as inherent methods. +//! Consumers share this single implementation instead of re-matching the enum in every model //! -//! Each kernel carries its own evaluation logic (`compute` / `compute_matrix`) as inherent -//! methods. Consumers share this single implementation instead of re-matching the enum in -//! every model +//! [`SVC`]: crate::machine_learning::svm::svc::SVC +//! [`KernelPCA`]: crate::machine_learning::decomposition::kernel_pca::KernelPCA use crate::math::squared_euclidean_distance_row; use crate::parallel_gates::{cheap_map_f64_parallel_threshold, exp_map_f64_parallel_threshold}; diff --git a/src/math.rs b/src/math.rs index e2589bfb..1ef41e97 100644 --- a/src/math.rs +++ b/src/math.rs @@ -1,10 +1,10 @@ //! Shared low-level numeric primitives used across estimators and metrics //! -//! - [`matmul`](crate::math::matmul) provides `gemmkit`-backed matrix products with automatic +//! - [`matmul`] provides `gemmkit`-backed matrix products with automatic //! parallelism -//! - [`reduction`](crate::math::reduction) provides deterministic blocked parallel reductions -//! - [`distance`](crate::math::distance) holds the pairwise distance primitives and the -//! [`DistanceCalculationMetric`](crate::math::DistanceCalculationMetric) dispatcher +//! - [`reduction`] provides deterministic blocked parallel reductions +//! - [`distance`] holds the pairwise distance primitives and the +//! [`DistanceCalculationMetric`] dispatcher //! //! It also hosts the tunable exp-reduction parallel gate used by the logistic-regression //! log-loss. @@ -29,6 +29,11 @@ //! let sq = squared_euclidean_distance_row(&v1, &v2); //! let d = DistanceCalculationMetric::Euclidean.distance(v1.view(), v2.view()); //! ``` +//! +//! [`matmul`]: crate::math::matmul +//! [`reduction`]: crate::math::reduction +//! [`distance`]: crate::math::distance +//! [`DistanceCalculationMetric`]: crate::math::DistanceCalculationMetric /// `gemmkit`-backed matrix products with automatic parallelism. pub mod matmul; diff --git a/src/math/reduction.rs b/src/math/reduction.rs index 783ed904..c6f46c00 100644 --- a/src/math/reduction.rs +++ b/src/math/reduction.rs @@ -5,17 +5,21 @@ //! from run to run and with the thread count. The helpers here fix the grouping instead. The //! input is cut into fixed-size blocks. Each block folds serially, and rayon's indexed //! `collect` gathers the per-block results in block order before they merge sequentially. The -//! grouping depends only on [`DET_REDUCE_BLOCK`](crate::math::reduction::DET_REDUCE_BLOCK), +//! grouping depends only on [`DET_REDUCE_BLOCK`], //! never on scheduling or the `parallel` flag. Re-running on the same machine reproduces the //! result, though not necessarily bit-for-bit. The flag is a performance hint that only decides //! whether the blocks run on rayon or in sequence. A caller that pairs these helpers with some //! other serial kernel below a size threshold makes that switch part of its own reproducibility //! surface. //! -//! [`det_reduce`](crate::math::reduction::det_reduce) folds a slice. -//! [`det_reduce_range`](crate::math::reduction::det_reduce_range) folds fixed blocks of an +//! [`det_reduce`] folds a slice. +//! [`det_reduce_range`] folds fixed blocks of an //! index range instead, for reductions that read several arrays at once or accumulate //! something richer than a scalar. +//! +//! [`DET_REDUCE_BLOCK`]: crate::math::reduction::DET_REDUCE_BLOCK +//! [`det_reduce`]: crate::math::reduction::det_reduce +//! [`det_reduce_range`]: crate::math::reduction::det_reduce_range use rayon::prelude::{IntoParallelIterator, ParallelIterator, ParallelSlice}; use std::ops::Range; diff --git a/src/metrics.rs b/src/metrics.rs index cd232a7e..f03945a5 100644 --- a/src/metrics.rs +++ b/src/metrics.rs @@ -14,11 +14,11 @@ //! constant bias) //! //! # Classification metrics -//! - [`ConfusionMatrix`](crate::metrics::ConfusionMatrix): binary TP/FP/TN/FN with derived +//! - [`ConfusionMatrix`]: binary TP/FP/TN/FN with derived //! accuracy, precision, recall, specificity, F1, error rate, balanced accuracy, and MCC -//! - [`MulticlassConfusionMatrix`](crate::metrics::MulticlassConfusionMatrix): KxK matrix with +//! - [`MulticlassConfusionMatrix`]: KxK matrix with //! per-class precision/recall/F1/support and macro/micro/weighted aggregation via -//! [`Average`](crate::metrics::Average) +//! [`Average`] //! - **accuracy**, **roc_auc**, **log_loss**, **cohen_kappa**, **top_k_accuracy**, //! **average_precision**, and the **roc_curve** / **precision_recall_curve** threshold sweeps //! @@ -65,6 +65,10 @@ //! let scores = array![0.1, 0.4, 0.35, 0.8]; //! let auc = roc_auc(&labels.view(), &scores.view()); //! ``` +//! +//! [`ConfusionMatrix`]: crate::metrics::ConfusionMatrix +//! [`MulticlassConfusionMatrix`]: crate::metrics::MulticlassConfusionMatrix +//! [`Average`]: crate::metrics::Average /// Classification metrics: confusion matrices, accuracy, ROC/PR curves, log loss, /// and Cohen's kappa. diff --git a/src/neural_network.rs b/src/neural_network.rs index 5486a3b5..79bd0773 100644 --- a/src/neural_network.rs +++ b/src/neural_network.rs @@ -2,33 +2,24 @@ //! that tie them together //! //! The module builds, trains, and runs neural networks. -//! Layers, optimizers, and losses live in the [`layers`](crate::neural_network::layers), -//! [`optimizers`](crate::neural_network::optimizers), and -//! [`losses`](crate::neural_network::losses) submodules. -//! [`Sequential`](crate::neural_network::sequential::Sequential) stacks layers into a -//! chain, and [`Graph`](crate::neural_network::graph::Graph) wires them into a directed graph. -//! The shared interfaces live in [`traits`](crate::neural_network::traits). -//! Every tensor in the framework is a [`Tensor`](crate::neural_network::Tensor), an `f32` -//! n-dimensional array +//! Layers, optimizers, and losses live in the [`layers`], [`optimizers`], and [`losses`] +//! submodules. [`Sequential`] stacks layers into a chain, and [`Graph`] wires them into a directed +//! graph. The shared interfaces live in [`traits`]. Every tensor in the framework is a [`Tensor`], +//! an `f32` n-dimensional array //! //! A layer computes, and it holds no cache and no gradient. -//! [`forward`](crate::neural_network::traits::UnaryLayer::forward) and -//! [`backward`](crate::neural_network::traits::UnaryLayer::backward) take `&self` and a -//! [`Ctx`](crate::neural_network::Ctx). The context carries the training flag, the caches of the -//! pass, every parameter gradient, and the non-trainable state that a training pass changes. -//! [`Ctx::training`](crate::neural_network::Ctx::training) and -//! [`Ctx::inference`](crate::neural_network::Ctx::inference) pick the mode. A built model is -//! therefore `Send` and `Sync`, and several threads can run inference against 1 model +//! [`forward`] and [`backward`] take `&self` and a [`Ctx`]. The context carries the training flag, +//! the caches of the pass, every parameter gradient, and the non-trainable state that a training +//! pass changes. [`Ctx::training`] and [`Ctx::inference`] pick the mode. A built model is therefore +//! `Send` and `Sync`, and several threads can run inference against 1 model //! //! A layer can hold other layers, which are its sublayers. -//! [`LayerBase::sublayers`](crate::neural_network::traits::LayerBase::sublayers) lists them by -//! name, and the layer calls each sublayer inside -//! [`Ctx::sublayer`](crate::neural_network::Ctx::sublayer). A -//! [`LayerPath`](crate::neural_network::LayerPath) addresses each layer of that tree. The path -//! `2.forward` is the sublayer `forward` of the layer at model position 2. The gradient store, -//! the optimizer state, the state channel, and the checkpoint all key on this path. 2 sublayers -//! of 1 type therefore never share 1 address. A checkpoint path is the layer path and the name of -//! the array, such as `2.forward.kernel` +//! [`LayerBase::sublayers`] lists them by name, and the layer calls each sublayer inside +//! [`Ctx::sublayer`]. A [`LayerPath`] addresses each layer of that tree. The path `2.forward` is +//! the sublayer `forward` of the layer at model position 2. The gradient store, the optimizer +//! state, the state channel, and the checkpoint all key on this path. 2 sublayers of 1 type +//! therefore never share 1 address. A checkpoint path is the layer path and the name of the array, +//! such as `2.forward.kernel` //! //! # Core components //! @@ -58,12 +49,10 @@ //! - **Merge**: `Add`, `Subtract`, `Multiply`, `Average`, `Maximum`, and `Minimum` reduce several //! inputs element by element, and `Concatenate` joins them along 1 axis. Each layer takes //! several inputs and gives 1 output, so a graph model joins its branches with them. -//! [`Concatenate::new`](crate::neural_network::layers::Concatenate::new) never fails, and its -//! axis counts against the full rank. The axis 0 therefore joins the batch axis, and a -//! negative axis counts back from the end. -//! [`Average`](crate::neural_network::layers::Average) is the 1 name that 2 categories of the -//! crate give to an item. The root of [`prelude`](crate::prelude) keeps the averaging mode of -//! the classification scores, so a caller that wants the layer must write +//! [`Concatenate::new`] never fails, and its axis counts against the full rank. The axis 0 +//! therefore joins the batch axis, and a negative axis counts back from the end. [`Average`] is +//! the 1 name that 2 categories of the crate give to an item. The root of [`prelude`] keeps the +//! averaging mode of the classification scores, so a caller that wants the layer must write //! `prelude::neural_network::Average` or `neural_network::layers::Average` //! //! ## Optimizers @@ -79,29 +68,25 @@ //! - **CategoricalCrossEntropy** / **SparseCategoricalCrossEntropy**: multi-class classification //! //! ## Shapes -//! - [`Shape`](crate::neural_network::Shape): the shape of a tensor as a layer describes it, -//! with a free axis for an extent that no configuration fixes. Every layer maps an input -//! shape to an output shape through -//! [`compute_output_shape`](crate::neural_network::traits::UnaryLayer::compute_output_shape), -//! which reads the layer configuration alone and answers before any tensor exists +//! - [`Shape`]: the shape of a tensor as a layer describes it, with a free axis for an extent that +//! no configuration fixes. Every layer maps an input shape to an output shape through +//! [`compute_output_shape`], which reads the layer configuration alone and answers before any +//! tensor exists //! //! ## Weight initialization -//! - [`Initializer`](crate::neural_network::Initializer): how a layer draws the starting values -//! of a weight array. Every drawn array of the module comes from this closed set of 3 rules. -//! The layer supplies the [`Fans`](crate::neural_network::Fans) pair, because a kernel shape -//! alone cannot tell a plain convolution from a transposed one +//! - [`Initializer`]: how a layer draws the starting values of a weight array. Every drawn array of +//! the module comes from this closed set of 3 rules. The layer supplies the [`Fans`] pair, +//! because a kernel shape alone cannot tell a plain convolution from a transposed one //! //! ## Models -//! - [`Sequential`](crate::neural_network::sequential::Sequential): a linear stack of layers with -//! an integrated training loop, prediction, and weight save and load. `fit` and -//! `fit_with_batches` return a [`History`](crate::neural_network::sequential::History) with 1 -//! loss value per epoch. Both build on the public `train_batch` step. `evaluate` scores a model -//! without training it -//! - [`Graph`](crate::neural_network::graph::Graph): a directed graph of layers, with the same -//! training loop, prediction, and weight save and load. It takes several inlets and gives -//! several outlets, and it holds a layer arena. Several nodes can call 1 layer of that arena. -//! Those nodes then share 1 set of arrays, and the sum of their gradients updates that set. -//! [`GraphBuilder`](crate::neural_network::graph::GraphBuilder) is the only way to reach one +//! - [`Sequential`]: a linear stack of layers with an integrated training loop, prediction, and +//! weight save and load. `fit` and `fit_with_batches` return a [`History`] with 1 loss value per +//! epoch. Both build on the public `train_batch` step. `evaluate` scores a model without training +//! it +//! - [`Graph`]: a directed graph of layers, with the same training loop, prediction, and weight +//! save and load. It takes several inlets and gives several outlets, and it holds a layer arena. +//! Several nodes can call 1 layer of that arena. Those nodes then share 1 set of arrays, and the +//! sum of their gradients updates that set. [`GraphBuilder`] is the only way to reach one //! //! # Examples //! @@ -176,6 +161,31 @@ //! let predictions = model.predict(&[&left, &right]).unwrap(); //! assert_eq!(predictions[0].shape(), &[4, 2]); //! ``` +//! +//! [`layers`]: crate::neural_network::layers +//! [`optimizers`]: crate::neural_network::optimizers +//! [`losses`]: crate::neural_network::losses +//! [`Sequential`]: crate::neural_network::sequential::Sequential +//! [`Graph`]: crate::neural_network::graph::Graph +//! [`traits`]: crate::neural_network::traits +//! [`Tensor`]: crate::neural_network::Tensor +//! [`forward`]: crate::neural_network::traits::UnaryLayer::forward +//! [`backward`]: crate::neural_network::traits::UnaryLayer::backward +//! [`Ctx`]: crate::neural_network::Ctx +//! [`Ctx::training`]: crate::neural_network::Ctx::training +//! [`Ctx::inference`]: crate::neural_network::Ctx::inference +//! [`LayerBase::sublayers`]: crate::neural_network::traits::LayerBase::sublayers +//! [`Ctx::sublayer`]: crate::neural_network::Ctx::sublayer +//! [`LayerPath`]: crate::neural_network::LayerPath +//! [`Concatenate::new`]: crate::neural_network::layers::Concatenate::new +//! [`Average`]: crate::neural_network::layers::Average +//! [`prelude`]: crate::prelude +//! [`Shape`]: crate::neural_network::Shape +//! [`compute_output_shape`]: crate::neural_network::traits::UnaryLayer::compute_output_shape +//! [`Initializer`]: crate::neural_network::Initializer +//! [`Fans`]: crate::neural_network::Fans +//! [`History`]: crate::neural_network::sequential::History +//! [`GraphBuilder`]: crate::neural_network::graph::GraphBuilder use ndarray::ArrayD; @@ -184,7 +194,7 @@ pub type Tensor = ArrayD; /// The per-pass context that carries caches, gradients and state between layers pub mod ctx; -/// Neural-network error type, aggregated into the crate-wide [`Error`](crate::error::Error) +/// Neural-network error type, aggregated into the crate-wide `Error` pub mod error; /// A model whose layers form a directed graph, rather than a chain pub mod graph; diff --git a/src/neural_network/ctx.rs b/src/neural_network/ctx.rs index 4cbd1ee3..655c409f 100644 --- a/src/neural_network/ctx.rs +++ b/src/neural_network/ctx.rs @@ -1,7 +1,7 @@ //! The per-pass context that carries the values a forward pass and its backward pass share //! //! A layer computes. It does not remember. -//! [`UnaryLayer::forward`](crate::neural_network::traits::UnaryLayer::forward) takes `&self`, +//! [`UnaryLayer::forward`] takes `&self`, //! so the values that only exist between a forward pass and its backward pass have no place //! inside the layer. [`Ctx`] is that place. 1 context serves 1 pass, and it holds 4 channels: //! @@ -18,7 +18,7 @@ //! # Sublayers //! //! A layer can hold other layers. See -//! [`LayerBase`](crate::neural_network::traits::LayerBase) for that contract. A layer calls +//! [`LayerBase`] for that contract. A layer calls //! each of its sublayers inside [`Ctx::sublayer`]. The call appends 1 frame to the current //! [`LayerPath`] for the time of the closure. It removes the frame when the closure returns, //! and also when the closure panics. Each channel then keys on the full path: @@ -34,7 +34,9 @@ //! each record against the layer tree, so a call under the name of another sublayer stops the //! step. //! +//! [`LayerBase`]: crate::neural_network::traits::LayerBase //! [`ParamId`]: crate::neural_network::traits::ParamId +//! [`UnaryLayer::forward`]: crate::neural_network::traits::UnaryLayer::forward use crate::error::Error; use crate::neural_network::Tensor; diff --git a/src/neural_network/error.rs b/src/neural_network/error.rs index b3214c14..4c8d48ce 100644 --- a/src/neural_network/error.rs +++ b/src/neural_network/error.rs @@ -2,9 +2,12 @@ //! //! [`NnError`] enumerates the failures specific to the neural-network framework (layer state, //! weight shapes, model compilation). Callers receive it through the crate-wide -//! [`Error::NeuralNetwork`](crate::error::Error::NeuralNetwork) variant, into which it +//! [`Error::NeuralNetwork`] variant, into which it //! converts via `?` (a `#[from]` bridge). See [`crate::error`] for the unified -//! [`Error`](crate::error::Error) that aggregates the per-domain error enums +//! [`Error`] that aggregates the per-domain error enums +//! +//! [`Error::NeuralNetwork`]: crate::error::Error::NeuralNetwork +//! [`Error`]: crate::error::Error use crate::error::Error; diff --git a/src/neural_network/graph.rs b/src/neural_network/graph.rs index f033ce7b..59315966 100644 --- a/src/neural_network/graph.rs +++ b/src/neural_network/graph.rs @@ -1,9 +1,9 @@ //! A model whose layers form a directed graph, rather than a chain //! -//! [`Sequential`](crate::neural_network::sequential::Sequential) gives every layer exactly 1 +//! [`Sequential`] gives every layer exactly 1 //! input, and that input is the output of the layer before it. A residual connection, a model //! with 2 towers, a shared encoder, and any model with several inlets or several outlets need -//! more than a chain. [`Graph`](crate::neural_network::graph::Graph) is that model +//! more than a chain. [`Graph`] is that model //! //! A graph holds a layer arena and a node list. A node is a call of 1 layer on the outputs of //! other nodes, so several nodes can call 1 layer. That is what weight sharing is: 1 set of @@ -37,6 +37,9 @@ //! let prediction = model.predict(&[&inputs]).unwrap(); //! assert_eq!(prediction[0].shape(), &[4, 2]); //! ``` +//! +//! [`Sequential`]: crate::neural_network::sequential::Sequential +//! [`Graph`]: crate::neural_network::graph::Graph use crate::error::Error; use crate::neural_network::NnError; diff --git a/src/neural_network/initializer.rs b/src/neural_network/initializer.rs index ef438b95..97c5668c 100644 --- a/src/neural_network/initializer.rs +++ b/src/neural_network/initializer.rs @@ -1,7 +1,7 @@ //! Weight initializers: the value that decides how a layer draws a new array //! //! [`Initializer`] is a closed enum, in the same shape as the -//! [`Activation`](crate::neural_network::layers::Activation) enum. It is `Copy`, it compares by +//! [`Activation`] enum. It is `Copy`, it compares by //! value, and it serializes, so a layer holds it by value and a checkpoint can record it. //! //! 13 functions of the module draw a weight array, and all 13 draw through this type. 14 of @@ -20,16 +20,21 @@ //! # The draw order is part of the contract //! //! 5 layers thread 1 generator through more than 1 draw. -//! [`SeparableConv1D`](crate::neural_network::layers::SeparableConv1D) and -//! [`SeparableConv2D`](crate::neural_network::layers::SeparableConv2D) draw the depthwise kernel +//! [`SeparableConv1D`] and +//! [`SeparableConv2D`] draw the depthwise kernel //! first and the pointwise kernel second. -//! [`FusedGates`](crate::neural_network::layers::recurrent::gate::FusedGates) draws the fused +//! [`FusedGates`] draws the fused //! input kernel first and then 1 orthogonal block per gate, in gate order, for the 3 recurrent //! layers. //! //! 1 generator gives 1 stream, and each draw takes the next values of that stream. A different //! order changes every value from the second draw onward. Keep the draws of 1 layer in 1 //! function, against 1 generator. +//! +//! [`Activation`]: crate::neural_network::layers::Activation +//! [`SeparableConv1D`]: crate::neural_network::layers::SeparableConv1D +//! [`SeparableConv2D`]: crate::neural_network::layers::SeparableConv2D +//! [`FusedGates`]: crate::neural_network::layers::recurrent::gate::FusedGates use crate::{Deserialize, Serialize}; use ndarray::{Array, Array2, Dimension, ShapeBuilder}; diff --git a/src/neural_network/layers.rs b/src/neural_network/layers.rs index 9674f6b8..c60381b1 100644 --- a/src/neural_network/layers.rs +++ b/src/neural_network/layers.rs @@ -2,7 +2,7 @@ //! //! Declares every layer submodule and glob-re-exports the public layer types. It also defines //! the infrastructure shared across the subsystem. The -//! [`ParamCounts`](crate::neural_network::layers::ParamCounts) report says how many parameter +//! [`ParamCounts`] report says how many parameter //! elements a layer holds, split into trainable and non-trainable. //! //! 5 macros remove the boilerplate that a layer would otherwise repeat by hand. @@ -21,29 +21,49 @@ //! The submodules fall into a few categories: //! //! - Core layers (re-exported): -//! - [`activation`](crate::neural_network::layers::activation) -//! - [`border`](crate::neural_network::layers::border) -//! - [`convolution`](crate::neural_network::layers::convolution) -//! - [`dense`](crate::neural_network::layers::dense) -//! - [`embedding`](crate::neural_network::layers::embedding) -//! - [`flatten`](crate::neural_network::layers::flatten) -//! - [`identity`](crate::neural_network::layers::identity) -//! - [`merge`](crate::neural_network::layers::merge) -//! - [`permute`](crate::neural_network::layers::permute) -//! - [`pooling`](crate::neural_network::layers::pooling) -//! - [`recurrent`](crate::neural_network::layers::recurrent) -//! - [`regularization`](crate::neural_network::layers::regularization) -//! - [`repeat_vector`](crate::neural_network::layers::repeat_vector) -//! - [`rescaling`](crate::neural_network::layers::rescaling) -//! - [`reshape`](crate::neural_network::layers::reshape) -//! - [`reverse`](crate::neural_network::layers::reverse) -//! - [`upsampling`](crate::neural_network::layers::upsampling) +//! - [`activation`] +//! - [`border`] +//! - [`convolution`] +//! - [`dense`] +//! - [`embedding`] +//! - [`flatten`] +//! - [`identity`] +//! - [`merge`] +//! - [`permute`] +//! - [`pooling`] +//! - [`recurrent`] +//! - [`regularization`] +//! - [`repeat_vector`] +//! - [`rescaling`] +//! - [`reshape`] +//! - [`reverse`] +//! - [`upsampling`] //! - Shared (private) helpers: `conv_op_helpers` (the depthwise convolution kernel, shared by //! the depthwise and separable convolution layers) and `shape_helpers` (pooling/convolution //! output-shape calculators) //! - Validation: `validation` (shared input/weight checks) -//! - Serialization: [`checkpoint`](crate::neural_network::layers::checkpoint) +//! - Serialization: [`checkpoint`] //! (the named on-disk format, and the load that applies it) +//! +//! [`ParamCounts`]: crate::neural_network::layers::ParamCounts +//! [`activation`]: crate::neural_network::layers::activation +//! [`border`]: crate::neural_network::layers::border +//! [`convolution`]: crate::neural_network::layers::convolution +//! [`dense`]: crate::neural_network::layers::dense +//! [`embedding`]: crate::neural_network::layers::embedding +//! [`flatten`]: crate::neural_network::layers::flatten +//! [`identity`]: crate::neural_network::layers::identity +//! [`merge`]: crate::neural_network::layers::merge +//! [`permute`]: crate::neural_network::layers::permute +//! [`pooling`]: crate::neural_network::layers::pooling +//! [`recurrent`]: crate::neural_network::layers::recurrent +//! [`regularization`]: crate::neural_network::layers::regularization +//! [`repeat_vector`]: crate::neural_network::layers::repeat_vector +//! [`rescaling`]: crate::neural_network::layers::rescaling +//! [`reshape`]: crate::neural_network::layers::reshape +//! [`reverse`]: crate::neural_network::layers::reverse +//! [`upsampling`]: crate::neural_network::layers::upsampling +//! [`checkpoint`]: crate::neural_network::layers::checkpoint /// How many parameter elements a layer holds, split by whether training updates them /// diff --git a/src/neural_network/layers/activation/p_relu.rs b/src/neural_network/layers/activation/p_relu.rs index 421c9ad4..2b6c51ca 100644 --- a/src/neural_network/layers/activation/p_relu.rs +++ b/src/neural_network/layers/activation/p_relu.rs @@ -3,15 +3,17 @@ //! Defines the [`PReLU`] struct, which holds 1 negative-side slope per position of its build //! shape. [`PReLU::with_shared_axes`] then names axes that share 1 slope across a group of //! positions. Every other activation layer in this crate delegates its math to the shared -//! [`Activation`](crate::neural_network::layers::activation::Activation) enum. `PReLU` holds its -//! own forward and backward transforms instead, because a trainable array carries state that the -//! enum's parameter-free variants do not. +//! [`Activation`] enum. `PReLU` holds its own forward and backward transforms instead, because a +//! trainable array carries state that the enum's parameter-free variants do not. //! -//! The [`LayerBase`](crate::neural_network::traits::LayerBase) and -//! [`UnaryLayer`](crate::neural_network::traits::UnaryLayer) trait implementations at the bottom of -//! the module wire the layer into the rest of a model. They handle shape inference and validation, -//! parameter registration for the optimizer, and the forward and backward passes a model calls -//! during training and inference. +//! The [`LayerBase`] and [`UnaryLayer`] trait implementations at the bottom of the module wire the +//! layer into the rest of a model. They handle shape inference and validation, parameter +//! registration for the optimizer, and the forward and backward passes a model calls during +//! training and inference. +//! +//! [`Activation`]: crate::neural_network::layers::activation::Activation +//! [`LayerBase`]: crate::neural_network::traits::LayerBase +//! [`UnaryLayer`]: crate::neural_network::traits::UnaryLayer use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/cropping_1d.rs b/src/neural_network/layers/border/cropping_1d.rs index 74676610..18ff93fe 100644 --- a/src/neural_network/layers/border/cropping_1d.rs +++ b/src/neural_network/layers/border/cropping_1d.rs @@ -3,7 +3,9 @@ //! [`Cropping1D`] removes steps from the step axis of a `[batch, steps, features]` tensor. It //! holds no parameter. The forward pass and the backward pass both run through //! `pad_crop_engine`, which also backs this layer's inverse, -//! [`ZeroPadding1D`](crate::neural_network::layers::border::ZeroPadding1D). +//! [`ZeroPadding1D`]. +//! +//! [`ZeroPadding1D`]: crate::neural_network::layers::border::ZeroPadding1D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/cropping_2d.rs b/src/neural_network/layers/border/cropping_2d.rs index 738ad03c..cccb6e0d 100644 --- a/src/neural_network/layers/border/cropping_2d.rs +++ b/src/neural_network/layers/border/cropping_2d.rs @@ -3,7 +3,9 @@ //! [`Cropping2D`] removes rows and columns from the height axis and the width axis of a //! `[batch, height, width, channels]` tensor. It holds no parameter. The forward pass and the //! backward pass both run through `pad_crop_engine`, which also backs this layer's inverse, -//! [`ZeroPadding2D`](crate::neural_network::layers::border::ZeroPadding2D). +//! [`ZeroPadding2D`]. +//! +//! [`ZeroPadding2D`]: crate::neural_network::layers::border::ZeroPadding2D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/cropping_3d.rs b/src/neural_network/layers/border/cropping_3d.rs index 68950361..a4efa943 100644 --- a/src/neural_network/layers/border/cropping_3d.rs +++ b/src/neural_network/layers/border/cropping_3d.rs @@ -3,7 +3,9 @@ //! [`Cropping3D`] removes planes from the depth axis, the height axis, and the width axis of a //! `[batch, depth, height, width, channels]` tensor. It holds no parameter. The forward pass and //! the backward pass both run through `pad_crop_engine`, which also backs this layer's inverse, -//! [`ZeroPadding3D`](crate::neural_network::layers::border::ZeroPadding3D). +//! [`ZeroPadding3D`]. +//! +//! [`ZeroPadding3D`]: crate::neural_network::layers::border::ZeroPadding3D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/zero_padding_1d.rs b/src/neural_network/layers/border/zero_padding_1d.rs index 4ae585a1..d262821a 100644 --- a/src/neural_network/layers/border/zero_padding_1d.rs +++ b/src/neural_network/layers/border/zero_padding_1d.rs @@ -3,7 +3,9 @@ //! [`ZeroPadding1D`] adds zero steps to the step axis of a `[batch, steps, features]` tensor. It //! holds no parameter. The forward pass and the backward pass both run through //! `pad_crop_engine`, which also backs this layer's inverse, -//! [`Cropping1D`](crate::neural_network::layers::border::Cropping1D). +//! [`Cropping1D`]. +//! +//! [`Cropping1D`]: crate::neural_network::layers::border::Cropping1D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/zero_padding_2d.rs b/src/neural_network/layers/border/zero_padding_2d.rs index ee13bb76..aed2661b 100644 --- a/src/neural_network/layers/border/zero_padding_2d.rs +++ b/src/neural_network/layers/border/zero_padding_2d.rs @@ -3,7 +3,9 @@ //! [`ZeroPadding2D`] adds zero rows and columns to the height axis and the width axis of a //! `[batch, height, width, channels]` tensor. It holds no parameter. The forward pass and the //! backward pass both run through `pad_crop_engine`, which also backs this layer's inverse, -//! [`Cropping2D`](crate::neural_network::layers::border::Cropping2D). +//! [`Cropping2D`]. +//! +//! [`Cropping2D`]: crate::neural_network::layers::border::Cropping2D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/border/zero_padding_3d.rs b/src/neural_network/layers/border/zero_padding_3d.rs index 27cef6a0..c35fcfaf 100644 --- a/src/neural_network/layers/border/zero_padding_3d.rs +++ b/src/neural_network/layers/border/zero_padding_3d.rs @@ -3,7 +3,9 @@ //! [`ZeroPadding3D`] adds zero planes to the depth axis, the height axis, and the width axis of //! a `[batch, depth, height, width, channels]` tensor. It holds no parameter. The forward pass //! and the backward pass both run through `pad_crop_engine`, which also backs this layer's -//! inverse, [`Cropping3D`](crate::neural_network::layers::border::Cropping3D). +//! inverse, [`Cropping3D`]. +//! +//! [`Cropping3D`]: crate::neural_network::layers::border::Cropping3D use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/checkpoint.rs b/src/neural_network/layers/checkpoint.rs index 863a535b..e0bcbff0 100644 --- a/src/neural_network/layers/checkpoint.rs +++ b/src/neural_network/layers/checkpoint.rs @@ -4,7 +4,7 @@ //! `.`: the [`LayerPath`] of the layer that holds the array, and the name that //! the layer gives the array. A layer at model position 2 gives the path `2.kernel`. A sublayer //! `forward` of that layer gives the path `2.forward.kernel`. It is the same pair that -//! [`ParamId`](crate::neural_network::traits::ParamId) uses for optimizer state, so 1 address +//! [`ParamId`] uses for optimizer state, so 1 address //! serves the training loop and the file alike //! //! # What a file holds @@ -42,6 +42,7 @@ //! [`apply`]: crate::neural_network::layers::checkpoint::apply //! [`apply_partial`]: crate::neural_network::layers::checkpoint::apply_partial //! [`BatchNormalization`]: crate::neural_network::layers::BatchNormalization +//! [`ParamId`]: crate::neural_network::traits::ParamId use crate::error::{Error, IoError}; use crate::neural_network::Shape; diff --git a/src/neural_network/layers/convolution.rs b/src/neural_network/layers/convolution.rs index 773742b3..0d80335a 100644 --- a/src/neural_network/layers/convolution.rs +++ b/src/neural_network/layers/convolution.rs @@ -2,8 +2,10 @@ //! //! Re-exports the 1D, 2D, and 3D convolution layers, their transposed counterparts, and the //! depthwise and separable convolution layers in 1D and 2D. Defines -//! [`PaddingType`](crate::neural_network::layers::convolution::PaddingType), which controls spatial -//! padding, and re-exports [`ConvPadding`], which adds the causal rule that only [`Conv1D`] accepts +//! [`PaddingType`], which controls spatial padding, and re-exports [`ConvPadding`], which adds the +//! causal rule that only [`Conv1D`] accepts +//! +//! [`PaddingType`]: crate::neural_network::layers::convolution::PaddingType /// Padding method used by convolutional and pooling layers /// diff --git a/src/neural_network/layers/convolution/convolution_engine.rs b/src/neural_network/layers/convolution/convolution_engine.rs index a2214063..03ad5909 100644 --- a/src/neural_network/layers/convolution/convolution_engine.rs +++ b/src/neural_network/layers/convolution/convolution_engine.rs @@ -48,9 +48,11 @@ //! //! The backward pass parallelizes over batch items. It reduces their weight and bias partials in //! batch order, so rerunning on the same machine gives the same result. Both GEMMs route through -//! the crate's [`dot_par`](crate::math::matmul::dot_par). The per-item GEMMs stay parallel while +//! the crate's [`dot_par`]. The per-item GEMMs stay parallel while //! the batch fan is too short to fill the thread pool. They switch to serial once the batch alone //! fills the pool, so a batch task does not fork rayon again inside its own GEMM +//! +//! [`dot_par`]: crate::math::matmul::dot_par use super::PaddingType; use super::validation::validate_valid_kernel_fits; diff --git a/src/neural_network/layers/merge.rs b/src/neural_network/layers/merge.rs index 618a2011..a0fa4d96 100644 --- a/src/neural_network/layers/merge.rs +++ b/src/neural_network/layers/merge.rs @@ -3,8 +3,8 @@ //! [`Add`], [`Subtract`], [`Multiply`], [`Average`], [`Maximum`], and [`Minimum`] reduce their //! inputs element by element. [`Concatenate`] joins its inputs along 1 axis. No layer of the //! family holds a trainable array, and each of them implements -//! [`Layer`](crate::neural_network::traits::Layer) by hand. -//! [`UnaryLayer`](crate::neural_network::traits::UnaryLayer) takes 1 input alone, so no layer +//! [`Layer`] by hand. +//! [`UnaryLayer`] takes 1 input alone, so no layer //! of this family can implement it //! //! Each layer records `built: Option>`, which holds 1 shape per input. The layer @@ -44,6 +44,9 @@ //! shape algebra of the 6 elementwise layers. //! - [`Concatenate`] takes the first macro alone, and it calls //! `validation::start_build_many` from its own `build_many` +//! +//! [`Layer`]: crate::neural_network::traits::Layer +//! [`UnaryLayer`]: crate::neural_network::traits::UnaryLayer use crate::error::{Context, Error}; use crate::neural_network::layers::validation::start_build_many; diff --git a/src/neural_network/layers/pooling/average_pooling_1d.rs b/src/neural_network/layers/pooling/average_pooling_1d.rs index 40dd4a3b..26791822 100644 --- a/src/neural_network/layers/pooling/average_pooling_1d.rs +++ b/src/neural_network/layers/pooling/average_pooling_1d.rs @@ -3,10 +3,12 @@ //! [`AveragePooling1D`] slides a window along the length axis of a `[batch, length, channels]` //! tensor and replaces each window with the mean of its elements. [`AveragePooling1D::new`] sets //! the stride to `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`AveragePooling1D::with_stride`] and [`AveragePooling1D::with_padding`] override those //! defaults. The backward pass spreads each output gradient evenly over the in-bounds elements of //! its window. The forward pass therefore caches only the input shape it must restore. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/pooling/average_pooling_2d.rs b/src/neural_network/layers/pooling/average_pooling_2d.rs index 1d1c7750..d6774ed7 100644 --- a/src/neural_network/layers/pooling/average_pooling_2d.rs +++ b/src/neural_network/layers/pooling/average_pooling_2d.rs @@ -3,10 +3,12 @@ //! [`AveragePooling2D`] slides a window across the height and width axes of a `[batch, height, //! width, channels]` tensor. The layer replaces each window with the mean of its elements. //! [`AveragePooling2D::new`] sets the strides to `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`AveragePooling2D::with_strides`] and [`AveragePooling2D::with_padding`] override those //! defaults. The backward pass spreads each output gradient evenly over the in-bounds elements of //! its window. The forward pass therefore caches only the input shape it must restore. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/pooling/average_pooling_3d.rs b/src/neural_network/layers/pooling/average_pooling_3d.rs index cae284f8..cd523959 100644 --- a/src/neural_network/layers/pooling/average_pooling_3d.rs +++ b/src/neural_network/layers/pooling/average_pooling_3d.rs @@ -3,10 +3,12 @@ //! [`AveragePooling3D`] slides a window across the depth, height, and width axes of a `[batch, //! depth, height, width, channels]` tensor. The layer replaces each window with the mean of its //! elements. [`AveragePooling3D::new`] sets the strides to `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`AveragePooling3D::with_strides`] and [`AveragePooling3D::with_padding`] override those //! defaults. The backward pass spreads each output gradient evenly over the in-bounds elements of //! its window. The forward pass therefore caches only the input shape it must restore. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/pooling/max_pooling_1d.rs b/src/neural_network/layers/pooling/max_pooling_1d.rs index 520fec6d..86113774 100644 --- a/src/neural_network/layers/pooling/max_pooling_1d.rs +++ b/src/neural_network/layers/pooling/max_pooling_1d.rs @@ -3,10 +3,12 @@ //! [`MaxPooling1D`] slides a window along the length axis of a `[batch, length, channels]` tensor //! and keeps the largest element of each window. [`MaxPooling1D::new`] sets the stride to //! `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`MaxPooling1D::with_stride`] and [`MaxPooling1D::with_padding`] override those defaults. The //! forward pass records the arg-max of every window. The backward pass then routes each output //! gradient back to the input element that produced it. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/pooling/max_pooling_2d.rs b/src/neural_network/layers/pooling/max_pooling_2d.rs index 8dd876fa..ca159817 100644 --- a/src/neural_network/layers/pooling/max_pooling_2d.rs +++ b/src/neural_network/layers/pooling/max_pooling_2d.rs @@ -3,10 +3,12 @@ //! [`MaxPooling2D`] slides a window across the height and width axes of a `[batch, height, width, //! channels]` tensor and keeps the largest element of each window. [`MaxPooling2D::new`] sets the //! strides to `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`MaxPooling2D::with_strides`] and [`MaxPooling2D::with_padding`] override those defaults. The //! forward pass records the arg-max of every window. The backward pass then routes each output //! gradient back to the input element that produced it. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/pooling/max_pooling_3d.rs b/src/neural_network/layers/pooling/max_pooling_3d.rs index 0407ac47..bd6f2024 100644 --- a/src/neural_network/layers/pooling/max_pooling_3d.rs +++ b/src/neural_network/layers/pooling/max_pooling_3d.rs @@ -3,10 +3,12 @@ //! [`MaxPooling3D`] slides a window across the depth, height, and width axes of a `[batch, depth, //! height, width, channels]` tensor. The layer keeps the largest element of each window. //! [`MaxPooling3D::new`] sets the strides to `pool_size` and the padding to -//! [`PaddingType::Valid`](crate::neural_network::layers::convolution::PaddingType). +//! [`PaddingType::Valid`]. //! [`MaxPooling3D::with_strides`] and [`MaxPooling3D::with_padding`] override those defaults. The //! forward pass records the arg-max of every window. The backward pass then routes each output //! gradient back to the input element that produced it. +//! +//! [`PaddingType::Valid`]: crate::neural_network::layers::convolution::PaddingType use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/recurrent.rs b/src/neural_network/layers/recurrent.rs index a0e97005..01a71887 100644 --- a/src/neural_network/layers/recurrent.rs +++ b/src/neural_network/layers/recurrent.rs @@ -13,7 +13,9 @@ //! This module also provides 2 shared helpers. A numerically stable sigmoid serves the GRU and //! LSTM gates, and 1 index map serves the `go_backwards` option of all 3 layers. All 3 layers //! draw their recurrent kernels from -//! [`Initializer::Orthogonal`](crate::neural_network::Initializer::Orthogonal). +//! [`Initializer::Orthogonal`]. +//! +//! [`Initializer::Orthogonal`]: crate::neural_network::Initializer::Orthogonal use ndarray::Array2; diff --git a/src/neural_network/layers/regularization.rs b/src/neural_network/layers/regularization.rs index 6a30bc55..3a6878fb 100644 --- a/src/neural_network/layers/regularization.rs +++ b/src/neural_network/layers/regularization.rs @@ -4,25 +4,31 @@ //! of parameter and input-shape checks shared across the layers. //! //! The families are: -//! - dropout: [`Dropout`](crate::neural_network::layers::Dropout) and the spatial variants -//! [`SpatialDropout1D`](crate::neural_network::layers::SpatialDropout1D), -//! [`SpatialDropout2D`](crate::neural_network::layers::SpatialDropout2D), and -//! [`SpatialDropout3D`](crate::neural_network::layers::SpatialDropout3D) -//! - noise injection: [`GaussianNoise`](crate::neural_network::layers::GaussianNoise) and -//! [`GaussianDropout`](crate::neural_network::layers::GaussianDropout) -//! - normalization: [`BatchNormalization`](crate::neural_network::layers::BatchNormalization), -//! [`LayerNormalization`](crate::neural_network::layers::LayerNormalization), -//! [`GroupNormalization`](crate::neural_network::layers::GroupNormalization), -//! [`InstanceNormalization`](crate::neural_network::layers::InstanceNormalization), and -//! [`UnitNormalization`](crate::neural_network::layers::UnitNormalization) -//! +//! - dropout: [`Dropout`] and the spatial variants [`SpatialDropout1D`], [`SpatialDropout2D`], and +//! [`SpatialDropout3D`] +//! - noise injection: [`GaussianNoise`] and [`GaussianDropout`] +//! - normalization: [`BatchNormalization`], [`LayerNormalization`], [`GroupNormalization`], +//! [`InstanceNormalization`], and [`UnitNormalization`] //! Every dropout layer, every noise layer, and `BatchNormalization` change what forward computes //! between training and inference. `LayerNormalization`, `GroupNormalization`, //! `InstanceNormalization`, and `UnitNormalization` give the same output either way. Each one //! takes its statistic from a single sample, never from the batch. The mode is not a field of a -//! layer. It is the training flag of the [`Ctx`](crate::neural_network::Ctx) that the pass -//! carries, and every layer of this module reads it with -//! [`Ctx::is_training`](crate::neural_network::Ctx::is_training) +//! layer. It is the training flag of the [`Ctx`] that the pass carries, and every layer of this +//! module reads it with [`Ctx::is_training`] +//! +//! [`Dropout`]: crate::neural_network::layers::Dropout +//! [`SpatialDropout1D`]: crate::neural_network::layers::SpatialDropout1D +//! [`SpatialDropout2D`]: crate::neural_network::layers::SpatialDropout2D +//! [`SpatialDropout3D`]: crate::neural_network::layers::SpatialDropout3D +//! [`GaussianNoise`]: crate::neural_network::layers::GaussianNoise +//! [`GaussianDropout`]: crate::neural_network::layers::GaussianDropout +//! [`BatchNormalization`]: crate::neural_network::layers::BatchNormalization +//! [`LayerNormalization`]: crate::neural_network::layers::LayerNormalization +//! [`GroupNormalization`]: crate::neural_network::layers::GroupNormalization +//! [`InstanceNormalization`]: crate::neural_network::layers::InstanceNormalization +//! [`UnitNormalization`]: crate::neural_network::layers::UnitNormalization +//! [`Ctx`]: crate::neural_network::Ctx +//! [`Ctx::is_training`]: crate::neural_network::Ctx::is_training /// Dropout layers for neural networks pub mod dropout; diff --git a/src/neural_network/layers/regularization/dropout.rs b/src/neural_network/layers/regularization/dropout.rs index 4c39f3b2..fbee83d1 100644 --- a/src/neural_network/layers/regularization/dropout.rs +++ b/src/neural_network/layers/regularization/dropout.rs @@ -1,14 +1,9 @@ //! Dropout-family regularization layers and the shared helpers that build them //! //! Re-exports the plain -//! [`Dropout`](crate::neural_network::layers::regularization::dropout::Dropout) layer and its -//! spatial variants -//! [`SpatialDropout1D`](crate::neural_network::layers::regularization::dropout::SpatialDropout1D), -//! [`SpatialDropout2D`](crate::neural_network::layers::regularization::dropout::SpatialDropout2D), -//! and -//! [`SpatialDropout3D`](crate::neural_network::layers::regularization::dropout::SpatialDropout3D). -//! Each layer has its own submodule: `dropout`, `spatial_dropout_1d`, `spatial_dropout_2d`, and -//! `spatial_dropout_3d`. +//! [`Dropout`] layer and its spatial variants [`SpatialDropout1D`], [`SpatialDropout2D`], and +//! [`SpatialDropout3D`]. Each layer has its own submodule: `dropout`, `spatial_dropout_1d`, +//! `spatial_dropout_2d`, and `spatial_dropout_3d`. //! //! Defines the infrastructure these layers share: //! @@ -20,6 +15,11 @@ //! - `spatial_dropout_scale` and `spatial_dropout_backward` - apply the per-channel //! inverted-dropout scale to a `[batch, *spatial, channels]` tensor from a small //! `[batch, channels]` mask without building a full-size mask +//! +//! [`Dropout`]: crate::neural_network::layers::regularization::dropout::Dropout +//! [`SpatialDropout1D`]: crate::neural_network::layers::regularization::dropout::SpatialDropout1D +//! [`SpatialDropout2D`]: crate::neural_network::layers::regularization::dropout::SpatialDropout2D +//! [`SpatialDropout3D`]: crate::neural_network::layers::regularization::dropout::SpatialDropout3D use crate::error::Error; use crate::neural_network::Tensor; diff --git a/src/neural_network/layers/regularization/dropout/dropout.rs b/src/neural_network/layers/regularization/dropout/dropout.rs index bf4ca905..a340316a 100644 --- a/src/neural_network/layers/regularization/dropout/dropout.rs +++ b/src/neural_network/layers/regularization/dropout/dropout.rs @@ -1,15 +1,18 @@ //! Dropout layer that randomly zeroes input units during training to reduce overfitting //! //! [`Dropout`] holds the drop rate, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), an optional +//! [`UnaryLayer::build`], an optional //! `noise_shape` that makes the sampled mask coarser than the input, and the random number //! generator that draws it. By default the layer draws 1 independent value per input element. //! [`Dropout::with_noise_shape`] shares 1 draw across a chosen axis instead, and //! [`Dropout::with_random_state`] reseeds the generator for reproducible masks. //! //! The forward pass samples a keep/drop mask at the resolved noise shape, scales the kept elements -//! by `1 / (1 - rate)`, and parks the mask in the [`Ctx`](crate::neural_network::Ctx) cache for +//! by `1 / (1 - rate)`, and parks the mask in the [`Ctx`] cache for //! `dropout_backward` to reuse on the backward pass. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build +//! [`Ctx`]: crate::neural_network::Ctx use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/dropout/spatial_dropout_1d.rs b/src/neural_network/layers/regularization/dropout/spatial_dropout_1d.rs index d5d91fdb..7e79617a 100644 --- a/src/neural_network/layers/regularization/dropout/spatial_dropout_1d.rs +++ b/src/neural_network/layers/regularization/dropout/spatial_dropout_1d.rs @@ -1,13 +1,15 @@ //! 1D spatial dropout layer that drops whole channels of `(batch_size, length, channels)` inputs //! //! [`SpatialDropout1D`] holds the drop rate, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), and the random number +//! [`UnaryLayer::build`], and the random number //! generator that draws the per-channel mask. [`SpatialDropout1D::with_random_state`] reseeds that //! generator for reproducible masks. //! //! The forward pass samples 1 keep/drop value per `(batch, channel)` pair, scales the kept //! channels by `1 / (1 - rate)` with `spatial_dropout_scale`, and parks the small mask for //! `spatial_dropout_backward` to reuse on the backward pass. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/dropout/spatial_dropout_2d.rs b/src/neural_network/layers/regularization/dropout/spatial_dropout_2d.rs index 1683021d..9778fbf3 100644 --- a/src/neural_network/layers/regularization/dropout/spatial_dropout_2d.rs +++ b/src/neural_network/layers/regularization/dropout/spatial_dropout_2d.rs @@ -2,13 +2,15 @@ //! channels)` inputs //! //! [`SpatialDropout2D`] holds the drop rate, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), and the random number +//! [`UnaryLayer::build`], and the random number //! generator that draws the per-channel mask. [`SpatialDropout2D::with_random_state`] reseeds that //! generator for reproducible masks. //! //! The forward pass samples 1 keep/drop value per `(batch, channel)` pair, scales the kept //! channels by `1 / (1 - rate)` with `spatial_dropout_scale`, and parks the small mask for //! `spatial_dropout_backward` to reuse on the backward pass. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/dropout/spatial_dropout_3d.rs b/src/neural_network/layers/regularization/dropout/spatial_dropout_3d.rs index 5402958b..d8b41033 100644 --- a/src/neural_network/layers/regularization/dropout/spatial_dropout_3d.rs +++ b/src/neural_network/layers/regularization/dropout/spatial_dropout_3d.rs @@ -2,13 +2,15 @@ //! width, channels)` inputs //! //! [`SpatialDropout3D`] holds the drop rate, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), and the random number +//! [`UnaryLayer::build`], and the random number //! generator that draws the per-channel mask. [`SpatialDropout3D::with_random_state`] reseeds that //! generator for reproducible masks. //! //! The forward pass samples 1 keep/drop value per `(batch, channel)` pair, scales the kept //! channels by `1 / (1 - rate)` with `spatial_dropout_scale`, and parks the small mask for //! `spatial_dropout_backward` to reuse on the backward pass. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/noise_injection.rs b/src/neural_network/layers/regularization/noise_injection.rs index 5792714d..760d6a9d 100644 --- a/src/neural_network/layers/regularization/noise_injection.rs +++ b/src/neural_network/layers/regularization/noise_injection.rs @@ -1,14 +1,17 @@ //! Noise-injection regularization layers //! //! Groups the 2 noise-injection layers and re-exports them. Multiplicative -//! [`GaussianDropout`](crate::neural_network::layers::GaussianDropout) scales inputs by -//! `N(1, rate/(1 - rate))` during training. Additive -//! [`GaussianNoise`](crate::neural_network::layers::GaussianNoise) adds zero-mean `N(0, stddev^2)` -//! noise during training. Both are identity maps at inference. +//! [`GaussianDropout`] scales inputs by `N(1, rate/(1 - rate))` during training. Additive +//! [`GaussianNoise`] adds zero-mean `N(0, stddev^2)` noise during training. Both are identity maps +//! at inference. //! //! This file defines no shared infrastructure of its own. The 2 layers read the training flag -//! of the [`Ctx`](crate::neural_network::Ctx), and they reuse the validation helpers from the -//! parent [`regularization`](crate::neural_network::layers::regularization) module +//! of the [`Ctx`], and they reuse the validation helpers from the parent [`regularization`] module +//! +//! [`GaussianDropout`]: crate::neural_network::layers::GaussianDropout +//! [`GaussianNoise`]: crate::neural_network::layers::GaussianNoise +//! [`Ctx`]: crate::neural_network::Ctx +//! [`regularization`]: crate::neural_network::layers::regularization /// Gaussian Dropout layer for neural networks pub mod gaussian_dropout; diff --git a/src/neural_network/layers/regularization/noise_injection/gaussian_dropout.rs b/src/neural_network/layers/regularization/noise_injection/gaussian_dropout.rs index 5932aa6e..6dd1c8b6 100644 --- a/src/neural_network/layers/regularization/noise_injection/gaussian_dropout.rs +++ b/src/neural_network/layers/regularization/noise_injection/gaussian_dropout.rs @@ -1,11 +1,14 @@ //! Gaussian Dropout layer that applies multiplicative Gaussian noise during training //! //! [`GaussianDropout`] holds the drop rate, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), and the random number +//! [`UnaryLayer::build`], and the random number //! generator that draws the noise. The forward pass multiplies the input by `N(1, rate / (1 - //! rate))` noise. The backward pass reuses the exact draw parked in the -//! [`Ctx`](crate::neural_network::Ctx) cache, since the multiplier does not depend on the input +//! [`Ctx`] cache, since the multiplier does not depend on the input //! value. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build +//! [`Ctx`]: crate::neural_network::Ctx use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/noise_injection/gaussian_noise.rs b/src/neural_network/layers/regularization/noise_injection/gaussian_noise.rs index cf63d046..6da387e3 100644 --- a/src/neural_network/layers/regularization/noise_injection/gaussian_noise.rs +++ b/src/neural_network/layers/regularization/noise_injection/gaussian_noise.rs @@ -1,10 +1,12 @@ //! Gaussian noise regularization layer that injects zero-mean normal noise during training //! //! [`GaussianNoise`] holds the standard deviation, the shape recorded by -//! [`UnaryLayer::build`](crate::neural_network::traits::UnaryLayer::build), and the random number +//! [`UnaryLayer::build`], and the random number //! generator that draws the noise. The forward pass adds `N(0, stddev^2)` noise to the input. The //! backward pass passes the gradient through unchanged, because the noise does not depend on the //! input value. +//! +//! [`UnaryLayer::build`]: crate::neural_network::traits::UnaryLayer::build use crate::error::Error; use crate::neural_network::layers::ParamCounts; diff --git a/src/neural_network/layers/regularization/normalization/group_normalization.rs b/src/neural_network/layers/regularization/normalization/group_normalization.rs index 135b22c8..af92175b 100644 --- a/src/neural_network/layers/regularization/normalization/group_normalization.rs +++ b/src/neural_network/layers/regularization/normalization/group_normalization.rs @@ -251,8 +251,7 @@ impl LayerBase for GroupNormalization { impl UnaryLayer for GroupNormalization { /// Allocates the per-channel arrays from the trailing axis of the input /// - /// The channel axis is the last axis. The build refuses an input of a rank below - /// [`MIN_RANK`] + /// The channel axis is the last axis. The build refuses an input of a rank below 2 fn build(&mut self, input: &Shape) -> Result<(), Error> { let Some(built) = start_build(&self.built, "GroupNormalization", input)? else { return Ok(()); diff --git a/src/neural_network/layers/regularization/normalization/instance_normalization.rs b/src/neural_network/layers/regularization/normalization/instance_normalization.rs index a9d91bfb..2c616e37 100644 --- a/src/neural_network/layers/regularization/normalization/instance_normalization.rs +++ b/src/neural_network/layers/regularization/normalization/instance_normalization.rs @@ -250,8 +250,7 @@ impl LayerBase for InstanceNormalization { impl UnaryLayer for InstanceNormalization { /// Allocates the per-channel arrays from the trailing axis of the input /// - /// The channel axis is the last axis. The build refuses an input of a rank below - /// [`MIN_RANK`] + /// The channel axis is the last axis. The build refuses an input of a rank below 3 fn build(&mut self, input: &Shape) -> Result<(), Error> { let Some(built) = start_build(&self.built, "InstanceNormalization", input)? else { return Ok(()); diff --git a/src/neural_network/layers/regularization/validation.rs b/src/neural_network/layers/regularization/validation.rs index 8420855c..e5a73cda 100644 --- a/src/neural_network/layers/regularization/validation.rs +++ b/src/neural_network/layers/regularization/validation.rs @@ -1,13 +1,16 @@ //! Shared parameter and shape validation helpers for regularization layers //! //! Every function checks 1 constructor argument or input shape and returns -//! [`Error::InvalidParameter`](crate::error::Error::InvalidParameter) or -//! [`Error::InvalidInput`](crate::error::Error::InvalidInput) on failure. None of them panics. +//! [`Error::InvalidParameter`] or +//! [`Error::InvalidInput`] on failure. None of them panics. //! //! The dropout and noise-injection layers use `validate_rate`, `validate_rate_exclusive`, //! `validate_stddev`, and `validate_input_ndim`. The normalization layers use //! `validate_epsilon`, `validate_momentum`, `validate_num_groups`, //! `validate_num_groups_positive`, and `validate_min_input_ndim`. +//! +//! [`Error::InvalidParameter`]: crate::error::Error::InvalidParameter +//! [`Error::InvalidInput`]: crate::error::Error::InvalidInput use crate::error::Error; diff --git a/src/neural_network/optimizers/ada_grad.rs b/src/neural_network/optimizers/ada_grad.rs index a9584e64..89c3d6d4 100644 --- a/src/neural_network/optimizers/ada_grad.rs +++ b/src/neural_network/optimizers/ada_grad.rs @@ -2,8 +2,10 @@ //! gradients //! //! Holds the [`AdaGrad`] struct, its constructor, its [`AdaGrad::with_global_clipnorm`] builder -//! method, and its [`Optimizer`](crate::neural_network::traits::Optimizer) implementation. Weight +//! method, and its [`Optimizer`] implementation. Weight //! decay is decoupled (AdamW-style), and gradient clipping stays off until a caller opts in +//! +//! [`Optimizer`]: crate::neural_network::traits::Optimizer use crate::error::Error; use crate::neural_network::LayerPath; diff --git a/src/neural_network/optimizers/adam.rs b/src/neural_network/optimizers/adam.rs index f488f18a..66a7d1e4 100644 --- a/src/neural_network/optimizers/adam.rs +++ b/src/neural_network/optimizers/adam.rs @@ -1,9 +1,11 @@ //! Adam (Adaptive Moment Estimation) optimizer with classic coupled L2 weight decay //! //! Holds the [`Adam`] struct, its constructor, its [`Adam::with_global_clipnorm`] builder method, -//! and its [`Optimizer`](crate::neural_network::traits::Optimizer) implementation. The moment math -//! and the update loop live in the private `AdamCore` type, shared with -//! [`AdamW`](crate::neural_network::optimizers::AdamW) +//! and its [`Optimizer`] implementation. The moment math and the update loop live in the private +//! `AdamCore` type, shared with [`AdamW`] +//! +//! [`Optimizer`]: crate::neural_network::traits::Optimizer +//! [`AdamW`]: crate::neural_network::optimizers::AdamW use crate::error::Error; use crate::neural_network::LayerPath; diff --git a/src/neural_network/optimizers/adam_w.rs b/src/neural_network/optimizers/adam_w.rs index 56df008b..e4252ccd 100644 --- a/src/neural_network/optimizers/adam_w.rs +++ b/src/neural_network/optimizers/adam_w.rs @@ -1,9 +1,11 @@ //! AdamW optimizer: Adam with decoupled weight decay //! //! Holds the [`AdamW`] struct, its constructor, its [`AdamW::with_global_clipnorm`] builder method, -//! and its [`Optimizer`](crate::neural_network::traits::Optimizer) implementation. The moment math -//! and the update loop live in the private `AdamCore` type, shared with -//! [`Adam`](crate::neural_network::optimizers::Adam) +//! and its [`Optimizer`] implementation. The moment math and the update loop live in the private +//! `AdamCore` type, shared with [`Adam`] +//! +//! [`Optimizer`]: crate::neural_network::traits::Optimizer +//! [`Adam`]: crate::neural_network::optimizers::Adam use crate::error::Error; use crate::neural_network::LayerPath; diff --git a/src/neural_network/optimizers/kernels.rs b/src/neural_network/optimizers/kernels.rs index 77ffdc87..4d0359d2 100644 --- a/src/neural_network/optimizers/kernels.rs +++ b/src/neural_network/optimizers/kernels.rs @@ -3,8 +3,10 @@ //! Each optimizer's per-parameter math lives here as a single function. Every function operates //! on a mutable `f32` parameter slice, a gradient slice, and any needed optimizer state slices. //! Every layer exposes its parameters as flat slices through -//! [`LayerBase::parameters_mut`](crate::neural_network::traits::LayerBase::parameters_mut). +//! [`LayerBase::parameters_mut`]. //! These kernels work for any parameter shape +//! +//! [`LayerBase::parameters_mut`]: crate::neural_network::traits::LayerBase::parameters_mut use crate::parallel_gates::fused_slice_parallel_threshold; use rayon::prelude::*; diff --git a/src/neural_network/optimizers/rms_prop.rs b/src/neural_network/optimizers/rms_prop.rs index 72185f54..eec10764 100644 --- a/src/neural_network/optimizers/rms_prop.rs +++ b/src/neural_network/optimizers/rms_prop.rs @@ -1,8 +1,10 @@ //! RMSprop (Root Mean Square Propagation) optimizer //! //! Holds the [`RMSprop`] struct, its constructor, its [`RMSprop::with_global_clipnorm`] builder -//! method, and its [`Optimizer`](crate::neural_network::traits::Optimizer) implementation. Weight +//! method, and its [`Optimizer`] implementation. Weight //! decay is decoupled (AdamW-style), and gradient clipping stays off until a caller opts in +//! +//! [`Optimizer`]: crate::neural_network::traits::Optimizer use crate::error::Error; use crate::neural_network::LayerPath; diff --git a/src/neural_network/optimizers/sgd.rs b/src/neural_network/optimizers/sgd.rs index c99b6ee1..d19a663c 100644 --- a/src/neural_network/optimizers/sgd.rs +++ b/src/neural_network/optimizers/sgd.rs @@ -1,8 +1,10 @@ //! Stochastic Gradient Descent (SGD) optimizer //! //! Holds the [`SGD`] struct, its constructor, its [`SGD::with_global_clipnorm`] builder method, and -//! its [`Optimizer`](crate::neural_network::traits::Optimizer) implementation. Weight decay is +//! its [`Optimizer`] implementation. Weight decay is //! decoupled (SGDW-style), and gradient clipping stays off until a caller opts in +//! +//! [`Optimizer`]: crate::neural_network::traits::Optimizer use crate::error::Error; use crate::neural_network::LayerPath; diff --git a/src/neural_network/sequential.rs b/src/neural_network/sequential.rs index d35b47cf..8b7b6c1c 100644 --- a/src/neural_network/sequential.rs +++ b/src/neural_network/sequential.rs @@ -4,9 +4,9 @@ //! //! # 2 types, and only 1 of them trains //! -//! [`SequentialBuilder`](crate::neural_network::sequential::SequentialBuilder) collects -//! layers. [`Sequential`](crate::neural_network::sequential::Sequential) is a built model. -//! [`SequentialBuilder::build`](crate::neural_network::sequential::SequentialBuilder::build) is +//! [`SequentialBuilder`] collects +//! layers. [`Sequential`] is a built model. +//! [`SequentialBuilder::build`] is //! the only way to reach a built model, and it takes the //! shape of the input. It walks the stack once, gives every layer the shape that reaches it, //! and threads each output shape into the next layer @@ -15,10 +15,16 @@ //! methods of the built model alone. Training a model that was never built is therefore a //! compile error, and no run-time state says whether a model is ready //! -//! [`History`](crate::neural_network::sequential::History) is what `fit` and +//! [`History`] is what `fit` and //! `fit_with_batches` give back: 1 loss value per epoch, in epoch order. //! `save_to_path` and `load_from_path` read and write every array of the model, in the -//! format that [`checkpoint`](crate::neural_network::layers::checkpoint) defines +//! format that [`checkpoint`] defines +//! +//! [`SequentialBuilder`]: crate::neural_network::sequential::SequentialBuilder +//! [`Sequential`]: crate::neural_network::sequential::Sequential +//! [`SequentialBuilder::build`]: crate::neural_network::sequential::SequentialBuilder::build +//! [`History`]: crate::neural_network::sequential::History +//! [`checkpoint`]: crate::neural_network::layers::checkpoint use super::traits::{Layer, Loss, Optimizer, check_addresses, check_every_gradient_is_claimed}; use crate::error::{Error, IoError}; diff --git a/src/neural_network/traits.rs b/src/neural_network/traits.rs index d8b64f74..357859f2 100644 --- a/src/neural_network/traits.rs +++ b/src/neural_network/traits.rs @@ -1,27 +1,38 @@ //! Core traits for the neural network module: layers, losses, and optimizers, plus the //! address and view types that connect them. //! -//! [`LayerBase`](crate::neural_network::traits::LayerBase) holds what every layer has regardless of +//! [`LayerBase`] holds what every layer has regardless of //! its input count: its type name, its arrays, its sublayers, and its build state. -//! [`UnaryLayer`](crate::neural_network::traits::UnaryLayer) adds the forward and backward pass for +//! [`UnaryLayer`] adds the forward and backward pass for //! a layer with 1 input, and a blanket implementation gives it the general -//! [`Layer`](crate::neural_network::traits::Layer) interface that a model holds every layer -//! through. Implement [`Layer`](crate::neural_network::traits::Layer) directly only for a layer +//! [`Layer`] interface that a model holds every layer +//! through. Implement [`Layer`] directly only for a layer //! with several inputs, such as a merge layer. //! -//! [`ParamId`](crate::neural_network::traits::ParamId) names the address of 1 parameter tensor: -//! the [`LayerPath`](crate::neural_network::LayerPath) of the layer that holds it, plus the name +//! [`ParamId`] names the address of 1 parameter tensor: +//! the [`LayerPath`] of the layer that holds it, plus the name //! the layer gives the tensor. -//! [`ParamRef`](crate::neural_network::traits::ParamRef), -//! [`WeightRef`](crate::neural_network::traits::WeightRef), and -//! [`WeightMut`](crate::neural_network::traits::WeightMut) are the borrowed views that a layer +//! [`ParamRef`], +//! [`WeightRef`], and +//! [`WeightMut`] are the borrowed views that a layer //! exposes under that name, for an optimizer to update or a checkpoint to read and write. //! `check_addresses` and `check_every_gradient_is_claimed` are the build-time and pass-time //! checks that keep every address unique and every gradient reachable. //! -//! [`Loss`](crate::neural_network::traits::Loss) computes a scalar loss and its gradient. -//! [`Optimizer`](crate::neural_network::traits::Optimizer) reads the gradient store and updates the -//! parameters of a layer, keyed on [`ParamId`](crate::neural_network::traits::ParamId). +//! [`Loss`] computes a scalar loss and its gradient. +//! [`Optimizer`] reads the gradient store and updates the +//! parameters of a layer, keyed on [`ParamId`]. +//! +//! [`LayerPath`]: crate::neural_network::LayerPath +//! [`LayerBase`]: crate::neural_network::traits::LayerBase +//! [`UnaryLayer`]: crate::neural_network::traits::UnaryLayer +//! [`Layer`]: crate::neural_network::traits::Layer +//! [`ParamId`]: crate::neural_network::traits::ParamId +//! [`ParamRef`]: crate::neural_network::traits::ParamRef +//! [`WeightRef`]: crate::neural_network::traits::WeightRef +//! [`WeightMut`]: crate::neural_network::traits::WeightMut +//! [`Loss`]: crate::neural_network::traits::Loss +//! [`Optimizer`]: crate::neural_network::traits::Optimizer use crate::error::Error; use crate::neural_network::Shape; diff --git a/src/prelude/neural_network.rs b/src/prelude/neural_network.rs index e0c832cd..3cafde54 100644 --- a/src/prelude/neural_network.rs +++ b/src/prelude/neural_network.rs @@ -3,9 +3,12 @@ //! //! `Average` is the 1 name that 2 categories of the crate give to an item. This module gives //! the merge layer that averages its inputs, and the metrics category gives the averaging mode -//! of the classification scores. The root of [`prelude`](crate::prelude) keeps the averaging +//! of the classification scores. The root of [`prelude`] keeps the averaging //! mode, so a caller that wants the layer through `use rustyml::prelude::*` names it -//! [`layers::Average`](crate::neural_network::layers::Average) +//! [`layers::Average`] +//! +//! [`prelude`]: crate::prelude +//! [`layers::Average`]: crate::neural_network::layers::Average pub use crate::neural_network::Ctx; pub use crate::neural_network::Shape; diff --git a/src/random.rs b/src/random.rs index 1981f32a..51386853 100644 --- a/src/random.rs +++ b/src/random.rs @@ -6,7 +6,7 @@ //! entry point for: //! //! - neural-network components: weight initialization, dropout/noise masks, and the -//! [`Sequential`](crate::neural_network::sequential::Sequential) minibatch shuffle +//! [`Sequential`] minibatch shuffle //! - machine-learning estimators: k-means, SVC/LinearSVC, MeanShift, Isolation Forest, and others //! - utilities: `train_test_split`, t-SNE //! @@ -48,6 +48,8 @@ //! //! General rule: route a draw through this module only when an unseeded call would make a //! pseudo-random choice that changes the result +//! +//! [`Sequential`]: crate::neural_network::sequential::Sequential use ndarray_rand::rand::{RngCore, SeedableRng, rng, rngs::StdRng}; use std::cell::RefCell; diff --git a/src/traits.rs b/src/traits.rs index ac640d09..5d5819ca 100644 --- a/src/traits.rs +++ b/src/traits.rs @@ -3,19 +3,19 @@ //! These traits give the otherwise independent estimators a single uniform contract //! so that generic code can train, transform, and run predictions over any of them: //! -//! - [`Fit`](crate::traits::Fit) - train a model from input data (`(&X, &Y)` for supervised -//! models, `&X` for unsupervised ones) -//! - [`Predict`](crate::traits::Predict) - run inference on a fitted model, with the output -//! type chosen by each model (labels, probabilities, cluster ids, anomaly scores, ...) -//! - [`Transform`](crate::traits::Transform) - project new data through a fitted transformer -//! (`PCA`, `KernelPCA`, `StandardScaler`) -//! - [`FitTransform`](crate::traits::FitTransform) - fit and transform the training data in -//! one call (`PCA`, `KernelPCA`, `TSNE`, `StandardScaler`) +//! - [`Fit`] - train a model from input data (`(&X, &Y)` for supervised models, `&X` for +//! unsupervised ones) +//! - [`Predict`] - run inference on a fitted model, with the output type chosen by each model +//! (labels, probabilities, cluster ids, anomaly scores, ...) +//! - [`Transform`] - project new data through a fitted transformer (`PCA`, `KernelPCA`, +//! `StandardScaler`) +//! - [`FitTransform`] - fit and transform the training data in one call (`PCA`, `KernelPCA`, +//! `TSNE`, `StandardScaler`) //! //! The traits live at the crate root because implementors span features: the -//! estimators in [`machine_learning`](crate::machine_learning) and the preprocessing -//! transformers in [`utils`](crate::utils). Each module also re-exports them -//! (`machine_learning::Fit`, `utils::Fit`), and both preludes bring them into scope +//! estimators in [`machine_learning`] and the preprocessing transformers in [`utils`]. Each module +//! also re-exports them (`machine_learning::Fit`, `utils::Fit`), and both preludes bring them into +//! scope //! //! Every implementor also exposes the same operations as inherent methods //! (`model.fit(..)`, `model.predict(..)`). The trait implementations forward to them. @@ -44,6 +44,13 @@ //! // Call it with any estimator that fits the bounds, e.g.: //! // let preds = train_and_predict(&mut LinearRegression::default(), &x, &y); //! ``` +//! +//! [`Fit`]: crate::traits::Fit +//! [`Predict`]: crate::traits::Predict +//! [`Transform`]: crate::traits::Transform +//! [`FitTransform`]: crate::traits::FitTransform +//! [`machine_learning`]: crate::machine_learning +//! [`utils`]: crate::utils use crate::error::Error; #[cfg(feature = "machine_learning")] diff --git a/src/utils/scaler.rs b/src/utils/scaler.rs index 3b0a6ca1..619c35b9 100644 --- a/src/utils/scaler.rs +++ b/src/utils/scaler.rs @@ -18,16 +18,17 @@ //! //! [`Normalizer`] is the odd one out. It rescales samples (rows), not features (columns), so //! it learns nothing beyond the feature count. It exists so row normalization uses the same -//! [`Fit`](crate::traits::Fit) and [`Transform`](crate::traits::Transform) contract as the -//! rest. The other 4 are per-feature and stateful in the full sense. All but -//! [`RobustScaler`] also support `partial_fit`. Their statistics merge exactly across batches, -//! but quantiles do not. +//! [`Fit`] and [`Transform`] contract as the rest. The other 4 are per-feature and stateful in the +//! full sense. All but [`RobustScaler`] also support `partial_fit`. Their statistics merge exactly +//! across batches, but quantiles do not. //! //! Every scaler here is 2-D and per-feature, exactly like scikit-learn's. The free functions //! handle row-wise or whole-array standardization, and N-D arrays. //! //! [`standardize`]: crate::utils::standardize::standardize //! [`normalize`]: crate::utils::normalize::normalize +//! [`Fit`]: crate::traits::Fit +//! [`Transform`]: crate::traits::Transform use crate::error::Error; use crate::parallel_gates::{cheap_map_f64_parallel_threshold, scan_f64_parallel_min_elems}; diff --git a/src/utils/scaler/normalizer.rs b/src/utils/scaler/normalizer.rs index 7b10a7ab..04483069 100644 --- a/src/utils/scaler/normalizer.rs +++ b/src/utils/scaler/normalizer.rs @@ -1,11 +1,15 @@ //! Per-sample unit-norm scaling behind the fit/transform estimator pattern //! //! Provides [`Normalizer`], the `fit`/`transform` face of the stateless -//! [`normalize`](crate::utils::normalize::normalize) function restricted to rows. It is the one +//! [`normalize`] function restricted to rows. It is the one //! transformer in this module that learns nothing statistical, since a sample's norm comes -//! from that sample alone. It still takes part in the same [`Fit`](crate::traits::Fit) and -//! [`Transform`](crate::traits::Transform) contract as the per-feature scalers. It also pins +//! from that sample alone. It still takes part in the same [`Fit`] and +//! [`Transform`] contract as the per-feature scalers. It also pins //! the feature count, so a mis-shaped batch is an error rather than a silent wrong answer +//! +//! [`normalize`]: crate::utils::normalize::normalize +//! [`Fit`]: crate::traits::Fit +//! [`Transform`]: crate::traits::Transform use super::{validate_matrix, validate_transform_matrix}; use crate::error::Error; diff --git a/src/utils/scaler/robust_scaler.rs b/src/utils/scaler/robust_scaler.rs index 965f2f22..43000daf 100644 --- a/src/utils/scaler/robust_scaler.rs +++ b/src/utils/scaler/robust_scaler.rs @@ -3,9 +3,11 @@ //! Provides [`RobustScaler`], which centers on the median and divides by the interquartile //! range instead of using the mean and standard deviation. Both statistics are order-based, so //! a handful of extreme values moves them barely at all. A single outlier can drag -//! [`StandardScaler`](super::StandardScaler)'s mean and inflate its standard deviation. An -//! outlier can also stretch [`MinMaxScaler`](super::MinMaxScaler)'s denominator until every -//! other sample is squeezed into a sliver of the range +//! [`StandardScaler`]'s mean and inflate its standard deviation. An outlier can also stretch +//! [`MinMaxScaler`]'s denominator until every other sample is squeezed into a sliver of the range +//! +//! [`StandardScaler`]: super::StandardScaler +//! [`MinMaxScaler`]: super::MinMaxScaler use super::{ column_quantiles, fitted, for_each_row, handle_zero_scale, validate_matrix, diff --git a/src/utils/scaler/standard_scaler.rs b/src/utils/scaler/standard_scaler.rs index c44f94c8..9e55ab18 100644 --- a/src/utils/scaler/standard_scaler.rs +++ b/src/utils/scaler/standard_scaler.rs @@ -1,12 +1,14 @@ //! Standardization that remembers its training statistics //! //! Provides [`StandardScaler`], the fit and transform counterpart to the stateless -//! [`standardize`](crate::utils::standardize::standardize) function. It learns the per-feature +//! [`standardize`] function. It learns the per-feature //! mean and standard deviation once, on the training matrix. It reuses those frozen numbers //! for every later batch: the test split, a validation fold, or a single sample arriving at //! inference time. This is the scikit-learn `StandardScaler` contract, and it keeps a //! train-test boundary honest. Rescaling a test set by its own column statistics applies a //! different linear map than the one the model was trained under +//! +//! [`standardize`]: crate::utils::standardize::standardize use super::{fitted, for_each_row, validate_matrix, validate_transform_matrix}; use crate::error::Error; diff --git a/src/utils/standardize.rs b/src/utils/standardize.rs index 62001b53..f01d9b92 100644 --- a/src/utils/standardize.rs +++ b/src/utils/standardize.rs @@ -5,9 +5,10 @@ //! //! This is the one-shot form. Every call recomputes the statistics from the array it is //! handed. When the same statistics must apply to more than one batch, use -//! [`StandardScaler`](crate::utils::StandardScaler) instead. Examples include a test split -//! scaled by the training mean, or a single sample at inference time. It stores the -//! statistics at fit time +//! [`StandardScaler`] instead. Examples include a test split scaled by the training mean, or a +//! single sample at inference time. It stores the statistics at fit time +//! +//! [`StandardScaler`]: crate::utils::StandardScaler use crate::error::Error; use crate::math::reduction::det_reduce;