Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 40 additions & 22 deletions src/error.rs
Original file line number Diff line number Diff line change
@@ -1,52 +1,70 @@
//! Error types for RustyML
//!
//! Every fallible operation in the crate returns [`RustymlResult<T>`](crate::error::RustymlResult),
//! an alias for `std::result::Result<T, Error>`. [`Error`](crate::error::Error) is the single,
//! Every fallible operation in the crate returns [`RustymlResult<T>`],
//! an alias for `std::result::Result<T, Error>`. [`Error`] is the single,
//! unified error type.
//!
//! `Error` aggregates 2 domain-specific error enums: `NnError` (from `neural_network`) and
//! `TreeError` (from `machine_learning`). It also wraps the shared
//! [`IoError`](crate::error::IoError). Callers can `match` precisely, since a shared variant never
//! [`IoError`]. Callers can `match` precisely, since a shared variant never
//! carries a concern specific to one part of the crate
//!
//! # Categories
//!
//! - **Input validation**: [`Error::EmptyInput`](crate::error::Error::EmptyInput),
//! [`Error::DimensionMismatch`](crate::error::Error::DimensionMismatch),
//! [`Error::ShapeMismatch`](crate::error::Error::ShapeMismatch),
//! [`Error::NonFinite`](crate::error::Error::NonFinite),
//! [`Error::InvalidParameter`](crate::error::Error::InvalidParameter),
//! [`Error::InvalidInput`](crate::error::Error::InvalidInput)
//! - **Model state**: [`Error::NotFitted`](crate::error::Error::NotFitted), and the neural-network
//! - **Input validation**: [`Error::EmptyInput`],
//! [`Error::DimensionMismatch`],
//! [`Error::ShapeMismatch`],
//! [`Error::NonFinite`],
//! [`Error::InvalidParameter`],
//! [`Error::InvalidInput`]
//! - **Model state**: [`Error::NotFitted`], and the neural-network
//! states in `NnError`
//! - **Numerics / computation**: [`Error::NotConverged`](crate::error::Error::NotConverged),
//! [`Error::Computation`](crate::error::Error::Computation)
//! - **Numerics / computation**: [`Error::NotConverged`],
//! [`Error::Computation`]
//! - **Domain-specific**: `Error::NeuralNetwork` (wrapping `neural_network`'s `NnError`) and
//! `Error::Tree` (wrapping `machine_learning`'s `TreeError`)
//! - **I/O and serialization**: [`Error::Io`](crate::error::Error::Io)
//! - **I/O and serialization**: [`Error::Io`]
//!
//! # Conventions
//!
//! - A non-finite **hyperparameter** from the user is an
//! [`Error::InvalidParameter`](crate::error::Error::InvalidParameter) (the reason names
//! [`Error::InvalidParameter`] (the reason names
//! finiteness). A non-finite value from the **data or a computation** is an
//! [`Error::NonFinite`](crate::error::Error::NonFinite)
//! - [`Error::DimensionMismatch`](crate::error::Error::DimensionMismatch) compares scalar counts
//! (e.g. number of features). Use [`Error::ShapeMismatch`](crate::error::Error::ShapeMismatch)
//! [`Error::NonFinite`]
//! - [`Error::DimensionMismatch`] compares scalar counts
//! (e.g. number of features). Use [`Error::ShapeMismatch`]
//! when whole tensor shapes differ
//!
//! # Constructing errors
//!
//! Prefer the smart constructors
//! ([`Error::dimension_mismatch`](crate::error::Error::dimension_mismatch),
//! [`Error::invalid_parameter`](crate::error::Error::invalid_parameter), and so on) over building
//! ([`Error::dimension_mismatch`],
//! [`Error::invalid_parameter`], and so on) over building
//! variants by hand. They accept flexible argument types and keep the wording consistent across the
//! crate. To attach context to a foreign error while preserving its source chain, use
//! [`Context::context`](crate::error::Context::context) /
//! [`Context::with_context`](crate::error::Context::with_context)
//! [`Context::context`] /
//! [`Context::with_context`]
//!
//! [`Error`](crate::error::Error) is `#[non_exhaustive]`. Match with a trailing `_` arm to stay
//! [`Error`] is `#[non_exhaustive]`. Match with a trailing `_` arm to stay
//! forward-compatible
//!
//! [`RustymlResult<T>`]: crate::error::RustymlResult
//! [`Error`]: crate::error::Error
//! [`IoError`]: crate::error::IoError
//! [`Error::EmptyInput`]: crate::error::Error::EmptyInput
//! [`Error::DimensionMismatch`]: crate::error::Error::DimensionMismatch
//! [`Error::ShapeMismatch`]: crate::error::Error::ShapeMismatch
//! [`Error::NonFinite`]: crate::error::Error::NonFinite
//! [`Error::InvalidParameter`]: crate::error::Error::InvalidParameter
//! [`Error::InvalidInput`]: crate::error::Error::InvalidInput
//! [`Error::NotFitted`]: crate::error::Error::NotFitted
//! [`Error::NotConverged`]: crate::error::Error::NotConverged
//! [`Error::Computation`]: crate::error::Error::Computation
//! [`Error::Io`]: crate::error::Error::Io
//! [`Error::dimension_mismatch`]: crate::error::Error::dimension_mismatch
//! [`Error::invalid_parameter`]: crate::error::Error::invalid_parameter
//! [`Context::context`]: crate::error::Context::context
//! [`Context::with_context`]: crate::error::Context::with_context

/// The unified error type for all fallible RustyML operations
///
Expand Down
48 changes: 33 additions & 15 deletions src/machine_learning.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
//! reduction, and anomaly detection
//!
//! This module groups models by algorithm family into submodules:
//! [`clustering`](crate::machine_learning::clustering),
//! [`decomposition`](crate::machine_learning::decomposition),
//! [`linear_model`](crate::machine_learning::linear_model),
//! [`manifold`](crate::machine_learning::manifold), [`svm`](crate::machine_learning::svm),
//! [`tree`](crate::machine_learning::tree), [`neighbors`](crate::machine_learning::neighbors),
//! [`discriminant_analysis`](crate::machine_learning::discriminant_analysis), and
//! [`ensemble`](crate::machine_learning::ensemble). This module also re-exports every estimator,
//! [`clustering`],
//! [`decomposition`],
//! [`linear_model`],
//! [`manifold`], [`svm`],
//! [`tree`], [`neighbors`],
//! [`discriminant_analysis`], and
//! [`ensemble`]. This module also re-exports every estimator,
//! so it is reachable directly as `machine_learning::<Model>`. Supervised and unsupervised
//! estimators implement the shared [`Fit`](crate::traits::Fit) /
//! [`Predict`](crate::traits::Predict) traits. The dimensionality-reduction
//! transformers implement [`Transform`](crate::traits::Transform) /
//! [`FitTransform`](crate::traits::FitTransform).
//! estimators implement the shared [`Fit`] /
//! [`Predict`] traits. The dimensionality-reduction
//! transformers implement [`Transform`] /
//! [`FitTransform`].
//!
//! # Supervised learning
//!
Expand Down Expand Up @@ -47,12 +47,12 @@
//!
//! # Shared types
//!
//! - [`DistanceCalculationMetric`](crate::math::DistanceCalculationMetric):
//! - [`DistanceCalculationMetric`]:
//! Euclidean/Manhattan/Minkowski dispatcher, re-exported from [`crate::math`]
//! - [`RegularizationType`](crate::machine_learning::types::RegularizationType): L1 / L2
//! - [`RegularizationType`]: L1 / L2
//! regularization
//! - [`KernelType`](crate::machine_learning::types::KernelType) /
//! [`Gamma`](crate::machine_learning::types::Gamma): kernel selection and coefficient
//! - [`KernelType`] /
//! [`Gamma`]: kernel selection and coefficient
//!
//! # Examples
//!
Expand All @@ -65,6 +65,24 @@
//! let y = array![6.0, 9.0, 12.0];
//! model.fit(&x, &y).unwrap();
//! ```
//!
//! [`clustering`]: crate::machine_learning::clustering
//! [`decomposition`]: crate::machine_learning::decomposition
//! [`linear_model`]: crate::machine_learning::linear_model
//! [`manifold`]: crate::machine_learning::manifold
//! [`svm`]: crate::machine_learning::svm
//! [`tree`]: crate::machine_learning::tree
//! [`neighbors`]: crate::machine_learning::neighbors
//! [`discriminant_analysis`]: crate::machine_learning::discriminant_analysis
//! [`ensemble`]: crate::machine_learning::ensemble
//! [`Fit`]: crate::traits::Fit
//! [`Predict`]: crate::traits::Predict
//! [`Transform`]: crate::traits::Transform
//! [`FitTransform`]: crate::traits::FitTransform
//! [`DistanceCalculationMetric`]: crate::math::DistanceCalculationMetric
//! [`RegularizationType`]: crate::machine_learning::types::RegularizationType
//! [`KernelType`]: crate::machine_learning::types::KernelType
//! [`Gamma`]: crate::machine_learning::types::Gamma

pub use crate::math::DistanceCalculationMetric;
/// The crate-wide estimator traits, re-exported here for convenience. Their canonical
Expand Down
8 changes: 5 additions & 3 deletions src/machine_learning/decomposition.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@
//! `sklearn.decomposition`: [`PCA`] for linear dimensionality reduction and
//! [`KernelPCA`] for its nonlinear, kernelized counterpart. Both learn their
//! components from a feature matrix at `fit` time and project new data through
//! `transform`. Both implement the shared [`Fit`](crate::traits::Fit),
//! [`Transform`](crate::traits::Transform), and
//! [`FitTransform`](crate::traits::FitTransform) traits
//! `transform`. Both implement the shared [`Fit`], [`Transform`], and [`FitTransform`] traits
//!
//! [`Fit`]: crate::traits::Fit
//! [`Transform`]: crate::traits::Transform
//! [`FitTransform`]: crate::traits::FitTransform

pub use crate::machine_learning::{Gamma, KernelType};

Expand Down
7 changes: 5 additions & 2 deletions src/machine_learning/error.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
//! Decision-tree error type
//!
//! [`TreeError`] enumerates the failures specific to decision-tree models. Callers receive
//! it through the crate-wide [`Error::Tree`](crate::error::Error::Tree) variant, into which
//! it through the crate-wide [`Error::Tree`] variant, into which
//! it converts via `?` (a `#[from]` bridge). See [`crate::error`] for the unified
//! [`Error`](crate::error::Error) that aggregates the per-domain error enums
//! [`Error`] that aggregates the per-domain error enums
//!
//! [`Error::Tree`]: crate::error::Error::Tree
//! [`Error`]: crate::error::Error

/// Decision-tree-specific errors, surfaced through [`Error::Tree`](crate::error::Error::Tree)
#[non_exhaustive]
Expand Down
4 changes: 3 additions & 1 deletion src/machine_learning/linear_model/logistic_regression.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
//!
//! Provides the [`LogisticRegression`] model, trained by full-batch gradient descent on the
//! logistic loss, with optional L1 or L2 regularization. Fitting never panics: every precondition
//! failure and numerical issue during training surfaces as an [`Error`](crate::error::Error) from
//! failure and numerical issue during training surfaces as an [`Error`] from
//! [`LogisticRegression::fit`].
//!
//! Also provides the [`generate_polynomial_features`] helper, which expands a feature
//! matrix into its polynomial combinations up to a chosen degree. Use it to fit a
//! non-linear decision boundary with the same linear model.
//!
//! [`Error`]: crate::error::Error

use crate::error::Error;
pub use crate::machine_learning::RegularizationType;
Expand Down
11 changes: 7 additions & 4 deletions src/machine_learning/manifold.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
//! Manifold-learning estimators for nonlinear dimensionality reduction
//!
//! [`TSNE`] implements t-distributed Stochastic Neighbor Embedding (t-SNE) for visualizing
//! high-dimensional data. Unlike the [`decomposition`](crate::machine_learning::decomposition)
//! estimators, t-SNE learns no out-of-sample projection. It only embeds the data it is given.
//! It implements [`FitTransform`](crate::traits::FitTransform) but not
//! [`Transform`](crate::traits::Transform).
//! high-dimensional data. Unlike the [`decomposition`] estimators, t-SNE learns no out-of-sample
//! projection. It only embeds the data it is given. It implements [`FitTransform`] but not
//! [`Transform`].
//!
//! [`decomposition`]: crate::machine_learning::decomposition
//! [`FitTransform`]: crate::traits::FitTransform
//! [`Transform`]: crate::traits::Transform

/// t-SNE algorithm for dimensionality reduction
pub mod t_sne;
Expand Down
4 changes: 3 additions & 1 deletion src/machine_learning/svm/linear_svc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@
//!
//! Provides [`LinearSVC`], a hinge-loss classifier trained with stochastic gradient
//! descent and L1 or L2 regularization, along with the
//! [`RegularizationType`](crate::machine_learning::RegularizationType) enum that
//! [`RegularizationType`] enum that
//! selects the penalty
//!
//! [`RegularizationType`]: crate::machine_learning::RegularizationType

use crate::error::Error;
pub use crate::machine_learning::RegularizationType;
Expand Down
4 changes: 3 additions & 1 deletion src/machine_learning/svm/svc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@
//!
//! Provides the [`SVC`] binary classifier, trained with the Sequential Minimal
//! Optimization (SMO) algorithm, and the kernel types re-exported as
//! [`KernelType`](crate::machine_learning::KernelType)
//! [`KernelType`]
//!
//! Labels are `{0.0, 1.0}` on the public API, matching the crate's other binary
//! classifiers. Only `fit` uses the internal +/-1 encoding that the SMO dual needs
//!
//! [`KernelType`]: crate::machine_learning::KernelType

use crate::error::Error;
use crate::machine_learning::parallel::map_collect;
Expand Down
11 changes: 5 additions & 6 deletions src/machine_learning/types.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
//! Configuration enums shared across the machine-learning estimators
//!
//! - The linear models and the linear SVM use [`RegularizationType`]
//! - [`SVC`](crate::machine_learning::svm::svc::SVC) and
//! [`KernelPCA`](crate::machine_learning::decomposition::kernel_pca::KernelPCA) both use
//! [`Gamma`] and [`KernelType`]
//! - [`SVC`] and [`KernelPCA`] both use [`Gamma`] and [`KernelType`]
//! Each kernel carries its own evaluation logic (`compute` / `compute_matrix`) as inherent methods.
//! Consumers share this single implementation instead of re-matching the enum in every model
//!
//! Each kernel carries its own evaluation logic (`compute` / `compute_matrix`) as inherent
//! methods. Consumers share this single implementation instead of re-matching the enum in
//! every model
//! [`SVC`]: crate::machine_learning::svm::svc::SVC
//! [`KernelPCA`]: crate::machine_learning::decomposition::kernel_pca::KernelPCA

use crate::math::squared_euclidean_distance_row;
use crate::parallel_gates::{cheap_map_f64_parallel_threshold, exp_map_f64_parallel_threshold};
Expand Down
13 changes: 9 additions & 4 deletions src/math.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
//! Shared low-level numeric primitives used across estimators and metrics
//!
//! - [`matmul`](crate::math::matmul) provides `gemmkit`-backed matrix products with automatic
//! - [`matmul`] provides `gemmkit`-backed matrix products with automatic
//! parallelism
//! - [`reduction`](crate::math::reduction) provides deterministic blocked parallel reductions
//! - [`distance`](crate::math::distance) holds the pairwise distance primitives and the
//! [`DistanceCalculationMetric`](crate::math::DistanceCalculationMetric) dispatcher
//! - [`reduction`] provides deterministic blocked parallel reductions
//! - [`distance`] holds the pairwise distance primitives and the
//! [`DistanceCalculationMetric`] dispatcher
//!
//! It also hosts the tunable exp-reduction parallel gate used by the logistic-regression
//! log-loss.
Expand All @@ -29,6 +29,11 @@
//! let sq = squared_euclidean_distance_row(&v1, &v2);
//! let d = DistanceCalculationMetric::Euclidean.distance(v1.view(), v2.view());
//! ```
//!
//! [`matmul`]: crate::math::matmul
//! [`reduction`]: crate::math::reduction
//! [`distance`]: crate::math::distance
//! [`DistanceCalculationMetric`]: crate::math::DistanceCalculationMetric

/// `gemmkit`-backed matrix products with automatic parallelism.
pub mod matmul;
Expand Down
10 changes: 7 additions & 3 deletions src/math/reduction.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,21 @@
//! from run to run and with the thread count. The helpers here fix the grouping instead. The
//! input is cut into fixed-size blocks. Each block folds serially, and rayon's indexed
//! `collect` gathers the per-block results in block order before they merge sequentially. The
//! grouping depends only on [`DET_REDUCE_BLOCK`](crate::math::reduction::DET_REDUCE_BLOCK),
//! grouping depends only on [`DET_REDUCE_BLOCK`],
//! never on scheduling or the `parallel` flag. Re-running on the same machine reproduces the
//! result, though not necessarily bit-for-bit. The flag is a performance hint that only decides
//! whether the blocks run on rayon or in sequence. A caller that pairs these helpers with some
//! other serial kernel below a size threshold makes that switch part of its own reproducibility
//! surface.
//!
//! [`det_reduce`](crate::math::reduction::det_reduce) folds a slice.
//! [`det_reduce_range`](crate::math::reduction::det_reduce_range) folds fixed blocks of an
//! [`det_reduce`] folds a slice.
//! [`det_reduce_range`] folds fixed blocks of an
//! index range instead, for reductions that read several arrays at once or accumulate
//! something richer than a scalar.
//!
//! [`DET_REDUCE_BLOCK`]: crate::math::reduction::DET_REDUCE_BLOCK
//! [`det_reduce`]: crate::math::reduction::det_reduce
//! [`det_reduce_range`]: crate::math::reduction::det_reduce_range

use rayon::prelude::{IntoParallelIterator, ParallelIterator, ParallelSlice};
use std::ops::Range;
Expand Down
10 changes: 7 additions & 3 deletions src/metrics.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@
//! constant bias)
//!
//! # Classification metrics
//! - [`ConfusionMatrix`](crate::metrics::ConfusionMatrix): binary TP/FP/TN/FN with derived
//! - [`ConfusionMatrix`]: binary TP/FP/TN/FN with derived
//! accuracy, precision, recall, specificity, F1, error rate, balanced accuracy, and MCC
//! - [`MulticlassConfusionMatrix`](crate::metrics::MulticlassConfusionMatrix): KxK matrix with
//! - [`MulticlassConfusionMatrix`]: KxK matrix with
//! per-class precision/recall/F1/support and macro/micro/weighted aggregation via
//! [`Average`](crate::metrics::Average)
//! [`Average`]
//! - **accuracy**, **roc_auc**, **log_loss**, **cohen_kappa**, **top_k_accuracy**,
//! **average_precision**, and the **roc_curve** / **precision_recall_curve** threshold sweeps
//!
Expand Down Expand Up @@ -65,6 +65,10 @@
//! let scores = array![0.1, 0.4, 0.35, 0.8];
//! let auc = roc_auc(&labels.view(), &scores.view());
//! ```
//!
//! [`ConfusionMatrix`]: crate::metrics::ConfusionMatrix
//! [`MulticlassConfusionMatrix`]: crate::metrics::MulticlassConfusionMatrix
//! [`Average`]: crate::metrics::Average

/// Classification metrics: confusion matrices, accuracy, ROC/PR curves, log loss,
/// and Cohen's kappa.
Expand Down
Loading
Loading