|
75 | 75 | sparsity = nothing)</code></pre><p>and utilize the following signature:</p><pre><code class="language-julia hljs">forwarddiff_color_jacobian!(J::AbstractMatrix{<:Number}, |
76 | 76 | f, |
77 | 77 | x::AbstractArray{<:Number}, |
78 | | - jac_cache::ForwardColorJacCache)</code></pre><p><code>dx</code> is a pre-allocated output vector which is used to declare the output size, and thus allows for specifying a non-square Jacobian.</p><p>Also, it is possible retrieve the function value via <code>value(jac_cache)</code> or <code>value!(result, jac_cache)</code></p><p>If one is using an out-of-place function <code>f(x)</code>, then the alternative form ca be used:</p><pre><code class="language-julia hljs">jacout = forwarddiff_color_jacobian(g, x, |
| 78 | + jac_cache::ForwardColorJacCache)</code></pre><p><code>dx</code> is a pre-allocated output vector which is used to declare the output size, and thus allows for specifying a non-square Jacobian.</p><p>Also, it is possible retrieve the function value via <code>value(jac_cache)</code> or <code>value!(result, jac_cache)</code></p><p>If one is using an out-of-place function <code>f(x)</code>, then the alternative form ca be used:</p><pre><code class="language-julia hljs">jacout = forwarddiff_color_jacobian(g, x, |
79 | 79 | dx = similar(x), |
80 | 80 | colorvec = 1:length(x), |
81 | 81 | sparsity = nothing, |
82 | 82 | jac_prototype = nothing)</code></pre><p>Note that the out-of-place form is efficient and compatible with StaticArrays. One can specify the type and shape of the output Jacobian by giving an additional <code>jac_prototype</code> to the out-of place <code>forwarddiff_color_jacobian</code> function, otherwise it will become a dense matrix. If <code>jac_prototype</code> and <code>sparsity</code> are not specified, then the oop Jacobian will assume that the function has a <em>square</em> Jacobian matrix. If it is not the case, please specify the shape of output by giving <code>dx</code>.</p><p>Similar functionality is available for Hessians, using finite differences of forward derivatives. Given a scalar function <code>f(x)</code>, a vector value for <code>x</code>, and a color vector and sparsity pattern, this can be accomplished using <code>numauto_color_hessian</code> or its in-place form <code>numauto_color_hessian!</code>.</p><pre><code class="language-julia hljs">H = numauto_color_hessian(f, x, colorvec, sparsity) |
83 | | -numauto_color_hessian!(H, f, x, colorvec, sparsity)</code></pre><p>To avoid unnecessary allocations every time the Hessian is computed, construct a <code>ForwardColorHesCache</code> beforehand:</p><pre><code class="language-julia hljs">hescache = ForwardColorHesCache(f, x, colorvec, sparsity) |
84 | | -numauto_color_hessian!(H, f, x, hescache)</code></pre><p>By default, these methods use a mix of numerical and automatic differentiation, namely by taking finite differences of gradients calculated via ForwardDiff.jl. Alternatively, if you have your own custom gradient function <code>g!</code>, you can specify it as an argument to <code>ForwardColorHesCache</code>:</p><pre><code class="language-julia hljs">hescache = ForwardColorHesCache(f, x, colorvec, sparsity, g!)</code></pre><p>Note that any user-defined gradient needs to have the signature <code>g!(G, x)</code>, i.e. updating the gradient <code>G</code> in place.</p><h3 id="Jacobian-Vector-and-Hessian-Vector-Products"><a class="docs-heading-anchor" href="#Jacobian-Vector-and-Hessian-Vector-Products">Jacobian-Vector and Hessian-Vector Products</a><a id="Jacobian-Vector-and-Hessian-Vector-Products-1"></a><a class="docs-heading-anchor-permalink" href="#Jacobian-Vector-and-Hessian-Vector-Products" title="Permalink"></a></h3><p>Matrix-free implementations of Jacobian-Vector and Hessian-Vector products is provided in both an operator and function form. For the functions, each choice has the choice of being in-place and out-of-place, and the in-place versions have the ability to pass in cache vectors to be non-allocating. When in-place the function signature for Jacobians is <code>f!(du,u)</code>, while out-of-place has <code>du=f(u)</code>. For Hessians, all functions must be <code>f(u)</code> which returns a scalar</p><p>The functions for Jacobians are:</p><pre><code class="language-julia hljs">auto_jacvec!(dy, f, x, v, |
| 83 | +numauto_color_hessian!(H, f, x, colorvec, sparsity)</code></pre><p>To avoid unnecessary allocations every time the Hessian is computed, construct a <code>ForwardColorHesCache</code> beforehand:</p><pre><code class="language-julia hljs">hescache = ForwardColorHesCache(f, x, colorvec, sparsity) |
| 84 | +numauto_color_hessian!(H, f, x, hescache)</code></pre><p>By default, these methods use a mix of numerical and automatic differentiation, namely by taking finite differences of gradients calculated via ForwardDiff.jl. Alternatively, if you have your own custom gradient function <code>g!</code>, you can specify it as an argument to <code>ForwardColorHesCache</code>:</p><pre><code class="language-julia hljs">hescache = ForwardColorHesCache(f, x, colorvec, sparsity, g!)</code></pre><p>Note that any user-defined gradient needs to have the signature <code>g!(G, x)</code>, i.e. updating the gradient <code>G</code> in place.</p><h3 id="Jacobian-Vector-and-Hessian-Vector-Products"><a class="docs-heading-anchor" href="#Jacobian-Vector-and-Hessian-Vector-Products">Jacobian-Vector and Hessian-Vector Products</a><a id="Jacobian-Vector-and-Hessian-Vector-Products-1"></a><a class="docs-heading-anchor-permalink" href="#Jacobian-Vector-and-Hessian-Vector-Products" title="Permalink"></a></h3><p>Matrix-free implementations of Jacobian-Vector and Hessian-Vector products is provided in both an operator and function form. For the functions, each choice has the choice of being in-place and out-of-place, and the in-place versions have the ability to pass in cache vectors to be non-allocating. When in-place the function signature for Jacobians is <code>f!(du,u)</code>, while out-of-place has <code>du=f(u)</code>. For Hessians, all functions must be <code>f(u)</code> which returns a scalar</p><p>The functions for Jacobians are:</p><pre><code class="language-julia hljs">auto_jacvec!(dy, f, x, v, |
85 | 85 | cache1 = ForwardDiff.Dual{DeivVecTag}.(x, v), |
86 | 86 | cache2 = ForwardDiff.Dual{DeivVecTag}.(x, v)) |
87 | 87 |
|
88 | 88 | auto_jacvec(f, x, v) |
89 | 89 |
|
90 | 90 | # If compute_f0 is false, then `f(cache1,x)` will be computed |
91 | 91 | num_jacvec!(dy,f,x,v,cache1 = similar(v), |
92 | | - cache2 = similar(v); |
| 92 | + cache2 = similar(v), |
| 93 | + cache3 = similar(v); |
93 | 94 | compute_f0 = true) |
94 | 95 | num_jacvec(f,x,v,f0=nothing)</code></pre><p>For Hessians, the following are provided:</p><pre><code class="language-julia hljs">num_hesvec!(dy,f,x,v, |
95 | 96 | cache1 = similar(v), |
96 | 97 | cache2 = similar(v), |
97 | | - cache3 = similar(v)) |
| 98 | + cache3 = similar(v), |
| 99 | + cache4 = similar(v)) |
98 | 100 |
|
99 | 101 | num_hesvec(f,x,v) |
100 | 102 |
|
101 | 103 | numauto_hesvec!(dy,f,x,v, |
102 | 104 | cache = ForwardDiff.GradientConfig(f,v), |
103 | 105 | cache1 = similar(v), |
104 | | - cache2 = similar(v)) |
| 106 | + cache2 = similar(v), |
| 107 | + cache3 = similar(v)) |
105 | 108 |
|
106 | 109 | numauto_hesvec(f,x,v) |
107 | 110 |
|
|
111 | 114 | cache3 = ForwardDiff.Dual{DeivVecTag}.(x, v)) |
112 | 115 |
|
113 | 116 | autonum_hesvec(f,x,v)</code></pre><p>In addition, the following forms allow you to provide a gradient function <code>g(dy,x)</code> or <code>dy=g(x)</code> respectively:</p><pre><code class="language-julia hljs">num_hesvecgrad!(dy,g,x,v, |
| 117 | + cache1 = similar(v), |
114 | 118 | cache2 = similar(v), |
115 | 119 | cache3 = similar(v)) |
116 | 120 |
|
|
124 | 128 |
|
125 | 129 | numback_hesvec!(dy,f,x,v, |
126 | 130 | cache1 = similar(v), |
127 | | - cache2 = similar(v)) |
| 131 | + cache2 = similar(v), |
| 132 | + cache3 = similar(v)) |
128 | 133 |
|
129 | 134 | numback_hesvec(f,x,v) |
130 | 135 |
|
|
133 | 138 | cache2 = ForwardDiff.Dual{DeivVecTag}.(x, v), |
134 | 139 | cache3 = ForwardDiff.Dual{DeivVecTag}.(x, v)) |
135 | 140 |
|
136 | | -autoback_hesvec(f,x,v)</code></pre><h4 id="J*v-and-H*v-Operators"><a class="docs-heading-anchor" href="#J*v-and-H*v-Operators">J<em>v and H</em>v Operators</a><a id="J*v-and-H*v-Operators-1"></a><a class="docs-heading-anchor-permalink" href="#J*v-and-H*v-Operators" title="Permalink"></a></h4><p>The following produce matrix-free operators which are used for calculating Jacobian-vector and Hessian-vector products where the differentiation takes place at the vector <code>u</code>:</p><pre><code class="language-julia hljs">JacVec(f,x::AbstractArray;autodiff=true) |
| 141 | +autoback_hesvec(f,x,v)</code></pre><h4 id="J*v-and-H*v-Operators"><a class="docs-heading-anchor" href="#J*v-and-H*v-Operators"><code>J*v</code> and <code>H*v</code> Operators</a><a id="J*v-and-H*v-Operators-1"></a><a class="docs-heading-anchor-permalink" href="#J*v-and-H*v-Operators" title="Permalink"></a></h4><p>The following produce matrix-free operators which are used for calculating Jacobian-vector and Hessian-vector products where the differentiation takes place at the vector <code>u</code>:</p><pre><code class="language-julia hljs">JacVec(f,x::AbstractArray;autodiff=true) |
137 | 142 | HesVec(f,x::AbstractArray;autodiff=true) |
138 | | -HesVecGrad(g,x::AbstractArray;autodiff=false)</code></pre><p>These all have the same interface, where <code>J*v</code> utilizes the out-of-place Jacobian-vector or Hessian-vector function, whereas <code>mul!(res,J,v)</code> utilizes the appropriate in-place versions. To update the location of differentiation in the operator, simply mutate the vector <code>u</code>: <code>J.u .= ...</code>.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="sparsedifftools/">API »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Tuesday 19 December 2023 00:42">Tuesday 19 December 2023</span>. Using Julia version 1.9.4.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html> |
| 143 | +HesVecGrad(g,x::AbstractArray;autodiff=false)</code></pre><p>These all have the same interface, where <code>J*v</code> utilizes the out-of-place Jacobian-vector or Hessian-vector function, whereas <code>mul!(res,J,v)</code> utilizes the appropriate in-place versions. To update the location of differentiation in the operator, simply mutate the vector <code>u</code>: <code>J.u .= ...</code>.</p></article><nav class="docs-footer"><a class="docs-footer-nextpage" href="sparsedifftools/">API »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 29 December 2023 08:28">Friday 29 December 2023</span>. Using Julia version 1.10.0.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html> |
0 commit comments