Add AD support for complex-valued DOFs

[zjwegert]

This PR adds basic autodiff support for problems with complex-valued DOFs. Currently, this is not supported natively because ForwardDiff does not fully support complex-valued functions.

The approach here is to split the computation into real and imaginary parts, and since we're assuming that the functional is differentiable, we can compute the derivative by only perturbing the real parts of the cell dofs. I thought the best approach would be to dispatch on T=eltype(get_vector_type(V)) and add new methods autodiff_array_gradient(::Type{<:Complex},...) and autodiff_array_gradient(::Type{<:Complex},...) that split the computation as above.

There are three cases that we consider:

1. Holomorphic maps from ℂ → ℂ. The standard approach in AD systems is to compute the conjugate gradient f'(z)ᴴ = ∂ᵣu + i ∂ₛu so that dF(z)(v) = <f'(z)ᴴ, v> = f'(z)v.
2. Non-holomorphic maps from ℂ → ℝ. Using CR-Calculus we have that the gradient is given by ∇z(f) = 2∂f/∂z* = ∂ᵣu + i ∂ₛu and the directional derivative is given by dF(z)(v)=Re{∇z(f),v}.
3. (NOT IMPLEMENTED) General non-holomorphic maps from ℂ → ℂ: requires splitting into derivative in z and derivative in z*. This is not implemented yet. To do this one properly, we should probably implement a new kind of FEOperator called NonholomorphicFEOperator that appropriately builds the Jacobian's in z and z* and implements a basic Newton method.

Because (1) and (2) are in the same form, we reuse the code below for both cases.

@JordiManyer, If we're happy with this PR, I'll make the appropriate changes to GridapDistributed as well.

[codecov]

Codecov Report

❌ Patch coverage is 92.85714% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.83%. Comparing base (9bc344b) to head (91bc250).

Files with missing lines	Patch %	Lines
src/Arrays/Autodiff.jl	76.92%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1325      +/-   ##
==========================================
- Coverage   88.83%   88.83%   -0.01%     
==========================================
  Files         228      228              
  Lines       30132    30177      +45     
==========================================
+ Hits        26768    26807      +39     
- Misses       3364     3370       +6     

Flag	Coverage Δ
drivers	39.48% <20.23%> (-0.05%)	⬇️
extensions	5.00% <0.00%> (-0.01%)	⬇️
unit-adaptivity	40.10% <10.71%> (-0.07%)	⬇️
unit-basics	14.40% <39.28%> (-0.02%)	⬇️
unit-celldata	20.84% <1.19%> (-0.04%)	⬇️
unit-fespaces-1	32.62% <17.85%> (-0.05%)	⬇️
unit-fespaces-2	39.47% <78.57%> (+0.06%)	⬆️
unit-fields	17.25% <28.57%> (-0.03%)	⬇️
unit-geometry	28.37% <0.00%> (-0.05%)	⬇️
unit-multifield	30.75% <78.57%> (+0.26%)	⬆️
unit-odes	28.33% <15.47%> (-0.03%)	⬇️
unit-referencefes	33.91% <10.71%> (-0.06%)	⬇️
unit-visualization	11.67% <0.00%> (-0.02%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[JordiManyer]

Hi @zjwegert , thanks for the contribution. Very interesting, I didn't know you could do this. I added some comments on the reusability of the code. I think it would help to make the whole AD module more maintainable. Cheers!

[zjwegert]

Hi @zjwegert , thanks for the contribution. Very interesting, I didn't know you could do this. I added some comments on the reusability of the code. I think it would help to make the whole AD module more maintainable. Cheers!

Thanks @JordiManyer ! I'll take a look when I get a chance.

[zjwegert]

Hey @JordiManyer, I've made the changes you requested and updated the NEWS. Cheers

[zjwegert]

Hi @JordiManyer, this should be ready now. I'm happy to make changes in GridapDistributed once you're happy with this PR.

Edit: I've also added adjoint for ArrayBlocks

[JordiManyer]

Hi @zjwegert I have a couple style comments, but otherwise I think the changes are good. I will merge after!