Representation of multivariate processes

Following PR #51, multivariate processes are represented as marked processes with i.i.d. integer marks, each mark representing one marginal process. This is fine for Poisson processes, since the marks do not influence the intensity of the process.

For Hawkes processes, however, marks may influence the intensity, so a univariate Hawkes process with integer marks is not equivalent to a multivariate Hawkes process. Currently I am treating any integer marks as really meaning the marginal process an event occurs, but I think this is not great.

This was shortly discussed in #43. I am not currently considering processes with vector-valued marks, so the options I see are:

Define that this package deals only with univariate processes, but because of the structure of the Poisson process, we can interpret this univariate process as a multivariate one.
Fields times and marks of History hold vectors of vectors, instead of vectors.
Add another field that holds a vector containing the marginal process corresponding to each event. If this field contains a vector filled with nothing, than it is a univariate process.
Have multiple subtypes of History, which may or may not have the fields marks and marginal_process (or whatever name).

I would not consider (1), since I started working on mutually exciting Hawkes processes. Between (2) and (3), I guess (2) is a more straightforward way to represent multivariate processes, but (3) has the benefit that the event times are already ordered, which is needed for pretty much every method related with multivariate Hawkes processes. (4) is possibly the most efficient (?), but a similar functionality can be obtained using aliases.

Thoughts? Other alternatives?

What I came up with, which should not change the interface too much, so we can still use what we have with small modifications is this (without the complete implementation of the constructors, so it is not too long)

struct History{N,T<:Real,M,D} # N is the number of dimensions, D can be Int or Nothing
    times::Vector{T}
    tmin::T
    tmax::T
    marks::Vector{M}
    dims::Vector{D}     # In which of the 'N' marginal process the event occurs

    # Add optional arguments check
    # Inner constructors are the standard for the methods to create a `History`
    function History{N}(times::Vector{<:Real}, tmin::Real, tmax::Real, marks::Vector, dims::Vector{<:Int}) where {N}
        new{N,T,M,D}(times, tmin, tmax, marks, dims)
    end

    function History{1}(times::Vector{<:Real}, tmin::Real, tmax::Real, marks::Vector, ::Any)
        new{1,T,M,Nothing}(times, tmin, tmax, marks, dims)
    end
end

const UnivariateHistory{T<:Real,M} = History{1,T,M,Nothing}

# Outer constructors are useful for inputing data
function History(times::Vector{<:Vector{<:Real}}, tmin::Real, tmax::Real, marks::Vector{<:Vector}) 
    # From a vector of vectors to the representation used in the `History` struct
    return History{N}(times_vec, tmin, tmax, marks_vec, dims)
end

function History(times::Vector{<:Real}, tmin::Real, tmax::Real)
    History([times], tmin, tmax; check=check)
end

function History(times::Vector{<:Real}, tmin::Real, tmax::Real, marks::Vector)
    History([times], tmin, tmax, [marks]; check=check)
end

function History(times::Vector{<:Vector{<:Real}}, tmin::Real, tmax::Real)
    marks = [fill(nothing, length(times[i])) for i in 1:length(times)]
    History(times, tmin, tmax, marks; check=check)
end

For univariate processes, I guess nothing changes. The call to the constructor and the fields are the same
, and methods should all work (maybe with some minor adjustments for the added dims)

This seems like a good approach to me. I like that it won'i interfere with the current implementation too much. If I understand correctly we could do something this for a bivariate process?

h = History{2}(times=[1.0, 1.5, 2.5, 3.7], tmin=0.0, tmax=10.0,
               marks=[m1, m2, m3, m4], dims=[1, 2, 1, 1])

Exactly. We have the two alternatives

History{2}(times=[1.0, 1.5, 2.5, 3.7], tmin=0.0, tmax=10.0, marks=[m1, m2, m3, m4], dims=[1, 2, 1, 1])
# or
History(times=[[1.0, 2.5, 3.7], [1.5]], tmin=0.0, tmax=10.0, marks=[[m1, m3, m4], [m2]])

And if times and marks are vectors instead of vectors of vectors in the second alternative, it constructs a univariate process.

Yeah i think this is good i say move forward with it

So, this was done in PR #77. All tests pass without changing the implementation, since, as expected, this does not change anything for univariate processes.

The issue now is that this requires changing how processes are implemented. Instead of using the Categorical distribution in mark_dist to mock a multivariate process, we can make them actually multivariate, so we need to change the implementation.

I guess the following is the obvious answer. The logic of the methods won't change much, but I guess everything would need to be adjusted.

abstract type AbstractPointProcess{N} end

# Both univariate and multivariate processes hold a vector of parameters, even for N=1
struct PoissonProcess{N,R<:Real} <: AbstractPointProcess{N}
λ::Vector{R}
mark_dist::Vector{<:Distribution}
end

hmm okay. What are the downstream ramifications you’re expecting if we switch to AbstractPointProcess{N}?

My worry is that to keep things type-stable we’d need to thread N through a lot of APIs (or enforce invariants everywhere at runtime), and that feels a bit cumbersome. Not saying that we shouldn't pursue--just thinking out loud/wondering how xpansive this would be

@gdalle

Do we really need to pass N as a type parameter ?
If I remember correctly, one of @gdalle's arguments against this implementation is that JIT is applied each time you modify the dimension of your processes. I guess that it is related to @rsenne's comment too.

I think it would be possible to pass N as a field of the struct instead. Don't you think ?

Definitely don't put N as a type parameter, you'll just strain the compiler needlessly

@gdalle

hmm okay. What are the downstream ramifications you’re expecting if we switch to AbstractPointProcess{N}?

My worry is that to keep things type-stable we’d need to thread N through a lot of APIs (or enforce invariants everywhere at runtime), and that feels a bit cumbersome. Not saying that we shouldn't pursue--just thinking out loud/wondering how xpansive this would be

I guess this will require changing a lot of stuff. As I mentioned, the logic will remain basically the same, but I would say all methods involving an AbstractPointProcess will need to be modified in some way.

Do we really need to pass N as a type parameter?
If I remember correctly, one of @gdalle's arguments against this implementation is that JIT is applied each time you modify the dimension of your processes. I guess that it is related to @rsenne's comment too.

I think it would be possible to pass N as a field of the struct instead. Don't you think?

That makes sense, but isn't that true for History too?
Of course we do not need to pass it as a type parameter, passing as a field is also fine. The only thing is that this enforces the history and the process to have the same dimensionality at compile time, instead of at runtime with manual checks.
And wouldn't this be an issue only for applications where there are many processes with different number of dimensions? I don't think this is usually the case?

The only thing is that this enforces the history and the process to have the same dimensionality at compile time, instead of at runtime with manual checks.

You don't need to perform any manual checks at runtime. If you try to compute an intensity for a 3-dimensional Poisson process on a 2-dimensional history, you'll trigger a DimensionMismatch error, which is much more informative to users than the MethodError you'd get from a type parameter.

In general, specializing on the length of an array is a bad idea because it gives rise to as many methods as you have array lengths and creates type instabilities, mostly without any benefits. At least, with StaticArrays.jl, you get the benefits of unrolled operations making things faster, but even then, it is only recommended for very small arrays (length < 100).

Ok, then. I see.
I guess we can add a field N in History instead of adding it as a type parameter. This will be needed in case the last marginal history is empty and to avoid having to compute minimum(h.dims) and maximum(h.dims) to get the dimensionality.
For the processes we don't actually even need this N, then. We can just add the method n_dims (or something like this) that returns the length of the vector of parameters of the process.
Thanks for the feedback!

So, I got back to working on this and in fact wrote the multivariate versions of the homogeneous Poisson process, but as I was writing yesterday I realized there is a better way to implement multivariate processes.
We just leave everything as it is, no need to rewrite anything, and multivariate processes will be implemented separately. Specifically I thought we could have something like

IndependentMultivariateProcess -> Takes a vector of processes. Simulation and estimation each process is done independently, based only on its own history. So any set of univariate processes would work here, which already gives us multivariate Poisson processes.
DependentMultivariateProcess -> Also takes a vector of processes,. In simulation, instead of the processes using only its own history, it uses the whole history from all processes combined. Any univariate process can be just plugged in here, but would probably require custom processes. Not sure about this one, but maybe it makes sense.
Other custom multivariate processes -> Such as the MultivariateHawkesProcess, which cannot really be expressed as a vector of separate processes, since parameters depend on how many dimensions are there

Implemented multivariate processes separately from univariate processes. In fact, added the abstract types AbstractUnivariateProcess and AbstractMultivariateProcess.
Still need to better define the interface for multivariate processes, add tests, and change the implementation of the priors for multivariate Poisson.
~~Also, I am getting an error from JET during testing. Not sure how to get rid of it, though~~ Adding AbstractVector type annotation to times solved it

 ┌ event_times(h::History, tmin::Real, tmax::Real, d::Int64) @ PointProcesses /home/jkling/projects/PointProcesses.jl/src/history.jl:
147
 │ no matching method found `Int64(::DataStructures.Tokens.IntSemiToken)` (1/2 union split): i_min = PointProcesses.Int(PointProcesses.searchsortedfirst(times, tmin::Real)::Union{Int64, DataStructures.Tokens.IntSemiToken})
 └────────────────────

I'm sorry, I no longer have the bandwidth to follow what happens here and give advice in a timely manner. This is your project now, so I trust your judgement! As long as you label the versions properly (and don't release something breaking as a patch version), nothing truly bad can happen anyway.

@gdalle

No worries, @gdalle.
I am trying to find some time here and there to continue the project. I just find it useful to have some progress on the issues documented, so also I know what is going on if I don't manage to work on it for a while.
Anyway, thanks for all the help so far.

Implemented in PR #79

@JoseKling

Hey @JoseKling sorry for not giving input on this. I've been busier than usual lately and probably can't give 100% until this summer. Feel free to request me for future code review though--I'll treat this as a priority.

@rsenne

Hey @rsenne. No worries.
I guessed you were also quite busy, and since what I did doesn't really affect what we already had I just went ahead and merged.
I also haven't had too much time lately, but I will try to keep it going.
If I feel I really need some input I will definitely tag you ; )

Navigation Menu

Repository navigation

Representation of multivariate processes #73

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Representation of multivariate processes #73

Description

Activity

JoseKling commented on Dec 19, 2025

rsenne commented on Dec 21, 2025

JoseKling commented on Dec 21, 2025

rsenne commented on Dec 21, 2025

JoseKling commented on Jan 14, 2026

rsenne commented on Jan 17, 2026

jucheval commented on Jan 18, 2026

gdalle commented on Jan 18, 2026

JoseKling commented on Jan 20, 2026

gdalle commented on Jan 21, 2026

JoseKling commented on Jan 21, 2026

JoseKling commented on Mar 28, 2026

JoseKling commented on Apr 8, 2026

gdalle commented on Apr 10, 2026

JoseKling commented on Apr 11, 2026

JoseKling commented on Apr 25, 2026

rsenne commented on Apr 27, 2026

JoseKling commented on Apr 27, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions