MTK: memory usage and out of memory???

[B-LIE]

Is there a memory problem with MTK❓

I am testing some components of pipes, etc. that I am developing, and have experienced that my Lunar Lake laptop with 32 GB runs out of memory and starts to swap on the disk, and then glitches or crashes completely. Seems like the ODEProblem (prob) and the solution (sol) become extraordinarily large.

Some numbers:

1. Rigid/lumped pipe, 3 unknowns, 30 observed. Base.summarysize():

• sol.t: 360 B
• sol.u: 2920 B
• prob: 1.50 MB
• sol: 1.52 MB

2. Distributed pipe with 100 cells, 300 unknowns, 931 observed. Base.summarysize():

• sol.t: 664 B
• sol.u: 0.18 MB
• prob: 8.67 GB
• sol: 8.67 GB

It gets worse when I connect several pipes, and other equipment.

Why does the prob and the sol become so large? Is there anything I can do to reduce the sizes? I generate the prob via:

prob = ODEProblem(sys, [pipe.md => 0.0], tspan)

sol = solve(prob, Rodas5P())

[oscardssmith]

The reason the sol is so big is that the sol contains a reference to the prob (i.e the 8.7gb of sol is the same memory as the 8.7gb of prob).

[B-LIE]

Why is prob so big, then? How can I avoid that? Or is that impossible?

[ChrisRackauckas]

Did you generate with a dense Jacobian?

[B-LIE]

OK, here is a slight update. This time a pipe component with mass and momentum balance, using second order Kurganov-Petrova scheme for discretization (a high quality MOL method). I am connecting a reservoir -> pipe 1 -> pipe 2 -> valve -> sink. This time, I have discretized both pipes with 100 cells:

julia> tspan = (0.0, 50.0)

julia> prob_hydro = ODEProblem(sys_hydro, [], tspan; sparse=true)

julia> Base.summarysize(prob_hydro)/1024^3
59.99

So: sparse Jacobian. Ca. 60 GByte of memory.

Ooops. There my computer crashed..

[B-LIE]

If anyone of developers are interested in taking a look at it, I can create a set-up that you can run in *.jl files. I would need to re-organize my files a little first, and is not a "minimal" kind of set-up. But perhaps easier to see what is really going on.

[lsablon]

What's your mtk-compiled system size ?
On my side, it's already quite large for bigger problem, and computing the symbolic jacobian is impossible due to the length of the equation that quickly grows when differentiating.

julia> csys
Model model:
Equations (383):
  383 standard: see equations(model)
Unknowns (383): see unknowns(model)
  (SO4(t))[44]
  (SO4(t))[43]
  (SO4(t))[42]
  ⋮
Parameters (104): see parameters(model)
  Ca_accel
  O2_accel
  sink_veloc
  ⋮
Observed (2313): see observed(model)

julia> Base.summarysize(csys)/1024^3
4.843502196483314

EDIT: missed the fact that ODEProblem is also inside the mtk system, hence the size (it's much smaller before creating the ODEProblem, here sparse without jac).