EOF-Library: Open-Source Elmer FEM and Openfoam Coupler For
Total Page:16
File Type:pdf, Size:1020Kb
EOF-Library: Open-source Elmer FEM and OpenFOAM coupler for electromagnetics and fluid dynamics (CC BY-SA 4.0) Motivation for MHD - controlling liquid metals Liquid metal pump rotating permanent pipe magnets F1 F2 F FL FL Mechanically FL FL Electromagnetically Required physics Modelling coupled ● Electromagnetics ● Fluid dynamics Extra ● Free surface ● Heat transfer Liquid metal pump - modelling requires two-way ● Solidification / melting coupled electromagnetics and fluid dynamics 1 Required numerical models Fluid dynamics ● 3D, transient Electromagnetics ● High Reynolds ● Complex geometries, turbulence models multi-regions ● Complex A−V ● Volume of Fluid magnetic vector ● Advanced pre- and potential (VOF) for free post-processing tools surface modelling formulation ● Parallelization with ● Conductivity ● Viscosity good scaling dependence on dependence on temperature ● User community, temperature documentation 2 Existing Open Source codes Best for Best for Electromagnetics (EM) Fluid dynamics (FD) FEM FVM getDP OpenFOAM Elmer SU2 ... ... 3 Choosing Electromagnetics solver (1) GetDP + Designed for EM in mind, great for eddy-current problems + Uses PETSc iterative solvers, PETSc is MPI parallelized - Matrix assembly is not parallel (neither MPI nor OpenMP) - Weak integration with external post-processing tools MaxFEM, FreeFem++, deal.II, … Pictures from getdp.info ● I have no opinion 4 Choosing Electromagnetics solver (2) Elmer by CSC (Finland) + Multiphysics out-of-the-box (>40 solvers) + Build-in linear & nonlinear iterative solvers, fully parallelized + Build-in computation of 3D electrical currents resulting from VxB term - very important for MHD and rotating magnets + Extensive documentation, active forum, responsive developers - Adding new functionality requires knowledge in FEM and Fortran 5 Choosing Fluid dynamics solver ● In terms of popularity and functionality OpenFOAM is leader + Most important - it has Volume of Fluid (VOF) + OpenFOAM = solvers + examples + community + impact 6 No one code solves our problem .., but two coupled codes can! Elmer + OpenFOAM ● Both can simulate real stuff ● Common pre/post processing (Salome, Paraview) ● Straight-forward mesh interpolation ● Most important - Parallel MPI-based coupling is feasible 7 Elmer OpenFOAM = EOF-Library Two-way coupling File based ● SLOW and does not SCALE on parallel computers MPI (Message Passing Interface) based ● Both codes already use MPI for parallelization ● Probably the most EFFICIENT coupling solution 8 MPI - Message Passing Interface Running on 16 cores: mpirun -n 16 Elmer : -n 16 OpenFOAM “Wrapper Elmer OpenFOAM (16 processes) (16 processes) that starts applications MPI_SEND( MPI_RECV( data, data, in parallel” type, type, size, size, to whom) from whom) MPI_RECV(...) MPI_SEND(...) 9 In the code Interpolating and sending fields from Elmer to OpenFOAM Checks whether Elmer sends message, if not then sleeps 10ms (saves CPU resources) Projector matrix (interpolation) is applied on variable 10 Structure of EOF-Library Patch for splitting MPI_COMM_WORLD on Coupler’s OpenFOAM part OpenFOAM side (every OF version has its own patch) ├── coupleElmer │ ├── Elmer.C │ ├── Elmer.H ├── commSplit │ └── Make │ ├── allReduce.H │ ├── files │ ├── allReduceTemplates.C │ └── options │ ├── Make │ │ ├── files │ │ └── options Coupler’s Elmer part │ ├── PstreamGlobals.C │ ├── PstreamGlobals.H ├── Elmer2OpenFOAM │ ├── UIPread.C │ └── Elmer2OpenFOAM.F90 │ ├── UOPwrite.C └── OpenFOAM2Elmer │ └── UPstream.C └── OpenFOAM2Elmer.F90 11 Example for modified OpenFOAM solver ├── mhdInterFoam │ ├── alphaCourantNo.H │ ├── alphaEqn.H Current field value │ ├── alphaEqnSubCycle.H │ ├── correctPhi.H Field value when │ ├── createFields.H Elmer was called │ ├── Make last time │ │ ├── files │ │ └── options │ ├── mhdInterFoam.C │ ├── pEqn.H │ ├── setDeltaT.H │ ├── setRDeltaT.H │ └── UEqn.H User is responsible about physics 12 Simulation workflow using open-source software EOF-Library CAD & meshing Post-processing Salome Elmer FEM OpenFOAM Paraview Possible* coupling combinations * not all of them are tested OpenFOAM Elmer FEM Liquid Gas Heat Induction Microwaves f(T) Parametric Rarified Particle Steady & dependence gas tracking transient Radiation Application examples Electromagnetic levitation melting Surface wave generation using low frequency EM field Interpolation test - unit cube Elmer & OpenFOAM Elmer-to-OpenFOAM meshes are different Tet size 0.1 0.05 0.025 Relative 8.0 2.3 0.57 Refinement level Error % ● 0.1 (4.7k tets) ● 0.05 (32k tets) ● 0.025 (245k tets) OpenFOAM-to-Elmer Tet size 0.1 0.05 0.025 Reference scalar field Relative 11 4.0 1.5 2 sin[2πx] cos[2πy] Error % Strong scaling test 13.1M cells / elements Conclusion: Coupler makes a negligible performance overhead comparing to computation time Finally - obtaining the code https://github.com/jvencels I typically use Salome for CAD and meshing Useful salome script for exporting mixed (tet+hex) meshes directly to Elmer Similar script for OpenFOAM https://github.com/nicolasedh/s alomeToOpenFOAM 13 EOF-Library.com ● Research ● Development ● Support [email protected].