The server ran out of metaspace, not heap
What causes it
Class metadata space filled up, which happens with very large modpacks or a -XX:MaxMetaspaceSize cap set too low.
How to fix it
Remove any MaxMetaspaceSize flag from the start command, or raise it. More -Xmx does not help this one.
Not sure this is your problem? Paste your log and Roastd will tell you which of these it actually matched, and what else it found.
Check your own log