Hi @jhalttu and welcome.
There can be a plethora of reasons why you’re experiencing this. Here are some quick wins you can look at Why is Magento 2 so slow? 12 EASY Tips to make Magento 2 faster.
If it’s server related, then you’re better off asking unix.stackexchange.com as a deep dive into server configuration is out of scope of this forum.