So, I just ran one of your pages through GTMetrix. There’s room for improvement but nothing obvious that would fix your loading times.
As you mentioned, your TTFB for the images is very high. I would assume that’s related to your NginX configuration (not Magento or the Droplet Specs).
But what doesn’t help is that you don’t have any Leverage Browser Caching for your images. Therefore, every time I reload the page my web browser re-downloads the images each time. With Leverage Browser Caching set, my browser would know not to bother downloading the images and just use what it has in its cache. By default, the Magento .htaccess file controls this but relies on you having PHP mod_header enabled to work.
You can see if you have this module installed by checking your PHP Info file. If you don’t see it listed, then Google how to enable both mod_headers and mod_expires for NginX. If it’s anything like Apache, then it should only take 2 minutes to enable.
Also, a CDN wouldn’t hurt.