Pretty sure that’s a RAM issue (i.e. not enough). What specs does your server have? I presume you’re running on something between 1-2GB?
There’s a couple of things you can do to get past this. But bear in mind, you’ll have to wipe your /var/www/html folder after every attempt otherwise composer will be like “I can’t install anything cause there’s stuff in your folder already!”, or something like that.
Solution 1: Upgrade
Upgrade VPS to a minimum of 4GB.
Solution 2: swap-file
Create yourself a swap-file for your VPS. This allocates some of your diskspace to help with machines will less RAM than is required. Simply run the following as root user, one step at a time:
fallocate -l 1G /swapfilechmod 600 /swapfilemkswap /swapfileswapon /swapfile
You’ll know if you’ve done this correctly, because if you type top in the CLI (“top” is the unix equivalent to Windows Task Manager), you’ll see KiB Swap. This will show ~1000000.
(Ctrl+C to exit this screen by-the-way)
This will give you 1GB of swap-file. It’ll reset after a reboot though. This is more or a temporary solution really, because you’ll run into similar issues whilst upgrading Magento in the future.
Disclaimer:
As always, create backups or test on dev machines before implementing on a Production Server.
Let me know how you get on.