I have perhaps the same problem. I used 0.54 with no trouble in a virtual machine allocated 3GB of memory, with Firefox and some other GUI applications running. I never had problems with that.
0.55.0 didn’t work with my site (“invalid taxonomy state” panic), but the 0.55.2 bugfix release does work. However, with 0.55.2, I could barely build the site with 3GB of memory allocated to the VM. Sometimes it would fail with a similar out of memory panic as what you’re experiencing. When it worked, it might take 5 minutes to build.
I increased memory allocation to 6GB, and now 0.55.2 works, but it’s still slow. And that’s more memory than I want to give to this VM.
Anyone else finding 0.55.x to be very memory hungry?
I have tests showing a more than 10% reduction in memory allocation and usage for general Hugo test building, but this of course depends on “what you use”. I may have introduced “something” in that release – which was not intended. I could have a look at it, but then I need to see a “prolbme site source”.