This is a known upstream issue:
https://github.com/yuin/goldmark/issues/347
And I suspect there are several closed issues related to this.
Historically, and understandably, the project lead has had little interest in working on the typographer extension.
My suggestion would be to use single quotes for contractions and possessive forms, and use double quotes everywhere else. Or you can bypass the typographer extension as needed by escaping the character:
Do it \'like this\' not 'like that'.
Which yields:
