Handling of `Retry-After` for 429 in `resources.GetRemote`

support
1

Currently a error 429 (Too many request) triggers a quadratic backoff, see Add retry in resources.GetRemote for temporary HTTP errors by bep · Pull Request #11326 · gohugoio/hugo · GitHub

But for several APIs this isn’t the best possible solution, like the ones provided by Wikipedia. In those APIs a response with error 429 also has a Retry-After which indicates after what wait time the next request should be made. I’m well aware that this might be longer then the configured timeout, so this might not help either. But it gives a better indication.

Has anyone here has an idea how to handle those? Since the isn’t a wait or sleep it might be hard to implement it in a template.

Since @irkode wrote this post on paged APIs, maybe he has an idea?

2

What are the typical values returned there? Seconds or hours? If it’s the first, it would make sense to add some support for it inside Hugo.

But if it’s hours, I would probably go for a bash scripter that has some retry logic (e.g. run hugo until no error of a given type).

3

Wikipedia returns seconds, I’ve seen up to a lower three-digit range (around 2 to 3 minutes). I would argue that those should only be respected if lower then the timeout setting otherwise fail with a message displaying both, like “ERROR: API expected to wait for X seconds, but timeout is Y seconds (which is less)”

4
5