Advanced GetRemote headers usage with multipart/form-data

[regis]

I’m trying to work with an API that does not take binary data, only parts (as in upload a file)

Which means the headers would look like this:

Content-Type: multipart/form-data; boundary=535e329ca936f79a19ac9a251f7d48f7
Authorization: HHMAC; key="xxx"; signature="xxx"; date="2015-03-05T03:00:27Z"
--535e329ca936f79a19ac9a251f7d48f7
Content-Type: application/json
Content-Disposition: form-data; filename=article.json; name=article.json
{
 content of file
}

There can be multiple file upload, each following a boundary string.

Two questions

1. This set of headers contains two Content-Type. As the headers option for GetRemote is a Map created with dict, I have no way (to my knowledge) to pass more than one header “entry” with the same key.
2. I’m not sure how to add the boundary, as it seems to be a new line with no key, just a string. I suppose I could just include it at the end of the value of the preceding header? But not sure how Hugo would process it.

[bep]

I’m not sure how feasible it would be to use GetRemote for multipart/form-data (esp. with the concept of … multi).

But I’m pretty sure:

resources.GetRemote "https:/foo" (dict "headers" "Content-Type: (slice "multipart/form-data" "boundary=535e329ca936f79a19ac9a251f7d48f7")

Would translate into something like:

Content-Type: multipart/form-data; boundary=535e329ca936f79a19ac9a251f7d48f7

[regis]

Hey thanks for jumping in!

I’m not sure I follow though. I might have not well communicated my predicament or misunderstood your explanations. I’ll take the liberty to clarify if this is the former. My problem is not with the semi-colon seperated list, this I can write as a string. My problem lays elsewhere.

Let’s say Content-Type: application/json is a header declaration and so is --535e329ca936f79a19ac9a251f7d48f7, the boundary.

I need to pass in the headers two declarations or more with the same key
Content-Type: multipart/form-data; boundar...
AND
Content-Type: application/json for the next part (the json file)`.

I also need to pass several declarations which have no KEY, the boundaries as they stand as lone strings --535e329ca936f79a19ac9a251f7d48f7

How could I achieve that using your suggestion?

[i40west]

The multipart boundary is not a header, it is part of the body.

The documentation does include an example of setting the request body with GetRemote. It looks like you will have to create the sub-part manually within the body.

[regis]

Hey thanks! That’s interesting. I’m often using the body to post application/json which is a basic json string, but in this case, do you think we can simply use the example strings posted above as the body

{{ $body := `

--535e329ca936f79a19ac9a251f7d48f7
Content-Type: application/json
Content-Disposition: form-data; filename=article.json; name=article.json
{
 content of file
}
etc...
`
{{ resrouce.GetRemote $url (dict "body" $body "headers" $headers") }}

I’ll definitely try something like this in the morning but looking at this it should work!

Thank you so much this is gold! Looking as those raw examples, I thought they were only headers and finding resources about concrete example is hard as any request client has its own method to generate body/headers etc…

In any case thanks a ton!

[i40west]

Yes, I think that should work.

[regis]

Alright, making progress but something’s off. I must admit it might be my misunderstanding of the API documentation or the whole multipart request. But some error messages are encouraging

In the off chance that this is linked to my formating of the body using Hugo, I’ll post here as a follow up…

This is how I construct my body:

{{ $body := printf
`--535e329ca936f79a19ac9a251f7d48f7\r\n
Content-Type: application/json\r\n
Content-Disposition: form-data; filename=article.json; name=article.json\r\n
%s\r\n
--535e329ca936f79a19ac9a251f7d48f7--
` $article_content }}

The $article_content contains a jsonified object but the resulting error is:

{
"code":"MISSING",
"keyPath":["article.json"]
}

Which by reading the doc means the article file is missing.

Before adding the \r\n to the boundaries I had a “INVALID_MIME_MULTIPART” code, so at least it means the body is not misformed… maybe the article is…

It might be that the printf of the json string is somehow not valid. But printing it on a page, it looks good (no escaped quotes etc…)

I realize, this might require much more that than the limited information I’m pasting here for in order to properly help, but maybe someone will notice a low hanging fruit I’m missing…

Thanks!

As a reference here is what the APPLE NEWS doc propose as an example:

POST /channels/63a75491-2c4d-3530-af91-819be8c3ace0/articles HTTP/1.1
Host: news-api.apple.com
Accept: application/json
Content-Type: multipart/form-data; boundary=535e329ca936f79a19ac9a251f7d48f7
Content-Length: 94348
Authorization: HHMAC; key="1e3gfc5e-e9f8-4232-a6be-17bf40edad09"; signature="gJYsk8qm+6wpONE8twX/yDlGRt0UzS7Qn32yHNpiwnM="; date="2015-03-05T03:00:27Z"
--535e329ca936f79a19ac9a251f7d48f7
Content-Type: application/json
Content-Disposition: form-data; filename=article.json; name=article.json
{
    "version": "1.7",
    "identifier": "SampleArticle",
    "language": "en",
    "title": "Apple News App",
    "subtitle": "A look at the features of the News iOS app",
    "layout": {
        "columns": 7,
        "width": 1024,
        "margin": 75,
        "gutter": 20
    },
    "components": [
        {
            "role": "title",
            "text": "Apple News App",
            "textStyle": "title"
        },
        {
            "role": "body",
            "text": "The Apple News Format allows publishers to craft beautiful editorial layouts. Galleries, audio, video, and fun interactions like animation make stories spring to life."
        },
        {
            "role": "photo",
            "URL": "bundle://image.jpg"
        }
    ],
    "documentStyle": {
        "backgroundColor": "#F7F7F7"
    },
    "componentTextStyles": {
        "default": {
            "fontName": "Helvetica",
            "fontSize": 13,
            "linkStyle": {
                "textColor": "#428bca"
            }
        },
        "title": {
            "fontName": "Helvetica-Bold",
            "fontSize": 30,
            "hyphenation": false
        },
        "default-body": {
        "fontName": "Helvetica",
        "fontSize": 13
        }
    }
}
--535e329ca936f79a19ac9a251f7d48f7
Content-Type: image/jpeg
Content-Disposition: form-data; filename=mountain-lions.jpg; name=my_image
{binary content of mountain-lions.jpg}
--535e329ca936f79a19ac9a251f7d48f7
Content-Type: application/octet-stream
Content-Disposition: form-data; filename=gradient.png; name=a_gradient
{binary content of gradient.png}
--535e329ca936f79a19ac9a251f7d48f7--

[i40west]

You’re putting a second boundary before $article_content. Replace that with another \r\n.

Edit: no, I misunderstood that it’s a printf. That’s not it.

[regis]

I don’t think so. The ending boundary the one with --x-- is after the %s string which printf will replace with $article_content.

Unless I’m missing something…

[i40west]

@regis

Unless I’m missing something…

You’re not, I edited my reply after realizing it was a printf with the substitution.

I’m wondering about the combination of \r\n and literal linefeeds. Maybe use one or the other?

[regis]

yes read somewhere (can’t find the link of course) that \r\n was the closest to CRLF which is advised here.

But I get what you mean, I tried without the litteral as well.

I tried with one or the other but get the same error.