Convert a slice to a map

I broke my head, but it looks like there’s no straight forward way to convert a slice to a map.
There’s dict accepting single elements but it would be cool to have something to expand the list in place and use for dict:

  $map = dict (flatten $list)

would ease up things

I can do

$map := dict "a" 1 "b" 2

{
  "a": 1,
  "b": 2
} 

sometimes a have a slice in a variable want to do cerate a map for it:

$list := slice "a" 1 "b" 2
$map = dict $list
{
  "a": 1,
  "b": 2
} 
that will raise: <dict $list>: error calling dict: invalid dictionary call

ofc, I can loop and manually build up, using dict and merge, maybe a store…
maybe I just could’nt work out a combination of collection functions…

any ideas or best practice?

Not sure if that is best practice (script kiddie solution), but I would range over the slice and hold a variable that is either true or false. Then toggle the variable at the end of the range. So it’s either an odd item (first, third, fifth, …) or an even. Put the odd item (key) into a holder variable, then in even ranges add to a second holder - the dict. This is probably using up time, but you also probably won’t do that on a regular base :wink:

Sample code, unchecked and just written down:

{{ $evenodd := false }}
{{ $dictcontainer := dict }}
{{ $keycontainer := "" }}
{{ $slice := "a" 1 "b" 2 }}
{{ range $item := $slice }}
  {{ if $evenodd }}
    {{ $dictcontainer := merge $dictcontainer (new dict with $keycontainer and . as key value pair) }}
  {{ else }}
    {{ $keycontainer = . }}
  {{ end }}
  {{ if eq true $evenodd }}
    {{ $evenodd = false }}
  {{ else }}
    {{ $evenodd = true }}
  {{ end }}
{{ end }}

As far as I read through the documentation (of golang) there is no way to do it with an internal command. They suggest using a for-loop that increases by 2 and then use $slice[0] and $slice[1] to merge into a dict. Basically what I scriptkiddily do above.

1 Like
{{/* map to slice */}}
{{ $s := slice }}
{{ range $k, $v := $m }}
  {{ $s = $s | append $k $v }}
{{ end }}

{{/* slice to map */}}
{{ $m := dict }}
{{ range seq 0 2 (sub (len $s) 1) }}
  {{ $m = merge $m (dict (index $s .) (index $s (add . 1))) }}
{{ end }}

For the last one you’ll want to make sure you have an even number of elements.

2 Likes

That’s the one I was thinking of with my for-loop comment.

1 Like

thank you @davidsneighbour , @jmooring

I had a similar approach as david using mod to calculate even but the flag is cool.
joes solution is a real neat one liner

thx @davidsneighbour for pointing me to Go. Read a little there and the big difference with these approaches is that Go has direct access methods and we don’t (same for array) and have to kind of copy/merge so creating new objects

  • Go: map["key"] = val
  • Hugo: $map = merge $map (dict "key" "val")

I tried Store.SetinMap with both variants

{{ range }}
   {{ $.Store.SetInMap "map" key val }}
{{ end }}
{{ $map = $.Store.Get "map" }}

For the ones interesting in full codes:

Scriptkiddy
{{ $map = dict }}
{{ $key := "" }}
{{ $even := true }}
{{ range $list }}
   {{ if $even }}
      {{ $key = . }}
   {{ else }}
      {{ $map = merge $map (dict $key .) }}
   {{ end }}
   {{ $even = (not $even) }}
{{ end }}
Neat One
{{ $map = dict }}
{{ range seq 0 2 (sub (len $list) 1) }}
   {{ $map = merge $map (dict (index $list .) (index $list (add . 1))) }}
{{ end }}
SetInMap - Scriptkiddy
{{ $even := true }}
{{ $key := "" }}
{{ range $list }}
   {{ if $even }}
      {{ $key = . }}
   {{ else }}
      {{ $.Store.SetInMap "map" $key . }}
   {{ end }}
   {{ $even = (not $even) }}
{{ end }}
{{ $map = $.Store.Get "map" }}

SetInMap - Neat One
{{ range seq 0 2 (sub (len $list) 1) }}
   {{ $.Store.SetInMap "map" (index $list .) (index $list (add . 1)) }}
{{ end }}
{{ $map = $.Store.Get "map" }}

To summarize that:

  • there’s no build in method to convert a slice to a map.
  • you will have to implement that on your own.

General finding

  • the Neat solution is a neglectable slower than ScriptKiddy
    guess due to generating a new seq for looping and, index access
  • the setinMap has a startup overhead but as data grows setInMap it is faster
    many merges are more expensive than later duplicating the Stores map

Performance

The performance depends on the size of the list and internal methods used.

  • if you don’t have large lists or complex structures it does not matter too much
  • if you have - the best implementation is that one for your data structures
  • don’t optimize unless necessary :wink:

So, in the original problem, it would be nice if dict could support a slice as a single argument …

1 Like

yeah…

would speed up map generation due to internal looping with direct element access in Go.

want me to file an enhancement request?

another option or even paralell would be a setter method to directly modify map values as within a store. SetInMap

mapset $map key value
mapget $map key

:smiling_face:

Please do.

That’s not happening, I’m afraid. Maps really needs to be immutable. The scratch struct is the exception here, but the .Values method (which gives you the backing map) comes with a big warning; doing {{ range $k, $v := .Page.Scratch.Values }} is not working in a concurrent build when other goroutines/threads are writing to that same map.

thx for the explanation, expected something like that

will file the ER for the list->map tomorrow. thx.

Out of curiosity, where would this be useful? I’ve needed to go from a map to a slice (e.g., before we changed collections.Querify to accept a map), but never the other way around.

Also, would this template function handle only the first level, or would we need to walk the input? For example, to what would this be transformed?

{{ $s := slice "a" 1 "b" (slice 2 3 4) "c" (dict "d" 5 "e" (slice 6 7 8)) }}

maps allow direct access to fields of stored objects by name, often you get a list back and want to convert that to a map to later address fields or add new fields. also extracting fields from multiple data files building up one object for later use …

due to the fact, that a map is immutable and you cannot add elements in one run using a single dict statement you will have to merge that. This needs a copy and is quite expensive especially with larger datasets.

seen often in code it is imho a common design pattern I to collect a set of options to a dict.

pushing all that attributes to a slice using append and later convert the slice to a map imho is much faster. So it might be mostly a thing for large datasets or large sites that do that very often

I found that using the SetInMap approach from above is about 3 times faster than the merge for a list of 50 objects 2times for 30 (just timer.Stop for a run of hugo )

moving such operation to the internals would I guess gain more pace also for lower numbers…

so it would look this

$map = {
  "a": 1,
  "b":  [ 2 3 4 ]
  "c":  {
     "d": 5
     "e": [6 7 8]
  }
}

ideally with dotted access with to possibility to use index $map "a.c.e"

real life

get Link from Response header : GetRemote: 0.143.0-dev
{{ define "partials/getLinks" }}
   {{ $links := dict }}
   {{ with try (index (index .Data.Headers "Link") 0) }}
      {{ with .Err }}
         {{ warnf "OOPS>  get links %v" . }}
      {{ else }}
         {{ with .Value }}
            {{ range split . "," }}
               {{ with split .  ";" }}
                  {{ $link := trim (index . 0) "<> " }}
                  {{ $rel :=  index (split (index . 1) `"`) 1 }}
                  {{ $links = merge $links (dict $rel $link) }}
               {{ end }}
            {{ end }}
         {{ else }}
            {{ warnf "NO VALUE>  %v" . }}
         {{ end }}
      {{ end }}
   {{ else }}
      {{ warnf "NO LINK HERE %v" . }}
   {{ end }}
   {{ return $links }}
{{ end }}

some more pseude code… examples

tl;tr;
# dict
$attrs := dict
range $name := get some attributes {
   $value := calculate attribute vale maybe with other data or a partial
   $attrs = merge $attrs (dict $name $value) 
}
# slice
$attrs = slice
range $name := get some attributes {
   $value := calculate attribute vale maybe with other data or a partial
   $attrs = $attrs | append $name $value
}
$attrs := dict $attrs   <-- impossible

the above is a common design pattern I found often to collect a set of options to a dict

# parse a list of objects and create a map (for direct access skipping where)

$dict = dict
range $objects {
  $dict = merge $dict (dict .key .)
}
# $map := cast.ObjectSliceToMapWithKey "KEYFIELD" $slice


# or create a list of properties (eg from a .properties file

range $properties {
  $prop = split '=' .
  $props = merge $props (dict $prop[0] $prop[1])
}
# What about a mapbuilder without need to create a new map always ;-)
# $map = mapbuilder.Start
# range $properties {
     $prop = split '=' .
     $map.Add $prop[0] $prop[1]
   }
   $map.End

looks like Go 1.23 has a maps package for that

In your real life example above:

  1. I’m not sure why you are using the try statement instead of a with/else construct. Maybe just a stylistic difference, but it seems complicated to me.
  2. Your map of links discards links. Cut and paste the example below.
{{ $url := "https://www.veriphor.com/shared/data/books.json" }}

{{ $links := dict }}
{{ $opts := dict "responseHeaders" (slice "link") }}
{{ with try (resources.GetRemote $url $opts) }}
  {{ with .Err }}
    {{ errorf "%s" . }}
  {{ else with .Value }}
    {{ with index .Data.Headers.Link 0 }}
      <pre>{{ jsonify (dict "indent" "  ") . }}</pre>
      {{ range split . "," }}
        {{ with split .  ";" }}
          {{ $link := trim (index . 0) "<> " }}
          {{ $rel :=  index (split (index . 1) "\u0022") 1 }}
          {{ $links = merge $links (dict $rel $link) }}
        {{ end }}
      {{ end }}
    {{ end }}
  {{ else }}
    {{ errorf "Unable to get remote resource %q" $url }}
  {{ end }}
{{ end }}

<pre>{{ jsonify (dict "indent" "  ") $links }}</pre>

A slice-to-map function could very well be useful… I’m just looking for a practical example of how it might be commonly used.

  1. yes you’re right, just played with the feature :wink:
  1. it’s just an extract for the whole thing, which uses the next link later to recursively load the paginated results. see hidden details below.

at the end of the getpage partial instead of returning a $pages array it would be nice to return dict $pages "name" which would create a map of the pages indexed with name as key :wink:

adding more fields to the pages/tags would need another set of merges

just aPoC hack
{{ define "main" }}
   {{ $url := "https://api.github.com/repos/gohugoio/hugo/tags" }}
   {{ $tags := partial "getpage" $url }}
   {{ range $tags }}
      {{ warnf "TAG: %s" .name }}
   {{ end }}
   {{ warnf "TAGS: %s" (debug.Dump $tags) }}
{{ end }}

{{ define "partials/getpage" }}
   {{ $pages := slice }}
   {{ $url := . }}
   {{ warnf "loading: %s" $url }}
   {{ $opts := dict "method" "get" "responseHeaders" (slice "Link" "Content-Security-Policy") }}
   {{ with try (resources.GetRemote $url $opts) }}
      {{ with .Err }}
         {{ errorf ".Err %s" . }}
      {{ else with .Value }}
         {{ $links := partial "getLinks" . }}
         {{/* warnf "LINKS: %s" (debug.Dump $links) */}}
         {{ $pages = .Content | transform.Unmarshal }}
         {{ with $links.next }}
            {{ warnf "> load next page from: %s" . }}
            {{ $pages = $pages | append (partial "getpage" .) }}
         {{ end }}
      {{ else }}
         {{ warnf "Unable to get remote resource %q" $url }}
      {{ end }}
   {{ end }}
   #### here it would be nice to do something like $tags := $pages "name"
   {{ return $pages }}
{{ end }}

{{ define "partials/getLinks" }}
   {{ $links := dict }}
   {{ with try (index (index .Data.Headers "Link") 0) }}
      {{ with .Err }}
         {{ warnf "OOPS>  get links %v" . }}
      {{ else }}
         {{ with .Value }}
            {{ range split . "," }}
               {{ with split .  ";" }}
                  {{ $link := trim (index . 0) "<> " }}
                  {{ $rel :=  index (split (index . 1) `"`) 1 }}
                  {{ $links = merge $links (dict $rel $link) }}
               {{ end }}
            {{ end }}
         {{ else }}
            {{ warnf "NO VALUE>  %v" . }}
         {{ end }}
      {{ end }}
   {{ else }}
      {{ warnf "NO LINK HERE %v" . }}
   {{ end }}
   {{ return $links }}
{{ end }}

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.