Good question. I think the answer depends on what the videos were referring to.
Usually in Hugo we mean with “template” a specific file that affects how Hugo renders content. For instance, an index.html template would contain code for how the homepage renders. sitemap.xml would be a template for the sitemap, and so on.
Here you can find a lot more templates and what they can do.
A theme consists out of a bunch of template files together that, when combined, create a complete Hugo website.
So it’s not a decision between creating a theme or making a template. You often have a theme (made by someone else or yourself) that has templates. So theme and template.