Variables from included Taskfiles cannot be overwritten

[Diaphteiros]

Description

The documentation states that in order to overwrite a variable from an included Taskfile, one has to use the syntax

MY_VAR: '{{.MY_VAR | default "my-default-value"}}'

That does not work, at least not for 'global' variables defined in the top-level vars field of the Taskfile.

Taskfile.yaml

version: 3

vars:
  FOOBAR: foo

includes:
  bar: Taskfile_bar.yaml

tasks:
  foobar:
    cmds:
    - cmd: echo '{{.FOOBAR}}'
      silent: true

Taskfile_bar.yaml

version: 3

vars:
  FOOBAR: '{{.FOOBAR | default "bar"}}'

tasks:
  foobar:
    cmds:
    - cmd: echo '{{.FOOBAR}}'
      silent: true

The Problem

Running either task foobar or task bar:foobar results in bar, but both should print foo instead.

---

I have to admit that I find the behavior weird in general, I would assume that in most use-cases, variables from the imported Taskfile (often generic/library) should be overwritten by the ones from the importing Taskfile (specialized for project). Having it work the other way around is somewhat confusing. I can work with that though, but if there is no way to overwrite the variables in the included Taskfile from the including one, that is a problem.

---

Version

v3.41.0

Operating system

darwin/arm64

Experiments Enabled

None

[Diaphteiros]

I just tried to set the variable in the vars field of the include

# Taskfile.yaml
version: 3

includes:
  bar:
    taskfile: Taskfile_bar.yaml
    vars:
      FOOBAR: foo

tasks:
  foobar:
    cmds:
    - cmd: echo '{{.FOOBAR}}'
      silent: true

but that doesn't work either, the result is still bar and not foo.

[Diaphteiros]

Just noticed that there is a new version, but the problem persists also with v3.42.0. 😞

[pd93]

This looks like another variable scoping issue to me. In this scenario, I would not expect your included taskfile to change any variables in your root taskfile at all - regardless of the default templating function. The fact that its adopting bar is confusing to me. I'm going to bundle this into the variables megathread for now.

I'd be interested to see a more complete example of what you're trying to do. I would generally avoid trying to set variables in a child Taskfile that you want to use in a parent. Instead, think of includes like a scope:

a := "foo"
{
    a = "bar"            <-- setting `a` here changes its value in this scope, but not in the parent scope
    print(a) // "bar"
}
print(a) // "foo"

In reality, this isn't how Task currently works and we have work planned to address this. However, generally, you will have less issues if you stick to these kind of principles.

[Diaphteiros]

Basically, I'm working on some library Taskfiles that we want to use for all of our projects. The idea is that the library Taskfiles reside in their own repo and contain all task definitions (code generation, validation, testing, release management, etc.). Each repo then imports the library repo as a submodule and has its own Taskfile, which includes the library Taskfile(s). The specific Taskfile should be as minimal as possible.

I want to have the task definitions in the library Taskfile generic and configurable via variables. The library Taskfile sets some defaults for the variables, but the specific Taskfile can (or in some cases has to) overwrite these defaults.

Example

I tried to copy a somewhat simple snippet to demonstrate the use-case:

tasks_tools.yaml

This file resides in the library repo, which is included in the project repo as a git submodule.

version: 3

tasks:
  localbin:
    desc: "Ensure that the folder specified in LOCALBIN exists."
    run: once
    requires:
      vars:
      - LOCALBIN
    status:
    - test -d {{.LOCALBIN}}
    cmds:
    - 'echo "localbin: {{.LOCALBIN}}"'
    - mkdir -p {{.LOCALBIN}}
    internal: true

  golangci-lint:
    desc: "Ensure that golangci-lint is installed."
    run: once
    requires:
      vars:
      - LINTER
      - LINTER_VERSION
    deps:
    - localbin
    status:
    - test -x {{.LINTER}}
    - '{{.LINTER}} --version | grep -q {{.LINTER_VERSION | trimPrefix "v"}}'
    cmds:
    - 'curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b {{.LOCALBIN}} {{.LINTER_VERSION}}'
    internal: true

tasks_validation.yaml

This file resides in the library repo, which is included in the project repo as a git submodule.

version: 3

vars:
  LOCALBIN: '{{ .LOCALBIN | default (print .ROOT_DIR "/bin") }}'
  LINTER: '{{ .LINTER | default (print .LOCALBIN "/golangci-lint") }}'
  LINTER_VERSION: '{{ .LINTER_VERSION | default "v1.64.4" }}'

includes:
  tools:
    taskfile: tasks_tools.yaml
    internal: true

tasks:
  lint:
    desc: "Run 'golangci-lint'."
    run: once
    deps:
    - tools:golangci-lint
    requires:
      vars:
      - LINTER
    cmds:
    - '"{{.LINTER}}" run -c "{{.ROOT_DIR}}/.golangci.yaml" {{.CODE_DIRS}}'

Taskfile.yaml

This file resides in the project repo. The library repo is imported as git submodule under hack/common.

version: 3

vars:
  CODE_DIRS: '{{.ROOT_DIR}}/cmd/... {{.ROOT_DIR}}/internal/... {{.ROOT_DIR}}/test/...'

includes:
  validation:
    taskfile: hack/common/tasks_validation.yaml

Explanation

The task definitions for both downloading the linter and running it are contained in the library repository.

The LINTER variable contains the path to the linter binary. It is defaulted in the library Taskfiles to {{.ROOT_DIR}}/bin/golangci-lint, but the importing repo should be able to overwrite it, if this is necessary for any reason. The same applies to the LINTER_VERSION variable.

The CODE_DIRS variable specifies which folders should be linted - this cannot be defaulted in the library repository, because it is highly dependent on the structure of the importing project repository. This is a value that must be specified in the project repo's Taskfile.yaml for the tasks to work.

---

I hope this example helps in making clear what I am trying to achieve.
I kind of try to use some Taskfiles like parameterized function definitions and the importing Taskfile provides the values for the parameters.

[Diaphteiros]

Maybe my first example was not the best one, I'm actually not trying to use variables from a child Taskfile in a parent one, but the other way around (with defaults in the child, if the parent doesn't set the variable).
Just wanted to give a minimal example of the fact that overwriting variables defined in the child with ones from the parent doesn't work. Would have been better to move the task definition in the child Taskfile, though. 😅

[Diaphteiros]

I just edited the initial post and added the foobar task also in the child Taskfile, to better reflect my actual use-case.

[pd93]

In your updated example, you'll notice that changing your include to:

includes:
  bar:
    taskfile: Taskfile_bar.yaml
    vars:
      FOOBAR: foo

in-fact, does work when you call task bar:foobar. The thing that is broken here is when you do:

includes:
  bar:
    taskfile: Taskfile_bar.yaml
    vars:
      FOOBAR: "{{.FOOBAR}}"

This just because we aren't resolving templates in included taskfile variables and is a relatively simple fix. If this were to work, would your use-case be satisfied?

Calling task foobar would still not work because of the aforementioned scoping reasons.

[Diaphteiros]

OK, I wasn't aware that this worked.

I can use that as a workaround. It is still slightly annoying, because I sometimes import multiple Taskfiles that take the same set of variables, which I now have to specify multiple times, so the second variant would be an even better workaround for me. But as long as I get it working, it will be fine.

That the original example still doesn't work does not bother me, I do not need this functionality (as far as I remember), it was just a badly chosen example.

[pd93]

@Diaphteiros Opened #2113 which should allow you to do what you want.