How to manage inter-test global variables?

[marco-brandizi]

I've just discovered bash_unit and it's great!

However, I've a problem: how to manage stateful variables, which keep their value across tests?
For instance, this isn't working:

echo "Code outside functions"
# Neither this works: export CTR=1

function setup_suite () {
  echo "All tests begin here"
  CTR=1
}

function teardown_suite () {
  echo "That's all, Folks!"
}

function setup () {
  CTR=$(($CTR+1))
  echo "--- Test Count $CTR"
}

function test_ctr () {
  echo "CTR is $CTR"
  assert_equals 2 $CTR "Wrong test counter!"
}

function test_ctr_1 () {
  echo "CTR is $CTR"
  assert_equals 3 $CTR "Wrong test counter!"
}

Results are:

$ bash_unit test_globals.sh
Running tests in test_globals.sh
Code outside functions
All tests begin here
--- Test Count 2
Running test_ctr... CTR is 2
SUCCESS ✓
--- Test Count 2
Running test_ctr_1... CTR is 2
FAILURE ✗
Wrong test counter!
 expected [3] but was [2]
test_globals.sh:25:test_ctr_1()
That's all, Folks!
$ 

What I would like to do is to see that CTR counts the tests. I know interfering tests are bad practice, but there might be cases like this (keeping a counter of done things, keeping a timestamp of the last operation, etc) and it would be good to have them working, somehow (some set_env function managed by bash_unit?). The only way I've found so far is to use temp files to store the values of variables like CTR, which is not exactly practical.

Note that this can be done shunit2 (but I prefer bash_unit, mainly cause it supports TAP format).

[pgrange]

Hi @marco-brandizi ,

Thanks for using bash_unit. Interesting to see that tap support is important for you, thanks for this feedback.

Regarding your issue, I understand your problem. The reason why the global variable behave that way is twofold.

First, the way environment variables are handled on unix systems: variables are copied from parent process to child processes but then, any change made to a variable in a child process is not observed in the parent process.

Second, the way bash_unit isolates tests execution from one another. Each test is run in its own child process and so any change made to an environment variable in one test can not be observed from another.

Ok, so I am just explaining why things are working the way you observe... So not helping so much and you probably already figured that out yourself ;)

I agree that interfering tests are bad practice :) I understand that there might be good reasons to do so but just want to stress that bash_unit does not give any garanty in the order it will run the tests. That might change from one version to another or even (not the case right now) from one test run to another. People are even asking for parallel test execution and that would be even harder to handle interferences between tests.

So... Your issue is quite a puzzle for me :)

I would have suggested to use tmp files as you did. But I understand this is not practical.

By the way, you may take a look at the documentation of the fake function where a similar problem is discussed with a not so practical solution:
https://github.com/pgrange/bash_unit/blob/master/README.adoc#fake-parameters

That would be helpful for me if you could give me an actual use case that you're stuck with. A usecase where you need to do something and the only way is to use tmp files. May be that would make us aware of a usecase that is missing in bash_unit itself and could be implemented so that you don't need to solve that test interference problem.

A second lead would be to imagine a way to make it easier to share data from one child process to another.

[pgrange]

That one is fun: https://stackoverflow.com/a/54622544

[pgrange]

Thinking out loud, I think that, if I had to manage that kind of global state between tests I think I would write it that way (still using temp files, no magic):

setup_suite () {
  GLOBAL_ENV="$(mktemp)"
}

teardown_suite () {
  rm -f "$GLOBAL_ENV"
}

setup () {
  load_env
  CTR=$((CTR+1))
}

teardown() {
  save_env CTR
}

load_env() {
  source "$GLOBAL_ENV"
}

save_env() {
  for var in $*
  do
    echo "export ${var}=${!var}" >> "$GLOBAL_ENV"
  done
}

test_ctr () {
  assert_equals 1 $CTR "Wrong test counter!"
}

test_ctr_1 () {
  assert_equals 2 $CTR "Wrong test counter!"
}

[marco-brandizi]

Hi @pgrange, many thanks for the detailed answer. Unfortunately I don't have a concrete use case of the type at issue, cause I am just evaluating a couple of tools like bash_unit. Cases where stateful inter-test variables would be needed are the ones I mentioned: keeping a count of tests run so far (maybe just to print, maybe to do things like auto-id generation), keeping the timestamp of the last operation (eg, for diagnostic and logging purposes).

I could live with doing it through tmp files. It would be useful to include in bash_unit functions like load_env/save_env (and automatic env initialisation from the bash_unit runner, instead of me having to set/dispose GLOBAL_ENV), which would abstract the temp files.

Thanks.

[pepdiz]

Sorry to comment about this private issue but IMHO tests should always be isolated black boxes and breaking this rule will get you more problems that advantages.

For this reason I think that use cases for state between tests should be handled by bash_unit itself, is its responsability, for example bash_unit has to maintain a counter of tests to export a TAP file ir order to write the plan. If needed it would be better bash_unit provide access inside tests to global internal state defined (or a subset) but not to allow to change state, state is only handled by bash_unit itself

So I think special needs that break test isolation is better handled outside the system with special hacks such as tmp files or similar, those hacks may get complicated or don't work with future implementations due to new features such as parallel execution but this is hacker responsability ;-)