Category: shell

When writing shell scripts, one of the things I used to find more problematic was how to return values from a function. If writing a function, I try to use local variables as far as possible, so I do not like the idea of using global variables to return values. There are a few well known options that can help with this, but none of them completely convinced me:

• You can use the return code (f; $?) of the function, but you are limited to integers and to an 8-bit range for them. Also, using it conflicts with the philosophy of function return codes, which are expected to be zero for success and otherwise an error code. As an example of how this can be a problem, if you run your shell with sh -e, the shell will exit when a function returns a non-zero code.
• You can take the function output to stdout as return value, by using command substitution (your_var=$(f)). The disadvantages are that (1) you need to make sure that the output from the function does not include anything undesired, so you end up redirecting output of the programs you call to /dev/null if you are not interested on it, and (2) that a sub-shell is created, which is costly and can be problematic too (for instance, if you want some side effect to still happen in the calling process).
• You can pass the names of the output variables to the function and then write the return values in there using eval. An advantage of this approach compared to the other methods is that you can easily return more than one value to the caller. But, you are actually writing in variables unknown to the function and that are then of global scope, which is what I wanted to avoid… or maybe not?

I actually liked more the third approach, but the problem with some variables still being global did not satisfy me. Concretely, I was worried with the case of a function calling another function, like

#!/bin/dash -e

return_two() {
    local ret_var_1=$1
    local ret_var_2=$2
    eval "$ret_var_1"=one
    eval "$ret_var_2"=two
}

calls_return_two() {
    return_two ret1 ret2
    echo "$ret1" "$ret2"
}

ret1=oldvalue1
ret2=oldvalue2
calls_return_two
echo "$ret1" "$ret2"

Here return_two() indeed overwrites $ret1 and $ret2, so the output of the script is

one two
one two

But, it turns out there is a way to avoid this problem. I had unconsciously assumed that shell interpreters do, as most languages these days, static (also called lexical) scoping. So, when the interpreter tried to find $ret1 and $ret2, it would not find them in the local variables for return_two(), and it would overwrite/create them in the global scope. But that is not necessarily the case. This script:

#!/bin/dash -e

return_two() {
    local ret_var_1=$1
    local ret_var_2=$2
    eval "$ret_var_1"=one
    eval "$ret_var_2"=two
}

calls_return_two() {
    local ret1 ret2
    return_two ret1 ret2
    echo "$ret1" "$ret2"
}

ret1=oldvalue1
ret2=oldvalue2
calls_return_two
echo "$ret1" "$ret2"

Has as output:

one two
oldvalue1 oldvalue2

The $ret1 and $ret2 variables are not being overwritten by return_two(), because shells have dynamic scoping! That means that when the interpreter does not find a variable, it goes up in the stack until it finds a parent that owns a variable with that name. As the variable names that calls_return_two() provides to return_two() are local, those are the variables actually modified and not the global ones.

So, in the end, I found what I wanted: a way to return multiple variables from a shell function without polluting the global namespace. The solution is quite obvious once you find how dynamic scoping behaves, but I had the static scoping principles so deeply burned into my brain that it was a bit of a surprise to find this.

It is interesting how in this case dynamic scoping comes to the rescue, taking into account that is quite controversial, and for good reasons – it can make the code less readable as you need to take into account the dynamic behavior of the program to understand which variables are available at a given execution path. Think for instance on something like

#!/bin/dash -e

const=2

mult_by_my_const() {
    echo $((const * $1))
}

call_mult_by_my_const() {
    local const=4
    mult_by_my_const 2
}

call_mult_by_my_const

which ‘overrides’ with a local variable a global constant! (the output is indeed 8 and not 4). The eval command is also something that needs to be handled with care, there are a few reasons for that too. So, it is a bit ironic that I had to use them to be able to return values in a clean way. But, I also think that this idiom is relatively safe to use, or that at least it is better than not using it:

• eval is used only to set the output variables for the function
• Even though dynamic binding is used, in this case it is a sort of passing by reference method and should make the program easier to understand, not harder
• All the information about the variable names is kept encapsulated in the caller function

Something to note is that I am assuming that the shell interpreter supports local variables. This is unfortunately not part of the POSIX standard, but it turns out that most of the interpreters around these days support it, including dash and bash. However, be aware of the different syntax in each case.

Anyway, I hope you enjoyed this small shell scripting pill!