Lua unpack bug?

Question

I Have stumbled on a weird behavior in Lua unpack function

table1 = {true, nil, true, false, nil, true, nil}
table2 = {true, false, nil, false, nil, true, nil}

a1,b1,c1,d1,e1,f1,g1 = unpack( table1 )
print ("table1:",a1,b1,c1,d1,e1,f1,g1)

a2,b2,c2,d2,e2,f2,g2 = unpack( table2 )
print ("table2:",a2,b2,c2,d2,e2,f2,g2)

Output:

table1: true nil true false nil nil nil
table2: true false nil nil nil nil nil

The second unpack delivers parameters up to the first nil value. I could live with that. The first table delivers 4? parameters with one being nil in the middle. It has 4 parameters that are not nil, but they aren't the one that are shown.

Could anyone explain this? This was tried with codepad.org and lua 5.1

Answer 1

2.2 - Values and Types

[...] The type table implements associative arrays, that is, arrays that can be indexed not only with numbers, but with any value (except nil). Tables can be heterogeneous; that is, they can contain values of all types (except nil). [...]

Given nil to an entry will break the table enumeration and your variables wont be init properly.

Here is a simple example that demonstrates a problematic behavior:

table1 = {true, false, nil, false, nil, true, nil}
for k,v in ipairs(table1) do
  print(k, v)
end

output:

1   true
2   false
>Exit code: 0

Answer 2

Simply put, setting a value of a table to nil will create undefined and inconsistent behavior.

Lua cleans up nil values in a table at garbage collect time, so variables in a table can be nil for a short time and then completely disappear, breaking the array portion of the table. This is why you would see true,nil,true for the first table and true,false,nil,nil... for the second table. So unless you want to get rid of a value in a table, never use nil as a value. If you'd like to create spaces, some other flag value might work, such as 0 instead of true/false (since 0 is neither true no false).

Answer 3

The problem can be resolved simply by specifying the beginning and ending indexes to unpack() and using the table.maxn() as the ending index:

table1 = {true, nil, true, false, nil, true, nil}

a1,b1,c1,d1,e1,f1,g1 = unpack( table1, 1, table.maxn(table1) )
print ("table1:",a1,b1,c1,d1,e1,f1,g1)
-->table1: true    nil     true    false   nil     true    nil

The true reason for the discrepancy on how the two tables are handled is in the logic of determining the length of the array portion of the table.

The luaB_unpack() function uses luaL_getn() which is defined in terms of lua_objlen() which calls luaH_getn() for tables. The luaH_getn() looks at the last position of the array, and if it is nil performs a binary search for a boundary in the table ("such that t[i] is non-nil and t[i+1] is nil"). The binary search for the end of the array is the reason that table1 is handled differently then table2.

This should only be an issue if the last entry in the array is nil.

From Programming in Lua (pg.16) (You should buy this book.): When an array has holes--nil elements inside it--the length operator may assume any of these nil elements as the end marker. Therefore, you should avoid using the length operator on arrays that may contain holes.

The unpack() is using the length operator lua_objlen(), which "may assume any of [the] nil elements as the end" of the array.