Will all key "widths" and "heights" be a multiple of the smallest key width/height? If so, you can nest horizontal and vertical LinearLayouts as necessary to describe the keyboard, using layout_weight to make your double-wide or double-high keys.
For example, if I look at the keypad to my workstation, it is essentially a 5x4 grid of keys, with a double-high "+" and "enter" key in the right-most column and a double-wide 0 in the bottom-left position (fairly standard configuration, refer to your own in this description). I can make vertical linear layouts for each of the columns "on top of" the 0 {numlock,7,4,1} and {/,8,5,2}. These can next be put into a horizontal linear layout to group all 8 keys. This assembly can be combined with the "0" button in another vertical layout, with weight "4" to the 8 key assembly and "1" to the 0 key. Let's call this full assembly of 9 keys assembly A. Now you can make vertical linear layouts for {*,9,6,3} and {-,+,enter}. In the second of these, give weightings of 2 to each of the + and enter key. Finally use a horizontal linear layout to group assembly A with each of the other 2 vertical layouts.