As this is more about the UI of the application than anything else, I don't think there is going to be a single right answer, as it will come down to a combination of what works (which is difficult without being able to see / play with things) and your personal preferences.
A few progressions I would run through:
- Visual feedback
Make you table more interactive by providing visual feedback to the user. At the most basic level, try adding some colour to the cells - a colour for those that are checked. This will allow the user to quickly see which options are "in play". It may be the reverse of this works better (highlighting unchecked cells) - but this all depends what the form is doing / intending to indicate - i.e. if it's more important to make clear that the unchecked state is bad, you may want these to be red.
The next level up is to add some dynamic highlighting. If the table is huge, you may want to highlight the row and column header cells that correspond the the cell under the cursor. You could also consider highlight the whole row / column (cross-hair style) to allow the user to examine 'companion' cells.
Dynamic table
Slightly more involved would be to add some spice to you table. Instead of showing rows and columns of check-boxes, use graphical icons / images. They are a lot easier on the eye, and will probably allow you to have tighter control on the dimensions of the table. The entire UI could then be done via Javascript and on-click - which is pretty easy these days if you employ something like JQuery.
Split the interface
This is based on the assumption that all combinations of Table A & Table B aren't setup in the pivot table to begin with - only when a user tries to relate A.item with B.item
Instead of showing all possible combinations, show only those which are active (have an entry in the pivot table). Then provide the user with a second form (probably of two drop-downs) that allows them to relate a record from the first table to the second.
Filter the interface
Provide the user with the ability to filter the interface - to show only the relationships between a single record from one of the tables. This would have the effect of restricting your table to a single column, making it a bit easier to accommodate in the design.
However, I would still allow the user to get to the "big view" of all records, as, depending on what you are doing, such as view can be very useful to quickly cross reference lots of records.