You can do this with an AWT clip. You'll need to know the bounds of the rectangle you want to exclude, and the outer bounds of your drawing area.
The following demo code opens a frame and displays a single panel in it. The panel's paint method sets up an example clip which looks like a rectangle with a rectangular hole in the middle, when in fact it's a polygon that describes the area around the area we want to exclude. The clip rectangle should be composed of the bounds of the excluded rectangle, and the outer edge of the drawing area, but I've left hard-coded values in to keep it simple and illustrate the workings better (I hope!)
+-------------------+
| clip drawing area |
+---+-----------+ |
| | excluded | |
| | area | |
| +-----------+ |
| |
+-------------------+
This method has the benefit over calculating the line intersection manually in that it prevents all AWT painting going into the excluded area. I don't know if that's useful to you or not.
My demo then paints a black rectangle over the whole area, and a single white diagonal line running through it, to illustrate the clip working.
public class StackOverflow extends JFrame {
public static void main(String[] args) {
new StackOverflow();
}
private StackOverflow() {
setTitle( "Clip with a hole" );
setSize( 320,300 );
getContentPane().add( new ClipPanel() );
setVisible( true );
}
}
class ClipPanel extends JPanel {
@Override
public void paintComponent(Graphics g) {
super.paintComponent(g);
Polygon clip = new Polygon(
new int[]{ 0, 100, 100, 0, 0, 20, 20, 80, 80, 0 },
new int[]{ 0, 0, 60, 60, 20, 20, 40, 40, 20, 20 },
10
);
g.setClip(clip);
g.setColor( Color.BLACK );
g.fillRect( 0,0,100,60 );
g.setColor( Color.WHITE );
g.drawLine( 0,0,100,60 );
}
}