I ran into the same issue. I never could find a very pleasing solution, but here is how I did it. Maybe someone else has a better way, I hate adding layouts that don't do anything.
My hack was to add a dummy linearlayout at the bottom of the scrollview that has fill_parent to take up all the room and force the scrollview to fill the screen. Then add whatever component I want to that linearlayout.
Here is one of my layouts that does this:
<ScrollView
android:layout_height="fill_parent"
android:layout_width="fill_parent"
android:fillViewport="true"
android:layout_weight="1">
<RelativeLayout
android:layout_height="fill_parent"
android:layout_width="fill_parent"
android:layout_marginTop="15px">
... bunch of components
<LinearLayout
android:layout_width="fill_parent"
android:layout_marginTop="5px"
android:paddingTop="2px"
android:layout_below="@+id/spinner"
android:layout_height="fill_parent"
android:gravity="center_horizontal|bottom">
<Button
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="Delete"
android:paddingLeft="20px"
android:paddingRight="20px" />
</LinearLayout>
</RelativeLayout>