I believe this is because the index is normally used to rebuild itself, and how physical rowid's are used in the index.
An index on an ordinary table stores physical rowid's, but because an index-organized table is fluid in structure, a secondary index is only able to store a physical rowid guess.
When you rebuild an index on an ordinary table, it doesn't need to reference the source table for physical rowid's, it has them already in the index. But on an index-organized table, the rebuild has to review the physical rowid's again and is thus forced to scan the base table.