See also rm on a directory with millions of files (the original question is not directly related, but we discuss this problem there nonetheless).
As far as I'm aware, at least on ext2/3, no, there's no (online) way to shrink them short of delete + recreate. Unmounted, several sources suggest e2fsck -D might work, though I can claim no personal experience with that option.
Some references for further reading:
