I think the ListView by default is designed to fill the available space, perhaps effectively as though it had preferredHeight: Infinity set.
I'm not certain the idea of the ListView auto-sizing to match N components is a good one. If you started it out with no items, it would vanish. If you added items, it would gradually expand. Generally I don't think that's behaviour people want for their ListViews (though with appropriate height and other settings you could probably get it to work that way if required).
Also note the leadingVisual property, which might need more space than a pair of items would. That's another case where you probably want it either to auto-fill the available space, or at least match whatever size you explicitly set, regardless of number of contained items.
Think of it as having spaceQuota set to a positive value...
What *I* would like is to have these auto-sizing behaviours actually documented in some clearer fashion, since I keep spending time reverse-engineering the behaviour of these things. I spent an hour yesterday trying to figure out why a Divider had a big space appearing above it. Turns out it effectively has topMargin and bottomMargin set to 20 by default, but that's not in the docs... same as however this ListView works, I guess. I mean, it does say "By default, the width and height are adjusted automatically to fill its parent container." but I'd like to know exactly how it is doing that (e.g. height = Infinity?) so that I could more readily override the behaviour when I wanted to.