What you are asking for is not supported by the underlying Win32 LISTVIEW system control that TListView wraps. There is no per-item visibility or per-item heights. So, deleting and re-adding items is the only way to "hide" and "show" them.
For what you are attempting to accomplish, consider using the TListView in virtual mode (OwnerData=true). That way, your data is physically stored outside of the ListView's memory, so you can access it however you want regardless of how you setup the TListView to display it.
Otherwise, use a different UI control, such as TVirtualTreeView, which can be setup to look like a ListView, and has many advanced features to customize the display (such as per-node heights).