The Storage Unit is where items are actually stored in a storage system. Terraria saves the list of items stored in a container in the record for the tiles where this container is physically placed, so Storage Units act as the containers for a storage system's list of items. However, the content of each Storage Unit cannot be seen directly: instead, all the content from all connected Storage Units are seen together in the storage system's Storage Heart, a connected Storage Access, or a Remote Storage Access. Items are still technically stored in the Storage Unit, however, and the unit is therefore protected from mining as long as it contains items.
To allow to remove a Storage Unit (so as to upgrade it, for example), it needs to be empty first. Right-clicking a Storage Unit will tell you its current use and capacity. A Storage Unit Wand can be used to deactivate or reactivate a Storage Unit. When a Storage Unit is deactivated, the Storage Heart will try to move all items it contains to other Storage Units, and will avoid putting more items in it unless all active Storage Units are full. If needed, remove items from the storage system until the deactivated Storage Unit reaches 0. It can then be mined, picked up, and upgraded or moved to a different position.