Yes, deleting multiple items from the Media Manager is possible if they are all in the same folder

It does require Administrator or Tenant Administrator privileges in the tenant.

To delete multiple items in the Media Manager

• In the Media Manager open the folder that has the items
• Start by clicking  on the first item you want to delete to the left of the thumbnail/icon
  • It will highlight blue and place a checkmark next to the item
• On your keyboard hold either the Shift or CTRL key while you click the other items
  • The Shift key will select all the items between the two
  • The CTRL key will only select/deselect the item you click on
    
• With your multiple items selected use the dropdown in the upper left to select the Delete action
  
• Then click the blue Apply button to delete your selections
• You will see the same warning/confirmation screen that is present when deleting 1 item