Solved. Go to Conditions => Inventory Status
Set the Stock Availability Equal or Smaller than 0
Then, the label/badge will show when product quantity = 0 or < 0
Note: This feature is only available on Pro ($5/month), Advanced ($10/month) and Platinum ($20/month) plan.