Solved. Go to Conditions => Inventory Status.
Here you choose the option "Out of stock".
Then, the label/badge will be automatically added to the products that are out of stock.
Note: This feature is only available on Pro ($5/month), Advanced ($10/month) and Platinum ($20/month) plan.