I don't think I'm really qualified to answer this. I will say yes there are some teas or herbs that you should be careful with for example St Johns wort. This herb can interact with other medications so best check with your doctor. Some herbs like chamomile I don't believe would have any effect on you regardless how much you drank as it's so mild. Also be careful with green tea. It may be herbal but it contains a lot of caffeine.