We present a holistic approach to building a robust and useful natural language classification system for real-world content moderation.