Arabic is the official language. While many locals will at least attempt to communicate with non-Arabic speakers in other languages, any visitor will almost certainly need at least some Arabic, particularly if traveling to locations outside the capital. Even within Sana'a, the bilingual signs common throughout most of the Middle East are commonly absent, with Arabic script and numbers predominating. This said, Yemenis are very open for communication, and hand-waving, making noises and smiling can get you very far, even if not always where you wanted to get (usually to a qat-chewing session).
Yemenis have a myriad of different accents, due to the historical inaccessibility of parts of the country. It is not unusual for a visitor to be told that his or her laborious attempts at speaking Arabic are in fact "Arabic" and not "Yemeni" or "Yemeni enough". The more vocal village children will almost certainly enjoy hearing a visitor's attempts at their language, and will show this appreciation either with peals of laughter or by asking questions about the visitor's homeland.