When faced with a dilemma of this type then you should go back to basics. Examine the purpose for which your awards program was established. It is the purpose of your awards program itself which will often be the determining factor in resolving your dilemma.
Let's assume for one moment that your sites purpose is to reward great design or to honour the skills of other webmasters in the presentation of art. In this situation it could be argued - and very successfully - that the evaluation of a website in English, by a French webmaster who is limited to one language, should not give rise to any dilemmas, even if he does not understand any of the accompanying text. A good design remains a good design irrespective of language. And art transcends all languages. (It could be argued that art is a language in itself but that is another discussion for another time.) In this situation the purpose is clearly used as the determining factor, and used to great effect.
However let us take a second example. You create an awards program which is clearly designed to reward webmasters who have managed to produce websites with excellent site contents. You can only speak German but the site with which you are presented is in Portuguese. The site appears wonderfully designed, employing all the latest technology to great effect. However, you cannot understand a word of the site contents. Here the dilemma becomes apparent. The site being evaluated may display skills that would amaze the most professional of designers, but you are unable to judge site contents. Should you nevertheless give the site an award?
The answer is an unequivocal no. It would be inappropriate to provide an award to a site whose language you do not understand and whose contents you cannot clearly judge in situations where contents is all. In addition, to do so would devalue your own award program.
But - I hear you cry . . . What about translation devices? Surely the availability of these devices means that all sites can now be easily translated and therefore the inability to speak more than one language is no longer a barrier. If only this were true. The reality is that online translation devices are extremely inaccurate at best, and it is difficult to rely on such devices for any more than the translation of a few words or paragraphs. Even here, meanings can be misinterpreted and nuances missed. If contents is the important issue in determining whether a site will receive an award then the evaluator, faced with a site in a language he does not understand, should seek an accurate version of the site, in a language with which he is comfortable, and not a poorly translated - and inaccurate version.
Ok so it is clear that there is less ambiguity where the award site places emphasis predominantly on contents or 'non content' related issues. But what about situations where there is equal emphasis on, lets say, navigation and content? What should be done in these situations? In fact there is no ambiguity. If contents are so important to the evaluator that they share equal billing with other issues then the site being evaluated should not receive an award if the contents cannot be understood.
It might be argued that to refuse to evaluate a site because its language is different from your own is unethical. After all is this not a form of discrimination? However such an argument is neither convincing nor accepted by the awards community as a reasonable argument. Award managers and evaluators should know their limitations and act within their level of competence. It cannot be said that an award manager or evaluator is acting unfairly if he acts within his level of competence.
Perhaps if your there is still doubt in mind then you should take comfort from the following words. Most, if not all award programs, disqualify sites that promote hate or explicit sexual content. This can only be judged if the site content can be understood. The most diligent of site evaluators can miss expletives and 'unsuitable content' when the site they are evaluating is in their native language. How must might slip past them if they attempt to evaluate a site whose language is incomprehensible to them?
So how is the dilemma to be resolved? Better still, how do you prevent the dilemma from occurring in the first place?
I would suggest that . . .
The award program should have a clearly defined and unambiguous purpose.
Where content is important the award manager should make clear the award program's linguistic limitations right from the start. If the only language the reviewer can understand is German, then make it clear that the submitted site must be in German or have an accurate German version available for review. And in case there is any further doubt, make it clear also that translation devices are not an acceptable alternative. If the statement is out in the public arena, then you need have no qualms about refusing to award a site presented in a language that is alien to you.
Where the contents are less important, then an award should be provided where it is clear that the content would not cause disqualification. In this situation a translation device may be sufficient here to ensure that the site contents meets the award program's basic criteria.
One final thought ... Many award managers now seek to collaborate with others that have knowledge of different languages. This is an excellent method of reducing the risk of such dilemmas occurring and there is an added bonus ... Your awards program becomes more inclusive, giving you and your evaluators the opportunity to review sites that have their roots in cultures that are often far removed from your own. Something most certainly to be relished.