I’ve been read­ing about Australia’s Best Blog Com­pe­ti­tion — the win­ner and runners-up were announced today.

The Singing Bridges Travel Diary won. Inter­est­ing site.

More inter­est­ing to me, how­ever, is that the organ­is­ers — Smar­ty­Host, who used the com­pe­ti­tion to mar­ket their blog host­ing ser­vices — have come under some fire for their judg­ing sys­tem. Accord­ing to a jour­nal­ist at The Age, who was on the panel, the var­i­ous judges seem to be mark­ing the entries on dif­fer­ent cri­te­ria, and no stan­dard­iza­tion of the scores was attempted. Appar­ently the win­ning entry was voted first by only two of the judges, and

“It took the prize only because one of the judges gave it a total of 40 points — by far the most gen­er­ous award by any judge — and the other 35 points.”

The judges (the report does not state how many there were) only saw the 11 final­ist entries. It sounds like not a lot of thought went into the exact scor­ing process.

This is some­thing that we were very con­scious of when organ­is­ing the WA Web Awards. The WAWAs were organ­ised by the com­mit­tee of Port 80, and nat­u­rally almost all of the com­mit­tee mem­bers, as active par­tic­i­pants in their local indus­try, wanted to enter their own client sites. In order to main­tain legit­i­macy and avoid any hint of bias, a very strict judg­ing process had to be set up.

Firstly, one com­mit­tee mem­ber was des­ig­nated the judg­ing chair­per­son. That per­son was not per­mit­ted to enter any sites them­selves. Our judg­ing chair­per­son was the only indi­vid­ual deal­ing with the judges and entry process. Of course, that meant a lot of work for that one per­son — Megs did an awe­some job — but ensured that no one could even acci­den­tally unfairly influ­ence the judges. We’ll get you an assis­tant next year, Megs!

Sec­ondly, a very detailed set of judg­ing cri­te­ria were devel­oped, cus­tomised for each of the 14 cat­e­gories, with guide­lines for what con­sti­tuted good and bad scores for each. The four judges all com­mented on the fair­ness and qual­ity of the cri­te­ria — which was very reas­sur­ing. Our tire­less judg­ing chair­per­son aver­aged the scores and deter­mined the win­ners using a math­e­mat­i­cal process — and in fact she was the only per­son who knew the win­ners ahead of time, even the judges did not know (although they could prob­a­bly have made some edu­cated guesses).

In the end each of the win­ning sites was very deserv­ing (espe­cially the ecom­merce cat­e­gory, smirk) and there was no snip­ing or gen­eral dis­agree­ment with the results that I heard about. So I guess our sys­tem was successful.

I’ll be inter­ested to see how the Best Blog Com­pe­ti­tion evolves… pre­sum­ably they will learn from their mis­takes and take it to big­ger and bet­ter things in the future. For me, it’s rein­forced just how impor­tant a good judg­ing process is to the per­ceived legit­i­macy of a com­pe­ti­tion — the last thing any­one wants is to receive com­men­da­tion in a com­pe­ti­tion that then has it’s valid­ity questioned.