A methodological note about our access panels

by Professor Martin Collins

Pre-recruitment to panels like ours brings some obvious advantages. In respect of cost, the investment in sampling and recruitment is shared. In respect of timeliness, a large sample - even of minority sub-groups - is there, ready to be used. Perhaps less obvious, the recruitment task is completed without the immediate time pressures of a single survey, as a shared investment, and can therefore be done better. We don't hang out our surveys on the Web and wait for anyone to respond (or get our interviewers to lurk near the Union entrance to catch the usual suspects). We recruit using UCAS to reach most of the relevant student population; we accept responses only from known panellists, and we reward them for being there when we need them.

We cannot offer the true probability sample that purists would love. But who can? Most commercial surveys use quota sampling - 'convenience' samples. Even those who stick to their guns (mostly in government and social research) have to face up to the huge problem of non-response - an undeniably non-random element of selection.

This was the problem that led me, in an invited paper to the 2003 Session of the International Statistical Institute, to suggest that it might be time to forget our reliance on classical theories of inference (assuming random selection) and focus instead on building on prior knowledge. My favoured solution was to pre-recruit a large panel of people willing to take part in surveys and approach them though whatever medium they preferred (face to face, by telephone or online). We would use other information we had collected about the panellists - and our topic knowledge - to enlighten our interpretation of their answers.

At that time I knew Ben Marks as former student but had no idea that he planned a new business venture with, at its heart, a large pre-recruited online panel of students. Ben knew me as a robust defender of survey methodology but had no idea that I had 'come out' in favour of just that approach. This was simply a beautiful coincidence. When Ben asked me to be involved as a mentor of theoretical standards - and especially when he asked me to be chairman - I was thrilled by the chance to be involved in developing in practice what had seemed a few months earlier to be no more than a challenging idea. I'm still thrilled to be part of a venture that has evolved so much over its first five years. It's easy to be a mentor of theoretical standards when those standards are as embedded in the business model as the underlying idea and technical standards of online panel management. Like the rest of the staff, I'm proud to be here and looking forward to the next five years of evolution.