Sunday, 5 January 2014

What Do You Think?

Sutton Council are asking people to fill out a survey giving their thoughts on the recent changes to the Central Road area in Worcester Park. You may have had one of these (shown right) through your door. There is also an electronic version which can be found here. However more than one person has contacted me to say they feel the survey itself is a bit biased. One person used the phrase:
“…don't get me started there on how biased that survey is.”
So I thought it was worth having a closer look…

Answer Options

I note the four possible answers for the questions are: ‘agree’, ‘agree somewhat’, ‘disagree’ and ‘not sure’. Does something appear to be missing?

Imagine an agreement scale with five positions, with Agree at one end and Disagree at the other end (just ignore the ‘not sure’ option for the moment).

Agree Disagree

If the scale is unbiased then the central position should be the neutral point, let’s call it ‘neither agree nor disagree’.

Agree Neither agree nor disagree  Disagree

‘Agree somewhat’ should then slot into place between 'Agree' and the neutral point:

Agree Agree somewhat Neither agree nor disagree  Disagree

Leaving the fourth spot for ‘Disagree somewhat’

Agree Agree somewhat Neither agree nor disagree      Disagree somewhat      Disagree

That would be a balanced unbiased survey that allows the full range of options for what people want to say.

In a three option unbiased survey (again ignoring ‘not sure’) then these options should be:

Agree  Neither agree nor disagree  Disagree

What looks to have happened here is that a five option survey has been cut down to a different three option survey by having a different two options removed leaving this:

Agree Agree somewhat Disagree

This means that anyone who wants to take a neutral position will have to choose ‘Agree somewhat’ as the closest available option.

Reporting the results

Now when these survey results are collated and published, what will they say?

One way statistics are often reported is to lump groups together so if equal numbers had ticked each box on the full five category survey question (shown above), you could say that 2 out of 5 (40%) of people ‘agree or agree somewhat’ with the statement. That is 1 in 5 or 20% who chose ‘agree’ added to the 1 in 5 (or 20%) who chose ‘agree somewhat’ making 40% in total. And in the same way 40% ‘disagreed or disagreed somewhat’ with it. You can then change the language a bit and say that 40% of people generally agreed with whatever is being asked.

However with this survey, if an equal amount of people tick each of the three boxes, it could be reported, quite honestly, using the same technique that two thirds (one third choosing 'Agree' plus another third choosing 'Agree somewhat') of people generally agreed that all these changes have ‘improved Worcester Park’ (perhaps rounded to 67%). Even though many of those who ‘generally agreed’ may have actually wanted to indicate a more neutral position that would have been better described as ‘Neither agree nor disagree’. Even those who might otherwise be inclined to ‘Disagree somewhat’ may choose the ‘Agree somewhat’ option rather than the ‘Disagree’ option, feeling that the ‘middle option’ better reflects that they don’t hold a particularly strong opinion even though it would fall on the disagreeing side.

So in fact we may end up with all the ‘neutral’ and some slightly negative responses being added to the ‘positive’ responses when the results are announced.

This begs the question, “Do they want to know what people actually think or are they looking to get a specific result for some reason?”

Choice Of Questions

Also I can’t help but feel that most of their questions focus only on the likely positive aspects of the works without asking about aspects that may not be so popular. Additionally the works are called improvements all the way through the survey – so if you so happen to think these changes have made the area worse – well you’re just wrong aren’t you! That’s not to say that they have made it worse or that the changes can’t be considered improvements, just that an impartial survey shouldn’t make value judgements up front about the very things it is asking people to comment on.

In order to rebalance what some have considered an inherent bias in the official council survey, I have devised my own survey. I have merely added the two missing response options and the questions which I think should have been on the official one but have been missed off for some reason. I have not included the questions from the main survey, only the missed out ones so you can do this one in conjunction with the official one.

Please let me know if you would like other questions added to this as I can still add some!

The missing questions:

11 COMMENTS (Add Yours Now!):

guest D said...

You are completely right in your assessment.

When I was studying statistics, this was what would be called a politician's survey. Most statistical text books use a drunk and a lamp post analogy. The drunk can use the lamp post in two ways, as illumination or support. A well designed study will aim to illuminate, not support preconceived ideas. A politician's survey will usually aim to support their ideas.

However, it must be added that this consulation does cover all of WP and not just those whose lives have been made worse by the changes. That is considered an even more dubious technique, as it easy to get a completely biased opinion, i.e. 9 out of 10 of team A's supporters thought the winning goal was offside, or the 9 out 10 users of a product thought it value for money.

At least by using non parametric techniques some (though extremely limited) value can be got out of it.

Alex said...

Of course this survey is heavily stacked so that the questions asked, the questions not asked and the possible answers available all point the respondent away from disagreement. However, there are three even more fundamental aspects:

1. If the Sutton OLF team genuinely wanted genuine feedback and a survey that actually had any credibility, it would be designed, undertaken and evaluated completely independently of Sutton Council.

2. This already stacked survey is an evaluation of the performance of Sutton Councillors and Sutton Council employees.... so why is it being undertaken by the same Sutton Council employees, overseen by the same councillors?
[Notice the contact email and web address for the survey team is exactly the same email and web address as the project team! And the snail mail address is directly to the Sutton OLF Team].
It's completely inappropriate that the evaluation is being undertaken by the same team being evaluated! And, conveniently, there is ample opportunity for a significant proportion of any negative views to be 'lost' for long enough that they miss the Monday January 13th deadline and fail ever to be registered. And nobody will know, because it's 'confidential'!.

3. The survey isn't just for local residents and traders or visitors affected by the changes - absolutely anyone can submit a response, and any opinion appears to be as valid as any other. Also, anyone with the ability to create any number of email addresses can complete and submit as many fake favourable opinions as they like! [Again, secure in the knowledge that the survey will remain 'confidential', within Sutton's OLF Team!] And though they have been wasting money on many OLF projects, I can't see anyone in Sutton Council questioning the validity of responses that say Sutton Council are doing a great job!

In summary: Obviously this survey will achieve what it's explicitly designed to do - perpetuate the myth that residents, traders and visitors are overwhelmingly happy with how the OLF project has been managed and implemented. Those of us who work, commute, shop and live here know that is far from true.

However, beyond this, there is rather sinister aspect to this survey: In response to widespread and growing criticism, the response by the OLF team has been the diversion of OLF funds into a privately administered, 'confidential' survey, that is simply designed to hide truth and criticism. This is not local democracy in action.

Barry said...

Q8 alone reveals that we're dealing with a bunch of morons... I rest my case!

Sarah said...

I note that once again those of us living in Kingston Borough have been left out again. Apparently the fact that I live and have lived in Worcester Park my entire life is not sufficient for me to have an opinion on the town I live in simply because I don't pay my Council tax to the council who maintain Central Road. Sutton Council, this is my town too - how many of you actually live in Worcester Park?

Howard said...

I live in RBK too,but picked up the survey in WP library.

Guest said...

I wonder how many of those responsible for these 'improvements' actually live in Worcester Park? That they (still) are unaware what a shambles they have made of the car park and surrounding traffic, suggests they lack local knowledge.

Maybe they can't see the gridlocked traffic queues, because their research consists of sitting on their hands, on one of their own 'back-to-front' benches, which face away from the roads, the traffic queues and those crumbling crossings?

Windsor Road said...

Biased survey? There is little doubt this is an example of the OLF board "gilding the lily".
Thank you WP blog for such an I would like to be proved wrong, but if the one-way system remains, this will be shameful example of local cllrs ignoring the negative experiences of local residents who are affected day in day out. 

I ask to what real gain? I would like to be informed how the imposition of some decisions has really had positive impact. 

That said the survey is the only voice being given, so don't be apathetic, please complete especially Q9 comments.

Of course as you are here reading this, do also complete the above fairer survey.

Thanks WP blog.

Alex said...

By the way, if you leave your email address at the end of the survey, you will automatically receive a PDF transcript of the entire survey results you have submitted. [This is most helpful, since there can be no subsequent Council/Councillor denying that your survey response was received].
So, if you want to have your views recorded (even if they are neither going to be considered, nor acted upon) please follow Windsor Road's link above. It takes only minutes and the timeframe is rapidly closing! So please take the opportunity to have your say about how wisely you think a one-off opportunity of £2,250,000 (or thereabouts) has been spent in our community.

guest ab said...

I live in Worcester Park and use the area daily ,but am excluded from the Worcester Park Residents Association , because i live in the Kingston part of WP.
But i contributed my comments to the people servicing the 'pop up shop' putting forward reasonable suggestions to their grandiose schemes in order to make it workable. only to be faced with what they did , much of which ignored what residents had proposed.
But i did add my email address to my comments and receive the questionaire.And have made my comments of opposition to much of what was done to WP. Even if the questions asked were biased and should have been carried out by an independant body.

KT4 Resident said...

This is exactly the same experience as myself and anyone who did the same thing.

I discovered a woman idly amusing herself on a laptop, who told me that the Pop Up Shop was a brilliant example of what her company can create in Sutton. I told her that her statement was completely untrue:

For those who are unaware, actually, the Pop Up Shop in Worcester Park had been opened before her company had arrived and it had been run by volunteers from the Friends of Worcester Park library. [Suddenly she didn't know what to say].

Her company (Meanwhile Space) did try copying the Friends' idea in North Cheam, and you can judge its success by the fact that the unit (next to the wedding shop) still lies abandoned and empty. [Well, empty except a blackboard with somebody's chalk scrawling "Make your idea happen here!"].

This is not a survey! said...

Worcester Park "improvement" survey...who composed this survey? Are they serious? Bias is not the word, it is almost criminal. Talk about tailoring to the response to what one wants. Can anyone tell me why for Q8 RE Longfellow Rd, has two questions that both address the same question, so one is doubling the response. Yet Windsor, Hampton, Moreton and Donnington are 'lumped' together? Also I am asked whether the changes improve the rat run (well yes of course), but am I asked whether the rat run diffuses traffic and is needed (I say yes), but I must answer this question positivity! Why not just be honest and simply ask, now the trial period is over (November 2013), a) has the one-way system encouraged you to use the high street more, b) has it negatively impacted your living standards.
It's a joke...but no one is laughing.

Post a Comment

The Worcester Park Blog welcomes your comments and opinions!