Notes from Poland is pleased to announce our cooperation with Ben Stanley, a political scientist at Warsaw’s SWPS University of Social Sciences and Humanities. We will regularly feature Dr Stanley’s pooling of Polish political polls. These provide a far better gauge of party support than do the eye-catching single polls that the media are often prone to give undue weight.
Dr Stanley’s model has proved extremely reliable: at last month’s parliamentary elections, its prediction of 236 seats for the Law and Justice (PiS) party was just one off the eventual result, and it was the most accurate among leading analysts.
There are few deterministic laws in political science, but one is that if you talk for long enough with a psephologist – a specialist in the science of polling – they will end up citing Twyman’s Law: “if a statistic looks interesting or unusual it is probably wrong.”
There is a good reason for this. Despite the best efforts of pollsters, aided by an increasing number of statistically literate journalists, the default lay response to any given poll is overreaction.
One day, a poll appears that gives Party X a five-point lead. Supporters of Party X retweet with enthusiasm. A few days later, another polling agency suggests Party Y has closed the gap to two points. Party X’s supporters develop an interest in “rogue polls”, while supporters of Party Y proclaim the fightback to be on.
Then another agency tells us that actually Party X has stretched its lead to seven points. Supporters of Party X forget about the concept of rogue polls, while those of Party Y mutter darkly about the political preferences of the polling agency. Meanwhile, supporters of Party Z insist that these polls are useless “because nobody has ever polled me or anyone I know, and what can a survey of 1000 people tell you about public opinion anyway?”
The rules of polling
The cumulative effect of all this? Everyone ends up distrusting the polls, while at the same time avidly consuming them. Unless they’re supporters of Party Z, in which case they’re as likely to call for polling to be banned. And yet by observing a few simple rules, we can extract useful signals from the blooming, buzzing confusion.
- Rule 1 – Polls are always wrong
The average poll is based on a sample of around 1000 people, and after we’ve excluded non-voters this can often fall to around 600 in countries like Poland, where turnout is – recent improvements notwithstanding – still behind the European average. This can still tell us something useful about relative party strengths, but the estimates we have are often based on a small sample.Random samples of a population always contain error, and the survey methods used by different pollsters introduce different types of error in different ways. However, while polls are always wrong, they are less wrong than licking our fingers and sticking them in the air or using our gut instinct. The signal provided by 1000 randomly selected Poles can still tell you more than one noisy pundit can. - Rule 2 – Individual polls can’t tell us much
No sports journalist would report only what happened between the 26th and 29th minutes of a football match. To assess the state of play, we need more information, and that means more points of reference. Data on voting intention are collected by a variety of agencies whose methods range from the traditional face-to-face model to telephone and internet surveys.Pollsters will have different techniques for weighting their sample to ensure representativeness, and different ways of phrasing questions. All these things have an impact on the results they obtain. For this reason, psephologists have developed methods of “pooling” polls to extract a more reliable estimate of vote intention that accounts for these differences. - Rule 3 – Watch medium-term trends, not short-term shifts
Public opinion is more like a supertanker than a jet-ski: rather than lurching about with every wave, it changes course gradually in response to actions taken some time ago. Comparing today’s survey with last week’s survey and concluding on the basis of a 5% difference that something momentous has happened is usually a mistake. What you are probably observing is simply the noise around the underlying trend.Dramatic events may cause rapid changes in support for a party, but this is the exception rather than the rule. Parties usually die slowly. Ryszard Petru’s decision to break off from the occupation of the Sejm for a winter holiday in Madeira with a party colleague who was not his wife did not cause the ratings of the liberal Modern (Nowoczesna) to plummet immediately, but the onset of the party’s decline can be dated back to that moment. - Rule 4 – Polls don’t tell you everything you need to know
Polls are usually conducted at the national level, but constituency-level preferences vary. The idiosyncrasies of electoral systems can further distort the ways in which national levels of support translate into shares of seats in parliament. It is shares of seats, not vote shares, that ultimately matter, and translating the former into the latter is not a simple task.
Working from these principles, Notes From Poland will be bringing you periodic updates of the evolving picture, rather than snapshots of striking polls. Poland is fortunate to have substantial and frequent political polling by a variety of agencies using internet, telephone and – less commonly – face-to-face methods. Several polls have already been conducted since October’s election, allowing us to begin to identify party strengths and evolving changes as the new parliament takes shape.
Our model
Our pooled polls model, which is based on code provided by political scientists Kai Arzheimer and Chris Hanretty, has already been put to the test in October’s election, where it came within four percentage points of PiS’s result and correctly predicted that the party would gain a majority of seats. The model uses data released over the last 150 days by nine polling agencies. The latest pooled estimates show the range of more or less credible estimates for each party, and the numbers give the median estimate.
We can see, for example, that on current polling, the ruling national-conservative PiS most probably has around 43% of the vote, but that there is a slight possibility that its actual support is in the high 30s or the high 40s. However, we can rule out actual support for PiS being over 50%.
Turning to trends, we can see that – in keeping with the generally stable results over the last year – the distribution of support has not changed much since the election. Support for PiS has slightly declined and support for the far-right Confederation (Konfederacja) has risen, but not by enough to warrant the conclusion that anything significant has changed overall during the post-electoral period.
However, even small changes in overall support can have significant consequences for shares of seats in parliament, and this is amplified by the differential patterns of competition across each of Poland’s 41 constituencies. When estimating shares of seats, our task is aided by the fact that relative regional levels of support are quite consistent over time.
Although the magnitudes of Poland’s east-west and urban-rural divides are often overstated, PiS tends to perform significantly better in more rural constituencies of the south-east, while the parties comprising the centrist Civic Coalition (KO, made up of Civic Platform, Modern and two smaller groups) usually perform better in large cities and in western constituencies.
Using these patterns of performance to adjust the national-level figures, it is relatively easy to estimate shares of support per party in each constituency and to calculate the likely share of seats gained. On current polling, PiS would lose four seats from the total it gained in October, giving it the slimmest possible majority of 231. The Left (Lewica) (+7) and Confederation (+14) would gain at the expense of KO (-12) and the alliance of the agrarian Polish People’s Party (PSL) and anti-establishment Kukiz’15 (-5).
Pooling polls is not a panacea for all the challenges faced by the psephologist. Just as all polls are wrong, so all estimates are wrong. However, by estimating carefully, we can extract a clearer signal from the information polls give us. And if there is one thing that can be concluded from the polls to date, it is that for all the noise and confusion of Polish politics, that signal is at present a remarkably stable one.
