So this is my first week really digging into this site and the first week I've done anything with polls. Now it could be an outlier, but it sure seems that some fishy stuff is going on at RCP. I haven't lost any money, so I'm not coming here as a sore loser.
It seems RCP has some crazy inconsistencies. They don't post polls at reliable times, they may choose to exclude a poll or just parts of it. They may post a poll and then not even include it on the poll type landing page so it never becomes part of the average.
Right now I have quite a bit of money riding on them actually moving a poll from the 'latest polls' page to the appropriate page it should be based on the group it's under in 'latest polls'. It sometimes takes them a few hours to update a poll from 'latest polls' to include it in the average. In this case, it's been more than a day.
Again, haven't lost money so not crying about this, but it's peculiar that they dropped Fox and added YouGov today. It was seeming like YouGov may have not been added at all and then for both events to happen so early in the morning was a bit curious (but likely coincidence) to me.
The reason for me posting is this: If we are going to do things that are based upon polls, perhaps we should get a more reliable, and more importantly, OPEN source. So instead of RCP, we could find another way to gather the data. Instead of wondering if RCP is going to choose to drop polls, the contracts would include all polls from a date range.
It doesn't seem right that so much of the market is dependent on inconsistent actions from a third party.
So for Obama approval next week we could say something like.....
All polls from X companies (Gallup, Rasmussen, Fox, CNN, etc) with polling dates from X to X, using X voters.
If PredictIt is willing, they could actually maintain an average similar to RCP, but more open. Yes, it's a little more work for them, but they are taking 10% on profit + 5% withdrawal so I think they can do it. They could update once a day (and would have to update by end of day on the last day) but we would know their possible sources so there would be no question on if or when something would post.
Look, I don't know if this is the best way, but I'm really not a fan of the power RCP has. I'm sure there are some F5ers that really love the unpredictability, and sure it can make for some nice short term gains, but I just feel there is too much mystery behind some of this stuff.
I agree it's not really in the spirit of political betting. Its website maintenance betting.
What if we have the weekly polls end on Thursday, not Friday? The funny stuff happens on Friday, and if it happens on the first day, it will have a much less adverse affect on the market than if it happens on the last day,.....
I just like the fact that you referred to some people as "F5ers." :)
I have a suggestion for the Obama approval multi-market:
Instead of using RCP, PredictIt should name the polls they will include if released before 11:59pm on Friday and either (a) the number of polls (e.g. the eight most recent of the aforementioned polls) or (b) the time period for which the polls will be included (for example, polls will no longer be included 14-21 days from either the start or end date of the poll.
As a suggestion, PredictIt could specify that it will use the following polls: (i) Rasmussen, (ii) Gallup, (iii) PPP, (iv) Reuters/ Ipsos, (v) Economist/YouGov, (vi) USA Today/Suffolk, (vii) Pew Research, (viii) NBC News/ Wall St. Jrnl, (ix) Fox News, (x) Bloomberg, (xi) Quinnipiac, (xii) CNN/ Opinion Research, (xiii) National Journal, (xiv) ABC News/ Wash Post, (xv) National Journal, and (xvi) McClatchy/ Marist. Other possible polls for consideration include Morning Consult and Gravis Marketing/ One America News.
(Sources: (a) http://www.realclearpolitics.com/epolls/other/president_obama_job_approval-1044.html#polls ; (b) http://elections.huffingtonpost.com/pollster/obama-job-approval ; and (c) http://fivethirtyeight.com/interactives/pollster-ratings/).
This would make the market more predictable and certain and clearly about Obama's approval rating rather than users having to guess about the decisions that RCP will make (including (i) what polls RCP will decide to use in its average; (ii) whether and when RCP will get around to updating its average; (iii) when RCP will drop polls from its average; and (iv) how many polls RCP will decide to include in its average).
I agree with OP. PredictIt should create its own polling aggregation page, with transparent rules about which polls get included and when it gets updated. This would 1) solve problems about RCP being inconsistent and opaque, and 2) drive more traffic to the site.
+1 for this. The prediction should be about Obama's approval ratings, not about how & when the guys at RCP choose to update. Any progress on this?
I completely agree on this. I would add that I saw this question asked in an old webinar on youtube from 2014. The CEO said something like "we have approached them about being more consistent. Averaging polls together ourself would be onerous". Obviously, we all know that it wouldn't be that difficult, as many here have pointed out. It wouldn't surprise me if there's an "arrangement" a foot here.
Just started betting this week to put money on McMullen being at 20% on the RCP average on 10/24. When I bought them I didn't realize that the bet was more along the lines of, "will RCP drop his single-digit polls from September by 10/24?" That is much less fun to predict and could turn me away from the cite completely.
These complaints come with the territory of betting on subjective things. Polling, despite all its attempts to be scientific, is limited to editorial judgements as to what gets polled when, how it gets polled, what models are used to weight the data, how the data is acquired. Double that with the choices of what polls to use in averaging, what ones are available to average, and how frequently do they update.
Worse are the "fact checking" markets. The numbers of "lies" told by one candidate or another depends on how many statements are parsed, in what detail, and not to mention the grey area of subjective judgements of statements that are treated as black and white true or false when there is a degree of uncertainty.
I wouldn't cry if these markets weren't used in the future, as the nature of what these statistics are in the moment change based on editorial choices of the source. In fact, I wouldn't be surprised in the future if RCP objected to being used in this manner.