Learning Lessons from: Experiment 11

Last Thursday I guinea pigged for an economics experiment at UCL. Unfortunately it wasn’t anything as exciting as Tim’s ones, no apples or chocolates were on offer but it was fascinating to see an economics lab from the inside. The invitation simply named it “Experiment 11” which I thought to be rather mysterious but it was probably just because economists don’t have an imagination. I’ll try and describe what it was like, but it will be very confusing so bear with me.

The first thing that I noticed was that there were only eight of us participating, so any one of us could change their results significantly. However, I’d imagine this to be repeated many times so as to overcome the limitations in having such a small sample size. We were each sat in our own little cubicle with a computer in front of us. There were three parts to the experiment, the first two parts ask us to assess the value of a good, while the last part allows us to trade this good.

Part 1:

The eight players were divided into two halves: A1, A2, A3, A4 and B1, B2, B3, B4. In Part 1 of the experiment, everyone is initially given the real values of ten goods. We’re told that the real value of each good is related to their respective Clue As and Clue Bs but we’re not told how. The A players are only given the ten Clue As while the B players are only given the ten Clue Bs. I was Player A1 and here are the values I was given:

Clue A Real Value
6.49 34.41
16.58 40.01
16.83 58.82
27.75 47.34
31.29 42.43
32.75 57.10
35.84 67.80
40.21 66.77
45.20 94.15
46.49 63.98

Originally they were in no particular order, I put their Clue As in ascending order so that I might be able to plot a pattern for their RVs. The first obvious thing was that Clue A is smaller than RV, and whatever their relationship is it doesn’t look to be linear.

After learning from these ten initially given values, the actual test started and we were given new Clue As and told to predict RVs from them. I tried to use the ten initial figures but the pattern was so erratic that I was often off by ten or more. This continued for fifteen rounds.

What I found interesting was their way of payment. They randomly pick a round between 11 and 15 and pay us £3.00 – 0.003x2 where x is the difference between our predication and the RV. The square in the formula means that we’re equally incentivised to overestimate or underestimate, but any large errors will be compounded exponentially.

Part 2:

After 15 rounds of Part 1 we are given the initial ten goods again but this time we have their Clue A, Clue B and their RVs. This is what I was given:

Clue A Clue B Real Value
6.49 26.11 34.41
16.58 20.59 40.01
16.83 40.01 58.82
27.75 22.34 47.34
31.29 13.37 42.43
32.75 23.99 57.10
35.84 32.21 67.80
40.21 25.91 66.77
45.20 46.16 94.15
46.49 16.70 63.98

After some mental arithmetic I realised that Clue A + Clue B ? RV ±<3.00. Now it makes a lot of sense that the relationship between Clue A and the RV seemed so erratic – there was another variable involved in the formula! It was pretty obvious by now and I’m sure everyone else realised the link too.

Now that I’ve got the formula the 15 predictions they asked us to make were a lot more accurate. I simply set my prediction to be Clue A + Clue B and played the law of averages with the ±<3.00: since sometimes I would overshoot and sometimes I would under, it’s better to just set it to be the sum and let the averages even out. As expected I earned more during this part of the experiment than the last part.

Part 3:
Now that we’ve learnt how to value the goods, the real experiment started. We had 30 rounds but only one clue again: Clue A for me. But from this Clue A, we each were asked for a buy price and a sell price. Each player was matched with the 4 players from the other team – i.e. player A1 was matched with B1, B2, B3, B4. Each round we were given 20p and then we had the possibility of earning more/less by trading. Two players trade if one player’s buy price is higher than the other player’s sell price, and at half-way between those prices. Since each of us had 4 trading partners, each round there’s the possibility of trading with 1, 2, 3, 4 people or with no one.

The reward formula for trading was Trade Price – Real Value + 5 pence. It means that every time trade is initiated, both players start off with a base profit of 5p, which I suppose modelled the consumer/producer surplus in trade, though of course in real life that’s never equal for both parties. That 5p profit can be altered by Trading Price – Real Value. Since the trading price was halfway between the buyer’s buy price and the seller’s sell price, one party profits (the buyer profits if TP < RV, while the seller profits if TP > RV).

For example, if my buy price is 30p and my sell price is 40p, while B1’s buy price is 50p and his sell price is 65p, trade occurs since I’m willing to sell for 40p while he’s willing to buy for 50p. The trade price at which it occurs is 45p, exactly halfway between 40p and 50p. But we’re then told that the real value is 38p. Now my total profit from this is (0.45 – 0.39) + 0.05 = 0.07 + 0.05 = 0.12. By selling something worth 38p at 45p, I’ve made 7p profit in addition to the 5p I was awarded purely for trading. This process may happen with each of my four trading partners and each round everyone gets 20p for participating in the round.

Sounds complicated? It damn was. Since I didn’t have the luxury of Clue B anymore, it was back to random guessing from Clue A. This time I knew the mechanics of how the Real Value was deduced, but then so did everyone else so I had to try and capitalise it more than them. Since I knew Clue B had to be between 0 and 50, I set it at an imaginary value of 25 (playing the law of averages again). I then set the ± 3.00 margin to be my buy/sell spread, so my buy price would be Clue A + 25 – 3 while my sell price would be Clue A + 25 + 3.

I thought I had it sussed until round 5 when I realised that the unknown Clue B really wasn’t behaving like I wanted it to – too often did my buy/sell spread not cover the RV. In addition, I wasn’t trading nearly as much as I thought I would be – most of the rounds were just spent with no trade happening and simply accumulating 20p. So I shaded down Clue B to a +20 modifier, purely on gut instinct and widened my spread to 10 (± 5.00 on estimated RV) – which may have been a mistake as I’ll explain later.

After each round we can see how our trading partners have decided to set their buy/sell prices. Since I thought I had the right idea, I thought others would have clocked onto it too, and followed a consistent formula. The fact that I saw my trading partners acting in completely unpredictable ways spooked me, and I ended up widening my spread since they all seemed to have it wider too. I guess they can test information diffusion on that – even wrong ones!

I spoke to a few other players after the experiment and someone else had made more than me with the strategy of Clue B + 20 with a spread of 5 (± 2.50). That got me thinking. It seems like having a low spread facilitates the most trade, so how about taking it to the extremes and minimising it to zero? Having the buy price = the sell price seems counter-intuitive but is that the best way of playing with these rules? That way you’ll be able to gain the most out of those 5p bonuses awarded simply for trading, and any bonus profits/losses will be ridden out over time.

If anyone has actually gotten to the end of this incomprehensible ramble, then get yourself a cookie, seriously. What are your ideas on an optimal pricing strategy?

Also, if anyone would like to copy the Part 2 data mine into a spreadsheet and figure out how the RVs are really generated feel free. It might be that the ±<3s are simply randomly generated, but it may follow a pattern such as odds/evens, sums of digits, Normal distribution etc. I know it’s only ten pieces of data but any insights would be helpful. And was it right for me to shade the unknown Clue B down to 20 simply because of my failures? The average still looks to be around 25…

[Editor’s note: this is the sixth instalment of the Learning Lessons series, detailing the author’s exploits on the London lecture circuit. For further information or to subscribe to the mailing list, contact arthurmauk@gmail.com]