Consider the 3 × 3 world shown below. 80% of the time the agent goes in the direction it selects; the rest of the time it moves at right angles to the intended direction.

r -1 +10
-1 -1 -1
-1 -1 -1

Implement value iteration for this world for each value of r below. Use discounted rewards with a discount factor of 0.99.
Show the policy obtained in each case. Explain intuitively why the value of r leads to each policy.

a) r = 100
b) r = −3
c) r = 0
d) r = +3