I come up with 0.567.
Probability of four heads and one tail out of five flips is P(head)^4*P(tail)^1*(5 choose 4).
5 choose 4 is 5 (i.e. HHHHT, HHHTH, HHTHH, HTHHH, THHHH).
For coin A, this probability of getting four heads and one tail in any order is 0.4^4*0.6^1*5 = 0.0768, for coin B it is 0.15625, and for coin C it is 0.36015.
The probability of getting a certain coin followed by four heads and one tail is the probability of getting that coin times one of the above probabilities. Call them Prob2(A), Prob2(B) and Prob2(C). I call it Prob2() to distinguish it from the probability of getting that coin in the first place, which is Prob(A)=0.5, Prob(B)=0.3 and Prob(C)=0.2.
The probability of getting coin A and four heads and one tail is Prob2(A) = 0.5 times 0.0768 = 0.0384.
For coin B, Prob2(B) = 0.3 times 0.15625 = 0.046875
For coin C, I get Prob2(C) = 0.07203.
After picking a coin and flipping it 5 times and observing the result, to determine the probability I've selected a certain coin in the first place, I take the probability of getting that result for A or B or C divided by the sum of Prob2(). I can simply sum those probabilities because they are mutually exclusive (i.e. I'm not going to get both coins A and B if I only pick one coin and then flip it 5 times). Whether I've picked coin A or coin B or coin C to begin with, I must end in Prob2(A) or Prob2(B) or Prob2(C), no more, no less.
Prob2(A)+Prob2(B)+Prob2(C) = 0.157305
Let us define Prob3(A) as the probability I've picked coin A, Prob3(B) for coin B and Prob3(C) for coin C. This is the probability of picking that coin given that I flipped it five times and got four heads and one tail.
Therefore, after observing that I got 4 heads and 1 tail, the probability I've picked coin A is Prob3(A) = Prob2(A)/0.157305 = 0.244112. For coin B, I get Prob3(B) = Prob2(B)/0.157305 = 0.297988, and for coin C, Prob3(C) = Prob2(C)/0.157305 = 0.4579.
To check my work, I add those three probabilities Prob3(A)+Prob3(B)+Prob3(C) = 0.244112+0.297988+0.4579 and I get 1.
Finally, to calculate the probability the next flip is heads, we sum the probability that I've picked a certain coin (based on the fact that we observed 4 heads and 1 tail) times the probability its next flip is heads. This would be Prob3(A) * 0.4 plus Prob3(B) * 0.5 plus Prob3(C) * 0.7, and the result is 0.567169.