Why does my 5 month old get fussy only 2 hours after nursing?

I try to hold him off till 3 hours before I nurse him. I have started cereal and now recently fruit but I haven't seen any change in his eating and sleeping habits. He takes 3 to 4 naps a day about 30-45 minutes each. He doesn't keep a pacifier in his mouth but seems to like it as long as someone is holding it in.