In my experience, the most influencing factors for mid day deer activity are the rut and human activity. As the rut is heating up this year we are seeing smaller bucks throughout the day and the big boys are starting to show themselves during daylight hours. This seems to be typical year to year for us. Human activity can also "move" deer during the day especially farmers harvesting crops. Food sources also play a role, but I've mostly seen does around them at mid day. My biggest buck to date was shot at 12;15 on opening day as he walked across an open field toward me (i was in a tree stand). He turned to head into the woods at 130yds offering a shot. Trail cams can be a big asset in determining which areas have any mid day activity. This guy finally showed himself during the day at 9:40 (8:40 daylight savings).
One things for sure - if you're not out there, you won't shoot one.
good luck,
Shawn