This is the first time this has happened to me, so it came as a mild shock.  I actually was surprised.  I had gone next door to borrow the use of a fellow builder's drill press (mine is down for repairs) and proceeded to drill out the F-1011D and F-1012D (pictured).  What happened was burrs built up on the underside of the stock.  See the ridges around the holes? 

These burrs can happen for several reasons.  One is that the bit has partially stopped cutting because it has aluminum welded to it.  And this happens because it's gotten too hot because I didn't use any lubricant.  The speed of the drill may have been off, but I doubt it.  In this case it was slow, which is better than extremely fast, however, make sure you check for correct speed. 

Get some lubricant, either Tap Magic for aluminum, or CoolTool II or Teflon lube.  And make sure you clear the bit by raising it at least once per hole.  Don't push hard.  Let the bit do the work. 

If you do get the burrs (sounds like a medical condition), then take a small flat file and smooth them down, followed by cleaning up any file marks.  Then go back and drill the holes again, since in filing you'll have pushed some of the burr back into the hole.  PITA.