So that can I understand what you want, you want to put a 3 mm hole in 4 mm stock. Leaving a half mm on each side of the hole.
Are you planing on doing this with a hand drill or a drill press? I will tell you the best drill bits, are the brad points from Lee Valley. Their drill bits have a lip, that cut a cleaner hole. With that tight tolerance, I would probably drill some practice pieces first.