My ref friend Dave Taylor then told me how they should have called it. The kick was an illegal kick because it was a loose ball. You can not kick a lose ball. If you remember the holder was fumbling the ball (he did not have possession) when the kicker kicked it. The penalty is 15 yards and lose of down. Here is the rule from the rule book.
Illegally Kicking Ball
ARTICLE 4. A player shall not kick a loose ball, a forward pass or a ball being held for a place kick by an opponent. These illegal acts do not change the status of the loose ball or forward pass; but if the player holding the ball for a place kick loses possession during a scrimmage down, it is a fumble and a loose ball; if during a free kick, the ball remains dead (A.R. 8-7-2-VII).
PENALTY-15 yards from the basic spot and loss of down if the loss of down is not in conflict with other rules (Rules 10-2-2-c, d, e and f) [S31 and S9] (Exception: No loss of down if the foul occurs when a legal scrimmage kick is beyond the neutral zone).