Check square between the table and spindle body. You *may* have to remove the spindle, and then the spindle mounting plate to get them parallel to each other, THEN put the spindle back on the Z & re-square.

The numbers you mention are well within the ballpark for woodworking & I would question your means of measurement. The only real way to get blueprinted/perfect adjustments is with a dial indicator. You don't need to take it that far. Start cutting and making mistakes...then worry about precision 6 months from now when you can appreciate it.

-B