You will rarely lose a harvest if you harvest at the same time every day. Crops that are ripe can often, though not always, still be properly harvested at the same time the following day.
Your crops crop will be ripe when the top of the crop (when groomed) is exactly level with the top of a [[small barrel]] placed in the tileturns yellow.
Your crops need to be harvested within about 24 hours of becoming ripe. If you harvest too early or too late you will get reduced yield. If you are only getting one potato, or whatever, anyway then you will get nothing from early or late harvesting.