Received Verbal offer but no written offer, how soon should i follow up?

The CEO (also owner) of a small company reached out to me directly for an HR position. He and the CFO interviewed me. The CFO (the position reports to him) told me that he would call me by Tuesday and let me know of their decision. Tuesday evening, he said, "as promised, i'm calling you to let you know i have good news, i pushed for decision in your favour, can you come in tomorrow, i will try to have your offer letter when we meet?". i went the next day and we discussed salary, benefits and start date, no letter as he didn't have time to prepare (since they have no current HR, he would have to do paperwork himself). He explained the org chart, resources, even offered to give me an offshore resource etc. he also told me : "between you and me, i didn't meet ant other candidate as you are most suitable". all seemed good, as i parted, he told me to look out for his email that day for offer letter along with other official documents on which he wanted me to start working. that was wednesday, today is friday and no news from them. should i be worried? how soon should i follow up? I'm currently unemployed and they know that i just started my search.