Those who break the new law face fines of up to 500 euros (£428), and businesses risk losing licences after several offences.
Greece is the EU's heaviest-smoking nation - 40% of the population smokes.
But defying the authorities is a national sport and there is no guarantee the ban will succeed, a BBC correspondent in Athens says.
'Moment of truth'
Small cafes and restaurants with premises measuring less than 70 square metres can choose either to be tobacco-free or to admit only patrons who smoke, our correspondent Malcolm Brabant says.
Bigger establishments can have clearly identified and ventilated smoking areas.
In order to give teeth to the new regulations, which came into force on 1 July, smokers who break the law can be fined up to 500 euros.
On the first offence, businesses will be fined 1,000 euros. Repeat offenders will face even bigger fines, and ultimately, on the fourth violation, could risk losing their licence.
Each year, 20,000 Greeks die from tobacco-related illnesses and the cost in terms of health care amounts to some 1.5bn euros.
Many workers are exposed to passive smoking.
Greek Health Minister Dimitris Avromopoulos has said the moment of truth has arrived, with the ban aiming to revolutionise people's outlook.
But many Greeks regard the new law as an infringement of civil liberties, our correspondent says.
He says that in the face of widespread civil disobedience, successive governments have always had a problem enforcing new legislation.
And so despite the tough talk, there is no guarantee that the new measures will succeed.