Neighbour accessing property through common garden

Hi, I hope I can articulate properly!

We own a ground floor apartment. Each ground floor apartment has a small grassed area  and garden off their verandah which is private but not exclusive use. “Our” area has a small opening to allow the gardener in to mow our section and the neighbours section. We have been told by the committee they don’t want us to access our property via this opening, which we happily abide by. Unfortunately, our neighbour does access there property through this opening despite our requests that they don’t. It intrudes greatly on our privacy, as it’s not only them but their friends, delivery drivers, tradies etc.

Is the owners committee able to stop access to their apartment via this piece of common property?

Any advice greatly appreciated.