Johor Bahru is not known for long sandy beaches. But it does actually have a beach, see the photo. This is over near the Zon (duty free area) where the ferry is. Singapore is visible across the water. Swimming might not be the greatest idea, though I did see 2 people swimming once.
You can also find some people selling fresh seafood along the walk. It actually is a nice walk along the water. If the city did a better job of featuring it, it is possible to make it into something really nice. I don’t see anything about the walk in Penang along Gurnsey drive that is any nicer.
But if you really want a beach Malaysia has plenty of great beaches. Langkawi is the best beach location I have been to yet.