The Sultan Abu Baker Mosque was constructed upon a hill overlooking the Johor Straits between 1892 and 1900 and is the state mosque for Johor. The mosque can accommodate 3,000 worshippers.
I recommended taking a short trip to visit and walking around the grounds. You can walk to old town from without much trouble (I would guess it is less than 2 km from old town), obviously you can drive or take a taxi if you prefer.
I asked about going into the Mosque but was told I couldn’t. I don’t know if that is always the case or if there are certain times for tourists to view inside. I didn’t see any signs with that information and the person I spoke to didn’t speak English.
The views are very nice and the architecture is an interesting English influence on a Mosque.
The Mosque is across the street from the Johor Zoo.
Address: Jalan Gertak Merah off Jalan Abu Bakar, Johor Bahru, 80100
Hours: I found a comment online that says 10:00-18:00 from Saturday-Thursday except during prayers, I don’t see any official information though.