Yes, but without exchanging the power supply (driver) you won't be able to fully use dimmable LEDs in this scenario. Classic halogen drivers have a valid power range for the output matching the amount of bulbs at a given power. If you get above or below these limit range, the driver could switch off the output. With LED bulbs instead of halogen ones, power consumption will only reach about 10% of the old value, so for example if your driver has a range of 35-105W (usually with 3 halogen bulbs of 35W each) you will only have around 12W after switching to LED bulbs, which lies below the limit. And if you try do dimm, consumption will even drop more, so the driver will switch off.
Such a driver can always break down and must be accessible in order to be exchanged. If it is mounted in the ceiling, it should be located near one of the bulb holes so it be pulled out.
WARNING: If you try to find this out, be sure to switch of the lights fuse first!!!
Best in this case would be to get help from an electrician.