I’m going to assume that your father lived in British Columbia. This would mean that the administration of his estate would be governed by BC law. (If your father lived in Alberta, the laws of Alberta would apply to the administration of his estate, and the following answer would be different.)
Under BC law, a person interested in the estate can apply to court to require the executor to provide “security” for the administration of the estate. This security, also known as a “bond”, is an amount of money deposited at the court. It protects the beneficiaries and creditors of the estate in the event the executor improperly administers the estate.
Before making such an application, you will want to see the terms of the will. In wills that I draw, there is always a clause covering bonds. The willmaker can require the executor to provide a bond. A bond can be expensive; it depends on the size and circumstances of the estate.
If the will does not require the executor to provide a bond, it can be challenging to convince a court to order one. On your summary of the facts and background, you may not have enough evidence to show your brother is likely to “mis-distribute” the estate (or otherwise be bad).
You may want to look at other avenues to monitor the executor’s work. If your fear is that he will not distribute the estate in accordance with the terms of the will, you may want to consider applying to court for an order replacing the executor or for an order adding you as a co-executor.