Slice bread into 1 inch cubes, place in a large bowl and set aside.
Whisk together eggs, milk, sugars, vanilla, cinnamon and nutmeg.
Add mashed bananas and mix together.
Pour mixture over the bread cubes and fully coat. Let sit for about an hour.
Preheat oven to 350 degrees and grease a 9x13" baking dish with butter.
After the bread has soaked with the custard mixture, add to the greased baking dish and bake for 50-60 minutes or until custard has set and the top is golden brown.
While the bread pudding is baking make the caramel sauce.
Serve warm with caramel sauce.
Notes
If the top of the bread pudding is starting to brown too much before it is done place some foil over the top.