Grease the bottom and sides of a 9x13 pan with oil. Cut the bread into roughly 1-inch cubes and line the bottom of the pan with the bread.
In a separate mixing bowl, whisk together the eggs, milk, cream, vanilla, sugar, and brown sugar. Pour evenly over the bread. Use a fork to push any dry bread underneath the egg mixture to coat. Cover and store overnight in the refrigerator.
In a zip top bag, add flour, brown sugar, cinnamon, salt, and nutmeg. Close it tightly and shake until the dry ingredients are combined. Store in the fridge.
When you're ready to bake, preheat the oven to 350℉. Sprinkle the sweet topping evenly over the top of the casserole. Bake for 45 minutes (for a softer bread pudding texture) to 1 hour (for a firm texture). Serve generous slices either plain or with a drizzle of maple syrup. Refrigerate leftovers.