Hello Blair Whitworth,
Good morning!
Thank you for publishing in Microsoft Community. We are happy to help you.
Based on the information you have shared; the outlook version you have shared (Version 2406 Build 16.0.17726.20078) need to be update. Reference: https://learn.microsoft.com/en-us/officeupdates/update-history-current-channel-preview
reference: Release Notes Current Channel (Preview) - Office release notes | Microsoft Learn

Please try to update your outlook version and check the issue. this version as some gap as this user. Reference: Outlook Calendar Reminders Full Screen - Microsoft Community
However, to better understand the problem and provide a solution, we need more details from you, please provide us with the following information. Thank you.
1-Please go change your work hours in options and reopen Outlook. Under Outlook Options, Calendar, Work Time. Check the start time you have set up.
And check the calendar view if it shows with your start time at the top.
2- After test i find out f I use my standard calendar account, I do not have this issue. If I select an addition calendar, and I Keep them separated, I do not have this issue
If I merge the external calendar with my standard calendar in a single view, I trigger the issue.
when I would change to "email" and then change back to Calendar. I had 2 calendars showing in 1 view. I removed Calendar 2 and checked my setting for work hours, I then added calendar #2, removed #1, and checked the setting for work hours. Both had the correct settings of 8am - 5pm. I then went back to show both calendars and put them both in the same view, and check the issue
3-try the below step as well:
- Highlight Calendar #1
- Open Calendar Options (I shortcut it to "View" then "Time Scale".) OK that screen.
- Highlight Calendar #2
- Open Calendar Options (I shortcut it to "View" then "Time Scale".) OK that screen.
Done. Go to emails, then back to Calendar, and Start time should be where the calendar starts and NOT midnight.
4-try adjusting the Time Scale under View at the top of the calendar. Changing it to 30 minutes instead of the default of 60 made my calendar the right size for me. and if the issue still persist unchecking the "shared calendar improvements" option under Account properties in Outlook. and check the issue.
We look forward to your reply. This requested information will help us narrow down the scenario and find the right troubleshooting. We will continue to assist you based on the information you provide. I sincerely appreciate your patience and cooperation.SincerelyEben Ezer Tres | Microsoft Community Moderator