Why is an IMAP Inbox Only Displaying 10,000 Items When There are Many More Emails on the Server?

I am afraid though, that instead of admitting to lying to many customers and reversing this change, Yahoo/AOL will just say - we do not support 3rd party Apps, talk to Microsoft about making the New API Call.

i THINK AOL gave that up a few years ago when they forced us to set an app password for third party apps. I forget if they had an explicit one for Outlook, or said that outlook used the generic one. So they acknowledged and assisted Outlook users. They can't now say they don't support it.

I am the Original Poster. First, I must thank everyone who has contributed to this thread. I don't know why the problem surfaced on my system a few weeks earlier than it affected everyone else's, but I am quite impressed by the persistence and effort of the community to get this problem corrected. After the MS moderator person blew off my requests to figure out why this was actually happening (see earliest posts on page 1 of thread) I figured this was going to be one of those threads where dozens of people said they had the same problem, the individual posts were ignored by the moderators and nothing ever happened. Not only have many of the respondents actually tried to put pressure on Yahoo/AOL to correct their coding, but it looks like Mark_302 has actually found the root cause. Superb!!

I have no technical background, so I have little of a practical nature to add. But I have one observation in response to YanTeper's post (another person who has done fantastic work in trying to track this down and fix it!) and a question for those who have more expertise than I.

Comment to YanTeper: While I have no illusions about Microsoft's business practices--does anyone else remember the antitrust suit?--I don't think MS was a conspirator here. I use Outlook 365, and have the same problem as everyone else. (I was contemplating going back to Outlook 2016 to see if that solved the problem, but it is now clear that it doesn't) I don't think MS did this to get more people to migrate to the subscription model. However, you may be right that if MS does update Outlook to support the IMAP UIDONLY extension, the updated version may only be made available to 365 subscribers. Although, isn't MS still updating some of the older versions of Outlook? You would think if they're going to patch this they'd make the change on all versions that they are currently supporting. (That may be wishful thinking on my part).

This leads to my Question--now that this thread in on its sixth page, how do we get Microsoft to take Mark_302's findings seriously? Again, I don't have any technical background or knowledge of how these boards are actually operated, but from my limited experience, the moderators check out the first post or two and then ignore a thread that's been going a while.

If that's so how will MS ever find these posts so it can do whatever is necessary to support the IMAP UIDONLY extension? Does someone need to start new threads so the moderators can report this information to the software developers? How do you get a company like MS do something now that we know what the problem is?

(Given that there is a solution for the third party developers, I have a hard time believing Yahoo/AOL will do anything. They've been doing their best for a couple of years to discourage the use of third party apps to read AOL mail. It clearly appears that they want you to use their webmail so they can sell more ads.)

Thanks again for everyone's help!

I have the full Office subscription with Outlook 365 and have exactly the same 10,000 email limitaiton on folders now. Happened in the past week or two.

Brutal.

Mark_302

This is an amazing research, thank you for doing this.

You pinpointed precisely what some of us have been guessing around intuitively.

This is a very concrete proof of a change in IMAP functionality that imposed Messagelimit onto all clients who have used special NEW API to get Unlimited messages. Yahoo caught red-handed!

I am afraid though, that instead of admitting to lying to many customers and reversing this change, Yahoo/AOL will just say - we do not support 3rd party Apps, talk to Microsoft about making the New API Call. And that is bad news for those of us who, like me, use versions of Outlook other that 365, that won't get any updates even if Microsoft created the fix.

In fact, it just occurred to me, Microsoft could have instigated the change by Yahoo/AOL to shake out the users of older Outlook version. Clearly, Microsoft wants you to buy 365 subscription rather than continue using old Outlook that was bought 10 years ago or so.

I have found some Yahoo documentation that sheds some light on the behavior at this location.

https://senders.yahooinc.com/developer/documentation

The page describes two possible modes of IMAP operation, the default with a limited number of messages; and a new mode of operation called UIDONLY that is described in the Yahoo documentation as "unlimited". This UIDONLY extension is in draft standardization with a technical specification here

https://www.ietf.org/archive/id/draft-melnikov-imap-uidonly-00.html

Some further experiments with direct IMAP interface to the Yahoo server using OpenSSL show both the origin of the message limit value of 10000 and further more allow the message count for the Inbox folder to give the true value when this new UIDONLY mode is enabled.

Partial logs from the session are copied below:

tag capability

* CAPABILITY IMAP4rev1 ID MOVE XYMHIGHESTMODSEQ UIDPLUS LITERAL+ CHILDREN UNSELECT X-MSG-EXT OBJECTID IDLE ENABLE UIDONLY X-ALL-MAIL X-PAGINATION X-UIDONLY LIST-EXTENDED LIST-STATUS SPECIAL-USE MESSAGELIMIT=10000 X-MESSAGEPAGELIMIT=10000 PARTIAL APPENDLIMIT=41697280

tag OK CAPABILITY completed

tag enable uidonly

* ENABLED UIDONLY

tag OK ENABLE completed

tag capability

* CAPABILITY IMAP4rev1 ID MOVE XYMHIGHESTMODSEQ UIDPLUS LITERAL+ CHILDREN UNSELECT X-MSG-EXT OBJECTID IDLE ENABLE UIDONLY X-ALL-MAIL X-PAGINATION X-UIDONLY LIST-EXTENDED LIST-STATUS SPECIAL-USE MESSAGELIMIT=1000 X-MESSAGEPAGELIMIT=1000 PARTIAL APPENDLIMIT=41697280

tag OK CAPABILITY completed

tag select Inbox

* 33934 EXISTS

* 0 RECENT

* OK [UIDVALIDITY 1] UIDs valid

* OK [UIDNEXT 248767] Predicted next UID

* FLAGS (\Answered \Deleted \Draft \Flagged \Seen $Forwarded $Junk $NotJunk)

* OK [PERMANENTFLAGS (\Answered \Deleted \Draft \Flagged \Seen $Forwarded $Junk $NotJunk)] Permanent flags

* OK [HIGHESTMODSEQ 57636]

* OK [MAILBOXID (1)] Ok

tag OK [READ-WRITE] SELECT completed; now in selected state

tag status Inbox (MESSAGES)

* STATUS "Inbox" (MESSAGES 33934)

tag OK STATUS completed

Note that the message count is now correct (33934).

The logs without use of the UIDONLY mode are

tag OK LOGIN completed

tag capability

* CAPABILITY IMAP4rev1 ID MOVE XYMHIGHESTMODSEQ UIDPLUS LITERAL+ CHILDREN UNSELECT X-MSG-EXT OBJECTID IDLE ENABLE UIDONLY X-ALL-MAIL X-PAGINATION X-UIDONLY LIST-EXTENDED LIST-STATUS SPECIAL-USE MESSAGELIMIT=10000 X-MESSAGEPAGELIMIT=10000 PARTIAL APPENDLIMIT=41697280

tag OK CAPABILITY completed

tag select Inbox

* 10000 EXISTS

* 0 RECENT

* OK [UIDVALIDITY 1] UIDs valid

* OK [UIDNEXT 248767] Predicted next UID

* FLAGS (\Answered \Deleted \Draft \Flagged \Seen $Forwarded $Junk $NotJunk)

* OK [PERMANENTFLAGS (\Answered \Deleted \Draft \Flagged \Seen $Forwarded $Junk $NotJunk)] Permanent flags

* OK [HIGHESTMODSEQ 57636]

* OK [MAILBOXID (1)] Ok

tag OK [READ-WRITE] SELECT completed; now in selected state

tag status "Inbox" (MESSAGES)

* STATUS "Inbox" (MESSAGES 10000)

tag OK STATUS completed

Note that this time the message count is wrong / limited.

So the situation may be that with this recent Yahoo server configuration change that the Mail client (i.e. Outlook in my case) must support the new IMAP UIDONLY extension to allow access to message counts more than 10000.

I would now have a question to Microsoft technical support: does Outlook support the IMAP UIDONLY extension? And, if so, in which version is this support introduced?