On 2023-05-06 17:25, mike wrote:
> On 06-05-2023 06:34 Paul <nospam@needed.invalid> wrote:
>
>>> Does this switch from MBOX to MAILDIR change the typical archive
>>> strategy?
>>
>> If your inbox had 9,000,000 messages, we know the Windows file system
>> does not handle such directories at all well. The Windows Search
>> Indexer is
>> rated for "1,000,000 files" by the designers.
>
> I never use Windows search for anything so I've always turned off that
> stupid wasteful useless indexer since day 1, which solves that problem.
>
> services.msc | Windows Search | Stop | Startup type => Disabled

The issue here is not Windows search. It is about having millions of
files in a single directory. Normal filesystems hate that. I only know
of one filesystem that loves that, and you can not use it.

>
>> TB has GLODA for indexing
>> messages (something to do with a Search feature?), and that's just
>> another
>> copy of the Windows Search Indexer in a sense.
>
> Turned off that long ago also.
> thunderbird.exe | Tools | Settings | General | Inexing | Uncheck "Enable
> Global Search and Indexer."

Nevertheless, Thunderbird has to index the "folder", and apparently does
that in a single file, probably something.msf. The flags will also be
probably stored inside each message as headers.

>
>> While NTFS has the
>> structure to hold 4 billion files, nobody said anything about actually
>> interacting with a folder of that size, which would suck. The only
>> practical
>> way for me to store 4 million files on an NTFS volume, was to arrange
>> them in a tree, and tree storage is not at all easy for humans
>> to wrangle by hand (when you want to move individual files).
>>
>> Since your box storage only had around 25000 messages, the "tax" at the
>> file system level should be manageable.
>
> Thank you for that advice as I am only using the MAILDIR format because of
> the compaction bugs in the MBOX format when switched from HTML to TEXT.
>
>> But to estimate the effect the change would make, it really depends on
>> what flags are stored (artificially) in the headers of the messages.
>> When a message comes into your Inbox, three lines are added to it
>> for message management. Whether those flags are a match for .msf or
>> .sqlite flags, I don't know. If those flags get updated on a regular
>> basis, there might not be that much difference between doing backups
>> on the two schemes.
>
> The problem there is I don't want to do any backups if I don't need to.
> I want Google to spend their money storing my old email on their servers.
> If I need to search the email, I can use Google's search mechanism.

Well, forget that and start doing mail backups. They are mandatory now
that you have changed to Maildir. The documentation warned you about that.

Besides that, remember that the cloud is known to evaporate and destroy
your data.

>
> By moving the SENT mail into a "Local Folders", now the onus is on me to
> have to back up and save that folder - but I had to do it due to quota.

You could have purchased more space.

--
Cheers,
Carlos E.R.

On 09-05-2023 02:51 gym <no_one@example.net> wrote:

> On 5/6/2023 3:08 PM, Carlos E. R. wrote:
>> Ah. It is too new, work in progress.
>
> While maildir is still a work in progress, it is not new. It has been
> available as an option since Thunderbird 38. Thunderbird's intention is
> to eventually make maildir the default. Of course they've been saying
> this for years, but supposedly significant progress was made in TB 102.
> Don't cancel the popcorn subscription just yet... :)

Thanks for responding to Carlos where I didn't respond because all I was
going to tell him was that he was basing his assumptions on a forum thread
from what seemed like 8 years ago, which is a long time for improvements.

So far the MAILDIR format with the huge (13GB) GMail inbox works fine.
No hangs reported by MemoryHogs & WhatIsHang for days in a row so far!

I'm using Betterbird (for my real email) and Thunderbird (in a test
configuration) in MAILDIR format and both are having no problems (but the
Betterbird test is more realistic).

I don't see any reason to go back to Thunderbird since I can barely tell
the difference if I don't look at the icons, so I'd say they're the same
for the things I do with mail on Betterbird that I had done on Thunderbird.

My recommendation to others is that the instant you get any hint that the
compaction had a problem, then to stop using that Thunderbird installation.

Either fix the compaction problem (not easy to do) or wipe out your profile
and start again (which only works with IMAP & if you do NOT use local
folders!).

After months of punishment, I'm happy to be rid of MBOX compaction forever.

On 09-05-2023 10:52 gym <no_one@example.net> wrote:

>> The whole selling point of GMail's 15GB is THEY do the backups for you.
>> How could anyone miss that is their major selling point over all others?
>> So you can't blame me for what likely BILLIONS of people do, can you?
>
> I'm certainly not blaming you for not wanting to do backups. Some of my
> clients won't do them even after multiple disk crashes and lost data.
> Personally, I would never trust a third-party to be solely responsible
> for my data. I've seen too many users get locked out of their accounts
> with no way back in. But that is me. You're absolutely right that many
> people do just that. Everyone's comfort level varies.

Well, the problem isn't really the SPACE used for backups. It's the FORMAT.

Have you ever gotten a Google "takeout" of your mail backup?
As far as I can tell, as a normal use, it's worthless.
It's 15GB of "something", but I don't know what format it's in.

If Thunderbird could READ that Google takeout format, that would be useful.
Can it?

Even a Thunderbird backup of the LOCAL folders isn't readable by anything
but Thunderbird, am I right? Or is a TB backup in a human-readable format?

If I saved my Thunderbird profile, and then twenty years from now I want to
read it, can I use a standard text editor to read my mail from a prior age?

On 2023-05-09 16:49, mike wrote:
> On 09-05-2023 10:52 gym <no_one@example.net> wrote:
>
>>> The whole selling point of GMail's 15GB is THEY do the backups for you.
>>> How could anyone miss that is their major selling point over all others?
>>> So you can't blame me for what likely BILLIONS of people do, can you?
>>
>> I'm certainly not blaming you for not wanting to do backups. Some of
>> my clients won't do them even after multiple disk crashes and lost
>> data. Personally, I would never trust a third-party to be solely
>> responsible for my data. I've seen too many users get locked out of
>> their accounts with no way back in. But that is me. You're absolutely
>> right that many people do just that. Everyone's comfort level varies.
>
> Well, the problem isn't really the SPACE used for backups. It's the FORMAT.
>
> Have you ever gotten a Google "takeout" of your mail backup?
> As far as I can tell, as a normal use, it's worthless.
> It's 15GB of "something", but I don't know what format it's in.
>
> If Thunderbird could READ that Google takeout format, that would be useful.
> Can it?

That's not the way I backup email. I simply mark all the emails in the
imap folder, and tell Thunderbird to copy them all to a local folder (or
use an external tool for that). Later, I backup the entire Thunderbird
tree to another disk.

>
> Even a Thunderbird backup of the LOCAL folders isn't readable by anything
> but Thunderbird, am I right? Or is a TB backup in a human-readable format?

Actually, it is. It is standard MBOX format, readable by almost any
_Linux_ mail tool. And many can be read with just a text editor.

>
> If I saved my Thunderbird profile, and then twenty years from now I want to
> read it, can I use a standard text editor to read my mail from a prior age?

Yes.

Actually, I can read with Thunderbird the emails that Netscape saved two
decades ago or more.

mbox is plain text, not binary. However, some mails are sent with encoding.

Trick:

For moving or copying mail from one mail client application to another,
you just need to upload the email to an imap server with one and
download with another.

--
Cheers, Carlos.

On 5/9/2023 10:49 AM, mike wrote:
> On 09-05-2023 10:52 gym <no_one@example.net> wrote:
>
>>> The whole selling point of GMail's 15GB is THEY do the backups for you.
>>> How could anyone miss that is their major selling point over all others?
>>> So you can't blame me for what likely BILLIONS of people do, can you?
>>
>> I'm certainly not blaming you for not wanting to do backups. Some of my clients won't do them even after multiple disk crashes and lost data. Personally, I would never trust a third-party to be solely responsible for my data. I've seen too many users get locked out of their accounts with no way back in. But that is me. You're absolutely right that many people do just that. Everyone's comfort level varies.
>
> Well, the problem isn't really the SPACE used for backups. It's the FORMAT.
>
> Have you ever gotten a Google "takeout" of your mail backup?
> As far as I can tell, as a normal use, it's worthless.
> It's 15GB of "something", but I don't know what format it's in.
>
> If Thunderbird could READ that Google takeout format, that would be useful.
> Can it?
>
> Even a Thunderbird backup of the LOCAL folders isn't readable by anything
> but Thunderbird, am I right? Or is a TB backup in a human-readable format?
>
> If I saved my Thunderbird profile, and then twenty years from now I want to
> read it, can I use a standard text editor to read my mail from a prior age?

https://helpdeskgeek.com/how-to/how-to-export-or-download-all-gmail-emails/

takeout-20230224T082142Z-001.zip <=== if a Takeout is big enough, expect >1 ZIP files

Takeout\
Mail\
allmail.mbox 12,345,678 bytes

There is also an HTML file, which records the details
of which account it is.

The ZIP extension, should tell you right away how to access it.

Inside the big file, a message has a custom header. This could be the "Label".
The "Label" is added by Google.

Here, I show some sample labels, so you can see how "allmail" can be
separated into "Inbox" and "Sent". I would think a user would have to
compare these, to the visual box structure in their GMail webmail,
to figure out what the labels mean, and whether a label has a "box implication".
For example, is "Archived" a box or is it "Trash" on the screen ?

X-Gmail-Labels: Inbox,Category Promotions,Unread
X-GMail-Labels: Sent,Opened
X-Gmail-Labels: Inbox,Important,Opened,Category Personal
X-Gmail-Labels: Archived,Sent,Important
X-Gmail-Labels: Inbox
X-Gmail-Labels: Sent

At this point, the trail grows cold, in that the source
for the Thunderbird program, does not contain "X-GMail-Labels".
It's not immediately apparent whether Thunderbird knows what
to do, or not. If Thunderbird opens it, it will become the
AllMail box and the labels won't do anything.

If you look at the helpdeskgeek.com article, there
is some option in there to select Takeout emails with a particular
label, which would allow doing a Takeout with Inbox, or
a Takeout with Sent. Personally, I'd just prefer to get the
AllMail with the least fuss, to start with. But the longer you
look at the content of some of those emails (some of the commercial
emails), the more of a puzzle it becomes.

Paul

On 09-05-2023 22:40 "Carlos E.R." <robin_listas@es.invalid> wrote:

> For moving or copying mail from one mail client application to another,
> you just need to upload the email to an imap server with one and
> download with another.

That might be useful. But how is it done to "upload" mail to IMAP?

Let's say I have a Local Folder in TB called "Sent" (which I do).
Let's say I create a new GMail account, let's call it "sent@gmail.com".

How would I upload thousands of those SENT mails to sent@gmail.com?

On 2023-05-10 00:34, mike wrote:
> On 09-05-2023 22:40 "Carlos E.R." <robin_listas@es.invalid> wrote:
>
>> For moving or copying mail from one mail client application to
>> another, you just need to upload the email to an imap server with one
>> and download with another.
>
> That might be useful. But how is it done to "upload" mail to IMAP?
>
> Let's say I have a Local Folder in TB called "Sent" (which I do).
> Let's say I create a new GMail account, let's call it "sent@gmail.com".
>
> How would I upload thousands of those SENT mails to sent@gmail.com?

Suppose you have your mail in local folders in Outlook, and you want to
migrate to Thunderbird, also local folders.

You need a friendly imap account, which can be gmail, or can be another
computer in your home, running Linux. Call that account bilbo@baggins.end

You register both Outlook and Thunderbird to that convenient imap
account, bilbo@baggins.end. You create in it a new empty folder, called
"temp".

Then in Outlook you go to the folder you want to migrate, mark all the
emails, and copy them, using the mouse, or the edit menu, to the temp
folder. You wait it to finish.

Then you go to Thunderbird. You also open the bilbo@baggins.end account.
You open the "temp" folder. Select all the email in temp, edit copy,
edit paste in the destination local folder of your wish.

Done.

It is more complicated to explain than to do.

--
Cheers, Carlos.