Archived Forum Post

Index of archived forum posts

Question:

How to Handle a 10GB Email?

Feb 18 '13 at 10:06

Lets say, i have an email file. This e-mail file is about 10GB. (With attachments 50mb each and many...)

em.LoadEml(‘C:myemail.eml’); em.GetAttachments().....

In this example, if i load this file with loademl method, will this email file loaded to memory? But what if i have 4GB of RAM?

What is the correct way of parsing and split to pieces for this email file?


Answer

Given that Chilkat Email and MIME were designed to hold the entire email (i.e. MIME) in-memory, it's not possible to load a 10GB email into an email object (or a MIME object).

One approach might be this: Assuming the top-level MIME header is multipart/mixed, write code to load enough of the MIME into a memory string to capture the 1st header (I'm sure no more than 5K would be needed). Then load this string into a Chilkat MIME object. This is only for the purpose of easily extracting the boundary string. Once you have the boundary string, you may discard the MIME header you just loaded.

With the boundary-string known, you may now write code that opens the 10GB file and scans for the next occurrence of the boundary string. Save each of the MIME parts delimited by the boundary strings to separate files. Don't include the boundary string in each file, and make sure each file begins with the 1st header line of the sub-part. At this point, each of these sub-part files are themselves MIME, and may be loaded individually into Chilkat MIME objects for processing.