login about faq

Currently I'm using the PHP IMAP libs to connect and download full imap accounts.

Current workflow;

  • Cronjob gets all folders with uidvalidity and inserts into DB. Will also list ALL UIDS (queue) found and insert into DB. (Taking 5 to ~100seconds depending on size (1MB to 10GB mail accounts)
  • Background process gets 15 UIDs from queue (which are in same imap folder). Connects to server and connects to the folder. Then it will fetch the messages from the (15) uids in the queue with the FetchSingle function ( http://www.chilkatsoft.com/refdoc/phpImapDoc.html#method031 ).
  • The background process will extract the needed information (subject, addresses, body text and attachments) from the email object and then save the Email object to an .eml file ( http://www.chilkatsoft.com/refdoc/phpEmailDoc.html#method128 )

Are there any ways to speed up the background process of downloading (this is taking the most time) the e-mails. It must be able to handle big accounts (>25GB, 50k messages). Currently it is just going at about 250 - 1k e-mails per hour.

I'm hoping someone could give me any tips about how to speed this process up.

Currently getting pretty much the same results @ 2 servers (1 x Amazon Ec2 with RDS and 1x a VPS with local mysql db).

asked May 17 '13 at 11:09

flo's gravatar image


Be the first one to answer this question!
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here



Answers and Comments

Markdown Basics

  • *italic* or __italic__
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported



Asked: May 17 '13 at 11:09

Seen: 1,269 times

Last updated: May 20 '13 at 22:47

powered by OSQA