Use existing IMAP server and IMAP component for that.
Creating your own IMAP server is a big security risk in my opinion.
There are free components, but most of them have problems with national characters and 'not standard' email messages...and there is none support (LumiSoft.Net is abandoned for almost 2 years)
using(Imap imap = new Imap())
{
imap.Connect("imap.server.com");
imap.Login("user", "password");
imap.SelectInbox();
List<long> uidList = imap.Search(Flag.Unseen);
foreach (long uid in uidList)
{
IMail email = new MailBuilder()
.CreateFromEml(imap.GetMessageByUID(uid));
Console.WriteLine(email.Subject);
Console.WriteLine(email.Attachments.Count);
}
imap.Close();
}
Mail.dll also includes SMTP component and template engine so you can send replies easily.
Disclaimer: I'm involved in the development of this commercial product.