How can I remove the oldest lines in a file when using a FileStream and StreamWriter?

Question

Based on Prakash's answer here, I thought I'd try something like this to remove the oldest lines in a file prior to adding a new line to it:

private ExceptionLoggingService()
{
    _fileStream = File.OpenWrite(GetExecutionFolder() + "\\Application.log");
    _streamWriter = new StreamWriter(_fileStream);
}

public void WriteLog(string message)
{
    const int MAX_LINES_DESIRED = 1000;

    StringBuilder formattedMessage = new StringBuilder();
    formattedMessage.AppendLine("Date: " + DateTime.Now.ToString());
    formattedMessage.AppendLine("Message: " + message);

    // First, remove the earliest lines from the file if it's grown too much
    List<string> logList = File.ReadAllLines(_fileStream).ToList();
    while (logList.Count > MAX_LINES_DESIRED)
    {
        logList.RemoveAt(0);
    }
    File.WriteAllLines(_fileStream, logList.ToArray());

    _streamWriter.WriteLine(formattedMessage.ToString());
    _streamWriter.Flush();
}

...but in my version of .NET (Compact Framework, Windows CE C# project in VS 2008), neither ReadAllLines() nor WriteAllLines() are available.

What is the ReadAllLines/WriteAllLines-challenged way of accomplishing the same thing?

UPDATE

This is doubtless kludgy, but it seems like it should work, and I'm going to test it out. I moved the "shorten the log file" code from the WriteLog() method to the constructor:

private ExceptionLoggingService()
{
    const int MAX_LINES_DESIRED = 1000;

    string uriPath = GetExecutionFolder() + "\\Application.log";
    string localPath = new Uri(uriPath).LocalPath;
    if (!File.Exists(localPath))
    {
        File.Create(localPath);
    }
    _fileStream = File.OpenWrite(localPath);
    // First, remove the earliest lines from the file if it's grown too much
    StreamReader reader = new StreamReader(_fileStream);
    List<String> logList = new List<String>();
    while (!reader.EndOfStream)
    {
        logList.Add(reader.ReadLine());
    }
    while (logList.Count > MAX_LINES_DESIRED)
    {
        logList.RemoveAt(0);
    }
    if (logList.Count > MAX_LINES_DESIRED)
    {
        _fileStream.Close();
        File.Delete(GetExecutionFolder() + "\\Application.log");
        File.Create(GetExecutionFolder() + "\\Application.log");
        _fileStream = File.OpenWrite(GetExecutionFolder() + "\\Application.log");
    }
    _streamWriter = new StreamWriter(_fileStream);
    foreach (String s in logList)
    {
        _streamWriter.WriteLine(s);
        _streamWriter.Flush();
    }
}

public void WriteLog(string message)
{
    StringBuilder formattedMessage = new StringBuilder();
    formattedMessage.AppendLine("Date: " + DateTime.Now.ToString());
    formattedMessage.AppendLine("Message: " + message);
    _streamWriter.WriteLine(formattedMessage.ToString());
    _streamWriter.Flush();
}

Shutter

ReadAllLines really is just a ReadAllText followed by a Split("\r\n") – Elbowroom

@Elbowroom Are you sure? StreamReader's ReadLine? – Elstan

@Elbowroom Not according to referencesource.microsoft.com/#mscorlib/system/io/… . Its actually doing my code below – Lohner

The implementation could be (and probably is) different, but the semantic result is a read and split. A loop on ReadLine like you have is probably more efficient. – Elbowroom

Answer 1

ReadAllLines and WriteAllLines are just hiding a loop from you. Just do:

StreamReader reader = new StreamReader(_fileStream);
List<String> logList = new List<String>();

while (!reader.EndOfStream)
   logList.Add(reader.ReadLine());

Note that this is nearly identical to the implementation of File.ReadAllLines (from MSDN Reference Source)

       String line;
        List<String> lines = new List<String>();

        using (StreamReader sr = new StreamReader(path, encoding))
            while ((line = sr.ReadLine()) != null)
                lines.Add(line);

        return lines.ToArray();

WriteAllLines is simialr:

StreamWriter writer = new StreamWriter(path, false); //Don't append!
foreach (String line in logList)
{
   writer.WriteLine(line);
}

Lohner

Would the downvoter care to comment? This is a valid approach to my knowledge – Lohner

Silent downvoters and closevoters are either craven cowards or lazyboy loungers. – Shutter

Bradley, I am not he downvoter but FileStream doesn't have ReadLine method. OK, your edit is good. – Elstan

I see how this adds the contents of the file to the list of Strings, but I'm not seeing how the list of String is written to the fileStream. – Shutter

@B.ClayShannon I thought it was similar enough to be obvious. Added a sample so that its clear :) – Lohner

@BradleyDotNET: But this just adds to the already overflowing _fileStream, right? My update has the idea I'm currently slogging through. – Shutter

@B.ClayShannon I'm not sure what you mean. What is overflowing? Your code seems reasonable (though you don't need to Flush after each write, and actually not at all as long as you close the stream down). If the code is too long, you could always put the loops in methods; even call them ReadAllText and WriteAllText if you want. – Lohner

@BradleyDotNET: What I mean is, the file has over max lines, and is being added to. I don't know if I'm doing it the right way in my update, but it's my attempt (close, delete, create, open). – Shutter

@B.ClayShannon I see what you mean now. Just open the StreamWriter in "Overwrite" mode. I've updated my answer. You'll need to use the "Path" mode instead of the "Stream" mode for this to work. – Lohner

Answer 2

I would write simple extension methods for this, that do the job lazily without loading whole file to memory.

Usage would be something like this:

outfile.MyWriteLines(infile.MyReadLines().Skip(1));

---

public static class Extensions
{
    public static IEnumerable<string> MyReadLines(this FileStream f)
    {
        var sr = new StreamReader(f);

        var line = sr.ReadLine();
        while (line != null)
        {
            yield return line;
            line = sr.ReadLine();
        }
    }

    public static void MyWriteLines(this FileStream f, IEnumerable<string> lines)
    {
        var sw = new StreamWriter(f);
        foreach(var line in lines)
        {
            sw.WriteLine(line);
        }
    }
}

Connatural