Implementing a fixed-size log file, or a circular buffer on disk

Question

I checked this question, but it's not what I'm looking for.

I'm trying to figure out how to cap a log file's size (say, 10MB), and as soon as it's hit, either:

• start writing to the beginning, rather than appending, or
• keep appending, but delete the contents from the beginning as I do so

Don't really care about language - as long as it's possible :)

---

Note: I am aware of the rolling log files approach (hit a target size, rename, and continue logging). I am looking to avoid such a roll.

Jacobjacoba

stackoverflow.com/a/9075604 – Whitewash

Answer 1

If you are implementing both the writer and the reader, then you can do something like this:

struct logentry {
    timestamp  ts;
    char       msg [4000];
};

class logger {
private:
    int write_recordnum;  // next record number to write
    int max_recordnum;  // controls maximum size of file
    FILE    *logfile;

public:
    logger (const char *filename, int max_records)
    {
        max_recordnum = max_records;
        logfile = fopen (filename, "a+");
    }

    void write_next_entry (const char *msg, ...)
    {
        struct logentry ent;
        // format message into entry
        va_list ap;
        va_start (ap, msg);
        vsnprintf (ent.msg, sizeof(ent.msg), msg, ap);
        va_end (ap);
        ent.ts = gettimestamp();

        // position logfile
        if (write_recordnum > max_recordnum)
            write_recordnum = 0;

        fseek (logfile, write_recordnum * sizeof (ent), 0);
        fwrite (&ent, 1, sizeof(ent), logfile);
    }

    bool read_entry (int recnum, char *msg)
    {
        struct logentry ent;
        if (recnum >= max_recordnum)
            return false;
        fseek (logfile, recnum * sizeof (ent), 0);
        fread (&ent, 1, sizeof(ent), logfile);
        strcpy (msg, ent.msg);
        return true;
    }
};

The idea is to manage a circular buffer by explicit fixed-size record numbers. Needed is logic to manage whether record N exists, and to check errors.

Wickman

a suggestion received out-of-band was to use fixed-size records and store a brief amount of meta-data at the front of the file, indicating which record to start from – Jacobjacoba

Answer 2

Why not do rolling logfiles? Does it have to be precisely 10MB? If 10MB is your quota, a common practice would be to write to blah.log, and when it hits, say 1MB, rename file to blah.log.1 and start writing to blah.log. Much simpler, and a very common practice. In fact, in Linux, if you use syslog, it's free.

Scribner

yes, I know about rolling logfiles (and logrotate, etc) - I'm looking more if I have an absolute do-not-exceed size, and I don't want to keep track of which file I'm in, having a self-rolling logfile would be nice – Jacobjacoba

Answer 3

If you are using Log4[j/net] there are options for a rolling log. See the RollingFileAppender. Also, there is an option when specifying the attributes of the log file to set the maximum file size for the log. (Param: MaxFileSize)

Calendre

this appears to automate rolling the logfiles, if I read it correctly - not rolling inside a logfile – Jacobjacoba