C read file line by line

Question

I wrote this function to read a line from a file:

const char *readLine(FILE *file) {

    if (file == NULL) {
        printf("Error: file pointer is null.");
        exit(1);
    }

    int maximumLineLength = 128;
    char *lineBuffer = (char *)malloc(sizeof(char) * maximumLineLength);

    if (lineBuffer == NULL) {
        printf("Error allocating memory for line buffer.");
        exit(1);
    }

    char ch = getc(file);
    int count = 0;

    while ((ch != '\n') && (ch != EOF)) {
        if (count == maximumLineLength) {
            maximumLineLength += 128;
            lineBuffer = realloc(lineBuffer, maximumLineLength);
            if (lineBuffer == NULL) {
                printf("Error reallocating space for line buffer.");
                exit(1);
            }
        }
        lineBuffer[count] = ch;
        count++;

        ch = getc(file);
    }

    lineBuffer[count] = '\0';
    char line[count + 1];
    strncpy(line, lineBuffer, (count + 1));
    free(lineBuffer);
    const char *constLine = line;
    return constLine;
}

The function reads the file correctly, and using printf I see that the constLine string did get read correctly as well.

However, if I use the function e.g. like this:

while (!feof(myFile)) {
    const char *line = readLine(myFile);
    printf("%s\n", line);
}

printf outputs gibberish. Why?

Yep

Use fgets instead of fgetc. You are reading character by character instead of line by line. – Mayonnaise

Note that getline() is a part of POSIX 2008. There may be POSIX-like platforms without it, especially if they don't support the rest of POSIX 2008, but within the world of POSIX systems, getline() is pretty portable these days. – Explore

line[count+1] is an automatic stack variable and you use a pointer to it as return value. This is UB. – Obsess

Answer 1

If your task is not to invent the line-by-line reading function, but just to read the file line-by-line, you may use a typical code snippet involving the getline() function (see the manual page here):

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

int main(void)
{
    FILE * fp;
    char * line = NULL;
    size_t len = 0;
    ssize_t read;

    fp = fopen("/etc/motd", "r");
    if (fp == NULL)
        exit(EXIT_FAILURE);

    while ((read = getline(&line, &len, fp)) != -1) {
        printf("Retrieved line of length %zu:\n", read);
        printf("%s", line);
    }

    fclose(fp);
    if (line)
        free(line);
    exit(EXIT_SUCCESS);
}

Zollie

More precisely, this getline is specific to GNU libc, i.e., to Linux. However, if the intent is to have a line-reading function (as opposed to learning C), there are several public domain line-reading functions available on the web. – Burier

also i think you wanna free(line) before the end of that while loop – Eductive

Why should I do that? Read the manual, buffer is reallocated at each call, then it should be freed at the end. – Zollie

The if(line) check is superfluous. Calling free(NULL) is essentially a no-op. – Padauk

@Padauk I agree, but this is an excerpt from the man page. "Better safe than sorry" they said. )) – Zollie

@gg.kaspersky I'm no specialist of finding divine messages in holy books. The fact is that in real life resources are freed. I think that short code excerpts provide concepts only and do not need to incorporate all the burden of initialization/deinitialization. – Zollie

@PhilipAdler - Then whoever wrote the spec did a silly/careless thing. Although the man page says that "if ptr is NULL, no operation is performed". That's a good enough definition for me. – Padauk

@PhilipAdler If you really wanna fight over free(NULL) being non specified (although I am pretty sure it is nowhere written like this), then you should know that even ls calls free(NULL). After checking, the man page says that free(ptr); free(ptr); is undefined, and that free(NULL) does nothing. @Zollie Then why do you bother freeing line then ? Still, this website is all about teaching or helping with the best solution possible, and freeing every allocated memory that isn't used anymore is actually the good practice to have. – Bigamy

@Jerksa; Can't argue with that! – Wally

For those who said that this getline is specific to GNU libc, "Both getline() and getdelim() were originally GNU extensions. They were standardized in POSIX.1-2008." – Thanasi

Frankly, allocating resources and leaving the responsability of deallocation to the user isn't a really smart idea in itself – Hambrick

To be completely honest this answer is directly from the man page. If a discussion about proper alloc/dealloc of memory is to be had, it should be posed as a separate question and surely it is already. The OP clearly only asked a question about the function at hand and its purpose - to read a file line by line. Therefore this question should have nothing to do with proper memory management or "best practices". Its irrelevant to the original question. – Terrellterrena

@PhilipAdler -- "free(NULL) is a non-defined operation...": from the C11 Standard, §7.22.3.3/2, "If ptr is a null pointer, no action occurs." The C89 and C99 Standards say the same thing. So, free(NULL) is a defined non-operation. – Horned

I no longer have access to the standards I was using four years to validate where my claim comes from, so @DavidBowling may well be right. – Wally

@PhilipAdler -- the link in my comment is to the C11 Draft Standard. But you can also read this and this. Apparently, before C89, some implementations would crash on a null pointer. – Horned

Interesting. I wonder which version of the C standard younger me was reading at my university library. Might be worth a visit to my alma-mater! – Wally

Even C90 §7.10.3.2 The free function said If ptr is a null pointer. no action occurs. Only pre-standard C libraries were at liberty to mishandle free(NULL). After the C90 standard was released, some libraries probably didn't get upgraded for a few years, but by the mid-90s, it was generally safe to use free(NULL) — or, more plausibly, free(ptr) where ptr happened to be a null pointer. – Explore

If you know the maximum length of a line, using fgets is a simpler approach that also doesn't need the heap. – Guardroom

@Nark it was already discussed like eight years ago and ruled out, check the documentation and comments above. – Zollie

Answer 2

FILE* filePointer;
int bufferLength = 255;
char buffer[bufferLength]; /* not ISO 90 compatible */

filePointer = fopen("file.txt", "r");

while(fgets(buffer, bufferLength, filePointer)) {
    printf("%s\n", buffer);
}

fclose(filePointer);

Plowman

For me this results in overwriting each line with the next. See this question based on the above answer. – Xl

If you're ok with lines being limited to a certain length, this is the best answer. Otherwise using getline is a good alternative. I agree the FILE * cast is unnecessary. – Guardroom

It should be const int bufferLength if pedantic:) – Nolde

\n in printf() is unnecessary as well, since we are already printing them line by line, and a line break already exists inside the txt file. – Nil

One of the great answer ever. Just by fgets function. \n is unnecessarily added because the file has already new lines. – Admissive

Answer 3

In your readLine function, you return a pointer to the line array (Strictly speaking, a pointer to its first character, but the difference is irrelevant here). Since it's an automatic variable (i.e., it's “on the stack”), the memory is reclaimed when the function returns. You see gibberish because printf has put its own stuff on the stack.

You need to return a dynamically allocated buffer from the function. You already have one, it's lineBuffer; all you have to do is truncate it to the desired length.

    lineBuffer[count] = '\0';
    realloc(lineBuffer, count + 1);
    return lineBuffer;
}

ADDED (response to follow-up question in comment): readLine returns a pointer to the characters that make up the line. This pointer is what you need to work with the contents of the line. It's also what you must pass to free when you've finished using the memory taken by these characters. Here's how you might use the readLine function:

char *line = readLine(file);
printf("LOG: read a line: %s\n", line);
if (strchr(line, 'a')) { puts("The line contains an a"); }
/* etc. */
free(line);
/* After this point, the memory allocated for the line has been reclaimed.
   You can't use the value of `line` again (though you can assign a new value
   to the `line` variable if you want). */

Stark

@Iron: I've added something to my answer, but I'm not sure what your difficulty is so it may be off the mark. – Burier

@Iron: the answer is that you don't free it. You document (in the API documentation) the fact that the returned buffer is malloc'd ansd needs to be freed by the caller. Then people who use your readLine function will (hopefully!) write code similar to the snippet that Gilles has added to his answer. – Dissociation

Answer 4

//open and get the file handle
FILE* fh;
fopen_s(&fh, filename, "r");

//check if file exists
if (fh == NULL){
    printf("file does not exists %s", filename);
    return 0;
}


//read line by line
const size_t line_size = 300;
char* line = malloc(line_size);
while (fgets(line, line_size, fh) != NULL)  {
    printf(line);
}
free(line);    // dont forget to free heap memory

Dicta

There are some problems with this code: fopen_s makes the code unportable. printf will look for format specifiers and not print percent signs and the following character(s) as they are. Null bytes will make all the characters in the rest of the line vanish. (Do not tell me null bytes cannot happen!) – Potation

And by the way, you do not solve the problem. The OP describes that the return value of his function disappears. I do not see you addressing this problem. – Potation

@Hartley I know this is an older comment, but I am adding this so that someone doesn't read his comment and try to free(line) in the loop. The memory for line is only allocated once before the loop begins, so it should only be free once after the loop ends. If you try freeing line inside the loop, you will get unexpected results. Depending on how free() treats the pointer. If it just deallocates memory and leaves the pointer pointing at the old location the code may work. If it assigns another value to the pointer then you will overwrite a different section of memory. – Harmonic

printf(line) is wrong! Do not do this. This opens your code to a string format vulnerability where you can freely read/write directly to memory via the stuff being printed. If I were to put %n/%p in the file and point the pointer back to an address in memory (in the string from the file) that I controlled, I could execute that code. – Subsequence

Answer 5

A complete, fgets() solution:

#include <stdio.h>
#include <string.h>

#define MAX_LEN 256

int main(void)
{
    FILE* fp;
    fp = fopen("file.txt", "r");
    if (fp == NULL) {
      perror("Failed: ");
      return 1;
    }

    char buffer[MAX_LEN];
    while (fgets(buffer, MAX_LEN, fp))
    {
        // Remove trailing newline
        buffer[strcspn(buffer, "\n")] = 0;
        printf("%s\n", buffer);
    }

    fclose(fp);
    return 0;
}

Output:

First line of file
Second line of file
Third (and also last) line of file

Remember, if you want to read from Standard Input (rather than a file as in this case), then all you have to do is pass stdin as the third parameter of fgets() method, like this:

while(fgets(buffer, MAX_LEN, stdin))

---

_Appendix

Removing trailing newline character from fgets() input

how to detect a file is opened or not in c

Lengthways

Hi, @Lengthways I think we can directly pass MAX_LEN to fgets. I found this piece of description in: linux.die.net/man/3/fgets ``` ``` – Ovary

Hey @juancortez, I am passing MAX_LEN - 1 at the 2nd argument of the method indeed! – Lengthways

There is no need for the -1 in MAX_LEN - 1, fgets(buffer, n, fp) already reads up to n-1 characters reserving space for the null termination. – Keddah

@マルちゃん だよ Yes you are right cplusplus.com/reference/cstdio/fgets, answer updated. – Lengthways

Answer 6

readLine() returns pointer to local variable, which causes undefined behaviour.

To get around you can:

1. Create variable in caller function and pass its address to readLine()
2. Allocate memory for line using malloc() - in this case line will be persistent
3. Use global variable, although it is generally a bad practice

Ferity

Answer 7

Use fgets() to read a line from a file handle.

Balzer

Answer 8

Some things wrong with the example:

• you forgot to add \n to your printfs. Also error messages should go to stderr i.e. fprintf(stderr, ....
• (not a biggy but) consider using fgetc() rather than getc(). getc() is a macro, fgetc() is a proper function
• getc() returns an int so ch should be declared as an int. This is important since the comparison with EOF will be handled correctly. Some 8 bit character sets use 0xFF as a valid character (ISO-LATIN-1 would be an example) and EOF which is -1, will be 0xFF if assigned to a char.

• There is a potential buffer overflow at the line

  lineBuffer[count] = '\0';
  

  If the line is exactly 128 characters long, count is 128 at the point that gets executed.

• As others have pointed out, line is a locally declared array. You can't return a pointer to it.

• strncpy(count + 1) will copy at most count + 1 characters but will terminate if it hits '\0' Because you set lineBuffer[count] to '\0' you know it will never get to count + 1. However, if it did, it would not put a terminating '\0' on, so you need to do it. You often see something like the following:

  char buffer [BUFFER_SIZE];
  strncpy(buffer, sourceString, BUFFER_SIZE - 1);
  buffer[BUFFER_SIZE - 1] = '\0';
  

• if you malloc() a line to return (in place of your local char array), your return type should be char* - drop the const.

Dissociation

Answer 9

Here is my several hours... Reading whole file line by line.

char * readline(FILE *fp, char *buffer)
{
    int ch;
    int i = 0;
    size_t buff_len = 0;

    buffer = malloc(buff_len + 1);
    if (!buffer) return NULL;  // Out of memory

    while ((ch = fgetc(fp)) != '\n' && ch != EOF)
    {
        buff_len++;
        void *tmp = realloc(buffer, buff_len + 1);
        if (tmp == NULL)
        {
            free(buffer);
            return NULL; // Out of memory
        }
        buffer = tmp;

        buffer[i] = (char) ch;
        i++;
    }
    buffer[i] = '\0';

    // Detect end
    if (ch == EOF && (i == 0 || ferror(fp)))
    {
        free(buffer);
        return NULL;
    }
    return buffer;
}

void lineByline(FILE * file){
char *s;
while ((s = readline(file, 0)) != NULL)
{
    puts(s);
    free(s);
    printf("\n");
}
}

int main()
{
    char *fileName = "input-1.txt";
    FILE* file = fopen(fileName, "r");
    lineByline(file);
    return 0;
}

Salver

Why are you using fgetc instead of fgets? – Guardroom

It would be better if it didnt realloc for every character, like if it used a global buffer and doubled the size every time a character would go over the size of the buffer. – Glide

Answer 10

const char *readLine(FILE *file, char* line) {

    if (file == NULL) {
        printf("Error: file pointer is null.");
        exit(1);
    }

    int maximumLineLength = 128;
    char *lineBuffer = (char *)malloc(sizeof(char) * maximumLineLength);

    if (lineBuffer == NULL) {
        printf("Error allocating memory for line buffer.");
        exit(1);
    }

    char ch = getc(file);
    int count = 0;

    while ((ch != '\n') && (ch != EOF)) {
        if (count == maximumLineLength) {
            maximumLineLength += 128;
            lineBuffer = realloc(lineBuffer, maximumLineLength);
            if (lineBuffer == NULL) {
                printf("Error reallocating space for line buffer.");
                exit(1);
            }
        }
        lineBuffer[count] = ch;
        count++;

        ch = getc(file);
    }

    lineBuffer[count] = '\0';
    char line[count + 1];
    strncpy(line, lineBuffer, (count + 1));
    free(lineBuffer);
    return line;

}


char linebuffer[256];
while (!feof(myFile)) {
    const char *line = readLine(myFile, linebuffer);
    printf("%s\n", line);
}

note that the 'line' variable is declared in calling function and then passed, so your readLine function fills predefined buffer and just returns it. This is the way most of C libraries work.

There are other ways, which I'm aware of:

• defining the char line[] as static (static char line[MAX_LINE_LENGTH] -> it will hold it's value AFTER returning from the function). -> bad, the function is not reentrant, and race condition can occur -> if you call it twice from two threads, it will overwrite it's results
• malloc()ing the char line[], and freeing it in calling functions -> too many expensive mallocs, and, delegating the responsibility to free the buffer to another function (the most elegant solution is to call malloc and free on any buffers in same function)

btw, 'explicit' casting from char* to const char* is redundant.

btw2, there is no need to malloc() the lineBuffer, just define it char lineBuffer[128], so you don't need to free it

btw3 do not use 'dynamic sized stack arrays' (defining the array as char arrayName[some_nonconstant_variable]), if you don't exactly know what are you doing, it works only in C99.

Refrain

note that the 'line' variable is declared in calling function and then passed, - you probably should have deleted the local declaration of line in the function then. Also, you need to tell the function how long the buffer is that you are passing and think of a strategy for handling lines that are too long for the buffer you pass in. – Dissociation

Answer 11

void readLine(FILE* file, char* line, int limit)
{
    int i;
    int read;

    read = fread(line, sizeof(char), limit, file);
    line[read] = '\0';

    for(i = 0; i <= read;i++)
    {
        if('\0' == line[i] || '\n' == line[i] || '\r' == line[i])
        {
            line[i] = '\0';
            break;
        }
    }

    if(i != read)
    {
        fseek(file, i - read + 1, SEEK_CUR);
    }
}

what about this one?

Whitacre

Answer 12

Implement method to read, and get content from a file (input1.txt)

#include <stdio.h>
#include <stdlib.h>

void testGetFile() {
    // open file
    FILE *fp = fopen("input1.txt", "r");
    size_t len = 255;
    // need malloc memory for line, if not, segmentation fault error will occurred.
    char *line = malloc(sizeof(char) * len);
    // check if file exist (and you can open it) or not
    if (fp == NULL) {
        printf("can open file input1.txt!");
        return;
    }
    while(fgets(line, len, fp) != NULL) {
        printf("%s\n", line);
    }
    free(line);
}

Imhoff

Answer 13

You should use the ANSI functions for reading a line, eg. fgets. After calling you need free() in calling context, eg:

...
const char *entirecontent=readLine(myFile);
puts(entirecontent);
free(entirecontent);
...

const char *readLine(FILE *file)
{
  char *lineBuffer=calloc(1,1), line[128];

  if ( !file || !lineBuffer )
  {
    fprintf(stderr,"an ErrorNo 1: ...");
    exit(1);
  }

  for(; fgets(line,sizeof line,file) ; strcat(lineBuffer,line) )
  {
    if( strchr(line,'\n') ) *strchr(line,'\n')=0;
    lineBuffer=realloc(lineBuffer,strlen(lineBuffer)+strlen(line)+1);
    if( !lineBuffer )
    {
      fprintf(stderr,"an ErrorNo 2: ...");
      exit(2);
    }
  }
  return lineBuffer;
}

Kellum

Answer 14

My implement from scratch:

FILE *pFile = fopen(your_file_path, "r");
int nbytes = 1024;
char *line = (char *) malloc(nbytes);
char *buf = (char *) malloc(nbytes);

size_t bytes_read;
int linesize = 0;
while (fgets(buf, nbytes, pFile) != NULL) {
    bytes_read = strlen(buf);
    // if line length larger than size of line buffer
    if (linesize + bytes_read > nbytes) {
        char *tmp = line;
        nbytes += nbytes / 2;
        line = (char *) malloc(nbytes);
        memcpy(line, tmp, linesize);
        free(tmp);
    }
    memcpy(line + linesize, buf, bytes_read);
    linesize += bytes_read;

    if (feof(pFile) || buf[bytes_read-1] == '\n') {
        handle_line(line);
        linesize = 0;
        memset(line, '\0', nbytes);
    }
}

free(buf);
free(line);

Baynebridge

Why are you using the heap (malloc) instead of the stack? It seems there's a simpler stack based solution with fgets that could be used. – Guardroom

Answer 15

Provide a portable and generic getdelim function, test passed via msvc, clang, gcc.

/*
 * An implementation conform IEEE Std 1003.1-2017:
 * https://pubs.opengroup.org/onlinepubs/9699919799/functions/getdelim.html
 *
 * <nio.h>:
 * https://github.com/junjiemars/c/blob/c425bd0e49df35a2649327664d3f6cd610791996/src/posix/nio.h
 * <nio.c>:
 * https://github.com/junjiemars/c/blob/c425bd0e49df35a2649327664d3f6cd610791996/src/posix/nio.c
 *
*/

#include <errno.h>
#include <stdlib.h>
#include <stdio.h>
#include <limits.h>


/*
 * LINE_MAX dependents on OS' implementations so check it first.
 * https://github.com/junjiemars/c/blob/c425bd0e49df35a2649327664d3f6cd610791996/src/posix/nlim_auto_check
 */
#define NM_LINE_MAX  4096       /* Linux */

#if (MSVC)
typedef SSIZE_T  ssize_t;
#  if !defined(SSIZE_MAX)
#    define SSIZE_MAX  ((ssize_t)((size_t)((ssize_t)-1) >> 1))
#  endif
#endif


ssize_t getdelim(char **restrict lineptr, size_t *restrict n, int delimiter,
                 FILE *restrict stream);

#if defined(getline)
#  undef getline
#endif
#define getline(lp, n, f)  getdelim((lp), (n), 0x0a, (f))


ssize_t
getdelim(char **restrict lineptr, size_t *restrict n, int delimiter,
         FILE *restrict stream)
{
    int       c;
    char     *p, *p1;
    ssize_t   len;

  if (NULL == lineptr || NULL == n || NULL == stream
      || (UCHAR_MAX < delimiter || delimiter < 0))
    {
      errno = EINVAL;
      return EOF;
    }

  if (feof(stream) || ferror(stream))
    {
      return EOF;
    }

    if (0 == *lineptr)
    {
      if (0 == *n)
        {
          *n = NM_LINE_MAX;
        }

      *lineptr = malloc(*n);
      if (0 == *lineptr)
        {
          return EOF;
        }
    }

  p = *lineptr;
  len = 0;

    while (EOF != (c = fgetc(stream)))
    {
      if (SSIZE_MAX == (ssize_t) len)
        {
          errno = EOVERFLOW;
          return EOF;
        }

      if ((size_t) len == (*n - 1))
        {
          *n <<= 1;
          p1 = realloc(*lineptr, *n);
          if (0 == p1)
            {
              return EOF;
            }
          *lineptr = p1;
          p = p1 + len;
        }
      *p++ = (char) c;
      len++;

      if (c == delimiter)
        {
          break;
        }
    }

  if (ferror(stream))
    {
      return EOF;
    }

  *p = 0;
    return len;
}



int
main(void)
{
  FILE     *fp;
  char     *line  =  NULL;
  size_t    len   =  0;
  ssize_t   read;

  fp = fopen("/some-file", "r");
  if (fp == NULL)
    exit(1);
  while ((read = getline(&line, &len, fp)) != -1) {
    printf("Retrieved line of length %zu :\n", read);
    printf("%s", line);
  }
  if (ferror(fp)) {
    /* handle error */
  }
  free(line);
  fclose(fp);

  return 0;

}

Ani

Why do this when fgets exists? – Guardroom

does fgets can customize line delimiters or customize what to do about current lines? – Cave

getdelim allows for customized delimiters. Also I do notice don't have a line length limit -- in this case you can use the stack with getline. (Both described here: man7.org/linux/man-pages/man3/getline.3.html) – Guardroom

do you talk about just Linux , the question is about how to read line in C, right? – Cave

This works for any standard c implementation (getdelim and getline were standardized in POSIX.1-2008, someone else mentions on this page). fgets is also standard c, and not linux specific – Guardroom

POSIX C != C standard, why you talking about POSIX? POSIX is another topic. – Cave

True, but getline, fgets, and getdelim I believe work on Mac, Linux, and Windows. – Guardroom

There are no getdelim on Windows. – Cave

Ah, you're right. This discussion helps me understand when your code would be useful. – Guardroom

Answer 16

You make the mistake of returning a pointer to an automatic variable. The variable line is allocated in the stack and only lives as long as the function lives. You are not allowed to return a pointer to it, because as soon as it returns the memory will be given elsewhere.

const char* func x(){
    char line[100];
    return (const char*) line; //illegal
}

To avoid this, you either return a pointer to memory which resides on the heap eg. lineBuffer and it should be the user's responsibility to call free() when he is done with it. Alternatively you can ask the user to pass you as an argument a memory address on which to write the line contents at.

Samora

There is a difference between illegal and undefine behavior ^^. – Palecek

Answer 17

I want a code from ground 0 so i did this to read the content of dictionary's word line by line.

char temp_str[20]; // you can change the buffer size according to your requirements And A single line's length in a File.

Note I've initialized the buffer With Null character each time I read line.This function can be Automated But Since I need A proof of Concept and want to design a programme Byte By Byte

#include<stdio.h>

int main()
{
int i;
char temp_ch;
FILE *fp=fopen("data.txt","r");
while(temp_ch!=EOF)
{
 i=0;
  char temp_str[20]={'\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0','\0'};
while(temp_ch!='\n')
{
  temp_ch=fgetc(fp);
  temp_str[i]=temp_ch;
  i++;
}
if(temp_ch=='\n')
{
temp_ch=fgetc(fp);
temp_str[i]=temp_ch;
}
printf("%s",temp_str);
}
return 0;
}

Franks

your program would work if your brackets were in the right places ;) e.g. int main() { – Sideline

Incidentally, you don't need to specify all 20 '\0'. You can just write: code char temp_str[20] = {'\0'}; code c will automatically fill each slot with a null terminator since the way array declarations work is that if an array is initialized with less elements that the array contains, the last element will be fill out the remaining elements. – Harmonic

I believe char temp_str[20] = {0} also fills up the entire character array with null terminators. – Guendolen