How to load file line by line in Nim?

Question

I want to load a large file line by line in Nim. I tried the following code snippet:

for line in lines "largefile.txt":
  echo line

However, this loads the entire file largefile.txt into memory which is not feasible when the file is very large > 30GB.

How can I iterate over a large file while holding only a single line in memory ?

Jaundiced

I'm not sure how that would happen, unless you import a non-standard lines iterator somewhere. The one in lib/system.nim reads the file line by line, just as you want it. And when I'm testing that with a large file, memory usage does remain constant, as expected. – Harte

Answer 1

Indeed, Reimer Behrends is right. The lines function works as expected.

The issue was that my file had only escaped newline characters. As a result, Nim (correctly) reads the file as one large line.

Jaundiced