I'm writing a program that supports input from stdin. To be able to do that I tend to rely on a simple loop
that tests the return of fgets(), such as the following example:

if (fgets (buffer, BUFFER_SIZE, stdin) != NULL)
{
// read the buffer
}


I've been doing this for a while but I'm not sure that this is the best way to handle it. A quick google on
the subject failed to return any meaningful result and searching this group's history through google groups
ended up being a very disappointing experience (a search for nothing more than "stdin" returns only 14
results that only go as far as June 8th). So, what's the best way to handle input from stdin?


Thanks in advance,
Rui Maciel

"Rui Maciel" <rui.maciel> wrote in message
news:d347
> I'm writing a program that supports input from stdin. To be able to do
> that I tend to rely on a simple loop
> that tests the return of fgets(), such as the following example:
>
> if (fgets (buffer, BUFFER_SIZE, stdin) != NULL)
> {
> // read the buffer
> }
>> I've been doing this for a while but I'm not sure that this is the best
> way to handle it.

What's the problem with doing things this way?

If you're worried about dealing with input lines of any conceivable length,
then everyone apparently writes their own 'getline' routines. You might try
searching for that, and fgetline, instead.

And in the code fragment you've given, I'm not sure how useful comparing the
result to NULL will be (assuming live input). When the user enters an empty
line, the buffer should contain "\n"; the function will succeed.

Rui Maciel wrote:
> I'm writing a program that supports input from stdin. To be able to do that I tend to rely on a simple loop
> that tests the return of fgets(), such as the following example:
>
> if (fgets (buffer, BUFFER_SIZE, stdin) != NULL)
> {
> // read the buffer
> }
>> I've been doing this for a while but I'm not sure that this is the best way to handle it. A quick google on
> the subject failed to return any meaningful result and searching this group's history through google groups
> ended up being a very disappointing experience (a search for nothing more than "stdin" returns only 14
> results that only go as far as June 8th). So, what's the best way to handle input from stdin?

You can't judge "best" without an objective of some kind.
The snippet you've shown looks perfectly plausible for many
situations, although there may be difficulty with input lines
that are too long (more than BUFFER_SIZE-2 "payload" characters,
plus the line-ending '\n', plus the string-ending '\0').

"Bless you, it all depends!" -- Pitti-Sing

"Rui Maciel" <rui.maciel> wrote
>
> So, what's the best way to handle input from stdin?
>
There's no decent way of writing a completely reusable getline() function.
The reason is that malciious users can potentially pass lines long enough to
exhaust memory, and there is no strategy for dealing with this that will be
right for all programs.
However as a general rule, fgets() with a big buffer(eg 4096 bytes) and some
wrapper code to check for the newline is OK. Most people do not know how to
use fgets() properly. The newline is a sentinel that tells you a full line
has been read. If it is not present, you need to take action, not treat the
half-read line as a full one, which could cause a nasty bug.

Eric Sosman wrote:

> You can't judge "best" without an objective of some kind.
> The snippet you've shown looks perfectly plausible for many
> situations, although there may be difficulty with input lines
> that are too long (more than BUFFER_SIZE-2 "payload" characters,
> plus the line-ending '\n', plus the string-ending '\0').

I intend to use the code in that snippet to simply fill a buffer from a file that is then used by a parser to
build a document tree. The parser then "waits" for more input once it reaches the buffer's end. Is it
possible that, in this case and judging by the example code, some problem may come up if the input lines are
too long?


Rui Maciel

Rui Maciel wrote:
> I'm writing a program that supports input from stdin. To be able to do that I tend to rely on a simple loop
> that tests the return of fgets(), such as the following example:
>
> if (fgets (buffer, BUFFER_SIZE, stdin) != NULL)
> {
> // read the buffer
> }
>> I've been doing this for a while but I'm not sure that this is the best way to handle it. A quick google on
> the subject failed to return any meaningful result and searching this group's history through google groups
> ended up being a very disappointing experience (a search for nothing more than "stdin" returns only 14
> results that only go as far as June 8th). So, what's the best way to handle input from stdin?

That depends entirely upon what the input looks like and what you need
to do with it. That's a key reason why there's several different ways to
do it. The approach you're currently using is fine for line-oriented
text files, completely inappropriate for binary files, and not
necessarily a good way to go if you're reading text files that are not
line-oriented.

bartc wrote:

> What's the problem with doing things this way?

I believe there isn't a problem and until now everything worked well. Nonetheless, as "getting something to
work" isn't exactly the same thing as "doing it right" then it is better to make sure that it is as good as
it appears to be.


> If you're worried about dealing with input lines of any conceivable
> length, then everyone apparently writes their own 'getline' routines. You
> might try searching for that, and fgetline, instead.

I've browsed some articles on getline() and it appears that it works like the code snippet I've posted here
with the added feature that the function also returns when stumbling on a specific delimiter character.

As that feature forces the getline() function to compare all characters with a given delimiter character
then it appears to be needlessly slower, as the input always needs to be parsed after being fed to a
buffer.

What is getline() used for?


> And in the code fragment you've given, I'm not sure how useful comparing
> the result to NULL will be (assuming live input). When the user enters an
> empty line, the buffer should contain "\n"; the function will succeed.

The NULL test is performed in order to check for EOF.


Rui Maciel

James Kuyper wrote:

> That depends entirely upon what the input looks like and what you need
> to do with it. That's a key reason why there's several different ways to
> do it. The approach you're currently using is fine for line-oriented
> text files,

That's good to hear.


> completely inappropriate for binary files

What's the best way to read data from a binary file?

> , and not
> necessarily a good way to go if you're reading text files that are not
> line-oriented.

How come?


Rui Maciel

Rui Maciel wrote:
> Eric Sosman wrote:
>
>> You can't judge "best" without an objective of some kind.
>> The snippet you've shown looks perfectly plausible for many
>> situations, although there may be difficulty with input lines
>> that are too long (more than BUFFER_SIZE-2 "payload" characters,
>> plus the line-ending '\n', plus the string-ending '\0').
>
> I intend to use the code in that snippet to simply fill a buffer from a file that is then used by a parser to
> build a document tree. The parser then "waits" for more input once it reaches the buffer's end. Is it
> possible that, in this case and judging by the example code, some problem may come up if the input lines are
> too long?

(The "example code" was a single fgets() call.)

If a line is too long for the buffer, fgets() will read as
much as it can, filling all but the last buffer position with
data from the input and placing a '\0' in the last position.
Whether that's a "problem" or not depends on what the program
then does with the partial line. Note that the un-read tail of
the line is still pending on the input, ready to be read as if
it were the "next" line -- another likely source of confusion
if the data in the lines is structured.

You can try strchr(buffer, '\n') to see whether a complete
line has been read, and this *almost* works. If it returns
non-NULL, fgets() has in fact read an entire line. But if it
returns NULL, there are two possibilities: fgets() may have
read a partial line, or the very last line of input may lack
a '\n' and fgets() has read all the way to end-of-file. (On
some systems this situation cannot arise, but on others it can.)

Rui Maciel wrote:
> bartc wrote:

>> If you're worried about dealing with input lines of any conceivable
>> length, then everyone apparently writes their own 'getline'
>> routines. You might try searching for that, and fgetline, instead.
>
> I've browsed some articles on getline() and it appears that it works
> like the code snippet I've posted here with the added feature that
> the function also returns when stumbling on a specific delimiter
> character.
>
> As that feature forces the getline() function to compare all
> characters with a given delimiter character then it appears to be
> needlessly slower, as the input always needs to be parsed after being
> fed to a buffer.

That character is likely to be '\n', which if present is going to be at the
end of a line so removing that is a quick operation, at least after strlen()
is used. Since this is file i/o, I don't think strlen is a significant
overhead.

> What is getline() used for?

The problem with fgets() is that you have to specify an upper line length,
the string returned *might* an '\n' at the end, and if it hasn't, then it
will have left unused characters in the line which will appear the next time
you call fgets as extra phantom lines, probably screwing up your program.

I believe getline() routines get around some of these problems.

if you're not worried about potential gigabyte-length lines, but only those
of a reasonable length, then you just need to cleanly strip the '\n' (if it
will affect your code), safely dump any text that will exceed your buffer,
and possibly signal the fact that you have an overflow line which could mean
a data error.

So use an existing getline() or write another variation.

>> And in the code fragment you've given, I'm not sure how useful
>> comparing the result to NULL will be (assuming live input). When the
>> user enters an empty line, the buffer should contain "\n"; the
>> function will succeed.
>
> The NULL test is performed in order to check for EOF.

Ok, then your code isn't necessarily just for stdin, but for any file.

Rui Maciel wrote:
>
> What's the best way to read data from a binary file?

What's the best way to breathe? (Give one answer only,
please, and make sure it's appropriate for all conceivable
and inconceivable circumstances: Swimming the English Channel,
hiding behind a door to escape detection by the Daleks on
the other side, making your Metropolitan Opera debut, doing
Lamaze exercises, being pepper-sprayed, ...)

In other words -- and for at least the third time in
this thread, hint, hint -- There is no "best" in isolation,
but only in relation to some goal or set of goals. Describe
your purposes, describe the sort of data you're reading, and
maybe someone will have suggestions.

Rui Maciel wrote:
> James Kuyper wrote:
....
> What's the best way to read data from a binary file?

fread(), unless you have access to a software package that knows the
format of the binary file, in which case you're better off using that
package rather than wasting time writing your own code to do the same
thing. Most of the files my programs read are in HDF format, for
instance, so I use the HDF C library to read them (for this purpose, all
you need to understand about HDF is that it's a specialize file format).

>> , and not
>> necessarily a good way to go if you're reading text files that are not
>> line-oriented.
>
> How come?

Getting a line at a time is a very good idea when lines have a special
significance. For a file in which a newline character is just another
whitespace character, no more or less significant than a blank or a tab
character, then depending upon what you're doing with the files, it
might be more appropriate to use either getchar() or scanf().

On 26 July, 13:45, "Malcolm McLean" <regniz> wrote:
> "Rui Maciel" <ruimac> wrote


> > So, what's the best way to handle input from stdin?
>
> There's no decent way of writing a completely reusable getline() function.
> The reason is that malciious users can potentially pass lines long enough to
> exhaust memory, and there is no strategy for dealing with this that will be
> right for all programs.
> However as a general rule, fgets() with a big buffer(eg 4096 bytes) and some
> wrapper code to check for the newline is OK. Most people do not know how to
> use fgets() properly. The newline is a sentinel that tells you a full line
> has been read. If it is not present, you need to take action, not treat the
> half-read line as a full one, which could cause a nasty bug.

and, if used carefully like this, fgets() is often perfectly ok for
a wide variety of programs.

Rui Maciel wrote:
> I'm writing a program that supports input from stdin. To be able to do that I tend to rely on a simple loop
> that tests the return of fgets(), such as the following example:
>
> if (fgets (buffer, BUFFER_SIZE, stdin) != NULL)
> {
> // read the buffer
> }
>> I've been doing this for a while but I'm not sure that this is the best way to handle it. A quick google on
> the subject failed to return any meaningful result and searching this group's history through google groups
> ended up being a very disappointing experience (a search for nothing more than "stdin" returns only 14
> results that only go as far as June 8th). So, what's the best way to handle input from stdin?

For the special case,
where the buffer size is definitely big enough
to hold the longest valid line
and where it is OK to discard any characters beyond that length,
I do it this way:


/* BEGIN fscanf_input.c */
/*
** There are only three different values
** that can be assigned to the int variable rc,
** from the fscanf calls in this program;
** They are:
** EOF
** 0
** 1
** If rc equals EOF, then the end of file was reached,
** or there is some other input problem;
** ferror and feof can be used to distinguish which.
** If rc equals 0, then an empty line
** (a line consisting only of one newline character)
** was entered, and the array contains garbage values.
** If rc equals 1, then there is a string in the array.
** Up to LENGTH number of characters are read
** from a line of a text stream
** and written to a string in an array.
** The newline character in the text line is replaced
** by a null character in the array.
** If the line is longer than LENGTH,
** then the extra characters are discarded.
*/
#include <stdio.h>

#define LENGTH 40
#define str(x) # x
#define xstr(x) str(x)

int main(void)
{
int rc;
char array[LENGTH + 1];

puts("The LENGTH macro is " xstr(LENGTH) ".");
do {
fputs("Enter any line of text to continue,\n"
"or just hit the Enter key to quit:", stdout);
fflush(stdout);
rc = fscanf(stdin, "%" xstr(LENGTH) "[^\n]%*[^\n]", array);
if (!feof(stdin)) {
getc(stdin);
}
if (rc == 0) {
array[0] = '\0';
}
if (rc == EOF) {
puts("rc equals EOF");
} else {
printf("rc is %d. Your string is:%s\n\n", rc, array);
}
} while (rc == 1);
return 0;
}

/* END fscanf_input.c */

.... otherwise I use my own get_line function
like everybody else does.