tags:

views:

89

answers:

2

I have an application which prints strings to a buffer using snprintf and vsnprintf. Currently, if it detects an overflow, it appends a > to the end of the string as a sign that the string was chopped and prints a warning to stderr. I'm trying to find a way to have it resume the string [from where it left off] in another buffer.

If this was using strncpy, it would be easy; I know how many bytes were written, and so I can start the next print from *(p+bytes_written); However, with printf, I have two problems; first, the formatting specifiers may take up more or less space in the final string as in the format string, and secondly, my valist may be partially parsed.

Does anyone have an easy-ish solution to this?

EDIT: I should probably clarify that I'm working on an embedded system with limited memory + no dynamic allocation [i.e., I don't want to use dynamic allocation]. I can print messages of 255 bytes, but no more, although I can print as many of those as I want. I don't, however, have the memory to allocate lots of memory on the stack, and my print function needs to be thread-safe, so I can't allocate just one global / static array.

+2  A: 

The C99 functions snprintf() and vsnprintf() both return the number of characters needed to print the whole format string with all the arguments.

If your implementation conforms to C99, you can create an array large enough for your output strings then deal with them as needed.

int chars_needed = snprintf(NULL, 0, fmt_string, v1, v2, v3, ...);
char *buf = malloc(chars_needed + 1);
if (buf) {
  snprintf(buf, chars_needed + 1, fmt_string, v1, v2, v3, ...);
  /* use buf */
  free(buf);
} else {
  /* no memory */
}
pmg
one could also use `asprintf()` for automatic allocation if the function is available; if you're stuck with the MS CRT, use `_scprintf()` to determine the string's size and fill the buffer with `sprintf()`
Christoph
good idea, but I can't do this: see my edit for the question
Mikeage
+2  A: 

I don't think you can do what you're looking for (other than by the straightforward way of reallocating the buffer to the necessary size and performing the entire operation again).

The reasons you listed are a couple contributors to this, but the real killer is that the formatter might have been in the middle of formatting an argument when it ran out of space, and there's no reasonable way to restart that.

For example, say there's 3 bytes left in the buffer, and the formatter starts working on a "%d" conversion for the value -1234567. It ll put "-1\0" into the buffer then do whatever else it needs to do to return the size of buffer you really need.

In addition to you being able to determine which specifier the formatter was working on, you'd need to be able to figure out that instead of passing in -1234567 on the second round you need to pass in 234567. I defy you to come up with a reasonable way to do that.

Now if there's a real reason you don't want to restart the operation from the top, you probably could wrap the snprintf()/vsnprintf() call with something that breaks down the format string, sending only a single conversion specifier at a time and concatenating that result to the output buffer. You'd have to come up with some way for the wrapper to keep some state across retries so it knows which conversion spec to pick up from.

So maybe it's doable in a sense, but it sure seems like it would be an awful lot of work to avoid the much simpler 'full retry' scheme. I could see maybe (maybe) trying this on a system where you don't have the luxury of dynamically allocating a larger buffer (an embedded system, maybe). In that case, I'd probably argue that what's needed is a much simpler/restricted scope formatter that doesn't have all the flexibility of printf() formatters and can handle retrying (because their scope is more limited).

But, man, I would try very hard to talk some sense into whoever said it was a requirement.

Edit:


Actually, I take some of that back. If you're willing to use a customized version of snprintf() (let's call it snprintf_ex()) I could see this being a relatively simple operation:

int snprintf_ex( char* s, size_t n, size_t skipChars, const char* fmt, ...);

snprintf_ex() (and its companion functions such as vsnprintf()) will format the string into the provided buffer (as usual) but will skip outputting the first skipChars characters.

You could probably rig this up pretty easy using the source from your compiler's library (or using something like Holger Weiss' snprintf()) as a starting point. Using this might look something like:

int bufSize = sizeof(buf);
char* fmt = "some complex format string...";

int needed = snprintf_ex( buf, bufSize, 0, fmt, arg1, arg2, etc, etc2);

if (needed >= bufSize) {
    // dang truncation...

    // do whatever you want with the truncated bits (send to a logger or whatever)

    // format the rest of the string, skipping the bits we already got
    needed = snprintf_ex( buf, bufSize, bufSize - 1, fmt, arg1, arg2, etc, etc2);

    // now the buffer contains the part that was truncated before. Note that 
    //  you'd still need to deal with the possibility that this is truncated yet
    //  again - that's an exercise for the reader, and it's probably trickier to
    //  deal with properly than it might sound...
}

One drawback (that might or might not be acceptable) is that the formatter will do all the formatting work over again from the start - it'll just throw away the first skipChars characters that it comes up with. If I had to use something like this, I'd think that would almost certainly be an acceptable thing (it what happens when someone deals with truncation using the standard snprintf() family of functions).

Michael Burr
I don't really mind wrapping early, or backing up a little if it's in the middle of a format string. I suspect I may need to rewrite my vsnprintf function slightly, to play games with the va_list [pass it a copy, and have it return the number used, plus the offset of the format string]. It's not a real requirement; right now, the requirement lets me cut off the string, but it'd be a very nice nice-to-have.
Mikeage
your edit was what I'd started coding this morning... but I found one more issue. I need to do a va_arg() to ignore the arguments that were already processed, but I need the format string to know how to handle them. Not because I care, but because I can't find any guarantee that they will all take up the same space on the stack; I can't just pop off a 4 byte type and expect it to cover a char or a 64 bit pointer [not that my environment has those yet...]
Mikeage
I'm not sure I follow your comment about the "need to do a va_arg() to ignore the arguments". What I was thinking is that each call to the `vsnprintf_ex()` function would process the format string right from the start (and any associated va_args in the va_list) - it's just that the first `skipChars` characters produced as 'output' to the buffer would be discarded. There's no reason skip any of the `va_args` - they each get processed again, it's just what they produce may get dropped.
Michael Burr
I was thinking of returning a pointer to a point midway through the format string. It might also save a bit of time; no need to iterate over the full string [right now, each vanilla snprintf takes ~200-500us].dropping is probably simpler, though; I'll work on both implementations and compare.
Mikeage
That would probably save some processing time, at the expense of being more complex to code up. You could even make it easier to use than my 'skipChars' idea if instead of passing in the # of characters to skip, the caller passes in an opaque context structure for the private use of the implementation. That way the caller doesn't need to keep track of where things got truncated - it can be manged internally by the implementation.
Michael Burr