It's hard to know exactly how much network traffic is occurring because of some unpredictable Ethernet traffic and that sort of thing, but essentially the gist of any HTTP request is:
method request-uri version
* (header : value)
CRLF
body
Given that, you can calculate out how long the request string will be:
HttpWebRequest req = ...;
StringBuilder requestText = new StringBuilder();
requestText.AppendFormat("{0} {1} HTTP/{2}.{3}", req.Method, req.RequestUri, req.ProtocolVersion.Major, req.ProtocolVersion.Minor);
requestText.AppendLine();
foreach (var header in req.Headers)
{
requestText.AppendFormat("{0}: {1}", v, webReq.Headers[v]);
requestText.AppendLine();
}
requestText.AppendLine();
// somehow add on the contents of the request stream, or just add that length later. I won't put that in this code because of stream positioning and all that
return System.Text.Encoding.UTF8.GetByteCount(requestText.ToString());
Then it's pretty similar for the response side. The format for an HTTP response is:
version status-code status-description
* (header : value)
CRLF
body
So,
HttpWebResponse resp = ...;
StringBuilder responseText = new StringBuilder();
responseText .AppendFormat("HTTP/{0}.{1} {2} {3}", resp.ProtocolVersion.Major, resp.ProtocolVersion.Minor, (int)resp.StatusCode, resp.StatusDescription);
responseText .AppendLine();
foreach (var header in resp.Headers)
{
responseText .AppendFormat("{0}: {1}", v, resp.Headers[v]);
responseText .AppendLine();
}
responseText.AppendLine();
Here, there's a bit of a decision to be made. You need to get the length of the response body. There might be more options, but what I'm thinking now are that you could:
- Write a
Stream wrapper that grabs the number of bytes copied. That way you don't have to worry about chunked transfers, and it could be kind of fun to write.
- Use the
content-length header, although then you won't get it if there isn't that header, as is the case for transfer-encoding: chunked.
- Read the stream using whatever method you want, then add the length of that in.
- Copy the stream to a
MemoryStream, then pass that off as the new response stream whilst grabbing the length on the way.
That all said, you have the extra frustration of worrying about content compression. I see you use it in yours, in fact. Given simplicity, I'm going to assume GZip and go with the fourth option. You might want to expand on what I write here to make it a little more comprehensive.
// webReq.AutomaticDecompression = DecompressionMethods.None; is required for this, since we're handling that decompression ourselves.
using (var respStream = resp.GetResponseStream())
using (var memStream = new MemoryStream())
{
respStream.CopyTo(memStream);
using (var gzip = new System.IO.Compression.GZipStream(respStream, System.IO.Compression.CompressionMode.Decompress))
using (var reader = new StreamReader(gzip))
{
var content = reader.ReadToEnd();
// you may or may not actually care about this, depending on whether this is just light testing or if you'll actually have these metrics in production
}
return System.Text.Encoding.UTF8.GetByteCount(responseText.ToString()) + memStream.Length;
}
Now, a few caveats that I'm seeing as I look now. Others might notice more, and I'll add them in if people comment.
- As I mentioned way back at the start of this, there can be more network traffic from one request than this will tell you.
- Depending on the client and server, you could find that actual headers could be listed with a different number of spaces. I don't believe that's beyond the HTTP spec, and even if it is people will do it. So there's a chance, for random example, that you'll see one server set
content-type: text/html and another set content-type : text/html. There's at least (well, exactly, since we're using UTF-8) one byte difference. Again, little, but that's a possible discrepancy.
This is just an estimate. It's a good one, but it's just an estimate.
If you're looking for extra accuracy at the cost of some simplicity, you could also write yourself a proxy. The idea of an HTTP proxy is that it simply gets the verbatim request, so it's "super easy" to then get the length. The biggest issue, of course, is that you then have to write a fully functioning proxy that, in all likelihood, parses and re-requests your incoming requests, and parses and relays all their respective responses. Certainly doable, but it's non-trivial. Particularly if you have to worry about SSL.
At the most basic level, of course, you could write, in essence, an alternative to a program like WireShark. I wouldn't know how to do that, and I wouldn't suggest you bother unless you really need it. That still wouldn't give you a perfect idea, either. Just closer.
And now, that all said, and boy, that was quite a bit said, if you're doing this for the purposes of profiling, there's a good chance the tools built into Visual Studio will let you do it. Admittedly I'm far from knowledgeable on them--I've never even opened them--but I believe there are network-traffic profilers available. Of course, I doubt they work on every platform like this would. But that would certainly be an easier approach.
Also, if anyone happens to notice any typos in here, I did copy-paste a couple times and I think I got them all, but feel free to let me know or just fix them.
System.Nettracing not an option? – Platysma