Complete Common Log example
Install PocketCsvReader.WebLogs. This input is one Common Log Format request; the response status is converted only when requested.
using System.Text;
using PocketCsvReader.WebLogs;
var input = "127.0.0.1 - - [10/Oct/2000:13:55:36 -0700] \"GET /orders HTTP/1.0\" 200 2326\n";
using var stream = new MemoryStream(Encoding.UTF8.GetBytes(input));
using var reader = new CommonLogReader().ToDataReader(stream);
while (reader.Read())
Console.WriteLine($"{reader.GetInt32(reader.GetOrdinal("StatusCode"))}: {reader.GetString(reader.GetOrdinal("Request"))}");
200: GET /orders HTTP/1.0
Install the PocketCsvReader.WebLogs extension to parse Common Log Format (CLF), Combined Log Format, and W3C Extended Log Format files.
Common Log Format
CommonLogReader exposes the standard seven CLF fields as RemoteHost, Identity, AuthenticatedUser, Timestamp, Request, StatusCode, and ResponseBytes.
using PocketCsvReader.WebLogs;
using var reader = new CommonLogReader().ToDataReader("access.log");
while (reader.Read())
{
var host = reader.GetString(reader.GetOrdinal("RemoteHost"));
var request = reader.GetString(reader.GetOrdinal("Request"));
var status = reader.GetInt32(reader.GetOrdinal("StatusCode"));
}
The parser recognizes bracketed timestamps and quoted request lines. A hyphen (-) is returned as a null value.
Built-in typed records
Use ToCommonLog() to stream CommonLog records without defining a custom model:
foreach (var entry in new CommonLogReader().ToCommonLog("access.log"))
{
DateTimeOffset? timestamp = entry.Timestamp;
int? status = entry.StatusCode;
long? bytes = entry.ResponseBytes;
Console.WriteLine($"{timestamp}: {entry.Request} ({status}, {bytes} bytes)");
}
CommonLog exposes RemoteHost, Identity, AuthenticatedUser, and Request as nullable strings, Timestamp as DateTimeOffset?, StatusCode as int?, and ResponseBytes as long?. Missing values represented by - become null; a missing timestamp uses [-] and a missing request uses "-" to retain CLF delimiters.
Timestamps are parsed using the CLF pattern dd/MMM/yyyy:HH:mm:ss zzz and invariant culture, preserving the original UTC offset (for example, -0700). Numbers also use invariant culture. Invalid typed values throw FormatException; numbers outside their target type’s range throw OverflowException during enumeration.
Both filename and Stream overloads return IEnumerable<CommonLog>. Records are parsed as enumeration advances; the reader disposes its input when enumeration completes, fails, or the enumerator is disposed early. The extension-reader filename overload opens the file when called, so enumerate the result and dispose the enumerator (for example, with foreach). The existing To<T>() remains available for custom models and mappers.
Combined Log Format
Select CombinedLogReader for the nine-field Combined format: the seven CLF fields followed by quoted Referrer and UserAgent fields. CommonLogReader continues to require exactly seven fields.
using PocketCsvReader.WebLogs;
using var reader = new CombinedLogReader().ToDataReader("access.log");
while (reader.Read())
Console.WriteLine(reader.GetString(reader.GetOrdinal("UserAgent")));
foreach (CombinedLog entry in new CombinedLogReader().ToCombinedLog("access.log"))
Console.WriteLine($"{entry.Timestamp}: {entry.Referrer} ({entry.UserAgent})");
CombinedLog exposes the same nullable properties and invariant conversions as CommonLog, plus nullable string Referrer and UserAgent properties. ToCombinedLog(string filename) and ToCombinedLog(Stream stream) return IEnumerable<CombinedLog> with the same streaming and disposal behavior as ToCommonLog(), including disposal on completion, conversion failure, or early exit. Timestamp offsets are preserved, and invalid typed values throw FormatException or OverflowException during enumeration.
Requests, referrers, and user agents retain spaces and backslash-escaped quote syntax, as in the CLF reader. Use "-" for missing quoted fields. Typed CombinedLog output converts these to null; generic and tabular outputs preserve quoted hyphens, matching existing CLF behavior. Unquoted hyphens become null. Malformed records throw InvalidDataException with the format name and physical line number.
W3C Extended Log Format
W3cExtendedLogReader discovers column names from the required #Fields: directive and retains directives such as #Software, #Version, and #Date as metadata.
using PocketCsvReader.WebLogs;
using var reader = new W3cExtendedLogReader().ToDataReader("u_ex.log");
while (reader.Read())
{
var method = reader.GetString(reader.GetOrdinal("cs-method"));
var path = reader.GetString(reader.GetOrdinal("cs-uri-stem"));
}
foreach (var directive in reader.Directives)
Console.WriteLine($"{directive.Name}: {directive.Value}");
Data records must have the number of fields declared by #Fields:. Repeated identical field directives are accepted; changing the schema within one stream produces an InvalidDataException because an IDataReader has a stable result shape.
Fluent configuration
Use CommonLogReaderBuilder, CombinedLogReaderBuilder, or W3cExtendedLogReaderBuilder when you need the shared PocketCsvReader schema, resource, parser, or optimization settings. A custom WebLogProfile passed to CombinedLogReader must select WebLogFormat.Combined.
using PocketCsvReader.WebLogs.Configuration;
var reader = new W3cExtendedLogReaderBuilder()
.WithParserOptimizations(new ParserOptimizationOptions(BufferSize: 16 * 1024))
.Build();
All three readers support ToDataReader, ToDataTable, ToArrayString, and To<T>.
Streaming storage
CLF and W3C readers frame records over shared pooled character buffers and reuse field metadata for their fixed result schema. A record contained in one buffer is read as a view over that memory; a record spanning buffers is consolidated once into pooled storage. The current record’s character memory and metadata are valid until the next read or reader disposal. Strings and typed objects returned to callers own their materialized values and remain valid after the reader advances.
Synchronous and asynchronous reads use the same framing and field-parsing logic. W3C directive names and values remain retained strings, and string/object output still allocates the values required by the public API.
Boundaries and output ownership
Use the stream ownership table for disposal and lazy creation, output models for returned shapes, and validation for the distinction between syntax errors and conversion failures. Final complete records may lack a newline. Malformed format syntax is rejected; a CSV bad-data policy does not enable log-format recovery.