I’m having to process an Amazon orders report (TSV flat file from the MWS Report API) and am needing to split the report based on sales channels (I could request them separately but need to keep API request counts as low as possible) whilst editing the contents of one of the columns (mapping a virtual SKU to its real counterpart). I’m using the CsvHelper library to handle the reading and writing, and would like to handle the file a line at a time to avoid loading the entire file into memory at once, which means as far as I know I can’t use the using block on the StreamWriter and CsvWriter instances, so I made a wrapper class that implements IDisposable and calls Dispose on both writers as part of its Dispose method. I store a Dictionary of these classes using sales channel as the key, then once the file has been completed I loop through the dictionary calling Dispose on each element.
Wrapper Class:
internal class AmazonCsvWriter : IDisposable { private bool _disposed; private readonly StreamWriter _streamWriter; private readonly CsvWriter _csv; public AmazonCsvWriter(string folder) { if (!Directory.Exists(folder)) { Directory.CreateDirectory(folder); } _streamWriter = new StreamWriter(Path.Combine(folder, $ "{DateTime.Now:yyyyMMddHHmmss}.txt"), false, Encoding.GetEncoding("Windows-1252")); _csv = new CsvWriter(_streamWriter); _csv.Configuration.Delimiter = "\t"; _csv.Configuration.RegisterClassMap(new AmazonOrderRowMap()); _csv.WriteHeader<AmazonOrderRow>(); _csv.NextRecord(); } public void WriteRecord(AmazonOrderRow record) { _csv.WriteRecord(record); _csv.NextRecord(); } public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { if (_disposed) { return; } if (disposing) { _csv.Dispose(); _streamWriter.Dispose(); } _disposed = true; } }
Used in this function:
public void ProcessFbaOrders(string reportFile) { try { using (var sr = new StreamReader(reportFile, Encoding.GetEncoding("Windows-1252"))) using (var csv = new CsvReader(sr)) { csv.Configuration.Delimiter = "\t"; csv.Configuration.RegisterClassMap(new AmazonOrderRowMap()); IEnumerable<AmazonOrderRow> records = csv.GetRecords<AmazonOrderRow>(); foreach (AmazonOrderRow record in records) { if (!_csvWriters.ContainsKey(record.SalesChannel)) { _csvWriters.Add(record.SalesChannel, new AmazonCsvWriter(Path.Combine(_platform.OutputFolder, record.SalesChannel))); } record.Sku = GetRealSku(record.Sku); _csvWriters[record.SalesChannel].WriteRecord(record); } } } finally { foreach (KeyValuePair<string, AmazonCsvWriter> kvp in _csvWriters) { kvp.Value.Dispose(); } } File.Delete(reportFile); }
_csvWriters is defined as:
private readonly Dictionary<string, AmazonCsvWriter> _csvWriters;
Is this safely closing the Stream/CsvWriters or is there a better method for handling this?