Your First ML.NET Model: MLContext, IDataView, Pipelines and Predicting Delivery Time

Version note: The code in this post was built and run with .NET 10, ML.NET 6.0 with the Microsoft.ML 6.0.0 package, and Microsoft.ML.FastTree 6.0.0. The data is synthetic Bookstore data produced by the series' seeded generator.

Guess First

I have two questions. First: when the LoadFromTextFile call in ML.NET returns, has the 24-thousand-line CSV file been read into memory? Second: what value does a number cell left empty in the file, such as the delivery time of a cancelled order, have when it enters ML.NET?

Knowing the answer to the first question matters for performance. Not knowing the answer to the second means silently training a wrong model. In this post we open up the pipeline we used as a box in the previous post and build the first ML.NET model step by step for Bookstore, the online bookstore used as the shared example across this site's series.

Project and Packages

All experiments of the series live in a .NET 10 console application called Bookstore.ML.Lab. The core of ML.NET is the Microsoft.ML package; tree-based trainers (FastTree, FastForest) are in a separate package. On the day I wrote this post the latest stable version on NuGet was 6.0.0, published on 11 November 2026:

dotnet add src/Bookstore.ML.Lab package Microsoft.ML --version 6.0.0
dotnet add src/Bookstore.ML.Lab package Microsoft.ML.FastTree --version 6.0.0

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Data.Analysis" Version="0.24.0" />
    <PackageReference Include="Microsoft.ML" Version="6.0.0" />
    <PackageReference Include="Microsoft.ML.FastTree" Version="6.0.0" />
  </ItemGroup>
</Project>

Every piece of ML.NET code starts with an MLContext. Loading data, transforms, trainers, evaluation and saving models are all called from the catalogs under this object: ml.Data, ml.Transforms, ml.Regression, ml.Model. The seed passed to the constructor makes every operation that shuffles or uses random starting values repeatable. But as we will see below, it is not enough on its own.

The Input Class and [LoadColumn]

We describe the shape of the data to ML.NET with a C# class. Each property is a column; the [LoadColumn] attribute says which column of the CSV to read it from. The output class, which receives the prediction, binds the Score column produced by the trainer to its own property:

/// <summary>The columns of data/orders_clean.csv that the delivery-time models use.</summary>
public class DeliveryInput
{
    [LoadColumn(0)] public string OrderId { get; set; } = "";
    [LoadColumn(1)] public string CustomerId { get; set; } = "";
    [LoadColumn(2)] public DateTime OrderedAtUtc { get; set; }
    [LoadColumn(3)] public string City { get; set; } = "";
    [LoadColumn(4)] public string District { get; set; } = "";
    [LoadColumn(5)] public string Warehouse { get; set; } = "";
    [LoadColumn(6)] public float DistanceKm { get; set; }
    [LoadColumn(7)] public string Carrier { get; set; } = "";
    [LoadColumn(8)] public string ShippingMethod { get; set; } = "";
    [LoadColumn(9)] public float ItemCount { get; set; }
    [LoadColumn(10)] public float TotalWeightKg { get; set; }
    [LoadColumn(11)] public float OrderTotal { get; set; }
    [LoadColumn(13)] public bool IsCampaign { get; set; }
    [LoadColumn(14)] public string PaymentMethod { get; set; } = "";
    [LoadColumn(15)] public bool HasBackorderItem { get; set; }
    [LoadColumn(16)] public string CustomerNote { get; set; } = "";
    [LoadColumn(17)] public bool IsCancelled { get; set; }
    [LoadColumn(18)] public float DeliveryDays { get; set; }
    [LoadColumn(19)] public float TrackingEventCount { get; set; }
}
public sealed class DeliveryPrediction
{
    [ColumnName("Score")]
    public float PredictedDays { get; set; }
}

Note three details. Number columns are float: ML.NET's number type is Single, and most trainers do not accept double. Column names come from the property names; we will refer to columns by these names in the pipeline. And LoadFromTextFile<T> expects a [LoadColumn] on every property of the class. In the previous post's experiments, when I added a property to this class that is not in the file, loading stopped with this error: Property 'CustomerAvgDays' is missing the LoadColumnAttribute attribute. That is why columns that are not in the file had to go into a separate class.

IDataView: Lazy and Schema-Aware

Let's load the data and look at the schema of the first ten columns:

MLContext ml = new(seed: 1);
// 1) Loading is lazy: nothing is read until someone iterates over the rows.
//    MissingRealsAsNaNs: an empty number becomes NaN instead of 0 (the default).
IDataView data = ml.Data.LoadFromTextFile<DeliveryInput>("data/orders_clean.csv", new TextLoader.Options
{
    HasHeader = true,
    Separators = [','],
    MissingRealsAsNaNs = true,
});
Console.WriteLine($"row count known before reading? {data.GetRowCount()?.ToString() ?? "no (null)"}");
foreach (DataViewSchema.Column column in data.Schema.Take(10))
{
    Console.WriteLine($"  {column.Index,2} {column.Name,-16} {column.Type}");
}

dotnet run --project src/Bookstore.ML.Lab -- first-model

row count known before reading? no (null)
   0 OrderId          String
   1 CustomerId       String
   2 OrderedAtUtc     DateTime
   3 City             String
   4 District         String
   5 Warehouse        String
   6 DistanceKm       Single
   7 Carrier          String
   8 ShippingMethod   String
   9 ItemCount        Single

The first line answers the first question from the opening: GetRowCount() returned null because the file has not been read yet. An IDataView is not a table; it is a recipe for reading a table. It has the schema, the column names and types, but rows are read only when someone starts iterating over them. Every step of a pipeline works the same way: it does not copy the data, it produces a view that adds new columns on top of the previous one. Thanks to this you can work with a file too big to fit in memory. The price is that an error in a transform may not show up until the data is actually read.

If you read the LINQ series, this behavior will feel familiar. Like deferred execution on IEnumerable, the work is done only when the result is requested.

An Empty Cell: 0 or NaN?

The delivery-time model must not use cancelled orders; their label is empty. FilterRowsByMissingValues drops the rows whose given column is missing (NaN):

// 2) Cancelled orders have an empty label (NaN): drop those rows
IDataView delivered = ml.Data.FilterRowsByMissingValues(data, "DeliveryDays");
Console.WriteLine($"rows with a label: {delivered.GetColumn<float>("DeliveryDays").Count():N0}");
DataOperationsCatalog.TrainTestData split = ml.Data.TrainTestSplit(
    delivered, testFraction: 0.2, samplingKeyColumnName: "CustomerId", seed: 7);

rows with a label: 23,105

23,105 rows remain, the number we expected. But this result depends on the MissingRealsAsNaNs = true line in the loading code. According to the ML.NET documentation the default value of this setting is false, and in that case an empty number field is read as 0. If you try the same filter with the default loader, this is what happens:

// 3) Empty cells: the default loader reads them as 0, not as missing
IDataView labels = ml.Data.FilterRowsByMissingValues(data, "DeliveryDays");
float[] days = [.. labels.GetColumn<float>("DeliveryDays")];
float[] weights = [.. labels.GetColumn<float>("TotalWeightKg")];
Console.WriteLine($"[empty]  default loader: {days.Length:N0} rows survive the filter, " +
                  $"{days.Count(d => d == 0):N0} have DeliveryDays = 0, {weights.Count(w => w == 0):N0} have weight 0");

dotnet run --project src/Bookstore.ML.Lab -- pitfalls

[empty]  default loader: 24,180 rows survive the filter, 1,075 have DeliveryDays = 0, 756 have weight 0

The filter dropped no rows. 1,075 cancelled orders entered training as "delivered in 0 days", and 756 orders with an unknown weight became "0 kilograms". That is the answer to the second question from the opening. I did not set this option in my first attempt either: the same pipeline gave a test MAE of 0.934 days and an R² of 0.167, and the test set contained fake zero-day deliveries as well. No exception, no warning; just a worse model and a wrongly measured metric. Read every CSV that may contain empty values with this setting, and after loading, check how many zeros a column contains.

Estimators, Transformers and the Pipeline

There are two core concepts in ML.NET. An estimator is the description of a step that has to learn something from data; it has not learned anything yet. When Fit is called, it reads the data and produces a transformer. A transformer is a learned transformation; it is applied to data with Transform. A pipeline is a chain that links estimators one after another with Append:

// 3) The pipeline: a chain of estimators, nothing is learned yet
var pipeline = ml.Transforms.Categorical.OneHotEncoding(
        [new("CarrierKey", "Carrier"), new("MethodKey", "ShippingMethod"), new("WarehouseKey", "Warehouse")])
    .Append(ml.Transforms.Conversion.ConvertType(
        [new("Campaign", "IsCampaign"), new("Backorder", "HasBackorderItem")], DataKind.Single))
    .Append(ml.Transforms.Concatenate("Features",
        "CarrierKey", "MethodKey", "WarehouseKey", "Campaign", "Backorder", "DistanceKm", "ItemCount"))
    .Append(ml.Transforms.NormalizeMinMax("Features"))
    .Append(ml.Regression.Trainers.Sdca(new SdcaRegressionTrainer.Options
    {
        LabelColumnName = "DeliveryDays",
        FeatureColumnName = "Features",
        NumberOfThreads = 1, // the same result on every run
    }));
Console.WriteLine($"pipeline: {pipeline.GetType().Name}");
// 4) Fit learns the one-hot vocabularies, the min/max values and the weights
Stopwatch sw = Stopwatch.StartNew();
ITransformer model = pipeline.Fit(split.TrainSet);
Console.WriteLine($"model:    {model.GetType().Name}, trained in {sw.ElapsedMilliseconds} ms");
IDataView scored = model.Transform(split.TestSet);
RegressionMetrics metrics = ml.Regression.Evaluate(scored, labelColumnName: "DeliveryDays");
Console.WriteLine($"test MAE {metrics.MeanAbsoluteError:F3} days, RMSE {metrics.RootMeanSquaredError:F3}, " +
                  $"R2 {metrics.RSquared:F3}");

pipeline: EstimatorChain`1
model:    TransformerChain`1, trained in 1973 ms
test MAE 0.873 days, RMSE 1.830, R2 0.229

The steps in the chain do the following in order. OneHotEncoding turns each categorical column into a vector with one slot per distinct value; it learns which values exist from the training data during Fit. ConvertType turns the boolean columns into 0 and 1. Concatenate collects all features into the single Features vector the trainer expects. NormalizeMinMax scales each slot into the 0–1 range; it also learns the min and max values from the training data. The last step is the trainer.

We chose SDCA, Stochastic Dual Coordinate Ascent, an optimization method that trains linear models quickly on large data. A linear model gives each feature a weight and predicts with their sum. We will look in detail at what it learns and why it works well or badly in the sixth post.

Look at the type names in the output: the pipeline is an EstimatorChain, and the model we get after Fit is a TransformerChain. What we call the model is not only the trainer's weights but the whole chain, including the one-hot vocabularies and the normalization values. This distinction matters; it will come up again in the next post when we discuss training-serving skew.

The NumberOfThreads = 1 line has a story too. Without it, running the same code three times gave three different results: 0.882, 0.883 and 0.884. Even though I gave MLContext a seed, when SDCA runs on several threads the order of the updates changes every time. With a single thread the result is the same on every run: 0.873. The difference is small, but when you compare two models you want to know whether a 0.001 difference comes from the model or from luck.

First Results

The model is off by 0.873 days on average on the test set. The output has two more metrics, RMSE and R². We will open them up in the sixth post, where we will also measure that R² stays as low as 0.229 mostly because of lost parcels that were found weeks later. For comparison: the rule from the first post was off by 1.05 days and the FastTree model from the previous post by 0.860 days on a random split. The test set in this post is a different split grouped by customer, so the numbers are not directly comparable. But the direction is clear: a simple linear model trained on seven columns beats the hand-written rule by a clear margin.

A Single Prediction with PredictionEngine

The evaluation scored the whole test set at once. In an application, though, we want one prediction for one order. That is what PredictionEngine is for:

// 5) One order at a time
PredictionEngine<DeliveryInput, DeliveryPrediction> engine =
    ml.Model.CreatePredictionEngine<DeliveryInput, DeliveryPrediction>(model);
DeliveryInput toVan = new()
{
    Carrier = "CarrierC", ShippingMethod = "Standard", Warehouse = "ANK-1",
    DistanceKm = 1220, ItemCount = 2,
};
DeliveryInput inIstanbul = new()
{
    Carrier = "CarrierA", ShippingMethod = "Express", Warehouse = "IST-1",
    DistanceKm = 25, ItemCount = 1,
};
Console.WriteLine($"Ankara -> Van, CarrierC, 2 books       : {engine.Predict(toVan).PredictedDays:F2} days");
Console.WriteLine($"Istanbul, Express, 1 book               : {engine.Predict(inIstanbul).PredictedDays:F2} days");
Console.WriteLine($"same, but one item is back-ordered      : " +
                  $"{engine.Predict(new DeliveryInput { Carrier = "CarrierA", ShippingMethod = "Express", Warehouse = "IST-1", DistanceKm = 25, ItemCount = 1, HasBackorderItem = true }).PredictedDays:F2} days");

Ankara -> Van, CarrierC, 2 books       : 3.83 days
Istanbul, Express, 1 book               : 1.10 days
same, but one item is back-ordered      : 6.25 days

3.83 days for a two-book order going from the Ankara warehouse to Van with CarrierC, 1.10 days for an Express order within Istanbul. If one book of the same Istanbul order is out of stock, the prediction rises to 6.25 days. The linear model learned this as "add about 5.15 days when an item is out of stock": we wrote 5 days in the rule in the first post, and the model found a similar number from the data.

When we created the examples we filled in only the properties the model uses. Other properties such as OrderId and City stayed empty strings and did not affect the model, because the pipeline never reads them.

Saving and Loading the Model

A trained model can be saved to a file and loaded in another process with another MLContext:

// 6) Save the whole chain together with the input schema
Directory.CreateDirectory("models");
ml.Model.Save(model, split.TrainSet.Schema, ModelPath);
Console.WriteLine($"saved {ModelPath} ({new FileInfo(ModelPath).Length / 1024.0:F1} KB)");
MLContext other = new();
ITransformer loaded = other.Model.Load(ModelPath, out DataViewSchema inputSchema);
PredictionEngine<DeliveryInput, DeliveryPrediction> engine2 =
    other.Model.CreatePredictionEngine<DeliveryInput, DeliveryPrediction>(loaded);
Console.WriteLine($"loaded model, Ankara -> Van              : {engine2.Predict(toVan).PredictedDays:F2} days " +
                  $"(input schema has {inputSchema.Count} columns)");

saved models/delivery-sdca.zip (6.4 KB)
loaded model, Ankara -> Van              : 3.83 days (input schema has 19 columns)

A 6.4 KB zip file. It contains the whole transformer chain and the input schema we passed to Save. The loaded model gives the same answer to the same order. The benefit of saving the schema with it is that the application using the model can learn from the file itself which columns, with which types, the model expects.

This file is not a data file; it is an executable chain of transformations. The Web API documentation on Microsoft Learn explicitly recommends loading models only from trusted sources. Loading a file that comes from a user as a model is like running code that comes from a user. We will see how to serve, version and safely replace this file in ASP.NET Core in the tenth post.

Common Pitfalls

Let's reproduce three errors you will meet in the first week on purpose. The first is a type mismatch: training the model with float and using a class with a double property for prediction.

/// <summary>The same input, but DistanceKm declared as double.</summary>
public sealed class DoubleInput
{
    public string Carrier { get; set; } = "";
    public string ShippingMethod { get; set; } = "";
    public string Warehouse { get; set; } = "";
    public double DistanceKm { get; set; }
    public float ItemCount { get; set; }
    public bool IsCampaign { get; set; }
    public bool HasBackorderItem { get; set; }
}
// 1) double instead of float
try
{
    var engine = ml.Model.CreatePredictionEngine<DoubleInput, DeliveryPrediction>(model);
    Console.WriteLine(engine.Predict(new DoubleInput { DistanceKm = 450 }).PredictedDays);
}
catch (Exception ex)
{
    Console.WriteLine($"[double] {ex.GetType().Name}: {ex.Message}");
}

[double] ArgumentOutOfRangeException: Schema mismatch for input column 'DistanceKm': expected Single, got Double (Parameter 'inputSchema')

The second is a typo in a column name. Writing DistanceKM instead of DistanceKm is not caught at compile time; column names are strings, and the error comes only when Fit starts reading the data:

// 2) A typo in a column name is only noticed when Fit runs
IDataView data = ml.Data.LoadFromTextFile<DeliveryInput>("data/orders_clean.csv", hasHeader: true, separatorChar: ',');
var typo = ml.Transforms.Concatenate("Features", "DistanceKM", "ItemCount")
    .Append(ml.Regression.Trainers.Sdca(labelColumnName: "DeliveryDays"));
try
{
    typo.Fit(data);
}
catch (Exception ex)
{
    Console.WriteLine($"[typo]   {ex.GetType().Name}: {ex.Message}");
}

[typo]   ArgumentOutOfRangeException: Could not find input column 'DistanceKM' (Parameter 'inputSchema')

Both messages use the same exception type and say clearly which column is the problem. Using nameof(DeliveryInput.DistanceKm) instead of typing column names as strings again and again moves most of these errors to compile time.

The third is the most dangerous because it does not always throw. PredictionEngine is not thread-safe. This is what happens when a single engine is used from several threads:

// 4) One PredictionEngine shared by many threads
PredictionEngine<DeliveryInput, DeliveryPrediction> shared =
    ml.Model.CreatePredictionEngine<DeliveryInput, DeliveryPrediction>(model);
DeliveryInput[] orders = [.. Enumerable.Range(0, 2000).Select(i => new DeliveryInput
{
    Carrier = "CarrierA", ShippingMethod = "Standard", Warehouse = "IST-1", DistanceKm = 20 + i % 1500, ItemCount = 1 + i % 5,
})];
float[] expected = [.. orders.Select(o => shared.Predict(o).PredictedDays)];
int wrong = 0, failed = 0;
string? firstError = null;
Parallel.For(0, orders.Length, i =>
{
    try
    {
        if (shared.Predict(orders[i]).PredictedDays != expected[i]) Interlocked.Increment(ref wrong);
    }
    catch (Exception ex)
    {
        Interlocked.Increment(ref failed);
        Interlocked.CompareExchange(ref firstError, ex.GetType().Name, null);
    }
});
Console.WriteLine($"[threads] 2000 parallel predictions on one engine: {wrong} wrong, {failed} exceptions (first: {firstError})");

[threads] 2000 parallel predictions on one engine: 383 wrong, 115 exceptions (first: ArgumentException)

Of 2,000 parallel predictions, 435 returned a wrong number and 124 threw an exception. The numbers change on every run; in my attempts the number of wrong results moved between 350 and 435. Wrong results are worse than exceptions: they show up in no log. The documentation on Microsoft Learn also says that PredictionEngine is not thread-safe and recommends PredictionEnginePool in ASP.NET Core. Remember the shared-state topic from the C# Async series: in a web application each request can run on a different thread, so registering the engine as a singleton carries this bug into production. We will solve it with the pool in the tenth post.

Add to these the trap from the middle of this post: without MissingRealsAsNaNs, empty number cells are read as 0, and nobody tells you.

A Small Experiment

★☆☆ Remove the NormalizeMinMax line from the pipeline and run the first-model command again. How much did the MAE change? Note the result; in the fifth post we will see when normalization is critical for SDCA.

★★☆ Write the column names in Concatenate with nameof and rename a property on purpose. Where is the error caught now?

★★★ Change the parallel test in Pitfalls so that each thread creates its own PredictionEngine (for example with ThreadLocal<T>). Do the wrong results and exceptions drop to zero? Measure the cost of creating an engine with Stopwatch.

In the next post we move on to feature engineering: which algorithms benefit from normalization, one-hot encoding versus hashing for categorical features such as city and district, building features from date and time with domain knowledge, text features from the customer's note, missing values, and why keeping all of this inside the pipeline is mandatory.