Дообучение LLM в Snowflake Cortex: практический тест через SQL

Level: intermediate · Reading time: 35 minutes · Updated: 2026-10-20

Fine-tuning a model on warehouse data without standing up GPUs, a training cluster or an export pipeline sounds like the obvious win. It is only a win if the tuned model beats a well-written prompt on data it has never seen. This guide gives you a SQL-only protocol for Snowflake Cortex: build a training set from a table, launch the job with SNOWFLAKE.CORTEX.FINETUNE, call the resulting model through COMPLETE, and compare it with the base model on a held-out set — with the decision rule written down before you look at the numbers.

The problem: "should we fine-tune?" is an empirical question

Teams usually arrive at fine-tuning with a reasonable intuition: "We have thousands of labelled rows in Snowflake, the general model keeps getting our categories slightly wrong, a model trained on our data should do better." Sometimes that is true. Sometimes a better prompt with a list of labels and two examples closes most of the gap at zero training cost. Sometimes the fine-tuned model is more accurate but emits output that breaks your parser on edge cases.

You cannot settle this by argument. You settle it by running both options on the same held-out set, with the same scoring code, and agreeing up front what difference would justify the extra moving part. Snowflake Cortex is convenient for this because data preparation, training, inference and scoring all stay in SQL inside one account — there is no export step where test rows can quietly leak into training.

The concrete case: support ticket routing

We use one task throughout, because a vague task makes any comparison meaningless.

• Input: a support ticket's subject and body (free text).
• Output: exactly one label from a fixed set, for example billing, login_access, bug_report, feature_request, data_export, other.
• Source: a table of historical tickets where a human agent already assigned the final category.
• Metric: exact-match accuracy on the held-out test split, plus the rate of invalid outputs (anything that is not one of the six labels).

This is a good first fine-tuning target: the output is short and checkable, the labels encode company-specific conventions a general model cannot know (what your team calls "data export" vs "bug report"), and the scoring is deterministic. Open-ended generation tasks — summaries, replies — are much harder to evaluate and should not be your first experiment.

The table names below (SUPPORT_TICKETS, schema AGENTLAB.EVAL) are examples. Replace them with your own; keep the column roles.

Prerequisites

1. A Snowflake account in a region where Cortex fine-tuning is available. Availability is region-dependent and has changed over time; confirm it in the documentation or by running SNOWFLAKE.CORTEX.FINETUNE('SHOW') (step 6) — an "unsupported" error answers the question quickly.
2. Privileges. The role that runs the job needs the SNOWFLAKE.CORTEX_USER database role, USAGE on the database and schema, CREATE MODEL on the schema where the tuned model will live, and SELECT on the source tables. Calling the tuned model later requires USAGE on that model object.
3. A warehouse for the data preparation and scoring queries. The fine-tuning job itself runs on Snowflake-managed compute; inference through COMPLETE is billed per token. Look up the current consumption table rather than estimating from this article.
4. Labelled data. A few hundred clean examples per task is a common starting point for classification; more matters less than consistency. If two agents would label the same ticket differently, the model will learn that noise too.

-- Run as an administrator once; role and object names are examples.
CREATE ROLE IF NOT EXISTS FT_EXPERIMENTER;
GRANT DATABASE ROLE SNOWFLAKE.CORTEX_USER TO ROLE FT_EXPERIMENTER;
GRANT USAGE ON DATABASE AGENTLAB TO ROLE FT_EXPERIMENTER;
GRANT USAGE ON SCHEMA AGENTLAB.EVAL TO ROLE FT_EXPERIMENTER;
GRANT CREATE MODEL ON SCHEMA AGENTLAB.EVAL TO ROLE FT_EXPERIMENTER;
GRANT CREATE TABLE, CREATE VIEW ON SCHEMA AGENTLAB.EVAL TO ROLE FT_EXPERIMENTER;
GRANT SELECT ON TABLE AGENTLAB.RAW.SUPPORT_TICKETS TO ROLE FT_EXPERIMENTER;
GRANT USAGE ON WAREHOUSE EVAL_WH TO ROLE FT_EXPERIMENTER;
GRANT ROLE FT_EXPERIMENTER TO USER <your_user>;

USE ROLE FT_EXPERIMENTER;
USE WAREHOUSE EVAL_WH;
USE SCHEMA AGENTLAB.EVAL;

Step 0. Write the decision rule before you train

The most common failure in this kind of test is not technical. It is looking at the results first and then deciding what counts as "better". Write the rule into a table or a ticket before step 6, for example:

We adopt the fine-tuned model if, on the test split, (1) its accuracy is at least 5 percentage points higher than the best prompted baseline, (2) its invalid-output rate is not higher than the baseline's, and (3) no single category drops by more than 10 points. Otherwise we keep the prompt.

The thresholds above are an example. Pick yours based on what an error costs in your routing process. The point is that they are fixed in advance.

Step 1. Inspect and clean the source data

Start by looking at the labels you actually have, not the ones you think you have.

SELECT category, COUNT(*) AS n
FROM AGENTLAB.RAW.SUPPORT_TICKETS
GROUP BY category
ORDER BY n DESC;

Typical things this query reveals: label spelling variants (Billing, billing , BILLING), retired categories, and a long tail of rare labels. Normalise them into a clean view and drop rows you cannot trust:

CREATE OR REPLACE VIEW TICKETS_CLEAN AS
SELECT
    ticket_id,
    TRIM(subject)                         AS subject,
    LEFT(TRIM(body), 4000)                AS body,      -- cap very long bodies
    LOWER(TRIM(category))                 AS label
FROM AGENTLAB.RAW.SUPPORT_TICKETS
WHERE body IS NOT NULL
  AND LENGTH(TRIM(body)) > 20
  AND LOWER(TRIM(category)) IN
      ('billing','login_access','bug_report','feature_request','data_export','other');

Personal data. Ticket bodies often contain names, emails, phone numbers and account IDs. Fine-tuning a model on raw text means the model can reproduce fragments of it. Decide with your data owner whether to mask before training. A minimal masking pass in SQL:

CREATE OR REPLACE VIEW TICKETS_MASKED AS
SELECT
    ticket_id,
    REGEXP_REPLACE(subject, '[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}', '<EMAIL>') AS subject,
    REGEXP_REPLACE(
        REGEXP_REPLACE(body, '[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}', '<EMAIL>'),
        '\\+?[0-9][0-9 ()-]{7,}[0-9]', '<PHONE>')                               AS body,
    label
FROM TICKETS_CLEAN;

Regex masking is incomplete by nature; treat it as a floor, not a guarantee. If your organisation already uses a classification or masking policy in Snowflake, use that instead.

Step 2. Make a deterministic three-way split

You need three disjoint sets:

• train — what the model learns from;
• validation — passed to the fine-tuning job so it can track loss on unseen rows;
• test — touched only once, for the final comparison. Neither the training job nor your prompt engineering may see it.

Use a hash of a stable key rather than RANDOM(), so the split is identical every time you rerun the query:

CREATE OR REPLACE TABLE TICKETS_SPLIT AS
SELECT
    t.*,
    CASE
        WHEN MOD(ABS(HASH(ticket_id)), 10) < 8 THEN 'train'
        WHEN MOD(ABS(HASH(ticket_id)), 10) = 8 THEN 'validation'
        ELSE 'test'
    END AS split
FROM TICKETS_MASKED t;

If your tickets have strong time effects (a new product launched, a pricing change), consider a time-based split instead: train on older tickets, test on the most recent month. That is a harder and more honest test of whether the model will hold up next month.

Leakage checks

Duplicate tickets are common: the same customer re-sends a message, or an automated alert creates near-identical rows. If one copy lands in train and another in test, the fine-tuned model gets an unfair advantage.

-- Exact duplicates of body text that cross split boundaries
SELECT body_hash, ARRAY_AGG(DISTINCT split) AS splits, COUNT(*) AS n
FROM (SELECT split, HASH(LOWER(REGEXP_REPLACE(body, '\\s+', ' '))) AS body_hash
      FROM TICKETS_SPLIT)
GROUP BY body_hash
HAVING COUNT(DISTINCT split) > 1
ORDER BY n DESC
LIMIT 50;

If this returns rows, either deduplicate before splitting or assign the split by the body hash instead of the ticket ID, so all copies land in the same set. Near-duplicates (same template, different order number) will not be caught by an exact hash; spot-check them manually.

Distribution check

SELECT split, label, COUNT(*) AS n,
       ROUND(100 * RATIO_TO_REPORT(COUNT(*)) OVER (PARTITION BY split), 1) AS pct
FROM TICKETS_SPLIT
GROUP BY split, label
ORDER BY label, split;

The label proportions should be similar across the three splits. If a category has only a handful of rows in test, its per-class accuracy will be too noisy to interpret — note that in your report rather than drawing conclusions from it.

Step 3. Fix one prompt template for training and inference

Cortex fine-tuning learns from pairs of columns named prompt and completion (you can alias any columns to those names in the training query). Whatever text you put in prompt during training is what you must send at inference time. A mismatch here is the single easiest way to make a good fine-tuned model look bad.

Keep the template in one place — a view — and reuse it everywhere:

CREATE OR REPLACE VIEW FT_PAIRS AS
SELECT
    ticket_id,
    split,
    'Classify the support ticket into one category.\n'
    || 'Subject: ' || COALESCE(subject, '') || '\n'
    || 'Body: '    || body                  || '\n'
    || 'Category:'                                   AS prompt,
    label                                            AS completion
FROM TICKETS_SPLIT;

Notice that this training prompt does not list the allowed labels. That is deliberate: one of the questions we are testing is whether the model can learn the label vocabulary from examples, so the inference prompt can stay short. You could include the label list as well; just keep it identical between training and inference.

Check prompt lengths

Fine-tuning jobs have context limits that depend on the base model, and rows that are too long may be truncated or rejected. Cortex provides a token counting function; run it on a sample to see your distribution:

SELECT
    APPROX_PERCENTILE(n_tokens, 0.5)  AS p50,
    APPROX_PERCENTILE(n_tokens, 0.95) AS p95,
    MAX(n_tokens)                     AS max_tokens
FROM (
    SELECT SNOWFLAKE.CORTEX.COUNT_TOKENS('llama3.1-8b', prompt || ' ' || completion) AS n_tokens
    FROM FT_PAIRS
    SAMPLE (500 ROWS)
);

If COUNT_TOKENS is not available for your chosen model or region, approximate with character length and stay well under the documented context limit. If p95 is close to the limit, shorten the body cap in step 1.

Step 4. Measure the baselines first

Run the baselines before training. If a prompted base model already meets your target, you can stop here and save the training cost.

We measure two baselines on the test split only:

• B0 — same prompt, base model. The exact training prompt, sent to the untuned base model. This isolates the effect of training.
• B1 — engineered prompt, base model. A prompt with the label list, one-line definitions and the instruction to answer with the label only. This is the fair competitor: it is what you would deploy if you did not fine-tune.

Use the array form of COMPLETE so you can set temperature to 0 and reduce run-to-run variation. Before running on the full test set, call it on one row and look at the raw response shape; the parsing below assumes the response is a JSON object with a choices array whose first element has a messages field.

SELECT SNOWFLAKE.CORTEX.COMPLETE(
    'llama3.1-8b',
    [{'role': 'user', 'content': 'Reply with the single word: ok'}],
    {'temperature': 0, 'max_tokens': 10}
) AS raw_response;

If the structure differs in your account (for example because a newer function such as AI_COMPLETE is the recommended entry point), adjust the extraction path in the queries below and keep it identical for all three variants.

B0

CREATE OR REPLACE TABLE RESULTS_B0 AS
SELECT
    ticket_id,
    completion AS gold,
    SNOWFLAKE.CORTEX.COMPLETE(
        'llama3.1-8b',
        [{'role': 'user', 'content': prompt}],
        {'temperature': 0, 'max_tokens': 10}
    ):choices[0]:messages::STRING AS raw_output
FROM FT_PAIRS
WHERE split = 'test';

B1

CREATE OR REPLACE TABLE RESULTS_B1 AS
SELECT
    ticket_id,
    label AS gold,
    SNOWFLAKE.CORTEX.COMPLETE(
        'llama3.1-8b',
        [{'role': 'system', 'content':
            'You route customer support tickets. Answer with exactly one label and nothing else.\n'
         || 'Labels:\n'
         || 'billing - invoices, charges, refunds, payment methods\n'
         || 'login_access - passwords, 2FA, locked accounts, SSO\n'
         || 'bug_report - something in the product behaves incorrectly\n'
         || 'feature_request - asks for new functionality\n'
         || 'data_export - exporting, downloading or migrating data\n'
         || 'other - anything else'},
         {'role': 'user', 'content':
            'Subject: ' || COALESCE(subject, '') || '\nBody: ' || body}],
        {'temperature': 0, 'max_tokens': 10}
    ):choices[0]:messages::STRING AS raw_output
FROM TICKETS_SPLIT
WHERE split = 'test';

Write the label definitions from your own team's routing guidelines. If you iterate on the B1 prompt, iterate on the validation split, not the test split — otherwise you are tuning the prompt to the test set and the comparison becomes unfair in the baseline's favour.

Step 5. Launch the fine-tuning job

The job is created with a single function call. Arguments, in order: the operation ('CREATE'), the fully qualified name for the new model, the base model, a query returning the training rows, and a query returning the validation rows. An optional object sets training options such as the number of epochs.

SELECT SNOWFLAKE.CORTEX.FINETUNE(
    'CREATE',
    'AGENTLAB.EVAL.TICKET_ROUTER_FT_V1',
    'llama3.1-8b',
    'SELECT prompt, completion FROM AGENTLAB.EVAL.FT_PAIRS WHERE split = ''train''',
    'SELECT prompt, completion FROM AGENTLAB.EVAL.FT_PAIRS WHERE split = ''validation''',
    {'max_epochs': 3}
) AS job_id;

Points that matter here:

• Use the same base model as the baselines. Otherwise you are comparing model families, not the effect of training.
• Fully qualify the table names inside the query strings. The job resolves them in its own context.
• Double the single quotes inside the query strings, as shown.
• The test split is not referenced anywhere in this call. Read the two query strings once more to confirm.
• Version the model name (_V1). You will almost certainly run a second job; overwriting names makes results tables ambiguous.
• The list of supported base models and allowed option values changes; check the documentation. If you pass an unsupported model, the call fails immediately rather than after hours.

Save the returned job ID somewhere durable — a results log table is better than a worksheet tab:

CREATE TABLE IF NOT EXISTS FT_RUN_LOG (
    job_id      STRING,
    model_name  STRING,
    base_model  STRING,
    train_rows  NUMBER,
    val_rows    NUMBER,
    options     VARIANT,
    created_at  TIMESTAMP_LTZ DEFAULT CURRENT_TIMESTAMP()
);

INSERT INTO FT_RUN_LOG (job_id, model_name, base_model, train_rows, val_rows, options)
SELECT '<job_id from above>', 'AGENTLAB.EVAL.TICKET_ROUTER_FT_V1', 'llama3.1-8b',
       (SELECT COUNT(*) FROM FT_PAIRS WHERE split = 'train'),
       (SELECT COUNT(*) FROM FT_PAIRS WHERE split = 'validation'),
       PARSE_JSON('{"max_epochs": 3}');

Step 6. Monitor the job

-- One job
SELECT SNOWFLAKE.CORTEX.FINETUNE('DESCRIBE', '<job_id>');

-- All jobs visible to your role
SELECT SNOWFLAKE.CORTEX.FINETUNE('SHOW');

-- Stop a job you launched by mistake
SELECT SNOWFLAKE.CORTEX.FINETUNE('CANCEL', '<job_id>');

DESCRIBE returns a JSON document with the job status (values such as pending, in progress, success, error, cancelled), progress information and, on failure, an error message. Duration depends on data size, base model and queue; do not plan around a specific number. When the status reaches success, the model appears as a schema object:

SHOW MODELS IN SCHEMA AGENTLAB.EVAL;

If the job reports training and validation loss, record them in your log. A validation loss that stops falling (or rises) while training loss keeps falling is the classic sign of overfitting; with a small dataset, fewer epochs is often the better next experiment.

Step 7. Call the fine-tuned model through COMPLETE

A tuned model is called exactly like a base model; you pass its fully qualified name instead of a model identifier. Send the same prompt column that was used for training.

-- Smoke test on a single validation row first
SELECT prompt, completion,
       SNOWFLAKE.CORTEX.COMPLETE(
           'AGENTLAB.EVAL.TICKET_ROUTER_FT_V1',
           [{'role': 'user', 'content': prompt}],
           {'temperature': 0, 'max_tokens': 10}
       ):choices[0]:messages::STRING AS raw_output
FROM FT_PAIRS
WHERE split = 'validation'
LIMIT 1;

If the smoke test returns a sensible label, run the test split:

CREATE OR REPLACE TABLE RESULTS_FT AS
SELECT
    ticket_id,
    completion AS gold,
    SNOWFLAKE.CORTEX.COMPLETE(
        'AGENTLAB.EVAL.TICKET_ROUTER_FT_V1',
        [{'role': 'user', 'content': prompt}],
        {'temperature': 0, 'max_tokens': 10}
    ):choices[0]:messages::STRING AS raw_output
FROM FT_PAIRS
WHERE split = 'test';

Other roles can only call the model if they have USAGE on it. If a colleague gets "model does not exist", check grants before suspecting the job.

Step 8. Score all three variants with the same code

Put the three result tables into one long table and normalise outputs with a single rule. Any output that is not one of the six labels after normalisation counts as INVALID — and as wrong.

CREATE OR REPLACE VIEW RESULTS_ALL AS
WITH unioned AS (
    SELECT 'B0' AS variant, ticket_id, gold, raw_output FROM RESULTS_B0
    UNION ALL
    SELECT 'B1', ticket_id, gold, raw_output FROM RESULTS_B1
    UNION ALL
    SELECT 'FT', ticket_id, gold, raw_output FROM RESULTS_FT
),
normalised AS (
    SELECT *,
           LOWER(TRIM(REGEXP_REPLACE(raw_output, '[^A-Za-z_]', ' '))) AS cleaned
    FROM unioned
)
SELECT
    variant, ticket_id, gold, raw_output,
    CASE WHEN SPLIT_PART(cleaned, ' ', 1) IN
              ('billing','login_access','bug_report','feature_request','data_export','other')
         THEN SPLIT_PART(cleaned, ' ', 1)
         ELSE 'INVALID'
    END AS pred
FROM normalised;

This takes the first word of the cleaned output. That is lenient toward a model that answers "billing." or "Billing - the user asks about a refund", and strict toward one that answers "The category is billing". Choose your rule, document it, and apply it identically to all variants. If you change it later, rescore everything.

Overall accuracy and invalid rate

SELECT
    variant,
    COUNT(*)                                             AS n,
    ROUND(100 * AVG(IFF(pred = gold, 1, 0)), 1)          AS accuracy_pct,
    ROUND(100 * AVG(IFF(pred = 'INVALID', 1, 0)), 1)     AS invalid_pct
FROM RESULTS_ALL
GROUP BY variant
ORDER BY variant;

All three rows must have the same n. If they don't, some calls failed or returned NULL; find them before interpreting anything:

SELECT variant, COUNT(*) AS null_outputs
FROM RESULTS_ALL
WHERE raw_output IS NULL
GROUP BY variant;

Per-category accuracy

SELECT
    gold AS category,
    variant,
    COUNT(*) AS n,
    ROUND(100 * AVG(IFF(pred = gold, 1, 0)), 1) AS accuracy_pct
FROM RESULTS_ALL
GROUP BY gold, variant
ORDER BY category, variant;

This is where fine-tuning often shows its real character: large gains on categories that depend on internal conventions, flat or worse results on rare categories with few training examples.

Confusion matrix for the tuned model

SELECT gold, pred, COUNT(*) AS n
FROM RESULTS_ALL
WHERE variant = 'FT'
GROUP BY gold, pred
ORDER BY gold, n DESC;

Paired comparison: FT vs B1

Both variants were scored on the same tickets, so compare them pairwise. What matters is the number of tickets where one is right and the other is wrong; tickets where both are right or both wrong say nothing about the difference.

WITH paired AS (
    SELECT
        f.ticket_id,
        IFF(f.pred = f.gold, 1, 0) AS ft_ok,
        IFF(b.pred = b.gold, 1, 0) AS b1_ok
    FROM RESULTS_ALL f
    JOIN RESULTS_ALL b
      ON f.ticket_id = b.ticket_id AND f.variant = 'FT' AND b.variant = 'B1'
)
SELECT
    SUM(IFF(ft_ok = 1 AND b1_ok = 0, 1, 0)) AS ft_only_right,
    SUM(IFF(ft_ok = 0 AND b1_ok = 1, 1, 0)) AS b1_only_right,
    SUM(IFF(ft_ok = 1 AND b1_ok = 1, 1, 0)) AS both_right,
    SUM(IFF(ft_ok = 0 AND b1_ok = 0, 1, 0)) AS both_wrong
FROM paired;

If ft_only_right and b1_only_right are both small (say, a dozen each), the accuracy difference is within noise no matter how it looks in percentages. You can run a McNemar test on these two counts if you want a formal p-value; for a go/no-go decision, the raw counts plus a read of the disagreeing tickets are usually more informative.

Read the disagreements

SELECT f.ticket_id, f.gold, f.pred AS ft_pred, b.pred AS b1_pred,
       LEFT(t.body, 300) AS body_preview
FROM RESULTS_ALL f
JOIN RESULTS_ALL b ON f.ticket_id = b.ticket_id AND b.variant = 'B1'
JOIN TICKETS_SPLIT t ON t.ticket_id = f.ticket_id
WHERE f.variant = 'FT' AND f.pred <> b.pred
ORDER BY f.gold
LIMIT 100;

Read at least a few dozen. You will usually find a mix of genuine model errors, ambiguous tickets that no model can get "right", and gold labels that are simply wrong. If a large share of disagreements are label errors, your accuracy numbers have a ceiling below 100% and small differences between variants mean even less.

Step 9. Fill in the report

The table below is a template. Leave the cells empty until your queries fill them; do not copy example numbers from anywhere, including this page.

Alongside the table, record: the job ID, base model, epochs, train/validation/test row counts, the date of the run, the normalisation rule, and the paired counts from step 8. Then apply the decision rule from step 0 literally. "FT is better by 2 points but the rule said 5" means you keep the prompt — or you write a new rule for a new experiment, not a retroactive one.

How to read typical outcome patterns

• B0 low, B1 and FT close. The model mostly needed to know the label set. A prompt solves it; fine-tuning buys little.
• B0 low, B1 medium, FT clearly higher. Your categories encode conventions the label definitions don't capture. Fine-tuning is earning its place — check that the gain holds on a fresh time-based slice before deploying.
• FT higher on accuracy but invalid rate is not zero. Keep output validation in your pipeline regardless. A tuned model is not a schema guarantee.
• FT better overall, much worse on one category. Usually a category with few training rows. Add examples for it or merge it into a neighbour, then retrain.
• FT worse than B1. Check the template match (step 3) and leakage (step 2) before concluding that training failed. Then look at label noise.

Failure cases and how to diagnose them

The CREATE call fails immediately

Usually privileges (missing CORTEX_USER or CREATE MODEL), an unsupported base model or region, an unqualified table name inside the query string, or unescaped single quotes. Run the two query strings on their own first; they must return columns named exactly prompt and completion.

The job ends in an error state

Read the error from DESCRIBE. Common causes: rows exceeding the context limit, NULL prompts or completions, too few rows. Filter NULLs and long rows in the view, not in an ad-hoc query you will forget.

The tuned model returns long sentences instead of labels

The inference prompt differs from the training prompt, or max_tokens is not set. Compare the exact strings byte for byte; trailing newlines and the final Category: cue matter.

Accuracy on test is suspiciously close to 100%

Look for leakage first: duplicates across splits, a column in the prompt that encodes the label (an internal queue name, an auto-tag), or tickets created by templates that contain the category word.

Results change between runs

Set temperature to 0 in every variant. If small variation remains, run each variant twice and report both; a difference smaller than the run-to-run spread is not a difference.

Some COMPLETE calls return NULL or errors in a large batch

Rate or quota limits, or individual inputs that exceed the context window. Re-run only the failed ticket_ids and merge, so all variants end with the same n.

A colleague cannot call the model

Grant USAGE on the model object and on its database and schema to their role.

Limitations of this protocol

• One task, one base model. A result on ticket routing with an 8B model says nothing about summarisation or about a larger base model. Rerun the protocol per task.
• Exact-match scoring fits classification only. For generation tasks you need a different evaluation (human review, rubric scoring, an LLM-as-judge setup with its own validation) and much more care.
• Gold labels are not ground truth. They are past human decisions, with their own error rate. The best possible score is bounded by their quality.
• Static snapshot. The test split is drawn from the same period as training unless you use a time-based split. Production drift — new products, new complaint types — will erode any model; plan periodic re-evaluation on fresh labelled rows.
• Cost is not measured here. Training and per-token inference are billed according to Snowflake's current consumption rules. Compare the tuned model's per-call cost and latency with the base model's on your own account before deciding; a shorter prompt can partly offset a higher per-token rate, or not.
• Platform details move. Function names, response formats, supported models and options in Cortex have changed before and will again. Everything marked "check the documentation" above is a place where this article may already be out of date by the time you read it.

Reproduction checklist

1. Decision rule written and dated before training.
2. Clean, masked view of the source data; label set fixed.
3. Deterministic split; duplicate and distribution checks passed.
4. One prompt template view used for training and FT inference.
5. Token lengths checked against the base model's limit.
6. B0 and B1 run on test with temperature = 0; B1 tuned only on validation.
7. Fine-tuning job launched with the same base model; job ID logged.
8. Tuned model smoke-tested, then run on test.
9. All variants scored by the same view; equal n confirmed.
10. Per-category accuracy, confusion matrix and paired counts reviewed.
11. At least a few dozen disagreements read by a human.
12. Decision rule applied as written; report saved with all run metadata.

Where to go next

If the tuned model wins, the next work is operational: output validation in the calling pipeline, a scheduled re-evaluation on newly labelled tickets, and a rollback path to the prompted baseline. If it does not win, you have a well-measured prompt baseline — which is a result in itself and cheaper to maintain.

More step-by-step material is in the Agent Lab Journal guides, and the terms used here are defined in the glossary.