Aimogen ProDocumentation
Product page Support

AI Model Training

Fine-tuning OpenAI models from Aimogen Pro, preparing datasets, and whether fine-tuning is the right answer.

Fine-tuning trains a model on your own examples, producing a custom model that follows your style more closely without being told every time.

Advanced Tools › AI Model Training

Creating A Custom Fine-Tuned Model With OpenAI AI modelsThe full fine-tuning workflow.Watch on YouTube

This tutorial was recorded before Aiomatic was renamed to Aimogen Pro. Some labels or interface elements may differ slightly from the current version.

Requirements#

An OpenAI API key. Fine-tuning is OpenAI-only in the plugin.

Trainable models in 2.8.8: gpt-4o-2024-08-06, gpt-4o-mini-2024-07-18 (the plugin default), gpt-4-0613, gpt-3.5-turbo-0125, gpt-3.5-turbo-1106.

Should you fine-tune?#

Often, no. Consider it only after the cheaper options.

Fine-tuning helps with

  • A consistent style or voice that prompting keeps drifting from
  • A consistent output format
  • A specialised task with many examples of correct output
  • Reducing prompt length, and therefore cost per request, at high volume

Fine-tuning does not help with

  • Teaching the model facts. That is what retrieval is for, and retrieval is cheaper, updatable and verifiable
  • General capability. A fine-tuned small model does not become a large one
  • Small datasets. A few dozen examples will not move it

The workflow#

TabPurpose
Dataset UploaderUpload a prepared JSONL dataset
Dataset Manual EntryBuild examples in the interface
Dataset ConverterConvert other formats to JSONL
DatasetsManage uploaded datasets
Model FinetunesStart, monitor and use fine-tuning jobs

There is also a CLI Data Preparation Tool reference for preparing datasets outside WordPress.

Datasets are stored as aiomatic_file and aiomatic_convert posts; fine-tunes as aiomatic_finetune posts.

Aimogen Pro > Advanced Tools > AI Model Training on the Model Finetunes tab

The dataset#

JSONL, one training example per line, each a conversation. Quality beats quantity, but you need enough: OpenAI suggests at least 50–100 well-chosen examples, and more for a meaningful effect.

What makes a good dataset:

  • Consistent. Every example should demonstrate the behaviour you want. Inconsistent examples teach inconsistency.
  • Representative. Cover the range of inputs you expect.
  • Correct. The model learns what you show it, including mistakes.
  • Genuinely yours. Fine-tuning on generic AI output teaches generic AI output.

Using a fine-tuned model#

Once training completes, the model appears in the model selectors like any other, identified by its fine-tune ID.

Cost#

Three separate costs:

  1. Training, charged per token in the dataset, once per job.
  2. Usage, at a higher per-token rate than the base model.
  3. Iteration, because the first fine-tune is rarely the last.

Set that against the saving from shorter prompts. At low volume, fine-tuning costs more than it saves.

Common problems#

The screen is unavailable. No OpenAI key.

Upload fails. PHP upload limits. The screen notes this.

Training fails immediately. Malformed JSONL. Use the Dataset Converter, and validate the file.

The fine-tuned model behaves no differently. Too few examples, or inconsistent ones.

The fine-tuned model is worse. Overfitted to a narrow dataset. Broaden the examples.

The model disappeared. Fine-tuned models are tied to the OpenAI account that created them.

Still stuck? Open a support ticket and include the diagnostics from Aimogen Pro › System & Logs › System Info.