OMN-1: configured context is not demonstrated context

Accepting a long sequence is different from using it reliably. We distinguish the positional range in code, coherent spans encountered during training and task performance at different lengths.

The reference configuration sets 32,768 maximum positions and RoPE theta 500,000; the historical data configuration uses 8,192-token sequences. Neither number alone establishes reliable retrieval or generation.

The internal record describes needle retrieval at six insertion depths: six successes at 4,096 tokens, none at 8,192 or 16,384, and one at 32,768. This is a small, prompt-specific diagnostic, not a comprehensive context benchmark.

A separate test of five approximately 8,191-token spans recorded one complete answer, three partial strings and one failure. The common span length reflected chunking; the spans should not be assumed to be independently verified natural documents.

These observations do not support reliable native 32K capability. Success in the small 4K suite also does not guarantee every task at 4K. Partial answers are clues, not proof that attention is correct and generation alone is responsible.

The pipeline inspection identified EOS-based attention boundaries for packed training. Such boundaries can correctly prevent unrelated documents from attending to one another. Removing them indiscriminately changes the training objective and can introduce irrelevant context.

The important question is whether training includes sufficiently long, coherent examples of the dependencies the model must use. Packed sequence length alone cannot answer it. Segmentation, EOS placement, positions and the attention implementation need inspection.

The failures motivate further investigation. These probes do not isolate an exclusive cause or prove continued pretraining will resolve it without other changes.

The analysis also records an evaluation-path change from manually materialized attention to PyTorch scaled-dot-product attention. Suitable efficient kernels can reduce memory requirements, but they do not add learned long-context competence.

Numerical parity, masking and backend selection require separate checks. Running without an out-of-memory error is an execution result, not a retrieval result.

Further evaluation should use held-out coherent documents, multiple tasks, depths and lengths, explicit answer scoring and repeatable decoding. Training changes must be compared with a baseline while preserving appropriate document isolation.

For the public OMN description, 32K remains a configuration rather than a verified capability. Context claims should describe evidence, not simply repeat a configuration limit.