Controlling LLM Drift in Patent Feature Representation

Date: 2026-07-10
Stage: Representation

Objective

Evaluate whether terminology constraints introduced to reduce LLM drift actually improve the consistency of patent feature representation, and determine whether such constraints may unintentionally distort the embedding space used for technology clustering.

Approach

After decomposing each patent feature into technological aspects such as Problem, Solution, Component, and Effect, the textual representation of each aspect is converted into an embedding vector for subsequent similarity analysis and clustering.

However, LLM-generated descriptions are not completely deterministic in their terminology or abstraction level. Similar technical concepts may therefore be described differently across patents.

For example, conceptually similar technical problems may be expressed as:

• preventing propagation of thermal runaway,
• suppressing heat transfer between adjacent cells,
• preventing thermal propagation,
• reducing inter-cell thermal influence.

  Although these descriptions may refer to closely related technical concepts, variation in wording and abstraction could cause their embeddings to occupy unnecessarily different positions in the technology space.

  To reduce this variation, an additional set of tags was introduced. These tags were intended to provide a more standardized terminology layer and thereby reduce representation drift caused by free-form LLM descriptions.

  The resulting representation therefore contains two different types of information:

  Natural-language aspect description

LLM-generated description preserving the contextual meaning of the technical feature.

Normalized tag representation

Short terminology intended to provide consistent anchors across different patents.

Both representations can be embedded and compared independently or in combination.

Observations

Initial experiments indicate that terminology normalization does reduce some variation in LLM-generated expressions. Features describing similar concepts are more likely to share identical or closely related terminology.

However, another problem becomes visible when the tags themselves are used as inputs for embedding-based clustering.

A tag is necessarily a compressed representation of the original technical description. During this compression, contextual information may be lost.

For example, a Problem description may contain information regarding:

• the object experiencing the problem,
• the physical mechanism involved,
• the operating condition,
• the technical consequence,
• and the scope of the problem.

  A normalized tag may preserve only one of these elements.

  Consequently, two features may receive similar tags even though their actual technical meanings differ. Conversely, two technically similar features may receive different tags because the LLM selects different terminology as the representative label.

  This creates a potential conflict between two objectives:

Reducing linguistic drift

and

Preserving semantic structure for embedding comparison

The tags appear useful as terminology anchors, but they may not necessarily constitute sufficiently representative semantic units for defining positions in the technology space.

This problem becomes particularly visible during clustering. Some clusters formed from tag embeddings appear to be dominated by shared terminology rather than by the complete technical meaning represented by the original aspect descriptions.

Interpretation

The experiment suggests that LLM drift and embedding representation should be treated as two different problems.

LLM drift concerns the stability of how the same technical concept is described.

Embedding representation concerns whether the text being embedded contains sufficient semantic information to position the concept correctly relative to other concepts.

A representation that performs well for terminology normalization does not necessarily perform well for semantic clustering.

This distinction is important.

The original assumption was:

normalized terminology → more consistent embedding → better clustering

The current observations suggest that the relationship may instead be:

normalized terminology → lower linguistic variation
but potentially → lower semantic resolution

Therefore, tags may be more appropriate as a control or annotation layer rather than as the primary representation used to construct the technology space.

The natural-language PSCE aspect descriptions may remain the primary semantic representation, while tags serve secondary functions such as:

• terminology normalization,
• filtering,
• taxonomy alignment,
• cluster interpretation,
• retrieval constraints,
• and detection of abnormal LLM outputs.

  Under this architecture, the tag does not determine where a feature is located in the embedding space. Instead, it provides an additional symbolic description of what the feature is expected to represent.

  This also suggests that representation should not necessarily collapse into a single vector or a single terminology system.

  A technical feature may instead be represented as a collection of parallel representations:

  Feature

  → Problem semantic vector
  → Solution semantic vector
  → Component semantic vector
  → Effect semantic vector
  → normalized terminology / tags

  The semantic vectors preserve contextual relationships, whereas the tags provide controlled terminology.

Upcoming Issues

Several questions remain unresolved.

First, it is necessary to determine whether the observed clustering problems originate from the tags themselves or from the way the tags are generated.

A more structured tag-generation rule may improve representativeness.

Second, the appropriate role of tags within the representation architecture must be clarified. Possible roles include:

• embedding input,
• clustering constraint,
• metadata,
• taxonomy mapping,
• cluster naming,
• or validation of LLM-generated descriptions.

  Third, representation quality should be evaluated separately for different technological aspects. A short normalized term may be sufficient for a Component, while a Problem or Solution usually contains relational or contextual information that cannot be adequately represented by a simple tag.

  This implies that the optimal representation strategy may differ between P, S, C, and E.

Next Step

Conduct controlled clustering experiments using different representations of the same patent features:

1. aspect-description embeddings only,
2. tag embeddings only,
3. combined aspect-description and tag embeddings,
4. aspect-description embeddings with tags used only as metadata or clustering constraints.

   The resulting clusters should then be compared in terms of semantic coherence, separation, recurring technical structures, and sensitivity to terminology variation.

   The purpose is not simply to identify which representation produces the cleanest clusters, but to determine which information should define the geometry of the technology space and which information should remain as an interpretive or control layer.