Skip to content

Add PPR relation token features to graph transformer - #700

Draft
mkolodner-sc wants to merge 4 commits into
mainfrom
mkolodner/gt-ppr-relation-features
Draft

Add PPR relation token features to graph transformer#700
mkolodner-sc wants to merge 4 commits into
mainfrom
mkolodner/gt-ppr-relation-features

Conversation

@mkolodner-sc

@mkolodner-sc mkolodner-sc commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds Graph Transformer support for generic PPR edge-attribute metadata.

PPR can emit multi-column edge_attr tensors where column 0 is the scalar PPR weight and the remaining columns contain sampler metadata. This PR lets Graph Transformer consume those extra columns as token-input features while preserving the existing scalar PPR path.

What This Enables

A PPR edge can now carry:

  • ppr_weight: the first edge_attr column, used as the scalar PPR score.
  • ppr_features: all remaining edge_attr columns, projected into the token embedding as continuous token features.

This allows PPR metadata such as hop distance, channel scores, channel min-hop values, or channel presence bits to flow into Graph Transformer tokens without changing how scalar PPR batches behave.

Changes

  • Adds reserved feature name ppr_features.
  • Supports scalar and multi-column PPR edge_attr.
  • Keeps ppr_weight mapped to edge_attr[..., 0].
  • Maps ppr_features to edge_attr[..., 1:].
  • Preserves raw metadata values in ppr_features, including hop values greater than 1.
  • Keeps ppr_weight clamped as a scalar score in [0, 1].
  • Sorts PPR tokens by descending edge_attr[..., 0] and carries feature columns through the same permutation.
  • Validates that ppr_features is used only as token input, not attention bias.
  • Updates token-input handling to support multi-column continuous features.
  • Adds transform and encoder tests for PPR feature extraction, ordering, and forward-pass support.

Toy Example

Suppose typed PPR emits PPR edges from an anchor to selected tokens.

Each PPR edge represents one selected (anchor, token) pair. The edge_attr row describes how PPR reached/scored that token:

  • Column 0 is the scalar PPR weight.
  • Column 1 is the global minimum hop from the anchor.
  • The next block contains per-channel PPR scores.
  • The next block contains per-channel minimum hops.
  • The final block contains per-channel presence bits.

For example, with two typed PPR channels:

data["user", "ppr", "item"].edge_attr = torch.tensor([
    # best_ppr, global_min_hop, ch0_score, ch1_score, ch0_min_hop, ch1_min_hop, ch0_present, ch1_present
    [0.90,     1.0,            0.90,      0.00,      1.0,         0.0,         1.0,         0.0],
    [0.75,     2.0,            0.10,      0.75,      2.0,         2.0,         1.0,         1.0],
    [0.40,     4.0,            0.00,      0.40,      0.0,         4.0,         0.0,         1.0],
])

Graph Transformer interprets this as:

ppr_weight = edge_attr[:, 0:1]
ppr_features = edge_attr[:, 1:]

A model can consume both as token-input features:

encoder = GraphTransformerEncoder(
    ...,
    sequence_construction_method="ppr",
    anchor_based_input_attr_names=[
        "ppr_weight",
        "ppr_features",
    ],
)

These continuous features are concatenated, passed through the learned token-input projection, and added to the corresponding token embeddings before the Transformer layers run.

Validation

  • Added focused unit coverage for multi-column PPR edge attrs and Graph Transformer token-input consumption.
  • Verified stale ppr_relation_features references were removed.
  • Verified ppr_features preserves raw metadata values rather than clamping hop-like columns.
  • Ran syntax and whitespace checks on the changed Graph Transformer files and tests.

@mkolodner-sc
mkolodner-sc force-pushed the mkolodner/gt-ppr-relation-features branch from 7fb45ce to 8ccc1a0 Compare July 28, 2026 10:43
PPR_WEIGHT_FEATURE_NAME,
PPR_RELATION_FEATURES_NAME,
}
if ppr_reserved_feature_names & set(pairwise_bias_attr_names):

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't ppr_reserved_feature_names always true since you set it above? Should it only be set when PPR mode?

)
if (
PPR_WEIGHT_FEATURE_NAME in anchor_bias_attr_names + anchor_input_attr_names
ppr_reserved_feature_names

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants