Multimodal User Interest Profiling for Display Ad Ranking

Models evolving user interests from browsing sequences and long-term multimodal interaction histories to predict campaign-category click preferences and improve display ad ranking, especially for ultra-long histories and long-tail items with sparse ID-based signals.

Business Blueprint

GROUNDED

Multimodal user-interest profiling improves display ad ranking by turning long browsing and engagement histories into more relevant campaign-category click predictions.

The Problem

Display ad ranking teams struggle to predict user interest when histories are very long, item signals are sparse, and ID-based features do not generalize well to long-tail items or capture enough semantic meaning from content and interactions.

Display ad ranking and recommender teams

Existing ID-heavy approaches make it harder to generalize to long-tail items and express user interest semantically, limiting relevance improvements in the ad ranker.

Advertisers and campaign owners

Lower ad relevance can reduce ad performance and advertiser ROI because the recommendation system is less able to serve relevant ads.

Ad platform operators

They must improve top-line ad metrics without adding material online latency to the serving path.

Cost of Inaction

Continued weak generalization on long-tail items, limited use of semantic and multimodal signals, and missed gains in ad relevance, conversions, and advertiser ROI.

Process Fit

Campaign management & targeting

As-Is

Campaign targeting and display ad ranking rely heavily on historical IDs, recent activity, and sparse engagement outcomes. Long histories and content-rich interactions are difficult to use consistently, so relevance can degrade for long-tail items and sparse campaign categories.

To-Be

The ad ranking workflow adds a multimodal interest profile that summarizes long-term browsing and interaction history, combines content signals with ID signals, and feeds the ranker with richer user-interest evidence before an ad impression is served.

Systems Touched

Display ad ranking systemAds recommendation modelsCampaign targeting and category systemsUser behavior and engagement data pipelinesAd content and creative feature storesExperimentation and measurement systems

Business Cycle

Upstream

  • Large-scale user behavior histories and high-quality multimodal embeddings must be available for profiling.
  • Ad content and user engagement data from ads and organic interactions feed the recommendation model.
  • The platform needs serving infrastructure that can use long sequence signals without material online latency overhead.

Downstream

  • Display advertising systems can model much longer user behavior histories for ranking decisions.
  • Ad recommendation relevance and advertiser ROI can improve by enhancing downstream ads recommendation models.
  • Top-line advertising metrics can improve while preserving the speed of the online ranking path.

Value Evidence

  • User behavior history length modeled in display advertisingINCREASED

    enabling 100K-length user behavior sequence modeling

  • Top-line advertising metricsIMPROVED

    delivering significant gains in top-line metrics with negligible online latency overhead

  • Online latency overheadREDUCED

    delivering significant gains in top-line metrics with negligible online latency overhead

  • Ad conversionsINCREASED

    GEM's launch across Facebook and Instagram has delivered a 5% increase in ad conversions on Instagram and a 3% increase in ad conversions on Facebook Feed in Q2.

  • Efficiency at driving ad performance gains for a given amount of data and computeIMPROVED

    A scalable model architecture that is now 4x more efficient at driving ad performance gains for a given amount of data and compute than our original ads recommendation ranking models.

ROI Estimator

Estimate

KPI

Ad conversions

Projected Annual Change — Ad conversions

—

Based on observed result at 1 operator — verify against your own baseline.

Adoption Journey

  1. LEVEL 3 — ADVANCED

    Gate: Prove the approach scales across longer histories, more campaign categories, and long-tail inventory without degrading measurement quality.

    Outcome: The business can apply richer interest profiles broadly across display advertising inventory, including users and items where ID-only signals are weak.

  2. LEVEL 4 — ENTERPRISE

    Gate: Prove the profile can become a shared ads intelligence layer reused by multiple downstream ranking models and channels.

    Outcome: The ad platform treats user-interest profiling as a reusable capability that transfers learning across ads models rather than a one-off ranking feature.

Detailed per-level builds in the solution spectrum below

Risk & Governance

  • Ranking improvements can be offset if the profile adds latency to the online ad-serving path.

    Posture: Set explicit latency guardrails, run staged online experiments, and block rollout if relevance gains do not hold under serving-time constraints.

  • Sparse clicks and conversions can make campaign-level lift hard to measure reliably, especially for long-tail items.

    Posture: Use pre-agreed experiment designs, minimum sample thresholds, and separate monitoring for long-tail segments before declaring business lift.

  • A shared profiling or foundation layer can propagate errors across multiple ad models if not governed centrally.

    Posture: Maintain model ownership, versioning, rollback plans, and cross-model impact reviews when profile signals are reused beyond the first ranker.

Operating Intelligence

How it works

AI runs the operating engine in real time.

Humans govern policy and overrides.

Measured outcomes feed the optimization loop.

Confidence86%
ArchetypeOptimize & Orchestrate
Shape6-step circular
Human gates1
Autonomy
67%AI controls 4 of 6 steps

Who is in control at each step

Each column marks the operating owner for that step. AI-led actions sit above the divider, human decisions and feedback loops sit below it.

Loop shapecircular

Step 1

Sense

Step 2

Optimize

Step 3

Coordinate

Step 4

Govern

Step 5

Execute

Step 6

Measure

AI lead

Autonomous execution

1AI
2AI
3AI
5AI
gate

Human lead

Approval, override, feedback

4Human
6↺ Loop
AI-led step
Human-controlled step
Feedback loop
TL;DR

AI senses, optimizes, and coordinates in real time. Humans set policy and override when needed. Measurements close the loop.

The Loop

6 steps

1 operating angles mapped

Operational Depth

Technologies

Technologies commonly used in Multimodal User Interest Profiling for Display Ad Ranking implementations:

Bi-Directional LSTMOther
4 mentions
IAB tier 2 page categoriesOther
4 mentions
Updated Parrot OptimizerOther
4 mentions
Variable-length sequence bucketingOther
3 mentions
Cosine similarity retrievalOther
2 mentions

Key Players

Companies actively working on Multimodal User Interest Profiling for Display Ad Ranking solutions:

static user-profile targetingnon-sequential campaign targeting modelsset-based interest modeling

Real-World Use Cases

MUSE multimodal lifelong user interest modeling for display ad ranking

The ad system remembers a shopper’s very long history and uses both item IDs and multimodal item meaning, such as visual/text embeddings, to find the past behaviors most relevant to the ad being ranked.

Two-stage retrieval and fine-grained sequence modeling for user-interest matching.production-deployed in taobao display advertising, with an accompanying arxiv paper and released code/data.
10.0

Campaign category click preference prediction from browsing sequences

The system looks at a user’s recent sequence of visited webpage categories and predicts which type of ad campaign the user is most likely to click.

Sequential user profiling and multi-class classificationresearch-stage proposed workflow evaluated in simulation.
10.0

Free access to this report