ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2503.18695
55
0

OCRT: Boosting Foundation Models in the Open World with Object-Concept-Relation Triad

24 March 2025
Luyao Tang
Yuxuan Yuan
C. L. P. Chen
Zeyu Zhang
Yue Huang
Kun Zhang
ArXivPDFHTML
Abstract

Although foundation models (FMs) claim to be powerful, their generalization ability significantly decreases when faced with distribution shifts, weak supervision, or malicious attacks in the open world. On the other hand, most domain generalization or adversarial fine-tuning methods are task-related or model-specific, ignoring the universality in practical applications and the transferability between FMs. This paper delves into the problem of generalizing FMs to the out-of-domain data. We propose a novel framework, the Object-Concept-Relation Triad (OCRT), that enables FMs to extract sparse, high-level concepts and intricate relational structures from raw visual inputs. The key idea is to bind objects in visual scenes and a set of object-centric representations through unsupervised decoupling and iterative refinement. To be specific, we project the object-centric representations onto a semantic concept space that the model can readily interpret and estimate their importance to filter out irrelevant elements. Then, a concept-based graph, which has a flexible degree, is constructed to incorporate the set of concepts and their corresponding importance, enabling the extraction of high-order factors from informative concepts and facilitating relational reasoning among these concepts. Extensive experiments demonstrate that OCRT can substantially boost the generalizability and robustness of SAM and CLIP across multiple downstream tasks.

View on arXiv
@article{tang2025_2503.18695,
  title={ OCRT: Boosting Foundation Models in the Open World with Object-Concept-Relation Triad },
  author={ Luyao Tang and Yuxuan Yuan and Chaoqi Chen and Zeyu Zhang and Yue Huang and Kun Zhang },
  journal={arXiv preprint arXiv:2503.18695},
  year={ 2025 }
}
Comments on this paper