arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Sihan Chen

115 papers · 2011–2026

2011: 12012: 12013: 02014: 02015: 02016: 12017: 22018: 22019: 22020: 52021: 82022: 72023: 192024: 272025: 142026: 26
2011peak 27 in 20242026

Frequent co-authors

Papers

showing 1–25 of 115 RSS

Health Communication Through News Media During the Early Stage of the COVID-19 Outbreak in China: Digital Topic Modeling Approach

Qian Liu, Zequan Zheng, Jiabin Zheng +9 more

General Social Sciences · Journal of Medical Internet Research · · cited 297× · DOI ↗ · more like this ·

BACKGROUND: In December 2019, a few coronavirus disease (COVID-19) cases were first reported in Wuhan, Hubei, China. Soon after, increasing numbers of cases were detected in other parts of China, eventually leading to a disease outbreak in China. As this dreadful disease spreads rapidly, the mass media has been active in community education on COVID-19 by delivering health information about this novel coronavirus, such as its pathogenesis, spread, prevention, and containment. OBJECTIVE: The aim of this study was to collect media reports on COVID-19 and investigate the patterns of media-directed health communications as well as the role of the media in this ongoing COVID-19 crisis in China. METHODS: We adopted the WiseSearch database to extract related news articles about the coronavirus from major press media between January 1, 2020, and February 20, 2020. We then sorted and analyzed…

VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset

Sihan Chen, Handong Li, Qunbo Wang +4 more

cs.CV · · cited 244× · arXiv ↗ · more like this ·

Vision and text have been fully explored in contemporary video-text foundational models, while other modalities such as audio and subtitles in videos have not received sufficient attention. In this paper, we resort to establish connections between multi-modality video tracks, including Vision, Audio, and Subtitle, and Text by exploring an automatically generated large-scale omni-modality video caption dataset called VAST-27M. Specifically, we first collect 27 million open-domain video clips and separately train a vision and an audio captioner to generate vision and audio captions. Then, we employ an off-the-shelf Large Language Model (LLM) to integrate the generated captions, together with subtitles and instructional prompts into omni-modality captions. Based on the proposed VAST-27M dataset, we train an omni-modality video-text foundational model named VAST, which can perceive and…

VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset

Jing Liu, Sihan Chen, Xingjian He +4 more

cs.LG · · cited 204× · arXiv ↗ · more like this ·

In this paper, we propose a Vision-Audio-Language Omni-peRception pretraining model (VALOR) for multi-modal understanding and generation. Different from widely-studied vision-language pretraining models, VALOR jointly models relationships of vision, audio and language in an end-to-end manner. It contains three separate encoders for single modality representations, and a decoder for multimodal conditional text generation. We design two pretext tasks to pretrain VALOR model, including Multimodal Grouping Alignment (MGA) and Multimodal Grouping Captioning (MGC). MGA projects vision, language and audio to the same common space, building vision-language, audio-language and audiovisual-language alignment simultaneously. MGC learns how to generate text tokens in conditions of vision, audio or their both. To promote vision-audio-language pretraining research, we construct a large-scale…

CPTR: Full Transformer Network for Image Captioning

Wei Liu, Sihan Chen, Longteng Guo +2 more

cs.CV · · cited 173× · arXiv ↗ · more like this ·

In this paper, we consider the image captioning task from a new sequence-to-sequence prediction perspective and propose CaPtion TransformeR (CPTR) which takes the sequentialized raw images as the input to Transformer. Compared to the "CNN+Transformer" design paradigm, our model can model global context at every encoder layer from the beginning and is totally convolution-free. Extensive experiments demonstrate the effectiveness of the proposed model and we surpass the conventional "CNN+Transformer" methods on the MSCOCO dataset. Besides, we provide detailed visualizations of the self-attention between patches in the encoder and the "words-to-patches" attention in the decoder thanks to the full Transformer architecture.

VL-Mamba: Exploring State Space Models for Multimodal Learning

Yanyuan Qiao, Zheng Yu, Longteng Guo +5 more

cs.CV · · cited 136× · arXiv ↗ · more like this ·

Multimodal large language models (MLLMs) have attracted widespread interest and have rich applications. However, the inherent attention mechanism in its Transformer structure requires quadratic complexity and results in expensive computational overhead. Therefore, in this work, we propose VL-Mamba, a multimodal large language model based on state space models, which have been shown to have great potential for long-sequence modeling with fast inference and linear scaling in sequence length. Specifically, we first replace the transformer-based backbone language model such as LLama or Vicuna with the pre-trained Mamba language model. Then, we empirically explore how to effectively apply the 2D vision selective scan mechanism for multimodal learning and the combinations of different vision encoders and variants of pretrained Mamba language models. The extensive experiments on diverse…

ChatBridge: Bridging Modalities with Large Language Model as a Language Catalyst

Zijia Zhao, Longteng Guo, Tongtian Yue +5 more

cs.CV · · cited 84× · arXiv ↗ · more like this ·

Building general-purpose models that can perceive diverse real-world modalities and solve various tasks is an appealing target in artificial intelligence. In this paper, we present ChatBridge, a novel multimodal language model that leverages the expressive capabilities of language as the catalyst to bridge the gap between various modalities. We show that only language-paired two-modality data is sufficient to connect all modalities. ChatBridge leverages recent large language models (LLM) and extends their zero-shot capabilities to incorporate diverse multimodal inputs. ChatBridge undergoes a two-stage training. The first stage aligns each modality with language, which brings emergent multimodal correlation and collaboration abilities. The second stage instruction-finetunes ChatBridge to align it with user intent with our newly proposed multimodal instruction tuning dataset, named…

APTNER: A Specific Dataset for NER Missions in Cyber Threat Intelligence Field

Xuren Wang, Songheng He, Zihan Xiong +4 more

Artificial Intelligence · 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD) · · cited 36× · DOI ↗ · more like this ·

This paper provides a new dataset for Named Entity Recognition (NER) missions in cyber threat intelligence (CTI) studying. To the best of our knowledge, the proposed dataset is the biggest and challenging one in the field to comply with the STIX 2.1 specification. We collected the APT (Advanced Persistent Threats) reports from different network security companies and manually annotated them. Then we constructed a dataset named APTNER, which can be used for NER joint and multi-task learning tasks in CTI. Apart from common labels like IP, URL, mal-ware, location and so on, APTNER contains 21 categories, which make APTNER more challenging than other NER datasets in CTI field and we have proved the rationality of the dataset. For ease of comparison studies, we realize several state-of-the-art baselines and report their analysis. To facilitate future work on fine-grained NER for CTI, we make…

VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending

Xingjian He, Sihan Chen, Fan Ma +7 more

cs.CV · · cited 32× · arXiv ↗ · more like this ·

Large-scale image-text contrastive pre-training models, such as CLIP, have been demonstrated to effectively learn high-quality multimodal representations. However, there is limited research on learning video-text representations for general video multimodal tasks based on these powerful features. Towards this goal, we propose a novel video-text pre-training method dubbed VLAB: Video Language pre-training by feature Adapting and Blending, which transfers CLIP representations to video pre-training tasks and develops unified video multimodal models for a wide range of video-text tasks. Specifically, VLAB is founded on two key strategies: feature adapting and feature blending. In the former, we introduce a new video adapter module to address CLIP's deficiency in modeling temporal information and extend the model's capability to encompass both contrastive and generative tasks. In the latter…

Evaluation of Sustainable Development of Tourism Cities Based on SDGs and Tourism Competitiveness Index: Analysis of 221 Prefecture-Level Cities in China

Junli Gao, Chaofeng Shao, Sihan Chen +1 more

Sociology and Political Science · Sustainability · · cited 29× · DOI ↗ · more like this ·

Based on the Sustainable Development Goals and competitiveness index, an evaluation index system for sustainable development of tourism cities was established. The sustainable development level of 221 outstanding tourism cities in 2018 was evaluated, and their sustainable development paths were designed accordingly. The results show the following: (1) There is a large gap in sustainable development scores. In general, no city has achieved a strong sustainable development model. Natural and cultural resources and protection systems are the shortcomings of the systems. (2) The weights of natural and cultural resources and protection systems are the largest, and the weights of natural and cultural resources endowment, degree of tourism infrastructure construction, and economic support for natural and cultural resources are larger. Nature reserve coverage index, network popularity, and…

Sounding Video Generator: A Unified Framework for Text-guided Sounding Video Generation

Jiawei Liu, Weining Wang, Sihan Chen +2 more

cs.CV · · cited 27× · arXiv ↗ · more like this ·

As a combination of visual and audio signals, video is inherently multi-modal. However, existing video generation methods are primarily intended for the synthesis of visual frames, whereas audio signals in realistic videos are disregarded. In this work, we concentrate on a rarely investigated problem of text guided sounding video generation and propose the Sounding Video Generator (SVG), a unified framework for generating realistic videos along with audio signals. Specifically, we present the SVG-VQGAN to transform visual frames and audio melspectrograms into discrete tokens. SVG-VQGAN applies a novel hybrid contrastive learning method to model inter-modal and intra-modal consistency and improve the quantized representations. A cross-modal attention module is employed to extract associated features of visual frames and audio signals for contrastive learning. Then, a Transformer-based…

Speech adapts to differences in dentition within and across populations

Caleb Everett, Sihan Chen

Linguistics and Language · Scientific Reports · · cited 24× · DOI ↗ · more like this ·

We test the hypothesis that a specific anatomical feature, the dental malocclusion associated with reduced dental wear, causes languages to adapt by relying more heavily on labiodental consonants. In contrast to previous work on this topic, we adopt a usage-based approach that directly examines the relative frequency of such labiodental sounds within phonetically transcribed word lists and texts from thousands of languages. Labiodentals are shown to be very infrequent in the languages of hunter gatherers, who tend to have edge-to-edge bites as opposed to the overbite and overjet observed in populations that consume softer diets and rely heavily on eating utensils. This strong tendency is evident after controlling for Galton's problem via multiple methods including frequentist and Bayesian linear mixed modeling. Additionally, we discuss data from Amazonian hunter gatherers with…

Health Communication Through News Media During the Early Stage of the COVID-19 Outbreak in China: A Digital Topic Modeling Approach

Qian Liu, Zequan Zheng, Jiabin Zheng +9 more

General Social Sciences · medRxiv · · cited 23× · DOI ↗ · more like this ·

Abstract Background In December 2019, some COVID-19 cases were first reported and soon the disease broke out. As this dreadful disease spreads rapidly, the mass media has been active in community education on COVID-19 by delivering health information about this novel coronavirus. Methods We adopted the Huike database to extract news articles about coronavirus from major press media, between January 1 st , 2020, to February 20 th , 2020. The data were sorted and analyzed by Python software and Python package Jieba. We sought a suitable topic number using the coherence number. We operated Latent Dirichlet Allocation (LDA) topic modeling with the suitable topic number and generated corresponding keywords and topic names. We divided these topics into different themes by plotting them into two-dimensional plane via multidimensional scaling. Findings After removing duplicates, 7791 relevant…

Linguistic correlates of societal variation: A quantitative analysis

Sihan Chen, David Gil, С. П. Гапонов +5 more

Cultural Studies · PLoS ONE · · cited 18× · DOI ↗ · more like this ·

Traditionally, many researchers have supported a uniformitarian view whereby all languages are of roughly equal complexity, facilitated by internal trade-offs between complexity at different levels, such as morphology and syntax. The extent to which the speakers' societies influence the trade-offs has not been well studied. In this paper, we focus on morphology and syntax, and report significant correlations between specific linguistic and societal features, in particular those relating to exoteric (open) vs. esoteric (close-knit) society types, characterizable in terms of population size, mobility, communication across distances, etc. We conduct an exhaustive quantitative analysis drawing upon WALS, D-Place, Ethnologue and Glottolog, finding some support for our hypothesis that languages spoken by exoteric societies tend towards more complex syntaxes, while languages spoken by esoteric…

Global-Local Propagation Network for RGB-D Semantic Segmentation

Sihan Chen, Xinxin Zhu, Wei Liu +2 more

cs.CV · · cited 16× · arXiv ↗ · more like this ·

Depth information matters in RGB-D semantic segmentation task for providing additional geometric information to color images. Most existing methods exploit a multi-stage fusion strategy to propagate depth feature to the RGB branch. However, at the very deep stage, the propagation in a simple element-wise addition manner can not fully utilize the depth information. We propose Global-Local propagation network (GLPNet) to solve this problem. Specifically, a local context fusion module(L-CFM) is introduced to dynamically align both modalities before element-wise fusion, and a global context fusion module(G-CFM) is introduced to propagate the depth information to the RGB branch by jointly modeling the multi-modal global context features. Extensive experiments demonstrate the effectiveness and complementarity of the proposed fusion modules. Embedding two fusion modules into a two-stream…

Linguistic and memory correlates of societal variation: A quantitative analysis

Sihan Chen, David Gil, С. П. Гапонов +5 more

Cultural Studies · PsyArXiv (OSF Preprints) · · cited 13× · DOI ↗ · more like this ·

Traditionally, researchers have proposed a uniformitarian view that all languages are roughly equally complex, via an internal trade-off between the complexity at different levels, such as morphology and syntax. The extent to which the societies where the speakers live influence the trade-off has not been well studied. In this paper, we focus on morphology and syntax, and report significant correlations between specific linguistic and societal features, in particular those relating to exoteric (open) vs. esoteric (close-knit) society types, characterizable in terms of population size, mobility, communication across distances, etc. We conducted an exhaustive quantitative analysis drawing upon WALS, D-Place, Ethnologue and Glottolog. Our results find some support for our hypothesis that languages spoken by exoteric societies tend towards more complex syntaxes, while languages spoken by…

PyOD 2: A Python Library for Outlier Detection with LLM-powered Model Selection

Sihan Chen, Zhuangzhuang Qian, Wingchun Siu +10 more

cs.LG · · cited 9× · arXiv ↗ · more like this ·

Outlier detection (OD), also known as anomaly detection, is a critical machine learning (ML) task with applications in fraud detection, network intrusion detection, clickstream analysis, recommendation systems, and social network moderation. Among open-source libraries for outlier detection, the Python Outlier Detection (PyOD) library is the most widely adopted, with over 8,500 GitHub stars, 25 million downloads, and diverse industry usage. However, PyOD currently faces three limitations: (1) insufficient coverage of modern deep learning algorithms, (2) fragmented implementations across PyTorch and TensorFlow, and (3) no automated model selection, making it hard for non-experts. To address these issues, we present PyOD Version 2 (PyOD 2), which integrates 12 state-of-the-art deep learning models into a unified PyTorch framework and introduces a large language model (LLM)-based pipeline…

UMT-Net: A Uniform Multi-Task Network With Adaptive Task Weighting

Sihan Chen, Lianqing Zheng, Libo Huang +3 more

Computer Vision and Pattern Recognition · IEEE Transactions on Intelligent Vehicles · · cited 9× · DOI ↗ · more like this ·

This article introduces a versatile multi-task learning framework (UMT-Net) and an adaptive task weighting (ATW) training method, specifically designed for resource-constrained scenarios that demand parameter-efficient networks. The adaptable UMT-Net architecture includes a global-shared backbone based on an encoder, task-specific self-attention modules, inter-task joint-attention fusion modules, and feature-aggregating decoders. The ATW technique accounts for both short-term variation and long-term statistics of task losses, leading to a more stable training process. Extensive experiments on CityScapes and NYUv2 datasets reveal that UMT-Net outperforms baseline methods while requiring fewer computations, model parameters, and inference latency. In addition, we also conducted experiments on the autonomous driving dataset BDD100 K and achieved state-of-the-art performance. Furthermore…

The absence of a trade-off between morphological and syntactic complexity

Antonio Benítez‐Burraco, Sihan Chen, David Gil

Cultural Studies · MPG.PuRe (Max Planck Society) · · cited 8× · DOI ↗ · more like this ·

The hypothesis that all languages are equally complex often invokes a trade-off principle, according to which if a language is more complex in one particular domain, it will be simpler in another different domain. In this paper, we use data from WALS to test the existence of a trade-off between two specific domains: morphology and syntax. Contrary to widespread views, we find a positive correlation between these two language domains. We discuss this finding in relation to ongoing research about language complexity, and in particular, the effects of factors external to language on linguistic structure.

Motivators and deterrents for early career female doctors applying to surgical training programmes in the UK National Health Service: a mixed-methods study

Kajal Ruparell, Rajas Barve, Rukiye N Tas +4 more

Gender Studies · BMJ Open · · cited 8× · DOI ↗ · more like this ·

OBJECTIVES: To perform a mixed-methods study identifying motivators and deterrents to female doctors interested in core surgical training (CST). To provide tangible implementations based on the findings. DESIGN: This study used quantitative (questionnaires) and qualitative (semistructured interviews (SSIs)) analyses. Participants completed online questionnaires on Qualtrics and SSIs were conducted remotely on Microsoft Teams. Questions were derived from previous studies and a novel term, the gender impact rating (GIR), was coined to assess the impact of gender on opportunities available during CST application. SETTING: Participants were working in the UK National Health Service and data collected from December 2020 to January 2021. PARTICIPANTS: A total of 100 female surgical trainees in the UK ranging from Foundation Year 2 to Core Training Year 2. MAIN OUTCOME MEASURES: Participants…

MagmaDNN

Daniel Nichols, Kwai Wong, Stanimire Tomov +3 more

Computer Vision and Pattern Recognition · Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (learning) · · cited 7× · DOI ↗ · more like this ·

MagmaDNN [17] is a deep learning framework driven using the highly optimized MAGMA dense linear algebra package. The library offers comparable performance to other popular frameworks, such as TensorFlow, PyTorch, and Theano. C++ is used to implement the framework providing fast memory operations, direct cuda access, and compile time errors. Common neural network layers such as Fully Connected, Convolutional, Pooling, Flatten, and Dropout are included. Hyperparameter tuning is performed with a parallel grid search engine.

VLAB: Enhancing Video Language Pretraining by Feature Adapting and Blending

Xingjian He, Sihan Chen, Fan Ma +7 more

Computer Vision and Pattern Recognition · IEEE Transactions on Multimedia · · cited 6× · DOI ↗ · more like this ·

Large-scale image-text contrastive pre-training models, such as CLIP, have been demonstrated to effectively learn high-quality multimodal representations. However, there is limited research on learning video-text representations for general video multimodal tasks based on these powerful features. Towards this goal, we propose a novel video-text pre-training method dubbed VLAB: <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</b>ideo <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">L</b>anguage pre-training by feature <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</b>dapting and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</b>lending, which transfers CLIP representations to video…

LLM2AutoML: Zero-Code AutoML Framework Leveraging Large Language Models

Sihan Chen, Weihong Zhai, Chen Chai +1 more

Artificial Intelligence · · cited 6× · DOI ↗ · more like this ·

Automated Machine Learning (AutoML) aims to make machine learning accessible to non-experts by minimizing technical barriers and enabling the creation of high-performance models without extensive programming knowledge. Despite advances, existing AutoML solutions often struggle with complexity and inefficiency in handling intricate tasks. Herein, we present LLM2AutoML, a zero-code AutoML framework that leverages Large Language Models (LLMs) to generate high-performance actionable ML models without coding and with explanations. LLM2AutoML enables bidirectional human-machine alignment by interpreting user intentions expressed in natural language, converting them into executable AutoML tasks, and delivering detailed analytical reports that foster user understanding and trust. We propose the template-bounded AutoML method to ensure the LLM-generated code is highly executable, enabling fully…

Why we fight: investigating the moral appeals in terrorist propaganda, their predictors, and their association with attack severity

Lindsay Hahn, Katherine Schibler, Tahleen A. Lattimer +6 more

Sociology and Political Science · Journal of Communication · · cited 6× · DOI ↗ · more like this ·

Abstract How do terrorists persuade otherwise decent citizens to join their violent causes? Guided by early mass communication research investigating propaganda’s efficacy and the model of intuitive morality and exemplars, we investigated the persuasive moral appeals employed by terrorist organizations known to be successful at recruiting others to their causes. We compiled a database of N = 873 propaganda items created by N = 73 violent terrorist organizations and content analyzed the moral appeals emphasized in each. Results revealed that terrorist groups’ ideologies and motivations predicted the moral values they emphasized in their propaganda, and that ingroup loyalty and fairness appeals featured prominently across all propaganda. Terrorist groups’ emphasis on purity in their propaganda was positively correlated with their attack frequency and with the number of human casualties…

MM21 Pre-training for Video Understanding Challenge

Sihan Chen, Xinxin Zhu, Dongze Hao +5 more

Computer Vision and Pattern Recognition · · cited 6× · DOI ↗ · more like this ·

The quality of video representation directly decides the performance of video related tasks, for both understanding and generation. In this paper, we propose single-modality pretrained feature fusion technique which is composed of reasonable multi-view feature extraction method and designed multi-modality feature fusion strategy. We conduct comprehensive ablation studies on MSR-VTT dataset to demonstrate the effectiveness of proposed method and it surpasses the state-of-the-art methods on both MSR-VTT and VATEX datasets. We further propose the multi-modality pretrained model finetuning technique and dataset augmentation scheme to improve the model's generalization capability. Based on these two proposed pretraining techniques and dataset augmentation scheme, we win the first place in the video captioning track of the MM21 pretraining for video understanding challenge.

Multi-Objective Optimal Trajectory Planning for Woodworking Manipulator and Worktable Based on the INSGA-II Algorithm

Jiaping Yi, Changqing Zhang, Sihan Chen +4 more

Computer Vision and Pattern Recognition · Applied Sciences · · cited 5× · DOI ↗ · more like this ·

The manipulator has been widely used in the wood processing industry; the main problem currently faced is optimizing the motion trajectory to enhance the processing efficiency and operational stability of the woodworking manipulator and worktable. A 5-7-5 piecewise polynomial interpolation method is proposed to construct the spatial trajectories of each joint. An improved non-dominated sorting genetic algorithm (INSGA-II) is proposed to achieve a time–jerk multi-objective trajectory planning that can meet the dual requirements of minimal processing time and reduced motion impact. In order to address the limitations of the standard NSGA-II algorithm, which is prone to local optima and exhibits slow convergence, we propose a good point set method for multi-objective optimization population initialization and a linear ranking selection method to refine the parent selection process within…