Author

Bo He

321 papers · 1990–2026

1990peak 31 in 20222026

Frequent co-authors

Papers

showing 1–25 of 321 RSS

GreenCloud

Liang Liu, Hao Wang, Xue Liu +4 more

Information Systems · 16 Jun 2009 · cited 368× · DOI ↗ · more like this ·

Nowadays, power consumption of data centers has huge impacts on environments. Researchers are seeking to find effective solutions to make data centers reduce power consumption while keep the desired quality of service or service level objectives. Virtual Machine (VM) technology has been widely applied in data center environments due to its seminal features, including reliability, flexibility, and the ease of management. We present the GreenCloud architecture, which aims to reduce data center power consumption, while guarantee the performance from users' perspective. GreenCloud architecture enables comprehensive online-monitoring, live virtual machine migration, and VM placement optimization. To verify the efficiency and effectiveness of the proposed architecture, we take an online real-time game, Tremulous, as a VM application. Evaluation results show that we can save up to 27% of the…

MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding

Bo He, Hengduo Li, Young Kyun Jang +5 more

cs.CV · 24 Apr 2024 · cited 319× · arXiv ↗ · more like this ·

With the success of large language models (LLMs), integrating the vision model into LLMs to build vision-language foundation models has gained much more interest recently. However, existing LLM-based large multimodal models (e.g., Video-LLaMA, VideoChat) can only take in a limited number of frames for short video understanding. In this study, we mainly focus on designing an efficient and effective model for long-term video understanding. Instead of trying to process more frames simultaneously like most existing work, we propose to process videos in an online manner and store past video information in a memory bank. This allows our model to reference historical video content for long-term analysis without exceeding LLMs' context length constraints or GPU memory limits. Our memory bank can be seamlessly integrated into current multimodal LLMs in an off-the-shelf manner. We conduct…

Align and Attend: Multimodal Summarization with Dual Contrastive Losses

Bo He, Jun Wang, Jielin Qiu +3 more

cs.CV · March 2023 · cited 132× · arXiv ↗ · more like this ·

The goal of multimodal summarization is to extract the most important information from different modalities to form output summaries. Unlike the unimodal summarization, the multimodal summarization task explicitly leverages cross-modal information to help generate more reliable and high-quality summaries. However, existing methods fail to leverage the temporal correspondence between different modalities and ignore the intrinsic correlation between different samples. To address this issue, we introduce Align and Attend Multimodal Summarization (A2Summ), a unified multimodal transformer-based model which can effectively align and attend the multimodal input. In addition, we propose two novel contrastive losses to model both inter-sample and intra-sample correlations. Extensive experiments on two standard video summarization datasets (TVSum and SumMe) and two multimodal summarization…

Human-Centered Reinforcement Learning: A Survey

Guangliang Li, Randy Gómez, Keisuke Nakamura +1 more

Artificial Intelligence · IEEE Transactions on Human-Machine Systems · 8 May 2019 · cited 125× · DOI ↗ · more like this ·

Human-centered reinforcement learning (RL), in which an agent learns how to perform a task from evaluative feedback delivered by a human observer, has become more and more popular in recent years. The advantage of being able to learn from human feedback for a RL agent has led to increasing applicability to real-life problems. This paper describes the state-of-the-art human centered RL algorithms and aims to become a starting point for researchers who are initiating their endeavors in human-centered RL. Moreover, the objective of this paper is to present a comprehensive survey of the recent breakthroughs in this field and provide references to the most interesting and successful works. After starting with an introduction of the concepts of RL from environmental reward, this paper discusses the origins of human-centered RL and its difference from traditional RL. Then we describe different…

ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localization

Bo He, Xitong Yang, Le Kang +3 more

cs.CV · 29 Mar 2022 · cited 104× · arXiv ↗ · more like this ·

Weakly-supervised temporal action localization aims to recognize and localize action segments in untrimmed videos given only video-level action labels for training. Without the boundary information of action segments, existing methods mostly rely on multiple instance learning (MIL), where the predictions of unlabeled instances (i.e., video snippets) are supervised by classifying labeled bags (i.e., untrimmed videos). However, this formulation typically treats snippets in a video as independent instances, ignoring the underlying temporal structures within and across action segments. To address this problem, we propose \system, a novel WTAL framework that enables explicit, action-aware segment modeling beyond standard MIL-based methods. Our framework entails three segment-centric components: (i) dynamic segment sampling for compensating the contribution of short actions; (ii) intra- and…

Towards Scalable Neural Representation for Diverse Videos

Bo He, Xitong Yang, Hanyu Wang +6 more

cs.CV · 24 Mar 2023 · cited 69× · arXiv ↗ · more like this ·

Implicit neural representations (INR) have gained increasing attention in representing 3D scenes and images, and have been recently applied to encode videos (e.g., NeRV, E-NeRV). While achieving promising results, existing INR-based methods are limited to encoding a handful of short videos (e.g., seven 5-second videos in the UVG dataset) with redundant visual content, leading to a model design that fits individual video frames independently and is not efficiently scalable to a large number of diverse videos. This paper focuses on developing neural representations for a more practical setup -- encoding long and/or a large number of videos with diverse visual content. We first show that instead of dividing videos into small subsets and encoding them with separate models, encoding long and diverse videos jointly with a unified model achieves better compression results. Based on this…

Underwater image enhancement using adaptive color restoration and dehazing

Tengyue Li, Shenghui Rong, Wenfeng Zhao +4 more

Computer Vision and Pattern Recognition · Optics Express · 4 Feb 2022 · cited 65× · DOI ↗ · more like this ·

Underwater images captured by optical cameras can be degraded by light attenuation and scattering, which leads to deteriorated visual image quality. The technique of underwater image enhancement plays an important role in a wide range of subsequent applications such as image segmentation and object detection. To address this issue, we propose an underwater image enhancement framework which consists of an adaptive color restoration module and a haze-line based dehazing module. First, we employ an adaptive color restoration method to compensate the deteriorated color channels and restore the colors. The color restoration module consists of three steps: background light estimation, color recognition, and color compensation. The background light estimation determines the image is blueish or greenish, and the compensation is applied in red-green or red-blue channels. Second, the haze-line…

Joint modeling of multivariate longitudinal measurements and survival data with applications to Parkinson’s disease

Bo He, Sheng Luo

Statistics and Probability · Statistical Methods in Medical Research · 16 Apr 2013 · cited 57× · DOI ↗ · more like this ·

In many clinical trials, studying neurodegenerative diseases including Parkinson's disease (PD), multiple longitudinal outcomes are collected in order to fully explore the multidimensional impairment caused by these diseases. The follow-up of some patients can be stopped by some outcome-dependent terminal event, e.g. death and dropout. In this article, we develop a joint model that consists of a multilevel item response theory (MLIRT) model for the multiple longitudinal outcomes, and a Cox's proportional hazard model with piecewise constant baseline hazards for the event time data. Shared random effects are used to link together two models. The model inference is conducted using a Bayesian framework via Markov Chain Monte Carlo simulation implemented in BUGS language. Our proposed model is evaluated by simulation studies and is applied to the DATATOP study, a motivating clinical trial…

DeepCC: Multi-Agent Deep Reinforcement Learning Congestion Control for Multi-Path TCP Based on Self-Attention

Bo He, Jingyu Wang, Qi Qi +5 more

Computer Networks and Communications · IEEE Transactions on Network and Service Management · 29 Jun 2021 · cited 53× · DOI ↗ · more like this ·

With the development of the Internet of Things (IoT) and 5G, there are ubiquitous smart devices and network functions providing emerging network services efficiently and optimally through building many network connections based on WiFi, LTE/5G, Ethernet, and etc. The Multipath TCP (MPTCP) protocol that enables these devices to establish multiple paths for simultaneous data transmission, has been a widely used extension of standard TCP in smart devices and network functions. On the other hand, more heavy and time-varying traffic loads are generated in an MPTCP network, so that an efficient congestion control mechanism that schedules the traffic between multiple subflows and avoids congestion is highly required. In this paper, we propose a decentralized learning approach, DeepCC, to adapt to the volatile environments and realize the efficient congestion control. The Multi-Agent Deep…

OmniVid: A Generative Framework for Universal Video Understanding

Junke Wang, Dongdong Chen, Chong Luo +4 more

cs.CV · 26 Mar 2024 · cited 44× · arXiv ↗ · more like this ·

The core of video understanding tasks, such as recognition, captioning, and tracking, is to automatically detect objects or actions in a video and analyze their temporal evolution. Despite sharing a common goal, different tasks often rely on distinct model architectures and annotation formats. In contrast, natural language processing benefits from a unified output space, i.e., text sequences, which simplifies the training of powerful foundational language models, such as GPT-3, with extensive training corpora. Inspired by this, we seek to unify the output space of video understanding tasks by using languages as labels and additionally introducing time and box tokens. In this way, a variety of video tasks could be formulated as video-grounded token generation. This enables us to address various types of video tasks, including classification (such as action recognition), captioning…

Feature Combination Meets Attention: Baidu Soccer Embeddings and Transformer based Temporal Detection

Xin Zhou, Le Kang, Zhiyu Cheng +2 more

cs.CV · 28 Jun 2021 · cited 43× · arXiv ↗ · more like this ·

With rapidly evolving internet technologies and emerging tools, sports related videos generated online are increasing at an unprecedentedly fast pace. To automate sports video editing/highlight generation process, a key task is to precisely recognize and locate the events in the long untrimmed videos. In this tech report, we present a two-stage paradigm to detect what and when events happen in soccer broadcast videos. Specifically, we fine-tune multiple action recognition models on soccer data to extract high-level semantic features, and design a transformer based temporal detection module to locate the target events. This approach achieved the state-of-the-art performance in both two tasks, i.e., action spotting and replay grounding, in the SoccerNet-v2 Challenge, under CVPR 2021 ActivityNet workshop. Our soccer embedding features are released at…

A Gaussian Mixture Unscented Rauch–Tung–Striebel Smoothing Framework for Trajectory Reconstruction

Jiacheng He, Bei Peng, Zhenyu Feng +3 more

Artificial Intelligence · IEEE Transactions on Industrial Informatics · 16 Feb 2024 · cited 42× · DOI ↗ · more like this ·

Trajectory reconstruction (TR) plays an important role in practical applications. The data collected for TR are often contaminated with non-Gaussian noise due to environmental factors, which reduces the accuracy of TR. This study proposes a novel method to suppress the effects of non-Gaussian measurement noise. The method consists of two steps: decomposing the measurement noise and performing a weighted fusion of the relevant states. In the first step, the measurement noise is decomposed into a weighted combination of multiple Gaussian distributions using the Gaussian mixture model. In the second step, the interacting multiple model is employed to perform the weighted fusion of the relevant states. The main idea of the proposed method is to transform the state estimation problem from a non-Gaussian and nonlinear case into a state estimation problem in the nonlinear and Gaussian case.…

Pre-Impact Fall Detection Based on a Modified Zero Moment Point Criterion Using Data From Kinect Sensors

Min Li, Guanghua Xu, Bo He +2 more

Computer Vision and Pattern Recognition · IEEE Sensors Journal · 7 May 2018 · cited 41× · DOI ↗ · more like this ·

Accidental falls have always been a serious problem for the elderly. There is considerable demand for pre-impact fall detection systems with long lead times. According to the zero moment point criterion, the zero moment point should be kept beneath the supporting foot for stability during humanoid robot standing or walking. However, the zero moment point in the human walk does not stay fixed under the supporting foot. In this paper, we define a dynamic supporting area containing both feet and the area between the two feet, and propose a method of fall prediction based on a modified zero moment point criterion using motion-monitoring data from a Kinect sensor. A fall event is predicted if the projection of the zero moment point locates outside of the dynamic supporting area. The proposed method is compared with a method identifying the imbalance state based on a support vector machine…

Intelligent VNFs Selection Based on Traffic Identification in Vehicular Cloud Networks

Jingyu Wang, Bo He, Jing Wang +1 more

Computer Networks and Communications · IEEE Transactions on Vehicular Technology · 12 Nov 2018 · cited 38× · DOI ↗ · more like this ·

Internet of Vehicles (IoV) has become a significant research area due to its specific features and applications such as efficient traffic management, road safety, and entertainment. Vehicles are expected to carry relatively more communication systems, onboard computing facilities, storage, and increased sensing power. Vehicular Cloud Network (VCN) is a hybrid technology that has a remarkable impact on IoV by instantly using vehicular resources. Merging software-defined networking with VCN is valuable to vehicular networks. Network function virtualization can be deployed across the servers and vehicles within the VCN to provide customized agile services. Its deployment requires an appropriate management strategy that often manifests as the difficult online decision making tasks. This paper proposes an intelligent Virtualized Network Functions (VNFs) selection strategy. For satisfying the…

DEEP NEURAL NETWORKS FOR APPLICATION AWARENESS IN SDN-BASED NETWORK

Jun Xu, Jingyu Wang, Qi Qi +2 more

Computer Networks and Communications · September 2018 · cited 35× · DOI ↗ · more like this ·

Accurate traffic classification is essential for traffic engineering and Quality of Service (QoS) guarantee, especially in Internet of Things (IoT). Different applications have different network resource requirements, so an excellent classification algorithm can realize application awareness in traffic engineering and significantly improve QoS. Software Defined Network (SDN) with centralized controlling of network resources provides opportunities for fine-grained resource allocation. However, there are many issues when deep learning is employed in SDN, for example, sampling and classifying traffic data consume a lot of IO and computing resources of the SDN controller. In this paper, we deploy the Deep Neural Network (DNN) on Virtualized Network Function (VNF) to solve the problems of applying deep learning in SDN. The experiments show that the proposed DNN model outperforms existing…

NUICNet: Non-Uniform Illumination Correction for Underwater Image Using Fully Convolutional Network

Xueting Cao, Shenghui Rong, Yongbin Liu +3 more

Computer Vision and Pattern Recognition · IEEE Access · 2020 · cited 34× · DOI ↗ · more like this ·

Absorption and scattering in aqueous media would attenuate light and make imaging difficult. Therefore, an artificial light source is usually utilized to assist imaging in the deep ocean. However, the artificial light source typically alters the light conditions to a large extent, resulting in the non-uniform illumination of images. To solve this problem, we propose a non-uniform illumination correction algorithm based on a fully convolutional network for underwater images. The proposed algorithm model the original image as the addition of the ideal image and a non-uniform light layer. We replace the traditional pooling layer with dilated convolution to expand the receptive field and achieve higher accuracy in non-uniform illumination recognition. To improve the perception ability of the network effectively, the original image and parameters which pre-trained on the ImageNet are…

Deep Learning-Assisted OFDM Channel Estimation and Signal Detection Technology

Jun Li, Zhichen Zhang, Yu–Kai Wang +3 more

Artificial Intelligence · IEEE Communications Letters · 16 Feb 2023 · cited 30× · DOI ↗ · more like this ·

The orthogonal frequency division multiplexing (OFDM) technique has received wide attention because of its high spectrum utilization. However, the drawback of inter-subcarrier interference in OFDM systems makes the channel estimation and signal detection performance of OFDM systems with few pilots and short cyclic prefixes (CP) poor. In this letter, we use deep learning to assist OFDM in recovering nonlinearly distorted transmission data. Specifically, we use a self-normalizing network (SNN) for channel estimation, combined with a convolutional neural network (CNN) and a bidirectional gated recurrent unit (BiGRU) for signal detection, thus proposing a novel SNN-CNN-BiGRU network structure (SCBiGNet). The simulation results show that the SCBiGNet model outperforms the existing techniques for the different numbers of pilots and lengths of CPs. The BER performance is improved by 0.2-9 dB.

GTA: Global Temporal Attention for Video Action Understanding

Bo He, Xitong Yang, Zuxuan Wu +3 more

cs.CV · December 2020 · cited 30× · arXiv ↗ · more like this ·

Self-attention learns pairwise interactions to model long-range dependencies, yielding great improvements for video action recognition. In this paper, we seek a deeper understanding of self-attention for temporal modeling in videos. We first demonstrate that the entangled modeling of spatio-temporal information by flattening all pixels is sub-optimal, failing to capture temporal relationships among frames explicitly. To this end, we introduce Global Temporal Attention (GTA), which performs global temporal attention on top of spatial attention in a decoupled manner. We apply GTA on both pixels and semantically similar regions to capture temporal relationships at different levels of spatial granularity. Unlike conventional self-attention that computes an instance-specific attention matrix, GTA directly learns a global attention matrix that is intended to encode temporal structures that…

Underwater image enhancement framework and its application on an autonomous underwater vehicle platform

Tengyue Li, Shenghui Rong, Xueting Cao +3 more

Computer Vision and Pattern Recognition · Optical Engineering · 10 Aug 2020 · cited 30× · DOI ↗ · more like this ·

Underwater imaging has been increasingly employed in vision-based marine research. However, the inappropriate installation of a light source and the complex underwater environment will result in the uneven illumination and overexposure on the captured images. To address these issues, an underwater image enhancement framework for autonomous underwater vehicles platform is proposed, which consists of underwater light source optimization and illumination nonuniformity correction. The light source optimization method improves the imaging quality by computing an appropriate angle of the light casting. In this way, the center of the field of view is always well lit. In addition, an adaptive filter-based illumination correction algorithm is proposed to solve the uneven illumination caused by the artificial light source. During this process, image block segmentation and the measure of image…

ROV-based Underwater Vision System for Intelligent Fish Ethology Research

Rui Nian, Bo He, Jia Yuan Yu +2 more

Computer Vision and Pattern Recognition · International Journal of Advanced Robotic Systems · September 2013 · cited 28× · DOI ↗ · more like this ·

Fish ethology is a prospective discipline for ocean surveys. In this paper, one ROV-based system is established to perform underwater visual tasks with customized optical sensors installed. One image quality enhancement method is first presented in the context of creating underwater imaging models combined with homomorphic filtering and wavelet decomposition. The underwater vision system can further detect and track swimming fish from the resulting images with the strategies developed using curve evolution and particular filtering, in order to obtain a deeper understanding of fish behaviours. The simulation results have shown the excellent performance of the developed scheme, in regard to both robustness and effectiveness.

Segmentation of Sidescan Sonar Imagery Using Markov Random Fields and Extreme Learning Machine

Yan Song, Bo He, Ying Zhao +6 more

Artificial Intelligence · IEEE Journal of Oceanic Engineering · 18 May 2018 · cited 26× · DOI ↗ · more like this ·

As a widely used segmentation scheme, Markov random field (MRF) utilizes k-means clustering to calculate the initial model for sidescan sonar image segmentation. However, for the noise and intensity inhomogeneity nature of the sidescan sonar images, the segmentation results of k-means clustering have low accuracy, motivating us to use machine learning methods to initialize MRF. Meanwhile, an extreme learning machine (ELM), a supervised learning algorithm derived from the single-hidden-layer feedforward neural networks, learns faster than randomly generated hidden-layer parameters and is superior to a support vector machine (SVM). Therefore, in this paper, we proposed a novel method for sidescan sonar image segmentation based on MRF and ELM. The proposed method segments sidescan sonar images in object-highlight, object-shadow, and sea-bottom reverberation areas. Specifically, we intend…

A Node Location Algorithm Based on Improved Whale Optimization in Wireless Sensor Networks

Pingzhang Gou, Bo He, Zhaoyang Yu

Computer Networks and Communications · Wireless Communications and Mobile Computing · 2021 · cited 25× · DOI ↗ · more like this ·

With the popularity of swarm intelligence algorithms, the positioning of nodes to be located in wireless sensor networks (WSNs) has received more and more attention. To overcome the disadvantage of large ranging error and low positioning accuracy caused by the positioning algorithm of the received signal strength indication (RSSI) ranging model, we use the RSSI modified by Gaussian to reduce the distance measurement error and introduce an improved whale optimization algorithm to optimize the location of the nodes to be positioned to improve the positioning accuracy. The experimental results show that the improved whale algorithm performs better than the whale optimization algorithm and other swarm intelligence algorithms under 20 different types of benchmark function tests. The positioning accuracy of the proposed location algorithm is better than that of the original RSSI algorithm…

PCA and Kernel-based extreme learning machine for side-scan sonar image classification

Mingcui Zhu, Yan Song, Jia Guo +4 more

Artificial Intelligence · 2017 · cited 24× · DOI ↗ · more like this ·

As an important role of oceanographic survey, side-scan sonar image classification has attracted much attention in the past two decades. Due to the special properties of sonar image, traditional approaches are difficult to get good classification accuracy, so their implementation in real world is blocked. In this paper, a novel classification system based on kernel-based extreme learning machine (KELM) and principle component analysis (PCA) is proposed. Experimental results demonstrate that the proposed method can get better stability and higher classification accuracy than traditional approaches such as support vector machine (SVM).

Distorted underwater image reconstruction for an autonomous underwater vehicle based on a self-attention generative adversarial network

Tengyue Li, Qianqian Yang, Shenghui Rong +2 more

Computer Vision and Pattern Recognition · Applied Optics · October 2020 · cited 23× · DOI ↗ · more like this ·

Imaging through the wavy air-water surface suffers from severe geometric distortions, which are caused by the light refraction effect that affects the normal operations of underwater exploration equipment such as the autonomous underwater vehicle (AUV). In this paper, we propose a deep learning-based framework, namely the self-attention generative adversarial network (SAGAN), to remove the geometric distortions and restore the distorted image captured through the water-air surface. First, a K-means-based image pre-selection method is employed to acquire a less distorted image that preserves much useful information from an image sequence. Second, an improved generative adversarial network (GAN) is trained to translate the distorted image into the non-distorted image. During this process, the attention mechanism and the weighted training objective are adopted in our GAN framework to get…

Underwater Single Image Dehazing Using the Color Space Dimensionality Reduction Prior

Yongbin Liu, Shenghui Rong, Xueting Cao +2 more

Computer Vision and Pattern Recognition · IEEE Access · 2020 · cited 22× · DOI ↗ · more like this ·

Underwater images suffer from low visibility and contrast caused by absorption and scattering, which leads to haze and some further limitations. The existing underwater single image dehazing methods cannot achieve a balance between the performance and computational complexity, and are difficult to produce satisfactory results in the regions with large distance. To overcome these problems, we propose a new underwater single image dehazing method, which includes an improved background light estimation based on the quad-tree subdivision iteration algorithm, and a novel transmission estimation method. For the background light estimation, we introduce a robust score for each region of the image, which can evaluate the region from both smoothness and color. For the transmission estimation, we propose the color space dimensionality reduction prior (CSDRP), which allows conversing an image from…