Author

Bo Liu

2,676 papers · 1753–2026

1967peak 248 in 20242026

Frequent co-authors

Papers

showing 1–25 of 2,676 RSS

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

DeepSeek-AI, Aixin Liu, Bei Feng +154 more

cs.CL · May 2024 · cited 1,487× · arXiv ↗ · more like this ·

We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and…

A Normalized Levenshtein Distance Metric

Yujian Li, Bo Liu

Artificial Intelligence · IEEE Transactions on Pattern Analysis and Machine Intelligence · 25 Apr 2007 · cited 945× · DOI ↗ · more like this ·

Although a number of normalized edit distances presented so far may offer good performance in some applications, none of them can be regarded as a genuine metric between strings because they do not satisfy the triangle inequality. Given two strings X and Y over a finite alphabet, this paper defines a new normalized edit distance between X and Y as a simple function of their lengths (|X| and |Y|) and the Generalized Levenshtein Distance (GLD) between them. The new distance can be easily computed through GLD with a complexity of O(|X|.|Y|) and it is a metric valued in [0, 1] under the condition that the weight function is a metric over the set of elementary edit operations with all costs of insertions/deletions having the same weight. Experiments using the AESA algorithm in handwritten digit recognition show that the new distance can generally provide similar results to some other…

DeepSeek-VL: Towards Real-World Vision-Language Understanding

Haoyu Lu, Wen Liu, Bo Zhang +12 more

cs.AI · 11 Mar 2024 · cited 896× · arXiv ↗ · more like this ·

We present DeepSeek-VL, an open-source Vision-Language (VL) Model designed for real-world vision and language understanding applications. Our approach is structured around three key dimensions: We strive to ensure our data is diverse, scalable, and extensively covers real-world scenarios including web screenshots, PDFs, OCR, charts, and knowledge-based content, aiming for a comprehensive representation of practical contexts. Further, we create a use case taxonomy from real user scenarios and construct an instruction tuning dataset accordingly. The fine-tuning with this dataset substantially improves the model's user experience in practical applications. Considering efficiency and the demands of most real-world scenarios, DeepSeek-VL incorporates a hybrid vision encoder that efficiently processes high-resolution images (1024 x 1024), while maintaining a relatively low computational…

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

DeepSeek-AI, :, Xiao Bi +85 more

cs.CL · 5 Jan 2024 · cited 859× · arXiv ↗ · more like this ·

The rapid development of open-source large language models (LLMs) has been truly remarkable. However, the scaling law described in previous literature presents varying conclusions, which casts a dark cloud over scaling LLMs. We delve into the study of scaling laws and present our distinctive findings that facilitate scaling of large scale models in two commonly used open-source configurations, 7B and 67B. Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source language models with a long-term perspective. To support the pre-training phase, we have developed a dataset that currently consists of 2 trillion tokens and is continuously expanding. We further conduct supervised fine-tuning (SFT) and Direct Preference Optimization (DPO) on DeepSeek LLM Base models, resulting in the creation of DeepSeek Chat models. Our evaluation results demonstrate…

SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering

Bo Liu, Li-Ming Zhan, Li Xu +3 more

cs.CV · 18 Feb 2021 · cited 729× · arXiv ↗ · more like this ·

Medical visual question answering (Med-VQA) has tremendous potential in healthcare. However, the development of this technology is hindered by the lacking of publicly-available and high-quality labeled datasets for training and evaluation. In this paper, we present a large bilingual dataset, SLAKE, with comprehensive semantic labels annotated by experienced physicians and a new structural medical knowledge base for Med-VQA. Besides, SLAKE includes richer modalities and covers more human body parts than the currently available dataset. We show that SLAKE can be used to facilitate the development and evaluation of Med-VQA systems. The dataset can be downloaded from http://www.med-vqa.com/slake.

Trustworthy AI: From Principles to Practices

Bo Li, Peng Qi, Bo Liu +5 more

cs.AI · October 2021 · cited 719× · arXiv ↗ · more like this ·

The rapid development of Artificial Intelligence (AI) technology has enabled the deployment of various systems based on it. However, many current AI systems are found vulnerable to imperceptible attacks, biased against underrepresented groups, lacking in user privacy protection. These shortcomings degrade user experience and erode people's trust in all AI systems. In this review, we provide AI practitioners with a comprehensive guide for building trustworthy AI systems. We first introduce the theoretical framework of important aspects of AI trustworthiness, including robustness, generalization, explainability, transparency, reproducibility, fairness, privacy preservation, and accountability. To unify currently available but fragmented approaches toward trustworthy AI, we organize them in a systematic approach that considers the entire lifecycle of AI systems, ranging from data…

LLM+P: Empowering Large Language Models with Optimal Planning Proficiency

Bo Liu, Yuqian Jiang, Xiaohan Zhang +4 more

cs.AI · April 2023 · cited 707× · arXiv ↗ · more like this ·

Large language models (LLMs) have demonstrated remarkable zero-shot generalization abilities: state-of-the-art chatbots can provide plausible answers to many common questions that arise in daily life. However, so far, LLMs cannot reliably solve long-horizon planning problems. By contrast, classical planners, once a problem is given in a formatted way, can use efficient search algorithms to quickly identify correct, or even optimal, plans. In an effort to get the best of both worlds, this paper introduces LLM+P, the first framework that incorporates the strengths of classical planners into LLMs. LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language. LLM+P does so by first converting the language description into a file written in the planning domain definition language (PDDL), then…

Conflict-Averse Gradient Descent for Multi-task Learning

Bo Liu, Xingchao Liu, Xiaojie Jin +2 more

cs.LG · October 2021 · cited 677× · arXiv ↗ · more like this ·

The goal of multi-task learning is to enable more efficient learning than single task learning by sharing model structures for a diverse set of tasks. A standard multi-task learning objective is to minimize the average loss across all tasks. While straightforward, using this objective often results in much worse final performance for each task than learning them independently. A major challenge in optimizing a multi-task model is the conflicting gradients, where gradients of different task objectives are not well aligned so that following the average gradient direction can be detrimental to specific tasks' performance. Previous work has proposed several heuristics to manipulate the task gradients for mitigating this problem. But most of them lack convergence guarantee and/or could converge to any Pareto-stationary point. In this paper, we introduce Conflict-Averse Gradient descent…

Satellite-relayed intercontinental quantum network

Sheng-Kai Liao, Wen-Qi Cai, Johannes Handsteiner +33 more

quant-ph · 13 Jan 2018 · cited 674× · arXiv ↗ · more like this ·

We perform decoy-state quantum key distribution between a low-Earth-orbit satellite and multiple ground stations located in Xinglong, Nanshan, and Graz, which establish satellite-to-ground secure keys with ~kHz rate per passage of the satellite Micius over a ground station. The satellite thus establishes a secure key between itself and, say, Xinglong, and another key between itself and, say, Graz. Then, upon request from the ground command, Micius acts as a trusted relay. It performs bitwise exclusive OR operations between the two keys and relays the result to one of the ground stations. That way, a secret key is created between China and Europe at locations separated by 7600 km on Earth. These keys are then used for intercontinental quantum-secured communication. This was on the one hand the transmission of images in a one-time pad configuration from China to Austria as well as from…

A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems

Bo Liu, Qingfu Zhang, Georges Gielen

Computational Theory and Mathematics · IEEE Transactions on Evolutionary Computation · 20 Feb 2013 · cited 570× · DOI ↗ · more like this ·

Surrogate model assisted evolutionary algorithms (SAEAs) have recently attracted much attention due to the growing need for computationally expensive optimization in many real-world applications. Most current SAEAs, however, focus on small-scale problems. SAEAs for medium-scale problems (i.e., 20-50 decision variables) have not yet been well studied. In this paper, a Gaussian process surrogate model assisted evolutionary algorithm for medium-scale computationally expensive optimization problems (GPEME) is proposed and investigated. Its major components are a surrogate model-aware search mechanism for expensive optimization problems when a high-quality surrogate model is difficult to build and dimension reduction techniques for tackling the “curse of dimensionality.” A new framework is developed and used in GPEME, which carefully coordinates the surrogate modeling and the evolutionary…

Learning active facial patches for expression analysis

Lin Zhong, Qingshan Liu, Peng Yang +3 more

Computer Vision and Pattern Recognition · June 2012 · cited 374× · DOI ↗ · more like this ·

In this paper, we present a new idea to analyze facial expression by exploring some common and specific information among different expressions. Inspired by the observation that only a few facial parts are active in expression disclosure (e.g., around mouth, eye), we try to discover the common and specific patches which are important to discriminate all the expressions and only a particular expression, respectively. A two-stage multi-task sparse learning (MTSL) framework is proposed to efficiently locate those discriminative patches. In the first stage MTSL, expression recognition tasks, each of which aims to find dominant patches for each expression, are combined to located common patches. Second, two related tasks, facial expression recognition and face verification tasks, are coupled to learn specific facial patches for individual expression. Extensive experiments validate the…

When Machine Learning Meets Privacy: A Survey and Outlook

Bo Liu, Ming Ding, Sina Shaham +3 more

cs.LG · 24 Nov 2020 · cited 360× · arXiv ↗ · more like this ·

The newly emerged machine learning (e.g. deep learning) methods have become a strong driving force to revolutionize a wide range of industries, such as smart healthcare, financial technology, and surveillance systems. Meanwhile, privacy has emerged as a big concern in this machine learning-based artificial intelligence era. It is important to note that the problem of privacy preservation in the context of machine learning is quite different from that in traditional data privacy protection, as machine learning can act as both friend and foe. Currently, the work on the preservation of privacy and machine learning (ML) is still in an infancy stage, as most existing solutions only focus on privacy problems during the machine learning process. Therefore, a comprehensive study on the privacy preservation problems and machine learning is required. This paper surveys the state of the art in…

Deep learning methods for super-resolution reconstruction of turbulent flows

Bo Liu, Jiupeng Tang, Haibo Huang +1 more

Computer Vision and Pattern Recognition · Physics of Fluids · February 2020 · cited 351× · DOI ↗ · more like this ·

Two deep learning (DL) models addressing the super-resolution (SR) reconstruction of turbulent flows from low-resolution coarse flow field data are developed. One is the static convolutional neural network (SCNN), and the other is the novel multiple temporal paths convolutional neural network (MTPC). The SCNN model takes instantaneous snapshots as an input, while the MTPC model takes a time series of velocity fields as an input, and it includes spatial and temporal information simultaneously. Three temporal paths are designed in the MTPC to fully capture features in different time ranges. A weight path is added to generate pixel-level weight maps of each temporal path. These models were first applied to forced isotropic turbulence. The corresponding high-resolution flow fields were reconstructed with high accuracy. The MTPC seems to be able to reproduce many important features as well…

A trusted-node-free eight-user metropolitan quantum communication network

Siddarth Koduru Joshi, Djeylan Aktas, Sören Wengerowsky +12 more

quant-ph · July 2019 · cited 306× · arXiv ↗ · more like this ·

Quantum communication is rapidly gaining popularity due to its high security and technological maturity. However, most implementations are limited to just two communicating parties (users). Quantum communication networks aim to connect a multitude of users. Here we present a fully connected quantum communication network on a city wide scale without active switching or trusted nodes. We demonstrate simultaneous and secure connections between all 28 pairings of 8 users. Our novel network topology is easily scalable to many users, allows traffic management features and minimises the infrastructure as well as the user hardware needed.

All in One: Multi-task Prompting for Graph Neural Networks

Xiangguo Sun, Hong Cheng, Jia Li +2 more

cs.SI · July 2023 · cited 300× · arXiv ↗ · more like this ·

Recently, ''pre-training and fine-tuning'' has been adopted as a standard workflow for many graph tasks since it can take general graph knowledge to relieve the lack of graph annotations from each application. However, graph tasks with node level, edge level, and graph level are far diversified, making the pre-training pretext often incompatible with these multiple tasks. This gap may even cause a ''negative transfer'' to the specific application, leading to poor results. Inspired by the prompt learning in natural language processing (NLP), which has presented significant effectiveness in leveraging prior knowledge for various NLP tasks, we study the prompting topic for graphs with the motivation of filling the gap between pre-trained models and various graph tasks. In this paper, we propose a novel multi-task prompting method for graph models. Specifically, we first unify the format of…

Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey

Xuesu Xiao, Bo Liu, Garrett Warnell +1 more

cs.RO · November 2020 · cited 297× · arXiv ↗ · more like this ·

Moving in complex environments is an essential capability of intelligent mobile robots. Decades of research and engineering have been dedicated to developing sophisticated navigation systems to move mobile robots from one point to another. Despite their overall success, a recently emerging research thrust is devoted to developing machine learning techniques to address the same problem, based in large part on the success of deep learning. However, to date, there has not been much direct comparison between the classical and emerging paradigms to this problem. In this article, we survey recent works that apply machine learning for motion planning and control in mobile robot navigation, within the context of classical navigation systems. The surveyed works are classified into different categories, which delineate the relationship of the learning approaches to classical methods. Based on…

Oriented Object Detection in Aerial Images with Box Boundary-Aware Vectors

Jingru Yi, Pengxiang Wu, Bo Liu +3 more

cs.CV · 29 Aug 2020 · cited 288× · arXiv ↗ · more like this ·

Oriented object detection in aerial images is a challenging task as the objects in aerial images are displayed in arbitrary directions and are usually densely packed. Current oriented object detection methods mainly rely on two-stage anchor-based detectors. However, the anchor-based detectors typically suffer from a severe imbalance issue between the positive and negative anchor boxes. To address this issue, in this work we extend the horizontal keypoint-based object detector to the oriented object detection task. In particular, we first detect the center keypoints of the objects, based on which we then regress the box boundary-aware vectors (BBAVectors) to capture the oriented bounding boxes. The box boundary-aware vectors are distributed in the four quadrants of a Cartesian coordinate system for all arbitrarily oriented objects. To relieve the difficulty of learning the vectors in the…

Small Object Detection Algorithm Based on Improved YOLOv8 for Remote Sensing

Yi Hao, Bo Liu, Bin Zhao +1 more

Computer Vision and Pattern Recognition · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · 5 Dec 2023 · cited 281× · DOI ↗ · more like this ·

Due to the limitations of small targets in remote sensing images such as background noise, poor information, and so on, the results of commonly used detection algorithms in small target detection is not satisfactory. To improve the accuracy of detection results, we develop an improved algorithm based on YOLOv8, called LAR-YOLOv8. First, in the feature extraction network, the local module is enhanced by using the dual-branch architecture attention mechanism, while the vision transformer block is used to maximize the representation of the feature map. Second, an attention-guided bi-directional feature pyramid network is designed to generate more discriminative information by efficiently extracting feature from the shallow network through a dynamic sparse attention mechanism, and adding top-down paths to guide the subsequent network modules for feature fusion. Finally, the RIOU loss…

Cluster synchronization in networks of coupled non-identical dynamical systems

Wenlian Lu, Bo liu, Tianping Chen

math.DS · 12 Dec 2009 · cited 241× · arXiv ↗ · more like this ·

In this paper, we study cluster synchronization in networks of coupled non-identical dynamical systems. The vertices in the same cluster have the same dynamics of uncoupled node system but the uncoupled node systems in different clusters are different. We present conditions guaranteeing cluster synchronization and investigate the relation between cluster synchronization and the unweighted graph topology. We indicate that two condition play key roles for cluster synchronization: the common inter-cluster coupling condition and the intra-cluster communication. From the latter one, we interpret the two well-known cluster synchronization schemes: self-organization and driving, by whether the edges of communication paths lie at inter or intra-cluster. By this way, we classify clusters according to whether the set of edges inter- or intra-cluster edges are removable if wanting to keep the…

Towards Playing Full MOBA Games with Deep Reinforcement Learning

Deheng Ye, Guibin Chen, Wen Zhang +15 more

cs.AI · November 2020 · cited 223× · arXiv ↗ · more like this ·

MOBA games, e.g., Honor of Kings, League of Legends, and Dota 2, pose grand challenges to AI systems such as multi-agent, enormous state-action space, complex action control, etc. Developing AI for playing MOBA games has raised much attention accordingly. However, existing work falls short in handling the raw game complexity caused by the explosion of agent combinations, i.e., lineups, when expanding the hero pool in case that OpenAI's Dota AI limits the play to a pool of only 17 heroes. As a result, full MOBA games without restrictions are far from being mastered by any existing AI system. In this paper, we propose a MOBA AI learning paradigm that methodologically enables playing full MOBA games with deep reinforcement learning. Specifically, we develop a combination of novel and existing learning techniques, including curriculum self-play learning, policy distillation, off-policy…

Deep Neural Networks for High Dimension, Low Sample Size Data

Bo Liu, Ying Wei, Yu Zhang +1 more

Artificial Intelligence · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 28 Jul 2017 · cited 195× · DOI ↗ · more like this ·

Deep neural networks (DNN) have achieved breakthroughs in applications with large sample size. However, when facing high dimension, low sample size (HDLSS) data, such as the phenotype prediction problem using genetic data in bioinformatics, DNN suffers from overfitting and high-variance gradients. In this paper, we propose a DNN model tailored for the HDLSS data, named Deep Neural Pursuit (DNP). DNP selects a subset of high dimensional features for the alleviation of overfitting and takes the average over multiple dropouts to calculate gradients with low variance. As the first DNN method applied on the HDLSS data, DNP enjoys the advantages of the high nonlinearity, the robustness to high dimensionality, the capability of learning from a small number of samples, the stability in feature selection, and the end-to-end training. We demonstrate these advantages of DNP via empirical results…

Distribution of high-dimensional entanglement via an intra-city free-space link

Fabian Steinlechner, Sebastian Ecker, Matthias Fink +5 more

quant-ph · December 2016 · cited 175× · arXiv ↗ · more like this ·

Quantum entanglement is a fundamental resource in quantum information processing and its distribution between distant parties is a key challenge in quantum communications. Increasing the dimensionality of entanglement has been shown to improve robustness and channel capacities in secure quantum communications. Here we report on the distribution of genuine high-dimensional entanglement via a 1.2-km-long free-space link across Vienna. We exploit hyperentanglement, that is, simultaneous entanglement in polarization and energy-time bases, to encode quantum information, and observe high-visibility interference for successive correlation measurements in each degree of freedom. These visibilities impose lower bounds on entanglement in each subspace individually and certify four-dimensional entanglement for the hyperentangled system. The high-fidelity transmission of high-dimensional…

SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning

Daoming Lyu, Fangkai Yang, Bo Liu +1 more

cs.AI · November 2018 · cited 174× · arXiv ↗ · more like this ·

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options.This framework features a planner -- controller -- meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components…

Finite-Sample Analysis of Proximal Gradient TD Algorithms

Bo Liu, Ji Liu, Mohammad Ghavamzadeh +2 more

cs.LG · June 2020 · cited 164× · arXiv ↗ · more like this ·

In this paper, we analyze the convergence rate of the gradient temporal difference learning (GTD) family of algorithms. Previous analyses of this class of algorithms use ODE techniques to prove asymptotic convergence, and to the best of our knowledge, no finite-sample analysis has been done. Moreover, there has been not much work on finite-sample analysis for convergent off-policy reinforcement learning algorithms. In this paper, we formulate GTD methods as stochastic gradient algorithms w.r.t.~a primal-dual saddle-point objective function, and then conduct a saddle-point error analysis to obtain finite-sample bounds on their performance. Two revised algorithms are also proposed, namely projected GTD2 and GTD2-MP, which offer improved convergence guarantees and acceleration, respectively. The results of our theoretical analysis show that the GTD family of algorithms are indeed…

PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Making

Fangkai Yang, Daoming Lyu, Bo Liu +1 more

cs.LG · April 2018 · cited 157× · arXiv ↗ · more like this ·

Reinforcement learning and symbolic planning have both been used to build intelligent autonomous agents. Reinforcement learning relies on learning from interactions with real world, which often requires an unfeasibly large amount of experience. Symbolic planning relies on manually crafted symbolic knowledge, which may not be robust to domain uncertainties and changes. In this paper we present a unified framework {\em PEORL} that integrates symbolic planning with hierarchical reinforcement learning (HRL) to cope with decision-making in a dynamic environment with uncertainties. Symbolic plans are used to guide the agent's task execution and learning, and the learned experience is fed back to symbolic knowledge to improve planning. This method leads to rapid policy search and robust symbolic plans in complex domains. The framework is tested on benchmark domains of HRL.