arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Shin

7,997 papers · 1941–2024

1965: 11966: 11967: 31968: 11969: 01970: 01971: 11972: 21973: 11974: 11975: 01976: 11977: 61978: 51979: 21980: 61981: 61982: 51983: 111984: 41985: 131986: 61987: 111988: 151989: 141990: 191991: 251992: 251993: 271994: 431995: 661996: 701997: 961998: 971999: 1172000: 1482001: 1402002: 3112003: 2662004: 2852005: 3302006: 3542007: 3382008: 3702009: 3762010: 4312011: 4452012: 4142013: 4272014: 4212015: 4592016: 2532017: 3162018: 3502019: 4122020: 3832021: 532022: 12023: 12024: 2
1965peak 459 in 20152024

Frequent co-authors

Papers

showing 1–25 of 7,997 RSS

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama +1 more

stat.ML · · cited 3,128× · arXiv ↗ · more like this ·

We propose a new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input. Virtual adversarial loss is defined as the robustness of the conditional label distribution around each input data point against local perturbation. Unlike adversarial training, our method defines the adversarial direction without label information and is hence applicable to semi-supervised learning. Because the directions in which we smooth the model are only "virtually" adversarial, we call our method virtual adversarial training (VAT). The computational cost of VAT is relatively low. For neural networks, the approximated gradient of virtual adversarial loss can be computed with no more than two pairs of forward- and back-propagations. In our experiments, we applied VAT to supervised and semi-supervised learning tasks on…

A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attacks

Kimin Lee, Kibok Lee, Honglak Lee +1 more

stat.ML · · cited 2,886× · arXiv ↗ · more like this ·

Detecting test samples drawn sufficiently far away from the training distribution statistically or adversarially is a fundamental requirement for deploying a good classifier in many real-world machine learning applications. However, deep neural networks with the softmax classifier are known to produce highly overconfident posterior distributions even for such abnormal samples. In this paper, we propose a simple yet effective method for detecting any abnormal samples, which is applicable to any pre-trained softmax neural classifier. We obtain the class conditional Gaussian distributions with respect to (low- and upper-level) features of the deep models under Gaussian discriminant analysis, which result in a confidence score based on the Mahalanobis distance. While most prior methods have been evaluated for detecting either out-of-distribution or adversarial samples, but not both, the…

Continual Learning with Deep Generative Replay

Hanul Shin, Jung Kwon Lee, Jaehong Kim +1 more

cs.AI · · cited 2,602× · arXiv ↗ · more like this ·

Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.

Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities

Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal

cs.DC · · cited 1,887× · arXiv ↗ · more like this ·

This keynote paper: presents a 21st century vision of computing; identifies various computing paradigms promising to deliver the vision of computing utilities; defines Cloud computing and provides the architecture for creating market-oriented Clouds by leveraging technologies such as VMs; provides thoughts on market-based resource management strategies that encompass both customer-driven service management and computational risk management to sustain SLA-oriented resource allocation; presents some representative Cloud platforms especially those developed in industries along with our current work towards realising market-oriented resource allocation of Clouds by leveraging the 3rd generation Aneka enterprise Grid technology; reveals our early thoughts on interconnecting Clouds for dynamically creating an atmospheric computing environment along with pointers to future community research…

Technological Pedagogical Content Knowledge (TPACK)

Denise Schmidt, Evrim Baran, Ann D. Thompson +3 more

Education · Journal of Research on Technology in Education · · cited 1,785× · DOI ↗ · more like this ·

Based in Shulman’s idea of Pedagogical Content Knowledge, Technological Pedagogical Content Knowledge (TPACK) has emerged as a useful frame for describing and understanding the goals for technology use in preservice teacher education. This paper addresses the need for a survey instrument designed to assess TPACK for preservice teachers. The paper describes survey development process and results from a pilot study on 124 preservice teachers. Data analysis procedures included Cronbach’s alpha statistics on the TPACK knowledge domains and factor analysis for each domain. Results suggest that, with the modification and/or deletion of 18 of the survey items, the survey is a reliable and valid instrument that will help educators design longitudinal studies to assess preservice teachers’ development of TPACK.

Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samples

Kimin Lee, Honglak Lee, Kibok Lee +1 more

stat.ML · · cited 971× · arXiv ↗ · more like this ·

The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their predictions, i.e., do not distinguish in- and out-of-distributions. Recently, to handle this issue, several threshold-based detectors have been proposed given pre-trained neural classifiers. However, the performance of prior works highly depends on how to train the classifiers since they only focus on improving inference procedures. In this paper, we develop a novel training method for classifiers so that such inference algorithms can work better. In particular, we suggest two additional terms added to the original loss (e.g., cross entropy). The first one forces samples…

Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications

Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo +3 more

cs.CV · · cited 957× · arXiv ↗ · more like this ·

Although the latest high-end smartphone has powerful CPU and GPU, running deeper convolutional neural networks (CNNs) for complex tasks such as ImageNet classification on mobile devices is challenging. To deploy deep CNNs on mobile devices, we present a simple and effective scheme to compress the entire CNN, which we call one-shot whole network compression. The proposed scheme consists of three steps: (1) rank selection with variational Bayesian matrix factorization, (2) Tucker decomposition on kernel tensor, and (3) fine-tuning to recover accumulated loss of accuracy, and each step can be easily implemented using publicly available tools. We demonstrate the effectiveness of the proposed scheme by testing the performance of various compressed CNNs (AlexNet, VGGS, GoogLeNet, and VGG-16) on the smartphone. Significant reductions in model size, runtime, and energy consumption are obtained…

Student Satisfaction: An alternative approach to assessing this important concept

Kevin M. Elliott, Dooyoung Shin

Education · Journal of Higher Education Policy and Management · · cited 917× · DOI ↗ · more like this ·

A student's overall satisfaction with his/her educational experience has traditionally been measured by a simple 'yes or no' question, or with one question assessing the degree of overall satisfaction. Even though this type of question is simple to answer and analyze, students may not thoroughly reflect upon their previous responses within a questionnaire regarding satisfaction with individual educational attributes when asked to assess their overall satisfaction with a university on the basis of a single question. The purpose of this article is to present an alternative approach to measuring student overall satisfaction using a multiple-item weighted gap score analysis approach. The results suggest that this approach may have more diagnostic value to both academics and practitioners.

DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation

Holger R. Roth, Le Lu, Amal Farag +4 more

cs.CV · · cited 885× · arXiv ↗ · more like this ·

Automatic organ segmentation is an important yet challenging problem for medical image analysis. The pancreas is an abdominal organ with very high anatomical variability. This inhibits previous segmentation methods from achieving high accuracies, especially compared to other organs such as the liver, heart or kidneys. In this paper, we present a probabilistic bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans, using multi-level deep convolutional networks (ConvNets). We propose and evaluate several variations of deep ConvNets in the context of hierarchical, coarse-to-fine classification on image patches and regions, i.e. superpixels. We first present a dense labeling of local image patches via $P{-}\mathrm{ConvNet}$ and nearest neighbor fusion. Then we describe a regional ConvNet ($R_1{-}\mathrm{ConvNet}$) that samples a set of bounding boxes around…

Embedding Watermarks into Deep Neural Networks

Yusuke Uchida, Yuki Nagai, Shigeyuki Sakazawa +1 more

cs.CV · · cited 805× · arXiv ↗ · more like this ·

Deep neural networks have recently achieved significant progress. Sharing trained models of these deep neural networks is very important in the rapid progress of researching or developing deep neural network systems. At the same time, it is necessary to protect the rights of shared trained models. To this end, we propose to use a digital watermarking technology to protect intellectual property or detect intellectual property infringement of trained models. Firstly, we formulate a new problem: embedding watermarks into deep neural networks. We also define requirements, embedding situations, and attack types for watermarking to deep neural networks. Secondly, we propose a general framework to embed a watermark into model parameters using a parameter regularizer. Our approach does not hurt the performance of networks into which a watermark is embedded. Finally, we perform comprehensive…

CSI: Novelty Detection via Contrastive Learning on Distributionally Shifted Instances

Jihoon Tack, Sangwoo Mo, Jongheon Jeong +1 more

cs.LG · · cited 755× · arXiv ↗ · more like this ·

Novelty detection, i.e., identifying whether a given sample is drawn from outside the training distribution, is essential for reliable machine learning. To this end, there have been many attempts at learning a representation well-suited for novelty detection and designing a score based on such representation. In this paper, we propose a simple, yet effective method named contrasting shifted instances (CSI), inspired by the recent success on contrastive learning of visual representations. Specifically, in addition to contrasting a given sample with other instances as in conventional contrastive learning methods, our training scheme contrasts the sample with distributionally-shifted augmentations of itself. Based on this, we propose a new detection score that is specific to the proposed training scheme. Our experiments demonstrate the superiority of our method under various novelty…

Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks

Hoo-Chang Shin, Neil A Tenenholtz, Jameson K Rogers +5 more

cs.CV · · cited 688× · arXiv ↗ · more like this ·

Data diversity is critical to success when training deep learning models. Medical imaging data sets are often imbalanced as pathologic findings are generally rare, which introduces significant challenges when training deep learning models. In this work, we propose a method to generate synthetic abnormal MRI images with brain tumors by training a generative adversarial network using two publicly available data sets of brain MRI. We demonstrate two unique benefits that the synthetic images provide. First, we illustrate improved performance on tumor segmentation by leveraging the synthetic images as a form of data augmentation. Second, we demonstrate the value of generative models as an anonymization tool, achieving comparable tumor segmentation results when trained on the synthetic data versus when trained on real subject data. Together, these results offer a potential solution to two of…

Steady State Thermodynamics of Langevin Systems

Takahiro Hatano, Shin-ichi Sasa

cond-mat.stat-mech · · cited 650× · arXiv ↗ · more like this ·

We study Langevin dynamics describing nonequilibirum steady states. Employing the phenomenological framework of steady state thermodynamics constructed by Oono and Paniconi [Prog. Theor. Phys. Suppl. {\bf130}, 29 (1998)], we find that the extended form of the second law which they proposed holds for transitions between steady states and that the Shannon entropy difference is related to the excess heat produced in an infinitely slow operation. A generalized version of the Jarzynski work relation plays an important role in our theory.

Performance guarantees for Web server end-systems: a control-theoretical approach

Tarek Abdelzaher, Kang G. Shin, N. Bhatti

Hardware and Architecture · IEEE Transactions on Parallel and Distributed Systems · · cited 595× · DOI ↗ · more like this ·

The Internet is undergoing substantial changes from a communication and browsing infrastructure to a medium for conducting business and marketing a myriad of services. The World Wide Web provides a uniform and widely-accepted application interface used by these services to reach multitudes of clients. These changes place the Web server at the center of a gradually emerging e-service infrastructure with increasing requirements for service quality and reliability guarantees in an unpredictable and highly-dynamic environment. This paper describes performance control of a Web server using classical feedback control theory. We use feedback control theory to achieve overload protection, performance guarantees, and service differentiation in the presence of load unpredictability. We show that feedback control theory offers a promising analytic foundation for providing service differentiation…

AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts

Taylor Shin, Yasaman Razeghi, Robert L. Logan IV +2 more

cs.CL · · cited 579× · arXiv ↗ · more like this ·

The remarkable success of pretrained language models has motivated the study of what kinds of knowledge these models learn during pretraining. Reformulating tasks as fill-in-the-blanks problems (e.g., cloze tests) is a natural approach for gauging such knowledge, however, its usage is limited by the manual effort and guesswork required to write suitable prompts. To address this, we develop AutoPrompt, an automated method to create prompts for a diverse set of tasks, based on a gradient-guided search. Using AutoPrompt, we show that masked language models (MLMs) have an inherent capability to perform sentiment analysis and natural language inference without additional parameters or finetuning, sometimes achieving performance on par with recent state-of-the-art supervised models. We also show that our prompts elicit more accurate factual knowledge from MLMs than the manually created…

Hop-count filtering

Cheng Jin, Haining Wang, Kang G. Shin

Computer Networks and Communications · · cited 473× · DOI ↗ · more like this ·

IP spoofing has been exploited by Distributed Denial of Service (DDoS) attacks to (1) conceal flooding sources and localities in flooding traffic, and (2) coax legitimate hosts into becoming reflectors, redirecting and amplifying flooding traffic. Thus, the ability to filter spoofed IP packets near victims is essential to their own protection as well as to their avoidance of becoming involuntary DoS reflectors. Although an attacker can forge any field in the IP header, he or she cannot falsify the number of hops an IP packet takes to reach its destination. This hop-count information can be inferred from the Time-to-Live (TTL) value in the IP header. Using a mapping between IP addresses and their hop-counts to an Internet server, the server can distinguish spoofed IP packets from legitimate ones. Base on this observation, we present a novel filtering technique that is immediately…

Distributional Smoothing with Virtual Adversarial Training

Takeru Miyato, Shin-ichi Maeda, Masanori Koyama +2 more

stat.ML · · cited 469× · arXiv ↗ · more like this ·

We propose local distributional smoothness (LDS), a new notion of smoothness for statistical model that can be used as a regularization term to promote the smoothness of the model distribution. We named the LDS based regularization as virtual adversarial training (VAT). The LDS of a model at an input datapoint is defined as the KL-divergence based robustness of the model distribution against local perturbation around the datapoint. VAT resembles adversarial training, but distinguishes itself in that it determines the adversarial direction from the model distribution alone without using the label information, making it applicable to semi-supervised learning. The computational cost for VAT is relatively low. For neural network, the approximated gradient of the LDS can be computed with no more than three pairs of forward and back propagations. When we applied our technique to supervised…

AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks

Chun-Chen Tu, Paishun Ting, Pin-Yu Chen +5 more

cs.CV · · cited 458× · arXiv ↗ · more like this ·

Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire the input-output correspondences of the target model; this is the so-called black-box attack setting. The major drawback of existing black-box attacks is the need for excessive model queries, which may give a false sense of model robustness due to inefficient query designs. To bridge this gap, we propose a generic framework for query-efficient black-box attacks. Our framework, AutoZOOM, which is short for Autoencoder-based Zeroth Order Optimization Method, has two novel building blocks towards efficient black-box attacks: (i) an adaptive random gradient estimation strategy…

Mathematics Achievement of Chinese, Japanese, and American Children: Ten Years Later

Harold W. Stevenson, Chuansheng Chen, Shin-Ying Lee

Education · Science · · cited 432× · DOI ↗ · more like this ·

A decade of heightened emphasis in the United States on mathematics and science education has had little influence on academic achievement or parental attitudes. American elementary school children in 1990 lagged behind their Chinese and Japanese peers to as great a degree as they did in 1980. Comparison of the performance of elementary and secondary school students between 1980 and 1990 reveals a decline from first to eleventh grade in the relative position of American students in mathematics. Parental satisfaction with American students' achievement and education remains high and standards remain low. Innate ability continues to be emphasized by Americans as a basis for achievement. American eleventh graders report more indications of stress than do their Chinese and Japanese counterparts.

Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning

Hamza Harkous, Kassem Fawaz, Rémi Lebret +3 more

cs.CL · · cited 427× · arXiv ↗ · more like this ·

Privacy policies are the primary channel through which companies inform users about their data collection and sharing practices. These policies are often long and difficult to comprehend. Short notices based on information extracted from privacy policies have been shown to be useful but face a significant scalability hurdle, given the number of policies and their evolution over time. Companies, users, researchers, and regulators still lack usable and scalable tools to cope with the breadth and depth of privacy policies. To address these hurdles, we propose an automated framework for privacy policy analysis (Polisis). It enables scalable, dynamic, and multi-dimensional queries on natural language privacy policies. At the core of Polisis is a privacy-centric language model, built with 130K privacy policies, and a novel hierarchy of neural-network classifiers that accounts for both…

Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision

Fei Pan, Inkyu Shin, Francois Rameau +2 more

cs.CV · · cited 415× · arXiv ↗ · more like this ·

Convolutional neural network-based approaches have achieved remarkable progress in semantic segmentation. However, these approaches heavily rely on annotated data which are labor intensive. To cope with this limitation, automatically annotated data generated from graphic engines are used to train segmentation models. However, the models trained from synthetic data are difficult to transfer to real images. To tackle this issue, previous works have considered directly adapting models from the source data to the unlabeled target data (to reduce the inter-domain gap). Nonetheless, these techniques do not consider the large distribution gap among the target data itself (intra-domain gap). In this work, we propose a two-step self-supervised domain adaptation approach to minimize the inter-domain and intra-domain gap together. First, we conduct the inter-domain adaptation of the model; from…

Collecting and Analyzing Multidimensional Data with Local Differential Privacy

Ning Wang, Xiaokui Xiao, Yin Yang +5 more

cs.CR · · cited 406× · arXiv ↗ · more like this ·

Local differential privacy (LDP) is a recently proposed privacy standard for collecting and analyzing data, which has been used, e.g., in the Chrome browser, iOS and macOS. In LDP, each user perturbs her information locally, and only sends the randomized version to an aggregator who performs analyses, which protects both the users and the aggregator against private information leaks. Although LDP has attracted much research attention in recent years, the majority of existing work focuses on applying LDP to complex data and/or analysis tasks. In this paper, we point out that the fundamental problem of collecting multidimensional data under LDP has not been addressed sufficiently, and there remains much room for improvement even for basic tasks such as computing the mean value over a single numeric attribute under LDP. Motivated by this, we first propose novel LDP mechanisms for…

Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation

Hoo-Chang Shin, Kirk Roberts, Le Lu +3 more

cs.CV · · cited 397× · arXiv ↗ · more like this ·

Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization techniques to circumvent the large normal-vs-diseased cases bias. Recurrent neural networks (RNNs) are then trained to describe the contexts of a detected disease, based on the deep CNN features. Moreover, we introduce a novel approach to use the weights of the already trained pair of CNN/RNN on the…

How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?

Junghwan Cho, Kyewook Lee, Ellie Shin +2 more

cs.LG · · cited 390× · arXiv ↗ · more like this ·

The use of Convolutional Neural Networks (CNN) in natural image classification systems has produced very impressive results. Combined with the inherent nature of medical images that make them ideal for deep-learning, further application of such systems to medical image classification holds much promise. However, the usefulness and potential impact of such a system can be completely negated if it does not reach a target accuracy. In this paper, we present a study on determining the optimum size of the training data set necessary to achieve high classification accuracy with low variance in medical image classification systems. The CNN was applied to classify axial Computed Tomography (CT) images into six anatomical classes. We trained the CNN using six different sizes of training data set (5, 10, 20, 50, 100, and 200) and then tested the resulting system with a total of 6000 CT images.…

Reducing Gender Bias in Abusive Language Detection

Ji Ho Park, Jamin Shin, Pascale Fung

cs.CL · · cited 388× · arXiv ↗ · more like this ·

Abusive language detection models tend to have a problem of being biased toward identity words of a certain group of people because of imbalanced training datasets. For example, "You are a good woman" was considered "sexist" when trained on an existing dataset. Such model bias is an obstacle for models to be robust enough for practical use. In this work, we measure gender biases on models trained with different abusive language datasets, while analyzing the effect of different pre-trained word embeddings and model architectures. We also experiment with three bias mitigation methods: (1) debiased word embeddings, (2) gender swap data augmentation, and (3) fine-tuning with a larger corpus. These methods can effectively reduce gender bias by 90-98% and can be extended to correct model bias in other scenarios.