Author

Krtin Kumar

5 papers · 2019–2024

2019peak 3 in 20192024

Frequent co-authors

Papers

showing 1–5 of 5 RSS

Laser damage attack against optical attenuators in quantum key distribution

Anqi Huang, Ruoping Li, Vladimir Egorov +3 more

quant-ph · May 2019 · cited 73× · arXiv ↗ · more like this ·

Many quantum key distribution systems employ a laser followed by an optical attenuator to prepare weak coherent states in the source. Their mean photon number must be pre-calibrated to guarantee the security of key distribution. Here we experimentally show that this calibration can be broken with a high-power laser attack. We have tested four fiber-optic attenuator types used in quantum key distribution systems, and found that two of them exhibit a permanent decrease in attenuation after laser damage. This results in higher mean photon numbers in the prepared states and may allow an eavesdropper to compromise the key.

Distilled embedding: non-linear embedding factorization using knowledge distillation

Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2 more

Artificial Intelligence · arXiv (Cornell University) · 2 Oct 2019 · cited 8× · OpenAlex ↗ · more like this ·

Word-embeddings are a vital component of Natural Language Processing (NLP) systems and have been extensively researched. Better representations of words have come at the cost of huge memory footprints, which has made deploying NLP models on edge-devices challenging due to memory limitations. Compressing embedding matrices without sacrificing model performance is essential for successful commercial edge deployment. In this paper, we propose Distilled Embedding, an (input/output) embedding compression method based on low-rank matrix decomposition with an added non-linearity. First, we initialize the weights of our decomposition by learning to reconstruct the full word-embedding and then fine-tune on the downstream task employing knowledge distillation on the factorized embedding. We conduct extensive experimentation with various compression rates on machine translation, using different…

Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition

Vasileios Lioutas, Ahmad Rashid, Krtin Kumar +2 more

cs.CL · October 2019 · cited 8× · arXiv ↗ · more like this ·

Word-embeddings are vital components of Natural Language Processing (NLP) models and have been extensively explored. However, they consume a lot of memory which poses a challenge for edge deployment. Embedding matrices, typically, contain most of the parameters for language models and about a third for machine translation systems. In this paper, we propose Distilled Embedding, an (input/output) embedding compression method based on low-rank matrix decomposition and knowledge distillation. First, we initialize the weights of our decomposed matrices by learning to reconstruct the full pre-trained word-embedding and then fine-tune end-to-end, employing knowledge distillation on the factorized embedding. We conduct extensive experiments with various compression rates on machine translation and language modeling, using different data-sets with a shared word-embedding matrix for both…

From Fully Trained to Fully Random Embeddings: Improving Neural Machine Translation with Compact Word Embedding Tables

Krtin Kumar, Peyman Passban, Mehdi Rezagholizadeh +2 more

cs.CL · April 2021 · cited 3× · arXiv ↗ · more like this ·

Embedding matrices are key components in neural natural language processing (NLP) models that are responsible to provide numerical representations of input tokens.\footnote{In this paper words and subwords are referred to as \textit{tokens} and the term \textit{embedding} only refers to embeddings of inputs.} In this paper, we analyze the impact and utility of such matrices in the context of neural machine translation (NMT). We show that detracting syntactic and semantic information from word embeddings and running NMT systems with random embeddings is not as damaging as it initially sounds. We also show how incorporating only a limited amount of task-specific knowledge from fully-trained embeddings can boost the performance NMT systems. Our findings demonstrate that in exchange for negligible deterioration in performance, any NMT model can be run with partially random embeddings.…

Evaluating the Retrieval Component in LLM-Based Question Answering Systems

Ashkan Alinejad, Krtin Kumar, Ali Vahdat

cs.CL · 10 Jun 2024 · cited 2× · arXiv ↗ · more like this ·

Question answering systems (QA) utilizing Large Language Models (LLMs) heavily depend on the retrieval component to provide them with domain-specific information and reduce the risk of generating inaccurate responses or hallucinations. Although the evaluation of retrievers dates back to the early research in Information Retrieval, assessing their performance within LLM-based chatbots remains a challenge. This study proposes a straightforward baseline for evaluating retrievers in Retrieval-Augmented Generation (RAG)-based chatbots. Our findings demonstrate that this evaluation framework provides a better image of how the retriever performs and is more aligned with the overall performance of the QA system. Although conventional metrics such as precision, recall, and F1 score may not fully capture LLMs' capabilities - as they can yield accurate responses despite imperfect retrievers - our…