arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Valentin Kuznetsov

45 papers · 1996–2026

1996: 21997: 01998: 11999: 02000: 02001: 12002: 02003: 02004: 02005: 12006: 12007: 02008: 32009: 02010: 22011: 12012: 12013: 12014: 12015: 22016: 32017: 32018: 52019: 12020: 62021: 32022: 12023: 02024: 22025: 22026: 2
1996peak 6 in 20202026

Frequent co-authors

Papers

showing 1–25 of 45 RSS

Predicting dataset popularity for the CMS experiment

Valentin Kuznetsov, Ting Li, Luca Giommi +2 more

physics.data-an · · cited 17× · arXiv ↗ · more like this ·

The CMS experiment at the LHC accelerator at CERN relies on its computing infrastructure to stay at the frontier of High Energy Physics, searching for new phenomena and making discoveries. Even though computing plays a significant role in physics analysis we rarely use its data to predict the system behavior itself. A basic information about computing resources, user activities and site utilization can be really useful for improving the throughput of the system and its management. In this paper, we discuss a first CMS analysis of dataset popularity based on CMS meta-data which can be used as a model for dynamic data placement and provide the foundation of data-driven approach for the CMS computing infrastructure.

Data Aggregation System - a system for information retrieval on demand over relational and non-relational distributed data sources

Gareth Ball, Valentin Kuznetsov, D. Evans +1 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 12× · DOI ↗ · more like this ·

We present the Data Aggregation System, a system for information retrieval and aggregation from heterogenous sources of relational and non-relational data for the Compact Muon Solenoid experiment on the CERN Large Hadron Collider. The experiment currently has a number of organically-developed data sources, including front-ends to a number of different relational databases and non-database data services which do not share common data structures or APIs {\\it (Application Programming Interfaces)}, and cannot at this stage be readily converged. DAS provides a single interface for querying all these services, a caching layer to speed up access to expensive underlying calls and the ability to merge records from different data services pertaining to a single primary key.

Exploiting CMS data popularity to model the evolution of data management for Run-2 and beyond

D. Bonacorsi, T. Boccali, Domenico Giordano +5 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 9× · DOI ↗ · more like this ·

During the LHC Run-1 data taking, all experiments collected large data volumes from proton-proton and heavy-ion collisions. The collisions data, together with massive volumes of simulated data, were replicated in multiple copies, transferred among various Tier levels, transformed/slimmed in format/content. These data were then accessed (both locally and remotely) by large groups of distributed analysis communities exploiting the WorldWide LHC Computing Grid infrastructure and services. While efficient data placement strategies - together with optimal data redistribution and deletions on demand - have become the core of static versus dynamic data management projects, little effort has so far been invested in understanding the detailed data-access patterns which surfaced in Run-1. These patterns, if understood, can be used as input to simulation of computing models at the LHC, to optimise…

Migration of CMSWEB cluster at CERN to Kubernetes: a comprehensive study

Muhammad Imran, Valentin Kuznetsov, Katarzyna Maria Dziedziniewicz-Wojcik +5 more

Computer Networks and Communications · Cluster Computing · · cited 7× · DOI ↗ · more like this ·

Abstract The Compact Muon Solenoid (CMS) experiment heavily relies on the CMSWEB cluster to host critical services for its operational needs. The cluster is deployed on virtual machines (VMs) from the CERN OpenStack cloud and is manually maintained by operators and developers. The release cycle is composed of several steps, from building RPMs to their deployment, validation, and integration tests. To enhance the sustainability of the CMSWEB cluster, CMS decided to migrate its cluster to a containerized solution based on Docker and orchestrated with Kubernetes (K8s). This allows us to significantly speed up the release upgrade cycle, follow the end-to-end deployment procedure, and reduce operational cost. In this paper, we give an overview of the CMSWEB VM cluster and the issues we discovered during this migration. We discuss the architecture and the implementation strategy in the CMSWEB…

Operational Intelligence for Distributed Computing Systems for Exascale Science

Alessandro Di Girolamo, F. Legger, Panos Paparrigopoulos +9 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 6× · DOI ↗ · more like this ·

In the near future, large scientific collaborations will face unprecedented computing challenges. Processing and storing exabyte datasets require a federated infrastructure of distributed computing resources. The current systems have proven to be mature and capable of meeting the experiment goals, by allowing timely delivery of scientific results. However, a substantial amount of interventions from software developers, shifters and operational teams is needed to efficiently manage such heterogeneous infrastructures. A wealth of operational data can be exploited to increase the level of automation in computing operations by using adequate techniques, such as machine learning (ML), tailored to solve specific problems. The Operational Intelligence project is a joint effort from various WLCG communities aimed at increasing the level of automation in computing operations. We discuss how…

Smart Caching at CMS: applying AI to XCache edge services

D. Spiga, D. Ciangottini, Mirco Tracolli +6 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 6× · DOI ↗ · more like this ·

The projected Storage and Compute needs for the HL-LHC will be a factor up to 10 above what can be achieved by the evolution of current technology within a flat budget. The WLCG community is studying possible technical solutions to evolve the current computing in order to cope with the requirements; one of the main focus is resource optimization, with the ultimate aim of improving performance and efficiency, as well as simplifying and reducing operation costs. As of today the storage consolidation based on a Data Lake model is considered a good candidate for addressing HL-LHC data access challenges. The Data Lake model under evaluation can be seen as a logical system that hosts a distributed working set of analysis data. Compute power can be “close” to the lake, but also remote and thus completely external. In this context we expect data caching to play a central role as a technical…

Job life cycle management libraries for CMS workflow management projects

Frank van Lingen, D.A. Evans, S. Metson +9 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 6× · DOI ↗ · more like this ·

Scientific analysis and simulation requires the processing and generation of millions of data samples. These processing and generation tasks are often comprised of multiple smaller tasks divided over multiple (computing) sites. This paper discusses the Compact Muon Solenoid (CMS) workflow infrastructure, and specifically the Python based workflow library which is used for so called task lifecycle management. The CMS workflow infrastructure consists of three layers: high level specification of the various tasks based on input/output datasets, life cycle management of task instances derived from the high level specification and execution management. The workflow library is the result of a convergence of three CMS subprojects that respectively deal with scientific analysis, simulation and real time data aggregation from the experiment.

Rapid web development using AJAX and Python

A Dolgert, L. Gibbons, Valentin Kuznetsov

Information Systems · Journal of Physics Conference Series · · cited 6× · DOI ↗ · more like this ·

We discuss the rapid development of a large scale data discovery service for the CMS experiment using modern AJAX techniques and the Python language. To implement a flexible interface capable of accommodating several different versions of the DBS database, we used a 'stack' approach. Asynchronous JavaScript and XML (AJAX) together with an SQL abstraction layer, template engine, code generation tool and dynamic queries provide powerful tools for constructing interactive interfaces to large amounts of data. We show how the use of these tools, with rapid development in a modern scripting language, improved the scalability and usability of the the search interface for different user communities.

A multi-dimensional view on information retrieval of CMS data

A Dolgert, L. Gibbons, Valentin Kuznetsov +2 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 6× · DOI ↗ · more like this ·

The CMS Dataset Bookkeeping System (DBS) search page is a web-based application used by physicists and production managers to find data from the CMS experiment. The main challenge in the design of the system was to map the complex, distributed data model embodied in the DBS and the Data Location Service (DLS) to a simple, intuitive interface consistent with the mental model of physicists analyzing the data. We used focus groups and user interviews to establish the required features. The resulting interface addresses the physicist and production manager roles separately, offering both a guided search structured for the common physics use cases as well as a dynamic advanced query interface.

CMS offline web tools

S. Metson, S. Belforte, Brian Bockelman +9 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 6× · DOI ↗ · more like this ·

We describe a relatively new effort within CMS to converge on a set of web based tools, using state of the art industry techniques, to engage with the CMS offline computing system. CMS collaborators require tools to monitor various components of the computing system and interact with the system itself. The current state of the various CMS web tools is described along side current planned developments.
\nThe CMS collaboration comprises of nearly 3000 people from all over the world. As well as its collaborators, its computing resources are spread all over globe and are accessed via the LHC grid to run analysis, large scale production and data transfer tasks. Due to the distributed nature of collaborators effective provision of collaborative tools is essential to maximise physics exploitation of the CMS experiment, especially when the size of the CMS data set is considered. CMS has…

The CMS DBS query language

Valentin Kuznetsov, Daniel Riley, Anzar Afaq +3 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 5× · DOI ↗ · more like this ·

The CMS experiment has implemented a flexible and powerful system enabling users to find data within the CMS physics data catalog. The Dataset Bookkeeping Service (DBS) comprises a database and the services used to store and access metadata related to CMS physics data. To this, we have added a generalized query system in addition to the existing web and programmatic interfaces to the DBS. This query system is based on a query language that hides the complexity of the underlying database structure by discovering the join conditions between database tables. This provides a way of querying the system that is simple and straightforward for CMS data managers and physicists to use without requiring knowledge of the database tables or keys. The DBS Query Language uses the ANTLR tool to build the input query parser and tokenizer, followed by a query builder that uses a graph representation of…

Using DODAS as deployment manager for smart caching of CMS data management system

Mirco Tracolli, Marica Antonacci, T. Boccali +8 more

Information Systems · Journal of Physics Conference Series · · cited 4× · DOI ↗ · more like this ·

Abstract DODAS stands for Dynamic On Demand Analysis Service and is a Platform as a Service toolkit built around several EOSC-hub services designed to instantiate and configure on-demand container-based clusters over public or private Cloud resources. It automates the whole workflow from service provisioning to the configuration and setup of software applications. Therefore, such a solution allows using “any cloud provider”, with almost zero effort. In this paper, we demonstrate how DODAS can be adopted as a deployment manager to set up and manage the compute resources and services required to develop an AI solution for smart data caching. The smart caching layer may reduce the operational cost and increase flexibility with respect to regular centrally managed storage of the current CMS computing model. The cache space should be dynamically populated with the most requested data. In…

HEP Software Foundation Community White Paper Working Group -- Data Organization, Management and Access (DOMA)

Dario Berzano, Riccardo Maria Bianchi, Ian Bird +35 more

physics.comp-ph · · cited 4× · arXiv ↗ · more like this ·

Without significant changes to data organization, management, and access (DOMA), HEP experiments will find scientific output limited by how fast data can be accessed and digested by computational resources. In this white paper we discuss challenges in DOMA that HEP experiments, such as the HL-LHC, will face as well as potential ways to address them. A research and development timeline to assess these changes is also proposed.

Gaining insight from large data volumes with ease

Valentin Kuznetsov

physics.data-an · · cited 4× · arXiv ↗ · more like this ·

Efficient handling of large data-volumes becomes a necessity in today's world. It is driven by the desire to get more insight from the data and to gain a better understanding of user trends which can be transformed into economic incentives (profits, cost-reduction, various optimization of data workflows, and pipelines). In this paper, we discuss how modern technologies are transforming well established patterns in HEP communities. The new data insight can be achieved by embracing Big Data tools for a variety of use-cases, from analytics and monitoring to training Machine Learning models on a terabyte scale. We provide concrete examples within context of the CMS experiment where Big Data tools are already playing or would play a significant role in daily operations.

Life in extra dimensions of database world or penetration of NoSQL in HEP community

Valentin Kuznetsov, D. Evans, S. Metson

Information Systems · Journal of Physics Conference Series · · cited 4× · DOI ↗ · more like this ·

The recent buzzword in IT world is NoSQL. Major players, such as Facebook, Yahoo, Google, etc. are widely adopted different "NoSQL" solutions for their needs. Horizontal scalability, flexible data model and management of big data volumes are only a few advantages of NoSQL. In CMS experiment we use several of them in production environment. Here we present CMS projects based on NoSQL solutions, their strengths and weaknesses as well as our experience with those tools and their coexistence with standard RDMS solutions in our applications.

MLaaS4HEP: Machine Learning as a Service for HEP

Valentin Kuznetsov, Luca Giommi, Daniele Bonacorsi

hep-ex · · cited 3× · arXiv ↗ · more like this ·

Machine Learning (ML) will play a significant role in the success of the upcoming High-Luminosity LHC (HL-LHC) program at CERN. An unprecedented amount of data at the exascale will be collected by LHC experiments in the next decade, and this effort will require novel approaches to train and use ML models. In this paper, we discuss a Machine Learning as a Service pipeline for HEP (MLaaS4HEP) which provides three independent layers: a data streaming layer to read High-Energy Physics (HEP) data in their native ROOT data format; a data training layer to train ML models using distributed ROOT files; a data inference layer to serve predictions using pre-trained ML models via HTTP protocol. Such modular design opens up the possibility to train data at large scale by reading ROOT files from remote storage facilities, e.g. World-Wide LHC Computing Grid (WLCG) infrastructure, and feed the data to…

Exploiting Apache Spark platform for CMS computing analytics

Marco Meoni, Valentin Kuznetsov, Luca Menichetti +3 more

physics.data-an · · cited 3× · arXiv ↗ · more like this ·

The CERN IT provides a set of Hadoop clusters featuring more than 5 PBytes of raw storage with different open-source, user-level tools available for analytical purposes. The CMS experiment started collecting a large set of computing meta-data, e.g. dataset, file access logs, since 2015. These records represent a valuable, yet scarcely investigated, set of information that needs to be cleaned, categorized and analyzed. CMS can use this information to discover useful patterns and enhance the overall efficiency of the distributed data, improve CPU and site utilization as well as tasks completion time. Here we present evaluation of Apache Spark platform for CMS needs. We discuss two main use-cases CMS analytics and ML studies where efficient process billions of records stored on HDFS plays an important role. We demonstrate that both Scala and Python (PySpark) APIs can be successfully used…

Three Case Studies of Large-Scale Data Flows

William Y. Arms, Selcuk Aya, Manuel Calimlim +9 more

Computer Networks and Communications · · cited 3× · DOI ↗ · more like this ·

We survey three examples of large-scale scientific workflows that we are working with at Cornell: the Arecibo sky survey, the CLEO high-energy particle physics experiment, and the Web Lab project for enabling social science studies of the Internet. All three projects face the same general challenges: massive amounts of raw data, expensive processing steps, and the requirement to make raw data or data products available to users nation- or world-wide. However, there are several differences that prevent a one-sizefits- all approach to handling their data flows. Instead, current implementations are heavily tuned by domain and data management experts. We describe the three projects, and we outline research issues and opportunities to integrate Grid technology into these workflows.

Implementation of New Security Features in CMSWEB Kubernetes Cluster at CERN

Aamir Ali, Muhammad Imran, Valentin Kuznetsov +4 more

cs.CR · · cited 2× · arXiv ↗ · more like this ·

The CMSWEB cluster is pivotal to the activities of the Compact Muon Solenoid (CMS) experiment, as it hosts critical services required for the operational needs of the CMS experiment. The security of these services and the corresponding data is crucial to CMS. Any malicious attack can compromise the availability of our services. Therefore, it is important to construct a robust security infrastructure. In this work, we discuss new security features introduced to the CMSWEB Kubernetes ("k8s") cluster. The new features include the implementation of network policies, deployment of Open Policy Agent (OPA), enforcement of OPA policies, and the integration of Vault. The network policies act as an inside-the-cluster firewall to limit the network communication between the pods to the minimum necessary, and its dynamic nature allows us to work with microservices. The OPA validates the objects…

Migration of CMSWEB Cluster at CERN to Kubernetes

Muhammad Imran, Valentin Kuznetsov, Lina Marcella +3 more

cs.DC · · cited 2× · arXiv ↗ · more like this ·

The CMS experiment heavily relies on the CMSWEB cluster to host critical services for its operational needs. The cluster is deployed on virtual machines (VMs) from the CERN OpenStack cloud and is manually maintained by operators and developers. The release cycle is composed of several steps, from building RPMs, their deployment to perform validation, and integration tests. To enhance the sustainability of the CMSWEB cluster, CMS decided to migrate its cluster to a containerized solution such as Docker, orchestrated with Kubernetes (k8s). This allows us to significantly reduce the release upgrade cycle, follow the end-to-end deployment procedure, and reduce operational cost. This paper gives an overview of the current CMSWEB cluster and its issues. We describe the new architecture of the CMSWEB cluster in Kubernetes. We also provide a comparison of VM and Kubernetes deployment approaches…

The evolution of the CMS monitoring infrastructure

Christian Ariza-Porras, Valentin Kuznetsov, F. Legger +3 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 2× · DOI ↗ · more like this ·

The CMS experiment at the CERN LHC (Large Hadron Collider) relies on a distributed computing infrastructure to process the multi-petabyte datasets where the collision and simulated data are stored. A scalable and reliable monitoring system is required to ensure efficient operation of the distributed computing services, and to provide a comprehensive set of measurements of the system performances. In this paper we present the full stack of CMS monitoring applications, partly based on the MONIT infrastructure, a suite of monitoring services provided by the CERN IT department. These are complemented by a set of applications developed over the last few years by CMS, leveraging open-source technologies that are industry-standards in the IT world, such as Kubernetes and Prometheus. We discuss how this choice helped the adoption of common monitoring solutions within the experiment, and…

Developments on the “Machine Learning as a Service for High Energy Physics” Framework and Related Cloud Native Solution

L. Giommi, D. Spiga, Mattia Paladino +2 more

Information Systems · IEEE Transactions on Cloud Computing · · cited 1× · DOI ↗ · more like this ·

Machine Learning (ML) techniques have been successfully used in many areas of High Energy Physics (HEP) and will play a significant role in the success of upcoming High-Luminosity Large Hadron Collider (HL-LHC) program at CERN. An unprecedented amount of data at the exascale will be collected by LHC experiments in the next decade, and this effort will require novel approaches to train and use ML models. The work presented in this paper is focused on the developments of a ML as a Service (MLaaS) solution for HEP, aiming to provide a cloud service that allows HEP users to run ML pipelines via HTTPs calls. These pipelines are executed by using MLaaS4HEP framework, which allows reading data, processing data, and training ML models directly using ROOT files of arbitrary size from local or distributed data sources. In particular, new features implemented on the framework will be presented as…

The CMS monitoring applications for LHC Run 3

B. K. Jashal, Valentin Kuznetsov, F. Legger +1 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 1× · DOI ↗ · more like this ·

Data taking at the Large Hadron Collider (LHC) at CERN restarted in 2022. The CMS experiment relies on a distributed computing infrastructure based on WLCG (Worldwide LHC Computing Grid) to support the LHC Run 3 physics program. The CMS computing infrastructure is highly heterogeneous and relies on a set of centrally provided services, such as distributed workload management and data management, and computing resources hosted at almost 150 sites worldwide. Smooth data taking and processing requires all computing subsystems to be fully operational, and available computing and storage resources need to be continuously monitored. During the long shutdown between LHC Run 2 and Run 3, the CMS monitoring infrastructure has undergone major changes to increase the coverage of monitored applications and services, while becoming more sustainable and easier to operate and maintain. The used…

Modern High-Performance Methods for Special Processing of Arms, Military and Special Equipment

Valentin Kuznetsov, P. E. Belyakov, S. A. Sharov +1 more

Development · Journal of NBC Protection Corps · · cited 1× · DOI ↗ · more like this ·

Carrying out special treatment is one of the main measures to maintain the combat capability of troops in conditions of chemical and biological contamination or radiation contamination of the area. The purpose of the work was to assess the current state of the means of special processing of military equipment and to identify possible ways to improve their effectiveness. The existing means of special processing of military equipment basically meet the requirements, however, in modern conditions they have a number of shortcomings and need to be improved. Thus, aqueous solutions of degassing and disinfecting substances and organic formulations of special treatment are used with a relatively high consumption rate of up to 4,5 l/m 2 and 0,6 l/m 2 , respectively, while the treatment time for 1 m 2 can be 1 minute, which is related as with the physicochemical properties of the solutions and…

The CMS monitoring infrastructure and applications

Christian Ariza-Porras, Valentin Kuznetsov, Federica Legger

cs.SE · · cited 1× · arXiv ↗ · more like this ·

The globally distributed computing infrastructure required to cope with the multi-petabytes datasets produced by the Compact Muon Solenoid (CMS) experiment at the Large Hadron Collider (LHC) at CERN comprises several subsystems, such as workload management, data management, data transfers, and submission of users' and centrally managed production requests. The performance and status of all subsystems must be constantly monitored to guarantee the efficient operation of the whole infrastructure. Moreover, key metrics need to be tracked to evaluate and study the system performance over time. The CMS monitoring architecture allows both real-time and historical monitoring of a variety of data sources and is based on scalable and open source solutions tailored to satisfy the experiment's monitoring needs. We present the monitoring data flow and software architecture for the CMS distributed…