arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Author

Colson Driemel

13 papers · 2017–2025

2017: 12018: 12019: 52020: 32021: 02022: 02023: 02024: 12025: 2
2017peak 5 in 20192025

Frequent co-authors

Papers

showing 1–13 of 13 RSS

High-Throughput Cloud Computing with the Cloudscheduler VM Provisioning Service

F. Berghaus, Kevin Casteels, Colson Driemel +9 more

Computer Networks and Communications · Computing and Software for Big Science · · cited 7× · DOI ↗ · more like this ·

Abstract We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large-scale production particle physics workloads for many years using thousands of cores on three continents. A decade after its initial design and implementation, cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The updated cloudscheduler is more resilient and scalable, with expanded capabilities. We present an overview of the original design and then describe the new version of the distributed compute cloud system. We conclude with a review…

Sim@P1: Using Cloudscheduler for offline processing on the ATLAS HLT farm

F. Berghaus, F Brasolin, Kevin Casteels +9 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 2× · DOI ↗ · more like this ·

The Simulation at Point1 (Sim@P1) project was established in 2013 to take advantage of the Trigger and Data Acquisition High Level Trigger (HLT) farm of the ATLAS experiment at the LHC. The HLT farm is a significant compute resource, which is critical to ATLAS during data taking. This large compute resource is used to generate and process simulation data for the experiment when ATLAS is not recording data. The Sim@P1 system uses virtual machines, deployed by OpenStack, in order to isolate the resources from the ATLAS technical and control network. During the upcoming long shutdown in 2019 (LS2), the HLT farm including the Sim@P1 infrastructure will be upgraded. A previous paper on the project emphasized the need for “simple, reliable, and efficient tools” to quickly switch between data acquisition operation and offline processing. In this contribution we assess various options for…

Using a dynamic data federation for running Belle-II simulation applications in a distributed cloud environment

M. Ebert, F. Berghaus, Kevin Casteels +8 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 2× · DOI ↗ · more like this ·

The dynamic data federation software Dynafed, developed by CERN IT, provides a federated storage cluster on demand using the HTTP protocol with WebDAV extensions. Traditional storage sites which support an experiment can be added to Dynafed without requiring any changes to the site. Dynafed also supports direct access to cloud storage such as S3 and Azure. We report on the usage of Dynafed to support Belle-II production jobs running on a distributed cloud system utilizing clouds across North America. Cloudscheduler, developed by the University of Victoria HEP Research Computing group , federates Openstack, OpenNebula, Amazon, Google, and Microsoft cloud compute resources and provides them as a unified Grid site which on average runs about 3500 Belle-II production jobs in parallel. The input data for those jobs is accessible through a single endpoint, our Dynafed instance. This Dynafed…

Federating distributed storage for clouds in ATLAS

Frank Berghaus, Kevin Casteels, Alessandro Di Girolamo +12 more

cs.DC · · cited 2× · arXiv ↗ · more like this ·

Input data for applications that run in cloud computing centres can be stored at distant repositories, often with multiple copies of the popular data stored at many sites. Locating and retrieving the remote data can be challenging, and we believe that federating the storage can address this problem. A federation would locate the closest copy of the data on the basis of GeoIP information. Currently we are using the dynamic data federation Dynafed, a software solution developed by CERN IT. Dynafed supports several industry standards for connection protocols like Amazon's S3, Microsoft's Azure, as well as WebDAV and HTTP. Dynafed functions as an abstraction layer under which protocol-dependent authentication details are hidden from the user, requiring the user to only provide an X509 certificate. We have setup an instance of Dynafed and integrated it into the ATLAS data distribution…

The Dynafed data federator as a grid site storage element

M. Ebert, F. Berghaus, Kevin Casteels +9 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 1× · DOI ↗ · more like this ·

The Dynafed data federator is designed to present a dynamic and unified view of a distributed file repository. We describe our use of Dynafed to construct a production-ready WLCG storage element (SE) using existing grid storage endpoints as well as object storage. Dynafed is used as the primary SE for the Canadian distributed computing cloud systems for the Belle-II experiment, where we use it in production for reading input files. We have run up to 6,000 Belle-II jobs simultaneously on distributed cloud resources, requiring the transfer of approximately 60 TB of input data per day. Similarly, we have been using a Dynafed-based SE in pre-production testing for the ATLAS experiment. We will describe the configuration of Dynafed to make it suitable as a WLCG SE. In particular, we will highlight the improvements within Dynafed that make it possible to do checksum based file verification…

The Canadian Belle II Tier1 Raw Data Centre

M. Ebert, Colson Driemel, R. Sobie +1 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 0× · DOI ↗ · more like this ·

We present our innovative design to hosting Canada’s share of the Belle II raw data and the necessary computing infrastructure for its processing. Our approach involves disk-only grid storage solution that utilizes XRootD and ZFS, complemented by a compute framework that deploys specialized virtual machines to process the raw data. We discuss the advantages of our approach as well as the challenges encountered during deployment and commissioning.

Using S3 storage with XRootD vs Dynafed

M. Ebert, Colson Driemel, R. Sobie +1 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 0× · DOI ↗ · more like this ·

The High Energy Physics Research Computing group at UVic used Dynafed to create federated storage clusters for Belle II and ATLAS that were used by worker nodes deployed on cloud computing facilities around the world. Since the development on Dynafed stopped, XRootD was tested with S3 as backend to replace Dynafed. We will show the similarities as well as the major differences between the two systems for data transfers, checksum calculations as well as clustering of different endpoints, which may help others to efficiently make use of S3 storage as a WLCG site SE.

Managing remote cloud resources for multiple HEP VOs with cloudscheduler

Colson Driemel, M. Ebert, R. Sobie +1 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 0× · DOI ↗ · more like this ·

Cloudscheduler is a system to manage resources of local and remote compute clouds and makes those resources available to HTCondor pools. It examines the resource needs of idle jobs, then starts virtual machines (VMs), sized accordingly, on allowed clouds with available resources. Using yaml files, cloudscheduler then provisions the VMs during the boot process with all necessary tools needed to register with HTCondor and run the experiment’s jobs. Although we have run cloudscheduler in its first version for ATLAS and Belle-II workloads successfully for more than 10 years, we developed cloudscheduler version 2 (CSV2), a complete overhaul and modernization of cloudscheduler. The new system is used successfully in production for Belle-II, ATLAS, DUNE, and BABAR . In addition to using cloudscheduler version 2 as a WLCG site, we also run it as a service for other WLCG sites, and the Canadian…

Cloudscheduler: a VM provisioning system for a distributed compute cloud

R. Sobie, F. Berghaus, Kevin Casteels +9 more

Computer Networks and Communications · EPJ Web of Conferences · · cited 0× · DOI ↗ · more like this ·

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large scale production particle physics workloads for many years using thousands of cores on three continents. Cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The result is a more resilient and scalable system, with expanded capabilities.

The Dynafed data federator as grid site storage element

Leavett-Brown, Colin Roy, Colson Driemel, Furano, Fabrizio +9 more

Computer Networks and Communications · Zenodo (CERN European Organization for Nuclear Research) · · cited 0× · DOI ↗ · more like this ·

The Dynafed data federator is designed to present a dynamic and unified view of a distributed file repository. We describe our use of Dynafed to construct a production-ready WLCG storage element (SE) using existing Grid storage endpoints as well as object storage. In particular, Dynafed is used as the primary SE for the Canadian distributed computing cloud systems. Specifically, we have been using a Dynafed-based SE in production for reading input files of Belle-II jobs. We have run up to 6000 Belle-II jobs simultaneously on distributed cloud resources, requiring the transfer of approximately 60 TB of input data per day. Similarly, we have been using a Dynafed-based SE in pre-production testing for the ATLAS experiment. We will describe the configuration of Dynafed to make it suitable as a WLCG SE. In particular, we will highlight the improvements within Dynafed that make it possible to…

CloudScheduler V2: Distributed Cloud Computing in the 21st century

Leavett-Brown, Colin Roy, Colson Driemel, Danika MacDonell +8 more

Information Systems · Figshare · · cited 0× · DOI ↗ · more like this ·

The cloudscheduler VM provisioning service has been running production jobs for ATLAS and Belle II for many years using commercial and private clouds in Europe, North America and Australia. Initially released in 2009, version 1 is a single Python 2 module implementing multiple threads to poll resources and jobs, and to create and destroy virtual machine. The code is difficult to scale, maintain or extend and lacks many desirable features, such as status displays, multiple user/project management, robust error analysis and handling, and time series plotting, to name just a few examples. To address these shortcomings, our team has re-engineered the cloudscheduler VM provisioning service from the ground up. The new version, dubbed cloudscheduler version 2 or CSV2, is written in Python 3 runs on any modern Linux distribution, and uses current supporting applications and libraries. The…

Quasi-online accounting and monitoring system for distributed clouds

Rolf Seuster, F. Berghaus, Kevin Casteels +5 more

Information Systems · EPJ Web of Conferences · · cited 0× · DOI ↗ · more like this ·

The HEP group at the University of Victoria operates a distributed cloud computing system for the ATLAS and Belle II experiments. The system uses private and commercial clouds in North America and Europe that run OpenStack, Open Nebula or commercial cloud software. It is critical that we record accounting information to give credit to cloud owners and to verify our use of commercial resources. We want to record the number of CPU-hours of the virtual machine. We continuously collect the CPU usage and an estimate of the HEPSpec06 units of the VM obtained during the boot of the VM and uploads it into an Elastic Search database. The information is processed and published as soon as it is available. The data is published in tables and plots in Kibana and as a cross check in ROOT. We have found the system to be useful beyond gathering accounting information and can be used for monitoring and…

Context-aware distributed cloud computing using CloudScheduler

R. Seuster, CR Leavett-Brown, Kevin Casteels +6 more

Computer Networks and Communications · Journal of Physics Conference Series · · cited 0× · DOI ↗ · more like this ·

The distributed cloud using the CloudScheduler VM provisioning service is one of the longest running systems for HEP workloads. It has run millions of jobs for ATLAS and Belle II over the past few years using private and commercial clouds around the world. Our goal is to scale the distributed cloud to the 10,000-core level, with the ability to run any type of application (low I/O, high I/O and high memory) on any cloud. To achieve this goal, we have been implementing changes that utilize context-aware computing designs that are currently employed in the mobile communication industry. Context-awareness makes use of real-time and archived data to respond to user or system requirements. In our distributed cloud, we have many opportunistic clouds with no local HEP services, software or storage repositories. A context-aware design significantly improves the reliability and performance of our…