# REDVID Collision Event Data – Tracks and Hits *dr. ir. Uraz Odyurt* ## Introduction REDuced VIrtual Detector (REDVID) is a simulation framework and a synthetic data generator written in Python. The generator simulates the propagation of subatomic particles, inspired by the detectors installed at the Large Hadron Collider (LHC). The simulation model is complexity-reduced and is intended for generating source data for Machine Learning (ML) algorithms. For further information, refer to the [REDVID website](https://virtualdetector.com/project/redvid/). ## Detector geometry A detector's geometry consists of multiple layers of sub-detectors of different shapes, belonging to different categories. Each category defines the relevant shape of a sub-detector. However, dimensions and the placement of different layers, relative to the detector origin, will vary within each category. These details, including which categories of sub-detectors are present, are provided in one or more configuration files. ### 2D structure As a result of the oversimplified design of this 2D variant, sub-detector categories have minute differences. In essence, all sub-detector layers are circles centred at the origin. Although the simulator supports filled or partially filled circle designs, the only filled sub-detector is the innermost circle, i.e., the Pixel sub-detector. The following sub-detector categories are available for a 2D variant: - Pixel - Short-strip - Long-strip ### 3D structure The 3D variant is a significant step towards closing the gap with real-world detector apparatus. The available sub-detectors take the form of disks and cylinders; a disk can be regarded as a special case of a cylinder. The following sub-detector categories are available for a 3D variant: - Pixel - Short-strip - Long-strip - Barrel The Pixel and the Barrel categories are of the cylindrical shape, with the Pixel being a filled shape and the Barrel being only a shell and without the end-caps. These two categories are centred at the origin. Both the Short-strip and the Long-strip categories consist of sub-detectors in the shape of disks. These disks are centred on the Z-axis. There are multiple such disks, which are mirrored relative to the XY-plane. In other words, each disk has an identical twin on the opposite side of the XY-plane. The shape, the size and the orientation of these pairs are the same. All sub-detector shapes, regardless of their category, are positioned around the Z-axis, i.e., the Z-axis goes through their centre. ## Data description The generated data includes information on geometry, tracks and hits from experiments. While the information on the geometry is included, the bulk of the data set covers data for tracks and hits. There are differences in the data generated for the 2D and the 3D structures. Tracks and hits belonging to the experiments performed on the 2D structure are given by means of function coefficients and point coordinates in the Cartesian coordinate system, respectively. Additionally, conversion to the polar coordinate system is included. Tracks and hits belonging to experiments performed on the 3D structure, on the other hand, are defined as parameters of line equations and point coordinates in the cylindrical coordinate system, respectively. ### Folder structure The generated folder structure, holding the data files, is as follows: ```text ANCHOR_PATH |-- detector_ |-- detector_ |-- experiment__ |-- events_all |-- hits__events_all.csv |-- hits_and_tracks__events_all.csv |-- hits_and_tracks_polar__events_all.csv |-- tracks__events_all.csv |-- events_individual |-- event__ |-- hits__.csv |-- tracks__.csv |-- ... |-- report |-- dataset__report.txt |-- geometry_ |-- geometry_.csv ``` ### File combinations The generated data is saved in multiple CSV files, with the same data being replicated in three file combinations. The user can opt for any of these combinations and will end up with a complete set. These file combinations are as follows: - The complete collection of files per event, i.e., all `hits__.csv` and `tracks__.csv` files under the `events_individual` folder tree are to be considered. - The files `hits__events_all.csv` and `tracks__events_all.csv`, residing inside the `events_all` folder, have to be considered. - The file `hits_and_tracks__events_all.csv`, residing inside the `events_all` folder, has to be considered. ### Conversions to polar coordinate system for the 2D structure A post-generation step exists, performing the conversion of the original hit point coordinates from the Cartesian system into the polar coordinate system, as well as the conversion of the track line slope to a degree given in radians. The extra headers generated as a result of this step are appended as new columns to the most comprehensive data file, `hits_and_tracks__events_all.csv`. The resulting extended CSV file is saved as `hits_and_tracks_polar__events_all.csv` in the same location. ### Data headers for the 2D structure Data headers, i.e., CSV column titles, apply to all CSV files. Different CSV files include different subsets of the headers, depending on the contained data. These headers are as follows: 1. `event_id` - An incremental identifier for events belonging to an experiment, which is unique within the scope of the experiment. **Type:** `integer` 2. `sub_detector_id` - An incremental identifier for different sub-detector layers belonging to a geometry, which is unique within the scope of the geometry. **Type:** `integer` 3. `sub_detector_type` - The type of the sub-detector layer recording a hit, which can be one of three available types: `pixel`, `short-strip`, or `long-strip`. **Type:** `string` 4. `track_id` - An incremental identifier for tracks belonging to an event, which is unique within the scope of the event. **Type:** `integer` 5. `track_type` - Indicates the type of function defining the track in terms of polynomial degree. At the moment, all tracks are `linear`. **Type:** `string` 6. `coefficient_1` - The first track polynomial function coefficient. Not applicable. **Type:** `float` 7. `coefficient_2` - The second track polynomial function coefficient. Not applicable. **Type:** `float` 8. `slope` - The third track polynomial function coefficient (`coefficient_3`), i.e., slope. **Type:** `float` 9. `y_intercept` - The fourth track polynomial function coefficient (`coefficient_4`), i.e., y-intercept. **Type:** `float` 10. `hit_id` - An incremental identifier for hits belonging to an event, which is unique within the scope of the event. **Type:** `integer` 11. `hit_x` - The X coordinate of the hit, in the Cartesian coordinate system. **Type:** `float` 12. `hit_y` - The Y coordinate of the hit, in the Cartesian coordinate system. **Type:** `float` 13. `track_theta` (polar) - The slope degree of the track line in radians. **Type:** `float` 14. `hit_r` (polar) - The vector radius, or in other words, the distance of the hit from the origin. **Type:** `float` 15. `hit_theta` (polar) - The hit vector slope degree in radians. As a result of the added random noise during data generation, this value is slightly different compared to the `track_theta` value. **Type:** `float` The header inclusion map for different files is as follows: - `hits__.csv` - 1, 2, 3, 4, 10, 11, 12 - `tracks__.csv` - 1, 4, 5, 6, 7, 8, 9 - `hits__events_all.csv` - 1, 2, 3, 4, 10, 11, 12 - `tracks__events_all.csv` - 1, 4, 5, 6, 7, 8, 9 - `hits_and_tracks__events_all.csv` - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 - `hits_and_tracks_polar__events_all.csv` - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ### Data headers for the 3D structure Data headers, i.e., CSV column titles, apply to all CSV files. Different CSV files include different subsets of the headers, depending on the contained data. These headers are as follows: 1. `event_id` - An incremental identifier for events belonging to an experiment, which is unique within the scope of the experiment. **Type:** `integer` 2. `sub_detector_id` - An incremental identifier for different sub-detector layers belonging to a geometry, which is unique within the scope of the geometry. **Type:** `integer` 3. `sub_detector_type` - The type of the sub-detector layer recording a hit, which can be one of four available types: `pixel`, `short-strip`, `long-strip`, or `barrel`. **Type:** `string` 4. `track_id` - An incremental identifier for tracks belonging to an event, which is unique within the scope of the event. **Type:** `integer` 5. `track_type` - Indicates the type of function defining the track in terms of polynomial degree. Available types are `linear`, `helical_uniform` and `helical_expanding`. **Type:** `string` 6. `r_0` or `radial_const` - The `r` coordinate of the `(r, theta, z)` tuple defining the point `P_0`, used in a track's parametric set of equations. The value will represent origin smearing for `r`. `r_0` and `radial_const` are applicable to `linear` and `helical_expanding` track types, respectively. **Type:** `float` 7. `theta_0` or `azimuthal_const` - The `theta` coordinate of the `(r, theta, z)` tuple defining the point `P_0`, used in a track's parametric set of equations. The value will represent origin smearing for `theta`. `theta_0` and `azimuthal_const` are applicable to `linear` and `helical_expanding` track types, respectively. **Type:** `float` 8. `z_0` or `pitch_const` - The `z` coordinate of the `(r, theta, z)` tuple defining the point `P_0`, used in a track's parametric set of equations. The value will represent origin smearing for `z`. `z_0` and `pitch_const` are applicable to `linear` and `helical_expanding` track types, respectively. **Type:** `float` 9. `r_d` - The `r` coordinate of the `(r, theta, z)` tuple defining the direction vector `V_d`, used in a track's parametric set of equations. `r_d` is applicable to the `linear` track type. **Or:** `radial_coeff` - The coefficient affecting the radius rate in the helical track. `radial_coeff` is applied to the free variable in the equation for `r`. `radial_coeff` is applicable to the `helical_expanding` track type. **Type:** `float` 10. `theta_d` - The `theta` coordinate of the `(r, theta, z)` tuple defining the direction vector `V_d`, used in a track's parametric set of equations. `theta_d` is applicable to the `linear` track type. **Or:** `azimuthal_coeff` - The coefficient affecting the clockwise/counter-clockwise extrusion direction of the helical track. `azimuthal_coeff` is applied to the free variable in the equation for `theta`. `azimuthal_coeff` is applicable to the `helical_expanding` track type. **Type:** `float` 11. `z_d` - The `z` coordinate of the `(r, theta, z)` tuple defining the direction vector `V_d`, used in a track's parametric set of equations. This value will be `1` or `-1`, depending on which side of the XY-plane the track is being directed to. `z_d` is applicable to the `linear` track type. **Or:** `pitch_coeff` - The coefficient affecting the pitch rate in the helical track. `pitch_coeff` is applied to the free variable in the equation for `z`. `pitch_coeff` is applicable to the `helical_expanding` track type. **Type:** `integer` 12. `hit_id` - An incremental identifier for hits belonging to an event, which is unique within the scope of the event. **Type:** `integer` 13. `hit_r` - The `r` coordinate of the `(r, theta, z)` tuple defining the recorded hit point on the relevant sub-detector. **Type:** `float` 14. `hit_theta` - The `theta` coordinate of the `(r, theta, z)` tuple defining the recorded hit point on the relevant sub-detector. **Type:** `float` 15. `hit_z` - The `z` coordinate of the `(r, theta, z)` tuple defining the recorded hit point on the relevant sub-detector. **Type:** `float` The header inclusion map for different files is as follows: - `hits__.csv` - 1, 2, 3, 4, 12, 13, 14, 15 - `tracks__.csv` - 1, 4, 5, 6, 7, 8, 9, 10, 11 - `hits__events_all.csv` - 1, 2, 3, 4, 12, 13, 14, 15 - `tracks__events_all.csv` - 1, 4, 5, 6, 7, 8, 9, 10, 11 - `hits_and_tracks__events_all.csv` - 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 ### Report A textual report file is generated and saved under the `report` folder within an experiment's folder tree. The automatically composed report lists many detector-specific configuration values directly from the relevant configuration file(s). Not every configuration field is needed by the user; only fields that help explain the data are included. In addition to important experiment parameters, basic statistical information about the data set is also included. These fields are most meaningful when dealing with variable event conditions, e.g., a variable number of tracks. ## Usage and citation If you use this data set in your research or any publication, we kindly request you to cite the following paper: ```bibtex @inproceedings{Odyurt:2024:REDVID, title = {{Reduced Simulations for High-Energy Physics, a Middle Ground for Data-Driven Physics Research}}, author = {Uraz {Odyurt} and Stephen Nicholas {Swatman} and Ana-Lucia {Varbanescu} and Sascha {Caron}}, booktitle = {Computational Science -- ICCS 2024}, editor = {Franco, Leonardo and de Mulatier, Cl{\'e}lia and Paszynski, Maciej and Krzhizhanovskaya, Valeria V. and Dongarra, Jack J. and Sloot, Peter M. A.}, year = {2024}, publisher = {Springer Nature Switzerland}, address = {Cham}, pages = {84--99}, isbn = {978-3-031-63751-3}, doi = {10.1007/978-3-031-63751-3_6} } @misc{Odyurt:2024:REDVID:arXiv, title = {{Reduced Simulations for High-Energy Physics, a Middle Ground for Data-Driven Physics Research}}, author = {Uraz {Odyurt} and Stephen Nicholas {Swatman} and Ana-Lucia {Varbanescu} and Sascha {Caron}}, year = {2024}, eprint = {2309.03780}, archivePrefix = {arXiv}, primaryClass = {hep-ex}, doi = {10.48550/arXiv.2309.03780} } ``` We put significant effort into curating and providing this data set and proper citation helps acknowledge and support the continued development of this resource. ## Support Note that this data set is being shared on an "as is" basis, without any express or implied warranties or obligations of support. While we have made efforts to ensure the accuracy and completeness of the data, we cannot guarantee its fitness for any particular purpose or provide any form of ongoing support. As the creators and sharers of this data set, we are unable to offer any dedicated support or assistance in working with or analysing the data. We do not commit to responding to inquiries, fixing issues, or providing additional documentation or guidance related to this data set. Should you encounter any challenges or have questions, we recommend referring to the existing documentation. ## Roadmap *Confidential* ## Authors and acknowledgements The REDVID simulation framework and the generated data sets are authored by: - *dr. ir. Uraz Odyurt* - Radboud University; Nikhef The collaborating team includes: - *dr. Sascha Caron* - Radboud University; Nikhef - *prof. dr. ir. Ana-Lucia Varbanescu* - University of Twente; University of Amsterdam - *dr. Roel Aaij* - Nikhef Previous collaborating members: - *MSc Stephen Nicholas Swatman* - University of Amsterdam; CERN ## Licence The data set is licenced under the [Creative Commons Attribution 4.0 International License (CC-BY-4.0)](https://creativecommons.org/licenses/by/4.0/legalcode), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited, as shown [above](#usage-and-citation). If you have any questions regarding the licence or usage of the data set, please contact the authors. **Note:** The licence applies only to the data set itself and not to any third-party content or software that may be included with the data set. Please review any licences or terms of use associated with those components separately. ---