MBTA standard offline search - O4a (May 2023 - Jan 2024)
This page documents the offline analysis of O4 Ligo and Virgo data to detect GW signals from coalescences of compact objects (black holes or neutron stars) using the MBTA code.The data analysed in this page were collected during the O4a observation run (May 2023 - Jan 2024) by Ligo (Virgo being expected to join the run for O4b).
The data were split in several chunks of 2 weeks length, analysed separately. The official chunk limits are given in this page. You can find in the table below the links to the pages presenting the mbta offline analysis results for each chunk. At the bottom of the page, you also find details on how the quality of the data used for the analysis was defined and obtained: we use ANALYSIS-READY segments and CAT1 veto information from the Ligo and Virgo DetChar groups (according to the CBC group recommendations), on top of which MBTA applies its own gating.
List of superchunks and links to the results
| Superchunk | Date range (UTC) | GPS time range | Chunk length (days) | Chunk length after CAT1 (days) | Chunk length after gating (days) | Results | ||
|---|---|---|---|---|---|---|---|---|
| ≥ 2 ifo OK | ≥ 1 ifo OK | ≥ 2 ifo OK | ≥ 1 ifo OK | |||||
| 01to04 | 2023-05-24 15:00 2023-07-18 10:33 |
1368975618 1373711624 |
54.81 | 31.05 (56.6 %) | 48.08 (87.7 %) | 30.98 (56.5 %) | 48.06 (87.7 %) | READY |
| 05to08 | 2023-07-18 10:33 2023-09-12 14:44 |
1373711624 1378565116 |
56.17 | 33.06 (58.9 %) | 47.69 (84.9 %) | 33.00 (58.7 %) | 47.67 (84.9 %) | READY |
| 09to12 | 2023-09-12 14:44 2023-11-07 15:03 |
1378565116 1383404656 |
56.01 | 33.60 (60.0 %) | 48.54 (86.6 %) | 33.55 (59.9 %) | 48.52 (86.6 %) | READY |
| 13to16 | 2023-11-07 15:03 2024-01-16 15:45 |
1383404656 1389455118 |
70.03 | 28.76 (41.1 %) | 52.52 (75.0 %) | 28.69 (41.0 %) | 52.49 (75.0 %) | READY |
Chunks production and checks
| Link to individual chunk monitoring page | Date range (UTC) | GPS time range | Chunk length (days) | Chunk length after CAT1 (days) | Production status | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| in at least | Preprocessing | Filtering | Postfiltering | |||||||||||
| 2 ifo | 1 ifo | DAT | INJ02 | INJ13 | DAT | INJ02 | INJ13 | DAT | INJ02 | INJ13 | ||||
| chunk01 | 2023-05-24 15:00 2023-06-06 14:30 |
1368975618 1370097052 |
12.98 | 5.66 | 11.77 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk02 | 2023-06-06 14:30 2023-06-20 14:21 |
1370097052 1371306087 |
13.99 | 7.55 | 12.19 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk03 | 2023-06-20 14:21 2023-07-04 22:47 |
1371306087 1372546081 |
14.35 | 8.75 | 12.55 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk04 | 2023-07-04 22:47 2023-07-18 10:33 |
1372546081 1373711624 |
13.49 | 9.08 | 11.57 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk05 | 2023-07-18 10:33 2023-08-01 14:44 |
1373711624 1374936316 |
14.17 | 7.71 | 11.22 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk06 | 2023-08-01 14:44 2023-08-15 14:25 |
1374936316 1376144770 |
13.99 | 6.65 | 11.17 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk07 | 2023-08-15 14:25 2023-08-29 14:44 |
1376144770 1377355516 |
14.01 | 9.97 | 12.70 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk08 | 2023-08-29 14:44 2023-09-12 14:44 |
1377355516 1378565116 |
14.00 | 8.74 | 12.59 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk09 | 2023-09-12 14:44 2023-09-26 14:44 |
1378565116 1379774716 |
14.00 | 9.63 | 12.57 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk10 | 2023-09-26 14:44 2023-10-10 14:44 |
1379774716 1380984316 |
14.00 | 9.13 | 12.40 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk11 | 2023-10-10 14:44 2023-10-24 15:04 |
1380984316 1382195103 |
14.01 | 8.19 | 12.25 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk12 | 2023-10-24 15:04 2023-11-07 15:03 |
1382195103 1383404656 |
14.00 | 6.66 | 11.32 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk13 | 2023-11-07 15:03 2023-11-21 07:59 |
1383404656 1384588801 |
13.71 | 7.15 | 11.74 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk14 | 2023-11-21 07:59 2023-12-05 14:46 |
1384588801 1385822808 |
14.28 | 5.84 | 9.38 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk15 | 2023-12-05 14:46 2023-12-21 15:15 |
1385822808 1387206946 |
16.02 | 5.15 | 12.85 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
| chunk16 | 2023-12-21 15:15 2024-01-16 15:45 |
1387206946 1389455118 |
26.02 | 10.61 | 18.55 | OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
OK |
Data quality and segments
Some instructions to produce the list of good data quality segments are given here.The list of good DQ segments is produced chunk by chunk and is available in /sps/virgo/USERS/mbta/O4/cat-flags/
1) The interferometer has to be in observing mode, also called DMT-ANALYSIS-READY in the segment database.
Recommendation of the ifo mode to be used for offline analysis is given in the ObsRun4 page.
The definitions of the ifo modes are given in the O4Flags page.
The DMT flags are defined in the dmt-runtime-configuration git page.
2) CAT1 flags are developped offline to reject additional bad data quality periods. The criteria are defined in a Veto Definer File (VDF) continuously updated during the run and can be found here.
The veto definer file for Burst and CBC analyses is now available for the full O4a period and tagged CAT1-HL-END_O4A-R2.
3) The good DQ segments and effective analysis times are given chunk by chunk in the links of the above table. They can also be found here for the whole O4a period, with the following summary numbers.
Three types of ascii files are produced:
- IFO_ANALYSIS_READY.txt : DMT-ANALYSIS_READY segments from segDB
- IFO_CAT1.txt : Bad CAT1 segments from veto definer files
- IFO_ANALYSIS_READY-CAT1.txt : Analysis ready segments minus bad CAT1 segments
H1_ANALYSIS_READY-CAT1.txt and L1_ANALYSIS_READY-CAT1.txt are the files used in the offline analysis.
4) During MBTA preprocessing, h(t) strain data are gated for times outside the good DQ segments (i.e. a filter is applied to smoothly set h(t) to 0). MBTA also applies its own gating when the BNS range drops below 0.6 times the median range (the ranges are calculated for equal mass neutron stars of 1.4 Msol over the whole frequency range).
The good data quality segments after preprocessing for the whole O4a period (chunks 1 to 16) are available here for H1 and L1.
The other segment files for OR and AND combination of detectors are in this directory as well as a summary of the different periods.
| chunk duration (in days) |
chunk duration after CAT1, in days | chunk duration after gating, in days (autogating loss in s) |
|||||||
|---|---|---|---|---|---|---|---|---|---|
| HL good | H good | L good | H or L good | HL good | H good | L good | H or L good | ||
| chunk01 | 12.98 | 5.66 (43.6 %) | 6.19 (47.7 %) | 11.24 (86.6 %) | 11.77 (90.6 %) | 5.65 (43.5 %) (-1142.2 s) |
6.18 (47.6 %) (-765.1 s) |
11.23 (86.5 %) (-881.6 s) |
11.76 (90.6 %) (-504.5 s) |
| chunk02 | 13.99 | 7.55 (54.0 %) | 8.45 (60.4 %) | 11.29 (80.7 %) | 12.19 (87.1 %) | 7.53 (53.8 %) (-1502.1 s) |
8.44 (60.3 %) (-961.6 s) |
11.28 (80.6 %) (-942.0 s) |
12.19 (87.1 %) (-401.5 s) |
| chunk03 | 14.35 | 8.75 (61.0 %) | 9.57 (66.7 %) | 11.74 (81.8 %) | 12.55 (87.5 %) | 8.74 (60.9 %) (-1525.0 s) |
9.55 (66.6 %) (-984.1 s) |
11.73 (81.7 %) (-897.2 s) |
12.55 (87.4 %) (-356.2 s) |
| chunk04 | 13.49 | 9.08 (67.3 %) | 9.52 (70.6 %) | 11.13 (82.5 %) | 11.57 (85.7 %) | 9.06 (67.2 %) (-1531.8 s) |
9.51 (70.5 %) (-959.5 s) |
11.12 (82.4 %) (-805.9 s) |
11.56 (85.7 %) (-233.7 s) |
| chunk05 | 14.17 | 7.71 (54.4 %) | 9.93 (70.0 %) | 9.00 (63.5 %) | 11.22 (79.2 %) | 7.69 (54.3 %) (-1458.0 s) |
9.91 (69.9 %) (-1085.2 s) |
9.00 (63.5 %) (-739.5 s) |
11.22 (79.1 %) (-366.7 s) |
| chunk06 | 13.99 | 6.65 (47.5 %) | 9.21 (65.9 %) | 8.61 (61.5 %) | 11.17 (79.9 %) | 6.63 (47.4 %) (-1232.8 s) |
9.20 (65.8 %) (-967.0 s) |
8.60 (61.5 %) (-681.5 s) |
11.17 (79.8 %) (-415.7 s) |
| chunk07 | 14.01 | 9.97 (71.1 %) | 11.08 (79.1 %) | 11.59 (82.7 %) | 12.70 (90.7 %) | 9.95 (71.0 %) (-1527.1 s) |
11.07 (79.0 %) (-955.9 s) |
11.58 (82.6 %) (-834.7 s) |
12.70 (90.6 %) (-263.5 s) |
| chunk08 | 14.00 | 8.74 (62.4 %) | 10.13 (72.3 %) | 11.20 (80.0 %) | 12.59 (89.9 %) | 8.72 (62.3 %) (-1328.4 s) |
10.12 (72.3 %) (-855.6 s) |
11.19 (79.9 %) (-754.4 s) |
12.59 (89.9 %) (-281.6 s) |
| chunk09 | 14.00 | 9.63 (68.8 %) | 10.40 (74.3 %) | 11.80 (84.3 %) | 12.57 (89.8 %) | 9.61 (68.7 %) (-1294.9 s) |
10.39 (74.2 %) (-807.4 s) |
11.79 (84.2 %) (-725.4 s) |
12.57 (89.8 %) (-237.9 s) |
| chunk10 | 14.00 | 9.13 (65.2 %) | 10.93 (78.1 %) | 10.60 (75.7 %) | 12.40 (88.5 %) | 9.12 (65.1 %) (-1266.1 s) |
10.92 (78.0 %) (-846.1 s) |
10.59 (75.7 %) (-689.5 s) |
12.39 (88.5 %) (-269.6 s) |
| chunk11 | 14.01 | 8.19 (58.4 %) | 10.87 (77.6 %) | 9.56 (68.2 %) | 12.25 (87.4 %) | 8.17 (58.3 %) (-1166.3 s) |
10.86 (77.5 %) (-785.4 s) |
9.56 (68.2 %) (-670.6 s) |
12.24 (87.4 %) (-289.7 s) |
| chunk12 | 14.00 | 6.66 (47.5 %) | 8.56 (61.1 %) | 9.42 (67.3 %) | 11.32 (80.9 %) | 6.64 (47.5 %) (-992.1 s) |
8.55 (61.1 %) (-744.7 s) |
9.40 (67.2 %) (-1001.9 s) |
11.31 (80.8 %) (-754.5 s) |
| chunk13 | 13.71 | 7.15 (52.2 %) | 10.00 (72.9 %) | 8.89 (64.9 %) | 11.74 (85.6 %) | 7.14 (52.1 %) (-1190.3 s) |
9.98 (72.9 %) (-955.5 s) |
8.89 (64.8 %) (-573.1 s) |
11.73 (85.6 %) (-338.2 s) |
| chunk14 | 14.28 | 5.84 (40.9 %) | 7.61 (53.3 %) | 7.61 (53.3 %) | 9.38 (65.7 %) | 5.83 (40.8 %) (-1056.0 s) |
7.60 (53.2 %) (-842.7 s) |
7.60 (53.2 %) (-578.2 s) |
9.38 (65.6 %) (-364.8 s) |
| chunk15 | 16.02 | 5.15 (32.2 %) | 11.87 (74.1 %) | 6.13 (38.3 %) | 12.85 (80.2 %) | 5.14 (32.1 %) (-855.4 s) |
11.86 (74.0 %) (-964.1 s) |
6.13 (38.2 %) (-566.9 s) |
12.85 (80.2 %) (-675.6 s) |
| chunk16 | 26.02 | 10.61 (40.8 %) | 15.51 (59.6 %) | 13.65 (52.5 %) | 18.55 (71.3 %) | 10.58 (40.7 %) (-2887.7 s) |
15.48 (59.5 %) (-2790.8 s) |
13.64 (52.4 %) (-1138.4 s) |
18.54 (71.2 %) (-1041.5 s) |
| Total | 237.03 | 126.47 (53.4 %) | 159.82 (67.4 %) | 163.47 (69.0 %) | 196.82 (83.0 %) | 126.22 (53.2 %) (-21956.2 s) |
159.63 (67.3 %) (-16270.7 s) |
163.33 (68.9 %) (-12480.8 s) |
196.74 (83.0 %) (-6795.2 s) |
Documentation and useful links
The preprocessing was performed with FdIOServer version v8r61p0 (chunks 1 to 10) and version v8r62p1 (chunks 11 to 16).Version v5r44 of MBTA was used for the filtering and postprocessing.
An MBTA documentation updated for its v5 version can be found here.
The documentation for the v4 version also contains some interesting details on the mbta processing.
FdIOServer is used for the preprocessing and the documentation of the Fd library can be consulted here.
Materials for the LVK internal review and signo-off for the O4 online and offline analysis are available on the O4ReviewMeetings wiki.
The production workflow, some instructions to produce the results and checks at all production steps are described in the mbta offline prod documentation.
Details about the injection set
The sets of injections are produced centrally by the Rate & Pop group for all the CBC pipelines.- injection distribution
- injection file formats
- number of injections to be produced to get stable inference results
- instructions to produce offline injections
The rate & pop injections have been produced in 4 trains, with a time interval of 24s between injections in each train. Each train is shifted by 6s with respect to the previous one. The injection h(t) are superposed to the h(t) data during the preprocessing step. Two runs are performed, one with the trains 0 and 2 spaced by 12s and another one with trains 1 and 3. Only train 3 contains BNS and NSBH events in addition to BBH. The number of injected events in one chunk is around 200 000 with a minority of NSBH (~4000) and BNS events (~5000).
Data files and logfiles
All output runs and logfiles are stored at the CCIN2P3 computing center.The organisation of the folders is described here.
Preprocessing files (config, log and gwf):
/sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaD/vYY/ (data) /sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaI/vYY/batchs-0-2/ (injection trains 0 and 2 spaced by 12s) /sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaI/vYY/batchs-1-3/ (injection trains 1 and 3 spaced by 12s) where chunXX is the chunk number and vYY a production version number
Filtering files (config, log and gwf):
/sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaD/runYY/ (data) /sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaI/runYY/batchs-0-2/ (injection trains 0 and 2 spaced by 12s) /sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaI/runYY/batchs-1-3/ (injection trains 1 and 3 spaced by 12s) where chunXX is the chunk number and runYY a production version number
Postprocessing files (config, log and gwf):
/sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaD/runYY/postfiltering/vZZ/ (data) /sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaI/runYY/batchs-0-2/postfiltering/vZZ/ (injection trains 0 and 2 spaced by 12s) /sps/virgo/USERS/mbta/O4/results/chunkXX/mbtaI/runYY/batchs-1-3/postfiltering/vZZ/ (injection trains 1 and 3 spaced by 12s) where chunXX is the chunk number, runYY the production version number of the filtering and vZZ the production version number of the postprocessing These different paths with the version numbers used are given in each individual chunk web page.
Files containing the analysis ready segments after CAT1 vetos:
/sps/virgo/USERS/mbta/O4/cat-flags/chunkXX/ where chunXX is the chunk number
The files containing the good data segments after mbta autogating are in the "segments" subdir of the preprocessing directories:
/sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaD/vYY/segments/ (data) /sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaI/vYY/batchs-0-2/segments/ (injection trains 0 and 2 spaced by 12s) /sps/virgo/USERS/mbta/O4/preprocessing/chunkXX/mbtaI/vYY/batchs-1-3/segments/ (injection trains 1 and 3 spaced by 12s) where chunXX is the chunk number and vYY a production version number
The interferometer h(t) files are available on cvmfs or locally:
/sps/virgo/USERS/mbta/O4/HLVoffline/cvmfs/analysis-ready /sps/virgo/USERS/mbta/O4/HLVoffline/local/analysis-ready
Injection files (characteristics of each individual injection event and h(t)):
/sps/virgo/USERS/mbta/O4/rpo4-offline
Template banks:
/sps/virgo/USERS/mbta/O4/banks/2023-02-01-preO4-HF-sorted-Q50-HF20ms
SNR Excess model:
sps/virgo/USERS/mbta/O4/snr-excess/2023-12-11-O4a-MainSearch/SNRexcessModel.gwf
pastro and FAR models are available in the superchunk folders:
/sps/virgo/USERS/mbta/O4/results/chunkXXtoYY/mbtaD/run07v01/FARofPastro.gwf