Dissertation/performance.tex

\chapter{Performance Studies}
\label{ch:performance}
The performance of the SciFi front-end electronics crucially depends on the digitisation of the SiPM signals as well as the error-free transmission of the data to the DAQ system.
Moreover, it relies on experiencing low noise levels in order to avoid the occurrence of false hits.

Therefore, in addition to the regular commissioning steps as discussed in \cref{ch:commissioning}, further studies in these areas were performed on individual \cframes{}.
These tests were mainly conducted on \cframe{}~2 that was commissioned at the beginning of 2021 (see \cref{tab:commissioning}).
During that time, the assembly of the next \cframe{} was on hold due to restrictions in the course of the COVID-19 pandemic that did not allow the required international personal to travel from abroad.
As a result, \cframe{}~2 was available for these performance studies at the system level within the commissioning setup (see \cref{sec:commissioning-setup}) for some time.
Due to the limitations in the available DAQ system (see \cref{sec:commissioning-daq}), the presented measurements mostly include the data from only half of the \cframe{} (i.e. one quadrant).
However, this still corresponds to 12 ROBs with a total of 25k channels, i.e. to about \qty{5}{\percent} of the final detector.


\section{PACIFIC Pulse Height Digitisation}
The digitisation of the SiPM pulses is a key aspect for the operation of the detector as it is the basis for the reconstruction of hit clusters.
A deep understanding of the analogue processing and digitisation in the PACIFIC ASICs is essential for an accurate calibration of the threshold DACs (see \cref{sec:th_scan}), which is required to achieve high hit efficiencies along with good noise rejection.


\subsection{Pedestal Stability}
The PACIFIC integrators play an important role in the analogue processing.
They integrate the signals within each period of \qty{25}{\nano\second} corresponding to the proton bunch crossing intervals.
Without any input signal, the output level is determined by the DC baseline of the integrator.
It is also referred to as pedestal since any additional signal is added to it.
Knowledge about the pedestal level is essential to determine the pulse height, i.e. amplitude, of a given signal.
In addition, it is required that it remains stable after being once determined by means of a PACIFIC threshold scan.

The stability of the pedestal first came into focus during the last test beam campaign that took place in the summer of 2018 at the CERN SPS (see \cref{sec:testbeam}).
In the course of the two test weeks, occasional shifts of the pedestal positions of up to 40 threshold DACs were noticed.
Compared to a typical photoelectron peak separation of about \qty{15}{DACs}, this far exceeded the expected variations.
Possible explanations are ranging from insufficient grounding to accidental changes in the PACIFIC configurations.
However, up to now, the effect seen in the test beam could not be reproduced or understood.
Among the extensive studies that have been performed in this context are several measurements within the SciFi commissioning setup that are presented in the following.

As a basis, one has to consider the general variations of the determined pedestal position within each channel.
Between two consecutive threshold scans, the pedestals show deviations as displayed in \vref{fig:pedestal-repeat} for the 25k channels.
The standard deviation of the distribution $\sigma=\qty{0.23}{DACs}$ can be interpreted as the overall uncertainty on the pedestal positions, which are determined according to the illustration in \vref{fig:pedestals}.

\begin{figure}
  \begin{subfigure}{0.49\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/Pedestal_difference_repeat}
    \caption{Consecutive scans}
    \label{fig:pedestal-repeat}
  \end{subfigure}
  \hfill
  \begin{subfigure}{0.49\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/Pedestal_difference_conditions_overview}
    \caption{Different conditions}
    \label{fig:pedestal-conditions}
  \end{subfigure}
  \caption{
    Variations of the determined pedestal positions between two consecutive threshold scans (a) and under further conditions (b) for the 25k channels of one \cframe{} half.
    The reference scan has been performed with the HV and LIS turned off.
    In the right figure (b), the mean values $\mu$ and standard deviations $\sigma$ of the underlying distributions are indicated by the markers and error bars, respectively.
  }
\end{figure}

By performing the threshold scans under different conditions, an attempt was made to reproduce similar pedestal shifts as observed during the 2018 test beam campaign.
Examples showing the largest displacements that could be achieved are given in \vref{fig:pedestal-conditions}.
While power cycling the front-end electronics virtually made no difference compared to two consecutive threshold scans, small biases could be produced in other cases.
When injecting a signal with the help of the internal charge injection mechanism (see following \cref{sec:charge-injection}), the pedestals are systematically shifted by \qty{0.13}{DACs}.
Larger displacements are achieved by biasing the SiPM arrays (\qty{0.28}{DACs}) and additionally enabling the LIS, although without injecting light pulses (\qty{0.43}{DACs}).
However, even these differences are negligible compared to the typical photoelectron peak separation of \qty{15}{DACs} and especially compared to the \qty{40}{DACs} test beam variations.
Therefore, there is no indication that the studied \cframe{} suffers from the problems observed in the test beam.

\subsection{Internal Charge Injection}
\label{sec:charge-injection}
An important feature for the characterisation of the PACIFICs is the internal charge injection, which is used to simulate the input of SiPM pulses with fixed amplitudes.
It consists of two small on-channel capacitors (nominal \qty{630}{\femto\farad} and \qty{1}{\pico\farad}) that can be discharged through two relatively large resistors (\qty{12}{\kilo\ohm} and \qty{9}{\kilo\ohm}, respectively).
The discharges can be triggered independently via two digital input signals that are controlled by two fast control commands, which are referred to as CalibA and CalibB.
Thereby, three different charge values can be injected into the pre-amplification stage of each channel (see \cref{sec:pacific_preamp}).
The capacitance and resistance value pairs were selected based on simulations that showed good approximations to actual SiPM signal shapes.
Using the slow control of the PACIFIC, any combination of channels can be selected for internal charge injection~\autocite{pacific-datasheet}.

The internal charge injection is an important debugging tool as it allows to inject signals into the PACIFIC channels without the need of additional hardware.
It is used in particular to examine the per-channel differences between the PACIFIC integrators and comparators as presented in the following \cref{sec:int_differences,sec:comp_differences}.

\begin{figure}
  \centering
  \begin{minipage}[t]{0.49\textwidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/Scurve_charge}
    \caption{
      Typical \Scurve{} when injecting a fixed charge \qty{50}{\percent} of the time using the internal charge injection mechanism for an exemplary PACIFIC channel.
    }
    \label{fig:scurve-charge}
  \end{minipage}
  \hfill
  \begin{minipage}[t]{0.49\textwidth}
    \centering
    \includegraphics[width=1.0\linewidth]{figures/performance/Charge_delayscan}
    \caption{
      Internal charge injection amplitudes depending on the delay for an exemplary PACIFIC channel.
      The amplitudes of the three possible charge values are shown.
    }
    \label{fig:charge-delayscan}
  \end{minipage}
\end{figure}

\Vref{fig:scurve-charge} shows a typical \Scurve{} in this context.
It is generated by triggering the discharge of the \qty{1}{\pico\farad} capacitor (CalibB) \qty{50}{\percent} of the time.
As a result, two transitions are visible corresponding to the pedestal and charge signal levels.
Thereby, the charge amplitude is given by the difference of the two transitions as illustrated in \vref{fig:scurve-charge}.

Similar to injected light pulses (see \cref{sec:delayscan-lite}), the total charge accumulated by the PACIFIC integrators strongly depends on the injection delay with respect to the internal clock.
\Vref{fig:charge-delayscan} shows the dependence for the three possible charge values of an exemplary PACIFIC channel.
Within a \qty{40}{\mega\hertz} clock cycle, the delay can be chosen to take eight different values in steps of \qty{3.125}{\ns}, as controlled by a corresponding register of the Cluster FPGA that is responsible for issuing the digital discharge signals to the PACIFICs.
For \qty{80}{\percent} of the examined channels, the maximum amplitude is reached at a delay of \qty{9.375}{\ns}, with the remaining ones peaking at \qty{12.5}{\ns}.
However, as can be seen in \vref{fig:charge-delayscan}, the charge difference between the two central points is relatively small and amounts to about \qty{5}{\percent} on average for all channels.

\begin{figure}
  \centering
  \includegraphics[width=0.9\textwidth]{figures/performance/Charge_amplitudes}
  \caption{
    Amplitude distributions of the three charge values available through the internal charge injection mechanism.
    The mean values are indicated by the dashed lines.
  }
  \label{fig:charge-amplitudes}
\end{figure}

At the overall best delay of \qty{9.375}{\ns}, the internal charge amplitudes for the 25k channels are distributed as shown in \vref{fig:charge-amplitudes}.
With mean values between \num{23.3} and \qty{55.3}{DACs} for the smallest and largest available charge values, the internal charge injection mechanism covers a similar range as the SiPM signals of 1-4 photoelectrons.
With the default operating thresholds $[1.5, 2.5, 3.5]\,\text{pe}$ lying in the similar range, it is therefore well suited for debugging of the detector.
Since no additional hardware is required, it can also be used in the final detector between normal data taking runs and without manual intervention.

\subsection{Integrator Differences}
\label{sec:int_differences}
The internal charge injection mechanism is mainly used to characterise the differences between the two integrators of each PACIFIC channel.
This is an important quantity for the operation of the detector since both integrators share the same set of three comparators.
Therefore, a uniform behaviour is required to ensure consistent performance of the two integrators and thus for even and odd events.

\begin{figure}
  \centering
  \includegraphics[width=1.0\textwidth]{figures/performance/Hysteresis}
  \caption{
    Illustration of the switching process of the PACIFIC comparator output depending on the input voltage in the presence of a hysteresis (top).
    Shown below are the consequences that arise from it during a threshold scan for the cases where the even and odd integrator levels are within the hysteresis (left) and exceeding it (right).
    Note: The output of the integrator (Int.) is fed to the input of the comparator (Comp.).
  }
  \label{fig:hysteresis}
\end{figure}

Due to the structure of the PACIFIC comparators, determining the integrator differences requires the use of the internal charge injection.
The reasoning behind that is illustrated in \vref{fig:hysteresis}.
To avoid repetitive switching of the comparator output when the signal level is close to the set threshold, the PACIFIC comparators feature a design with a \qty{10}{\milli\volt} hysteresis.
As a result, there exist two slightly different voltage levels causing the comparator output to change.
Going above $V_\uparrow$ changes its state from $0 \rightarrow 1$, while going below $V_\downarrow$ changes its state from $1 \rightarrow 0$.
The difference $V_\uparrow-V_\downarrow$ is referred to as hysteresis.

As a consequence, the pedestals might overlap perfectly when performing a threshold scan, even though the baselines might differ by up to \qty{10}{\milli\volt} (see bottom left part of \cref{fig:hysteresis}).
In the default configuration of the PACIFIC ASICs, a \qty{10}{\milli\volt} difference at the comparator input corresponds to about 4 DACs in the threshold scan (see \cref{sec:pacific_digitisation}).
To determine the integrator differences with a higher precision than that, a signal needs to be injected whose amplitude exceeds the level of the hysteresis in order to resolve the two \Scurves{} (bottom right part of \cref{fig:hysteresis}).
The internal charge injection mechanism is well suited for this as it does not require any additional hardware.
As shown in \cref{fig:charge-amplitudes}, any of the three available charge levels by far exceed the hysteresis and are in principle suitable for the task.
However, to minimise the effect of any potential and interfering effects like timing differences between the two integrators, the smallest possible charge amplitude (CalibA) is used.

\subsubsection{Trimming}
\label{sec:trimming}
A uniform behaviour of the PACIFIC integrators to ensure consistent performance is achieved through the trimming, which refers to the process of aligning the two integrator baselines per channel.
As discussed in \cref{sec:pacific_integration}, the trim DAC and thus the reference voltage of the integrators can take 31 different values.
On average, incrementing the trim DAC by 1 shifts the baseline by about \qty{7}{\milli\volt} (using the default operating settings).
When performing a threshold scan, these shifts of the baseline effectively move the \Scurves{} in the horizontal direction by about 2.8 threshold DACs per trim DAC.
The covered range by the trimming is illustrated in \cref{fig:trimming-illustration} for the two integrators of one PACIFIC channel.
The goal of the trimming is to align both integrator baselines such that the \Scurves{} are overlapping.
This is the prerequisite for registering the same pulse height for the same SiPM signal and thus achieve a consistent response for the even and odd integrators.

\begin{figure}
  \centering
  \includegraphics[width=1.0\textwidth]{figures/performance/Trimming_illustration}
  \caption{
    Amplitude distributions of the three charge values available through the internal charge injection mechanism.
    The mean values are indicated by the dashed lines.
  }
  \label{fig:trimming-illustration}
\end{figure}

The trimming is performed as part of the test procedure during the quality assurance (QA) of the \PBs{} (see \cref{sec:qa}).
In the first step, three threshold scans are performed with the trim DACs set to the minimum, maximum and central (default) values.
As discussed previously, this happens in each case under injection of the smallest charge (CalibA) with the help of the internal charge injection mechanism.
The covered range for each channel and integrator is determined with the three initial threshold scans.
Especially the overlap between the two integrators is of interest as it defines the available range where the two integrator baselines can be aligned (see \cref{fig:trimming-illustration}).

Within that range, a target value is defined to which the two integrators are trimmed.
To calculate the required change of the trim DAC value, the difference of the current \Scurve{} transition to the target value (in threshold DACs) is divided by 2.8, i.e. the average change in threshold DACs per trim DAC.
To achieve a good agreement of the two integrator baselines, this process is repeated multiple times.
After each iteration, another threshold scan is performed to evaluate the overlap of the \Scurves{}.
The trimming procedure is considered successful when the \Scurves{} of the two integrators are aligned within 2.5 threshold DACs for all channels of the tested \PB{}.
This is the level of accuracy that can be achieved given the resolution of the trim DACs.
In comparison to a typical photoelectron peak separation of 15 threshold DACs, this is an acceptable level of accuracy.

\Vref{fig:trimming-comparison} shows the integrator differences of the 192 \PBs{} that are mounted on \cframe{}~2 using the trimming constants as determined during the QA procedure for the boards.
By design, the variations are within 2.5 threshold DACs when verifying the trimming within the QA stand.
However, it was found that this result is not directly applicable to the case where the identical boards are installed in ROBs and mounted on the \cframe{}.
Instead, as can be seen, the trimming distribution is smeared out and shows a significantly larger spread.

\begin{figure}
  \begin{subfigure}{0.51\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/Trimming_comparison}
    \caption{Trimming distributions}
    \label{fig:trimming-comparison}
  \end{subfigure}
  \hfill
  \begin{subfigure}{0.47\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/Retrimming}
    \caption{Box plot}
    \label{fig:retrimming}
  \end{subfigure}
  \caption{
    Integrator differences of 192 \PBs{} (PBs) after the trimming as performed and measured during the PB QA and after being installed on \cframe{}~2 using the identical trimming constants (a).
    As illustrated in the box plot (b), one retrimming iteration is required on the \cframe{} to achieve the same performance as before in the QA stand.
    Additional trimmings only change the observed differences slightly.
    The box boundaries indicate the 25th and 75th percentiles, in addition to the median (50th percentile) as the orange lines inside the boxes.
    The whiskers mark the 5th and 95th percentiles.
    The dashed lines indicate the region in which the trimming is considered successful.
  }
\end{figure}

In order to mitigate the effect, an attempt was made to redo the trimming of the PACIFIC integrators within the commissioning setup.
As shown in the corresponding box plot in \vref{fig:retrimming}, after one iteration of the retrimming, the same performance can be achieved as before in the QA stand.
After the second iteration, this is even slightly exceeded.
Beyond that, no further improvements can be observed.

The tight schedule of the commissioning (see \cref{ch:commissioning}) did not allow for the retrimming to be performed for each \cframe{} during the commissioning itself.
However, as shown here, it is needed in order to restore or even exceed the intended quality.
Therefore, to ensure consistent performance between the two PACIFIC integrators, a retrimming of all channels will be necessary after the installation of the complete detector in the LHCb cavern and before starting to take data.


\subsection{Comparator Differences}
\label{sec:comp_differences}
Another fundamental property of the PACIFIC channels is the uniformity of the three comparators.
Although, in contrast to the integrator differences, it does not directly influence the performance, knowing the level of agreement can have an impact on the daily operation of the detector.
For instance, if a strong correlation between the three comparators is observed, only scanning the thresholds of one comparator would be required and could be applied to the remaining two (compare with \cref{sec:th_scan}).

The measurement is carried out analogously to the assessment of the integrator differences as presented previously in \cref{sec:int_differences}.
By using the internal charge injection mechanism, it is defined by the differences in the determined charge transition positions (see illustration in \cref{fig:scurve-charge}) between the different comparators.
The results as obtained from the 50k channels on \cframe{}~2 are shown in \vref{fig:comparator-differences}.
Compared to comparator 1, which is used as reference, a similar spread of $\sigma = \qty{1.6}{DACs}$ can be observed for both remaining comparators.
In addition, systematic biases up to \qty{0.9}{DACs} between comparator 3 and 1 is visible.

\begin{figure}
  \centering
  \includegraphics[width=0.9\textwidth]{figures/performance/Comparator_differences}
  \caption{
    Differences in the determined charge transitions between the different PACIFIC comparators.
    The underlying data is taken from the 50k channels on \cframe{}~2 using the internal charge injection mechanism.
    The differences to the (reference) comparator 1 in each channel are displayed.
    The mean values $\mu$ and standard deviations $\sigma$ of the distributions are indicated on the right.
  }
  \label{fig:comparator-differences}
\end{figure}

While the slight biases are negligible compared to the overall spread, the resulting differences are at the edge of being relevant compared to a typical photoelectron peak separation of \qty{15}{DACs}.
For the majority of channels, scanning one of the three comparators and assuming a similar behaviour for the remaining two can be sufficient for some applications.
However, for optimal performance, a full threshold scan of all comparators is advisable for the calibration of the PACIFIC thresholds prior to taking physics data.


\section{Clock Distribution}
In view of partially small margins of errors when operating the front-end electronics using a common set of clock timing settings as presented in \cref{sec:timing_scans}, concerns about the stability of the clock phase configurations arose.
This is to be investigated by means of a series of clock timing scans:
\begin{enumerate}
  \setlength\itemsep{0em}
  \item Initial scans (reference)
  \item Consecutive scans with identical conditions
  \item After power cycling the ROBs
  \item Change cooling water temperature by \qty{-5}{\celsius} ($T = \qty{15}{\celsius}$)
  \item Change cooling water temperature by \qty[retain-explicit-plus]{+5}{\celsius} ($T = \qty{25}{\celsius}$)
\end{enumerate}
The clock scans were carried out on the electronics located on the bottom half of \cframe{}~2 (12 ROBs, 25k channels).
It is expected that especially the variations in the cooling water and therefore the temperature of the electronics has an effect due to slight changes in the electrical conductivity.
Note that when changing the cooling water temperature by \qty{\pm 5}{\celsius}, the average temperatures of the boards changed by about \qty{\pm 4}{\celsius}.

\begin{figure}
  \centering
  \includegraphics[width=1.0\textwidth]{figures/performance/Timing_scans_stability}
  \caption{
    Differences in the optimal clock phases between the SYNC Pulse and PACIFIC clock for different run conditions compared to a reference (left).
    Measured by the centres of the error-free intervals.
    The right plot summarises the mean and standard deviation of the underlying distributions for the remaining clock phases within the SciFi front-end electronics as indicated by the markers and error bars, respectively.
  }
  \label{fig:timing-scans-stability}
\end{figure}

The clock phases are scanned in the smallest available step size of \qty{48.8}{\pico\second} in order to achieve the highest possible level of accuracy.
The determined error-free clock phase interval are each compared with the initial scans that serve as a reference.
In general, differences to the reference scans are found to be in the order of the step size resolution and therefore subject to relatively large uncertainties.
However, significant differences are observed in the phase between the SYNC Pulse and PACIFIC clock for the different cooling water temperature conditions.
This is shown in \vref{fig:timing-scans-stability} (left) that displays the phase differences between the centres of the error-free intervals.
While performing a consecutive run with identical conditions as well as power cycling the ROBs only results in minor variations around 0, systematic biases can be observed between different cooling water temperatures.
On average, a shift by about \qty{80}{\pico\second} is visible between the two scans taken with the water temperature set to \qty{15}{\celsius} and \qty{25}{\celsius}, which can be explained by the length of the corresponding clock and signal lines.
As shown in \vref{fig:timing-scans-stability} (right), a similar trend can be observed for the other clock phases, although not to the same extent.

In summary, any of the observed variations lie well below the margins of errors as found in \cref{sec:timing_scans} when operating the front-ends at a common set of clock timing settings.
As discussed, this also applies to rather unrealistic changes of the cooling water temperature.
After all, the temperature of the cooling water, as well as the ambient temperature, is more strictly controlled in the LHCb cavern as it is the case in the assembly hall on the surface.


\section{Dark Count Rate}
It has been discussed in \cref{sec:sipm} that the dark count rate (DCR) contributes significantly to the detector noise once the SiPM arrays are exposed to radiation.
To mitigate this, the detector is designed to allow for the cooling of the SiPMs down to \qty{-40}{\celsius} (see \cref{sec:novec}).
While the radiation dose cannot be varied within the commissioning setup consisting of new hardware, it can be used to study the influence of temperature and the applied overvoltage (\OV{}) on the DCR using the complete readout chain.

Because of the processing of the SiPM signals inside the PACIFIC ASICs, the total DCR is reduced compared to the DCR of the SiPM arrays alone.
This is due to the integration stage that accumulates the signals within each bunch crossing of \qty{25}{\nano\second}.
Thereby, a dark count event may fall below the applied threshold if it arrives between two integration windows.
In the following, the thresholds of the PACIFIC comparators are set to be just (\qty{0.5}{pe}) above the determined pedestal positions.
For instance, with a typical photoelectron peak separation of \qty{15}{DACs} at $\OV{} = \qty{3.5}{\volt}$, this corresponds to \qty{7.5}{DACs}.
For each condition, a total of 60M events are recorded.

\begin{figure}
  \begin{subfigure}{0.49\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/DCR_Temperatures_ExLink}
    \caption{SiPM array}
    \label{fig:dcr-temperatures-link}
  \end{subfigure}
  \hfill
  \begin{subfigure}{0.49\linewidth}
    \centering
    \includegraphics[width=1.0\textwidth]{figures/performance/DCR_Temperatures}
    \caption{Box plot}
    \label{fig:dcr-temperatures}
  \end{subfigure}
  \caption{
    Dark count rate (DCR) along the \num{128} channels of one SiPM array for different SiPM temperatures (a).
    The box plot (b) shows the measured rates for the \num{192} SiPMs located on the bottom half of \cframe{}~2.
    The box boundaries indicate the 25th and 75th percentiles, in addition to the median (50th percentile) as the orange lines inside the boxes.
    The whiskers mark the 5th and 95th percentiles.
  }
\end{figure}

Due to the thermal origin of the dark counts, the DCR strongly depends on the temperature of the SiPMs.
To verify this, a series of measurements were carried out at different temperatures: \qty{-40}{\celsius}, \qty{-10}{\celsius} and \qty{20}{\celsius}.
\Vref{fig:dcr-temperatures-link} shows the determined rates along the \num{128} channels of one exemplary SiPM array.
In addition, the box plot in \vref{fig:dcr-temperatures} summaries the results from the \num{192} arrays located on the bottom half of \cframe{}~2, revealing an exponential relationship between the temperature and DCR.
Due to the fact that unirradiated SiPMs are in use at this stage, the measured rates even at room temperature are still comparatively low and are in line with the expected values of a few \unit{\kilo\hertz}.

Analogous to this, measurements were also carried out at different overvoltages: \qty{2.5}{\volt}, \qty{3.5}{\volt} (nominal) and \qty{4.5}{\volt}.
However, due to the overall low rates no significant difference could be found between the three settings.



%\subsubsection{Noise Cluster Rate}
%Due to the process of the clustering (see \cref{sec:clustering}), individual dark counts only play a subordinate role.
%In order to result in actual noise clusters, they need to occur simultaneously and in neighbouring channels.