UNIVERSIDADE DE LISBOA
FACULDADE DE CIÊNCIAS
DEPARTAMENTO DE FÍSICA
Functional Tester for High Voltage Boards of the TILECAL
Calorimeter
Filipe Mendes de Oliveira Cuim
Mestrado Integrado em Engenharia Física
Dissertação orientada por:
Prof. Guiomar Gaspar de Andrade Evans
Prof. José António Soares Augusto
Acknowledgements
This work is dedicated to my family that gave me excellent conditions to continue my studies and always encouraged me to take full advantage of my years in the university.
I also want to thank all the professors of the Faculty of Sciences that helped me during my academic years be it for personal or scholar matters.
To the Department of Physics of the Faculty of Sciences of the University of Lisbon that gave me the space and material to help develop my work.
A big thanks to professor José Soares Augusto, professor Guiomar Evans and Filipe Martins, that followed my work and gave me a tremendous support during its development.
Finally, to my friends that have also supported and helped me and with whom I shared one of the greatest experiences of my life.
Abstract
This work was done in the scope of a project, called HV Remote, with its main goal being to build an improved high voltage regulation system for the ATLAS TileCal’s photomultiplier tubes (PMTs). This project consists on a remote system that comprises a HV distribution system, called HV Remote board, a high voltage and low voltage supplies board, called Power Supplies board, and several multi-conductor cables that distribute the high voltage to the PMTs. The Power supplies board provides the LV (low voltage) that the HV Remote needs to work and feeds a fixed HV (high voltage) value to it. The HV Remote individually regulates the HV provided by the Power supplies board and monitors the PMTs’ high voltage, being also able to disable any of them in the case of a malfunction.
The TileCal has approximately 10000 PMTs, so this new system has 256 HV Remote and dedicated supplies boards, where each pair of boards is associated to 48 PMTs. When these boards are produced, after the design phase, they will need to be the target of several functional tests, and high-temperature tests. This work focuses on designing a software interface for the HV Remote boards, that will be incorporated, in the future, in the DCS (Detector Control System) of ATLAS and a GUI (Graphical User Interface) to perform all the necessary tests to the boards, before they are sent to CERN.
Each group of 16 boards will be controlled by a FPGA, but in this work a Raspberry Pi will be used as the master controller for testing the control system architecture. The Raspberry Pi will use the standard Serial Peripheral Interface (SPI) protocol to establish the communication with the boards. The code of the software interface will be written with Python 3 and will be specifically designed for the Raspberry Pi. So, in the future, the code must be adapted to the FPGA. The GUI for the tests will be designed using the PyQt5 library of Python. At present, and during the development of this thesis’ work, the boards weren’t produced yet, so some of its main digital components (port expanders, DACs, ADC and MUXs) were bought and assembled, in order to test the code, the GUI and the overall communication interface of the Raspberry Pi. These components were assembled in a breadboard, and the connections mimicked the HV Remote board’s schematics.
Resumo
Este trabalho foi realizado no âmbito de um projeto de desenvolvimento de um sistema remoto de distribuição e controlo da alta tensão fornecida aos fotomultiplicadores usados no TileCal, na experiência ATLAS no CERN. O sistema consiste num conjunto de 256 pares de cartas eletrónicas: uma para controlo e distribuição das altas tensões individuais aos PMTs, denominada HV Remote, e outra de alimentação, denominada de Power Supplies. A carta de alimentação fornece um de dois possíveis valores de alta tensão para a HV Remote, sendo eles -830 V ou -950 V. Para além disso ela fornece também a baixa tensão que a HV Remote necessita para funcionar, que são ± 12 V para os componentes analógicos e 3.3 V para os componentes digitais, bem como as suas respetivas massas. A carta HV Remote tem as seguintes funções:
▪ Regular individualmente o valor de alta tensão fornecido aos fotomultiplicadores, sendo isto possível com um conjunto de conversores digitais-analógicos (DACs);
▪ Monitorizar as suas altas tensões, realizando leituras de tensão nos seus canais com um conjunto de multiplexadores (MUXs) e um conversor analógico-digital (ADC);
▪ Ativar/desativar os canais dos fotomultiplicadores, para o caso de haver alguma avaria. Isto permite que as medições realizadas nos fotomultiplicadores funcionais não sejam perturbadas pelos que apresentam um mau funcionamento;
▪ Realizar leituras de sensores de temperatura, que permitem ter um melhor controlo do aumento de temperatura das placas em regiões mais densamente povoadas por componentes. As 256 cartas serão agrupadas em grupos de 16, sendo cada grupo controlado por uma FPGA no sistema final, quando as cartas forem enviadas para o CERN. Neste momento as cartas estão em fase de produção, e posteriormente terão de ser alvo de testes funcionais e testes de temperatura. Nesta fase, um Raspberry Pi (o modelo usado é o 3b+), substitui as FPGA. O Raspberry Pi (e no futuro a FPGA) controlará as cartas, estabelecendo um protocolo de comunicação SPI, que é uma interface de comunicação em série baseada uma arquitetura mestre-escravo, em que o dispositivo mestre (o Raspberry Pi) controla toda a comunicação, que envolverá transferências de palavras de 8 bits. Como o controlador tem de comunicar com vários componentes, e o protocolo de comunicação escolhido é do tipo série, expansores série-paralelo (“port expanders”) foram usados para converter as palavras digitais enviadas em série, para 16 portas de saída que permitem a comunicação com todos os circuitos integrados. Alguns dos expansores foram também usados para implementar a funcionalidade de ativação/desativação dos canais dos fotomultiplicadores. O protocolo SPI assenta em 3 sinais principais: MOSI (“Master Output Slave Input” ou saída do mestre e entrada do escravo), MISO (“Master Input Slave Output” ou entrada do mestre e saída do escravo) e SCLK (sinal de relógio). Adicionalmente um sinal, geralmente representado pela sigla CS (“Chip Select” – selecionador de chip) ou SS (“Slave Select” – selecionar de dispositivo escravo), é usado para ativar a comunicação com os componentes digitais. Na carta HV Remote cada componente é associado a um sinal CS e os três sinais principais do SPI serão partilhados por todos eles. O Raspberry poderá posteriormente comunicar com um computador através de uma ligação ethernet, sendo assim controlado remotamente ou poderá ser ele próprio usado como um computador, conectando-o a um ecrã. Este trabalho foi sempre desenvolvido diretamente no Raspberry Pi, montando-o como um computador convencional.
Este trabalho é sobre o desenvolvimento do software que permitirá a comunicação entre o Raspberry e a HV Remote e no desenvolvimento de uma interface gráfica (GUI) que será executada no Raspberry Pi para testar as placas HV Remote quando forem produzidas. O
programa de controlo foi escrito usando a linguagem Python 3, seguindo o formalismo da programação orientada para objetos, que envolve estruturas de classes constituídas por um número de métodos/funções, que permitem definir e emular objetos numa linguagem de programação. Por isso, foram criadas classes que representam cada um dos componentes que participam no controlo digital da HV Remote, sendo eles: o MCP23S17 (expansor série-paralelo), o DAC7568 (conversor digital-analógico) e o MAX1240 (conversor analógico-digital). Os multiplexadores não têm uma classe associada pois não têm nenhuma interface de comunicação, pois podem ser manuseados apenas pela alteração dos valores lógicos dos seus endereços (função que é realizada pelos expansores). Após a criação destas três classes, foi desenhada uma classe para a HV Remote, com os métodos necessários para realizar as funções das cartas, que incorpora e complementa as outras classes. As classes mais complicadas de criar foram a do expansor e do DAC, pois a classe do ADC contem apenas um método, devido ao facto de este não ter mais nenhuma funcionalidade para além da leitura da tensão na sua entrada. O DAC e o expansor têm diferentes funcionalidades que permitem realizar diferentes formas de escrita de tensões ou aceder de diferentes formas à referência interna, no caso do DAC, e configurar de diferentes formas as portas de saída, no caso do expansor.
Após a escrita das classes, começou-se a escrever o código da interface gráfica. Esta GUI foi realizada usando um módulo do Python, denominado PyQt5, que é um conjunto de “atalhos” que ligam a uma biblioteca escrita em C++ denominada Qt5. Este módulo foi escolhido, pois disponibiliza uma série de funcionalidades que no momento pareceram ser úteis para este projeto. O PyQt5 baseia-se num conjunto de objetos denominados “layouts” e “widgets”. Os “widgets” são todas os objetos de interação que usualmente se observa em aplicações: botões, quadrados de selecionar (“checkboxes”), listas de selecção, retângulos de introdução de números ou palavras, etc… Os “layouts” são objetos que contêm os “widgets” e que permitem organizar as suas posições e os seus tamanhos. A janela da GUI foi desenhada com dois painéis, um para o ajuste de tensões e ativação/desativação de canais, e outro para a monitorização da alta tensão dos fotomultiplicadores.
Durante o período de realização deste trabalho, as placas ainda não tinham sido produzidas, e por isso, para testar o código e a comunicação com o Raspberry Pi, foram comprados os componentes acima mencionados. Estes foram montados numa placa de ligações (breadboard), e as ligações foram feitas de acordo com os esquemas elétricos do controlo digital da HV Remote. Estes testes tiveram como principal objetivo, verificar o funcionamento de todo o software. Alguns LEDs foram também conectados na saída dos expansores para simular a funcionalidade de ativação/desativação dos canais dos fotomultiplicadores. O software de comunicação foi terminado, e os testes realizados deram indicações de um bom funcionamento. A GUI tem a parte do ajuste de tensões e de ativação/desativação terminada, faltando desenvolver a parte da monitorização dos canais dos fotomultiplicadores. Nesta última parte, pretende-se inserir gráficos, para cada canal, que atualizam o valor da tensão em tempo real. Ficou também por realizar uma função para a leitura das tensões dos sensores de temperatura, cujos valores serão também mostrados num gráfico que será atualizado em tempo real.
Para além das funcionalidades que faltam implementar na interface gráfica, depois de serem realizados alguns testes funcionais às cartas HV Remote, o software terá de ser adaptado para funcionar numa FPGA.
Index
1 Introduction ... 1
1.1 Scope of this work ... 1
1.2 Objectives ... 1
1.3 Software implementation ... 1
1.4 State of the HV Remote board ... 2
1.5 Document structure ... 2
2 The Atlas Experiment... 3
2.1 Overview of the ATLAS detector ... 3
2.2 The Magnet System ... 5
2.3 The Inner Detector ... 6
2.3.1 The Pixel Detector ... 7
2.3.2 The semiconductor tracker (SCT) ... 7
2.3.3 The Transition Radiation Tracker (TRT) ... 8
2.4 The Muon Spectrometer ... 8
2.5 Calorimeters ... 9
3 The TileCal Calorimeter ... 10
3.1 Overview of the calorimeter ... 10
3.2 The Mechanical Structure of TileCal ... 10
3.3 Optical Elements: the scintillating tiles, the wavelength shifting fibres and the photomultipliers. ... 11
3.4 The old HV distribution system of the PMTs: HV Opto and HV Micro boards ... 13
3.5 The High-Luminosity Upgrade of the LHC ... 15
3.5.1 The ATLAS upgrade for the HL-LHC ... 16
3.5.2 The upgrades to the Tile Calorimeter ... 16
4 The HV Remote system ... 18
4.1 General Description of the HV Remote System ... 18
4.2 The SPI protocol... 23
4.3 The Digital Devices of the HV Remote ... 25
4.3.1 The port expander MCP23S17 ... 25
4.3.2 The DAC7568 ... 29
4.3.3 The ADC MAX1240 ... 34
4.3.4 The bus switch SN74CB3Q3245 ... 35
4.3.5 The MUX36S16 ... 36
4.4 Functional description of the digital control of HV Remote ... 37
5.1 General Description of the tester ... 42
5.2 The Raspberry Pi 3b+ ... 43
5.3 The HV Remote software interface ... 47
5.3.1 The MCP23S17 class ... 47
5.3.2 The DAC7568 class ... 53
5.3.3 The MAX1240 class ... 56
5.3.4 The HV Remote class ... 58
5.4 The GUI and the breadboard assemble ... 60
5.5 Test of the DAC and the of the ADC connected in tandem ... 66
6 Conclusion ... 73
6.1 Summary of the activities and goals achieved ... 73
6.2 Future work ... 73
6.3 Personal reflection ... 74
7 References ... 75
8 Annex ... 77
8.1 Tables of all the DAC7568 commands ... 77
8.2 Port Expander’s Connections ... 80
8.2.1 Port Expander A ... 80
8.2.2 Port Expander D ... 81
8.3 DAC1 connections ... 82
8.4 MUX1 Connections... 83
8.5 Mapping of the PMTs channels ... 84
8.6 Code with the implementation of the HVREMOTE class ... 86
8.7 Code of the GUI ... 90
Figures Index
Figure 1.1. Frontal picture of the HV Remote board on the left, and of the backward part on the
right. ... 2
Figure 2.1. The ATLAS Detector layout. ... 4
Figure 2.2. Geometry of the magnetic system. In the centre is the solenoid system and around it, is the barrel toroid. At the sides are the end-cap toroids. ... 5
Figure 2.3. Image of the central Solenoid in the factory. ... 5
Figure 2.4. Picture of the Barrel Toroid. ... 5
Figure 2.5. Picture of one of the end-cap toroids. ... 5
Figure 2.6. A representation of the sub-detectors of the inner detector and their radial positions inside the cylindrical cavity... 6
Figure 2.7. Mechanical structure of the inner detector. ... 6
Figure 2.8. Scheme of the pixel detector. In the centre are the 3 pixel-layers, concentric to the beam axis and in the extremities are the 5 end-cap disk layers. ... 7
Figure 2.9. Layout of the semiconductor tracker (SCT)... 7
Figure 2.10. Straw tubes mounted in one of the end-cap TRT wheels. ... 8
Figure 2.11. Layout of the muon chambers. ... 8
Figure 2.12. ATLAS’s Electromagnetic and Hadronic Calorimeters. ... 9
Figure 3.1. Schematic of a wedge-shaped module of the TileCal Calorimeter. At the rear of the module there is a support girder that houses a unit called drawer... 10
Figure 3.2. TileCal’s modules and the drawer units. The WLS fibres are connected to the PMTs through a circular opening in the drawer units that contain the photomultipliers and all the front-end electronics. ... 11
Figure 3.3. Segmentation in depth and η of the TileCal’s barrel module (left) and extended barrel (right)... 12
Figure 3.4. Glued fibres in the girder insertion tube. ... 12
Figure 3.5. Picture of the HV Micro board. (1) is the Motorola microcontroller, (2) are 256 KB flash memories, (3) are the 256 KB RAM, (4) is the 2 KB EEPROM, (5) are opto-couplers of the CANBus interface and (6) is the connector to the CANBus cable... 13
Figure 3.6. Picture of part of the HV Opto. (1) is one of the regulation loops, (2) is a opto-coupler, (3) are two 100 MΩ HV resistor, (4) are 6 DACs, (5) are two -5 V regulators, (6) is the ADC, (7) is a voltage reference and (8) is a 2 KB EEPROM. ... 14
Figure 3.7. Electric scheme of the regulation Loop. ... 14
Figure 3.8. Two partial views of the HV Bus. In the top, the low and high voltage input connector (1) and the connectors of the PMTs’ HV (2). In the bottom picture, the connectors that link the HV Opto to the HV Bus (3) and the connector that links the two HV Bus boards (4). ... 14
Figure 3.9. Graph showing the predicted (blue) and achieved (green) integrated luminosity along the year of 2017. Image taken from [9]. ... 15
Figure 3.10. Super-drawer concept. In yellow new electronics, in green: adaptor boards for compatibility with the legacy system and in blue legacy system electronics. ... 16
Figure 3.11. Part of the electrical scheme of the new regulation loop. ... 17
Figure 4.1. Control tree of the first version of the HV Remote system... 19
Figure 4.2. Analogue circuitry of the HV Remote boards. These circuits are designed for one channel each. ... 20
Figure 4.3. Current control hierarchy of the HV regulation system. The FPGAs are connected to the DCS via ethernet and communicate with the HV Remote and Power Supplies boards via SPI.
... 21
Figure 4.4. Simplified scheme of the HV Remote’s digital control circuit. The FPGA converts the commands from the PC to digital signals, that are sent to the digital chips (Expanders, ADC or DACs) via SPI. ... 22
Figure 4.5. Example of SPI Master connected to 3 Slaves. Source: Wikipedia page about the SPI interface. ... 23
Figure 4.6. SPI clock modes. When CPOL = 0 the clock is idle at ‘0’ and activates at ‘1’ and when CPOL = 1, the clock is idle at ‘1’ and activates at ‘0’. When CPHA = 0, bits are clocked at the leading edge, and when CPHA = 1, bits are clocked at the trailing edge of clock signal. .... 24
Figure 4.7. Pinout of the MCP23S17. ... 25
Figure 4.8. Format of the Control Byte (device Opcode). The A2, A1 and A0 bits are the address of the device. The first 4 bits are fixed. ... 26
Figure 4.9. Format of the control byte and the Address Byte that contains the address of the register to whom the (read or write) operation is intended ... 26
Figure 4.10. Pin Configuration of the DAC7568 ... 29
Figure 4.11. Overall architecture of DAC7568. ... 30
Figure 4.12. Format of the 32-bit input shift register. ... 31
Figure 4.13. Temporal diagram exemplifying the SYNC operation. The first write sequence is invalid, because SYNC is brought high before the 32nd falling clock edge. The second sequence is sent with success because SYNC stays in a low state along all the bits transfer. ... 32
Figure 4.14. Write sequence for enabling the internal reference (Static Mode). Corresponds to the hexadecimal word 08000001h. ... 32
Figure 4.15. Write sequence for disabling the internal reference (static mode). Corresponds to the hexadecimal word 08000000h. ... 32
Figure 4.16. Write sequence for enabling the internal reference (flexible mode). Corresponds to the hexadecimal word 09080000h. ... 33
Figure 4.17. Write sequence for the internal reference to be always enabled (flexible mode). Corresponds to the hexadecimal word 090A0000h. ... 33
Figure 4.18. Write sequence for the internal reference to be always disabled (flexible mode). Corresponds to the hexadecimal word 090C0000h. ... 33
Figure 4.19. Write sequence to switch from flexible mode to static mode. Corresponds to the hexadecimal word 09000000h. ... 33
Figure 4.20. Pin configuration of the MAX1240. ... 34
Figure 4.21. Serial interface timing sequence. ... 35
Figure 4.22. Pin configuration on the left, and functional block diagram of the bus switch on the right. ... 35
Figure 4.23. Pin configuration of the left and functional block diagram on the right. ... 36
Figure 4.24. Inputs (SCLK, DIN, CS_CARD, CS_SPA, CS_SPB, CS_SPC and CS_SPD) and outputs (DOUT and EXP_OUT) of the HV Remote. In the centre the 8-bit bus switch is shown, connected to the FPGA on its left. The black filled triangles are the digital grounds of the HV Remote. ... 37 Figure 4.25. Port Expander D connections. On the bottom left corner is the voltage supply V3 that supplies the port expanders with 3.3 V. The address code of this expander is 011, with A2 connected to DGND and A1 and A0 connected to 3.3 V. Pins 19 and 20 are left unconnected. 38
Figure 4.26. Connections of DAC1. The capacitors in the reference pin (pin 7) and in the power supply are recommendations from the datasheets. Pin 12 is the ground of the DAC which is connected to the common digital ground DGND. ... 39 Figure 4.27. Connections of the ADC. AIN is the analogue input voltage that is selected by the multiplexers. After reading and converting the analogue voltage to a 12-bit digital word, the ADC sends it to the master device (FPGA) via the DOUT line that is linked to the SPI MISO line. .. 40 Figure 4.28. Connections of MUX1. Pins 2, 3 and 13 are left unconnected. VP12 and VN12 are the analogue voltage supplies ±12 V of the HV Remote. ... 41 Figure 5.1. Raspberry Pi b+ pinout. The name of the pins is shown as BCMi, where i corresponds to the numbering of the GPIOs of BCM2835. ... 43 Figure 5.2. Raspberry Pi SPI test code. Highlighted by the rectangles are the initialization of the SpiDev object (blue), the connection to the SPI controller (orange), the definition of the max clock speed (green), the SPI transaction (red) and the closing of the SPI connection to the controller (white). A delay of 0.1 s is created and is used after each print, in the while cycle. ... 46 Figure 5.3. MCP23S17 class constructor. The arguments, in blue, represent the SpiDev instance (spiDevice), the port expander address code (device_id), the Pi’s pin number associated with the port expander’s RESET pin (pin_reset), the Pi’s pin number associated with the port expander’s CS pin (chip_select) and an instance of the RPi.GPIO class (iGPIO). ... 47 Figure 5.4. Content of the MCP23S17 constructor (the method __init__(self, spiDevice=spidev.SpiDev(), device_id=0x00, pin_reset=-1, chip_select=-1, iGPIO = GPIO) ). ... 48 Figure 5.5. This is the “__writeRegister” private method. The first line of code defines the name of the method and the arguments it must be given. The second line asserts if the flag “isInitialized” has the Boolean value “True”. The SPI transaction is performed by the “__spi.xfer2” line. ... 49 Figure 5.6. Content of the “setDirection” public method. Takes in as argument a pin number of the MCP23S17 GPIOs and one of the values DIR_INPUT and DIR_OUTPUT. The acceptable numbers for the “pin” argument range from 1 to 8 (GPBs) and from 21 to 28 (GPAs). ... 50 Figure 5.7. MCP23S17 class methods. The ones that are preceded by the underscore character, “__”, are private methods, and the others are public methods. ... 51 Figure 5.8. Content of the “writeGPIO” method. It takes in as argument a 16-bit value (“data”). ... 51 Figure 5.9. Content of the “digitalWrite” method. Takes in as arguments a pin number of the MCP23S17 GPIOs, and one of the class’ constants LEVEL_HIGH or LEVEL_LOW that correspond to ‘1’ and ‘0’ respectively. ... 52 Figure 5.10. DAC7568 class constructor. Each instance must receive as arguments a SpiDev instance (“spiDevice”), a MCP23S17 instance (“MCP”) and a MCP23S17 pin number (SYNC). ... 53 Figure 5.11. Content of the DAC7568 constructor. It assigns the instance’s arguments to attributes and creates additional attributes that act as flags to the remainder of the program or indicators of the state of the DAC’s registers (DAC channel’s register, internal reference’s register and power-down logic’s registers). ... 54 Figure 5.12. DAC7568 class methods. The most important are the “writeAndUpdateChannel” and “enableReference”. ... 55 Figure 5.13. Content of the “__sendWord” private method. This method is responsible for performing all the DAC7568 SPI operations. It receives as arguments the different groups of bits of the DAC7568 32-bit input shift register. All the arguments must be 4-bit words, except for the “data” argument that must be a 16-bit word, where the first 12 bits are the valid sequence and the 4 last bits are ignored. ... 55
Figure 5.14. Code of the “writeAndUpdateChannel()” method. This method sets the respective bit groups to the correct values in order to perform a write and update channel operation. The method takes in as argument a letter (between A and H) and a value of 12 bits. On line 173 the argument “data” is transformed to a 16-bit value and stored in the variable “data_bits” which is the one that is inserted as argument in the “__sendWord” method. ... 56 Figure 5.15. MAX1240 class constructor. Arguments are a SpiDev instance (“spiDevice”), a pin number for the SHDN channel (“SHDN”), a pin number for the ADC chip-select (“chip_select”) and a MCP23S17 instance (“MCP”). ... 57 Figure 5.16. Content of the “readVoltage” method. It returns the value read by the ADC in bits. ... 57 Figure 5.17. HVREMOTE class constants and methods. ... 58 Figure 5.18. HVREMOTE class constructor. It has a private attribute for each of the main components of the HV Remote boards (MCP, DAC and ADC). ... 59 Figure 5.19. SpiDev, GPIO and HVREMOTE initializations. ... 60 Figure 5.20. Set HV tab of the GUI. Now all channels are disabled, since all checkboxes are unchecked. The orange rectangle signals one checkbox, the blue rectangle signal the labels, the red signals the combo box that allows to select one of the 48 channels, the black rectangle highlights an entry that allows to insert a HV value and the green rectangles highlight the buttons. ... 61 Figure 5.21. The read HV tab. This will be where the real-time plots will be added. For now, only a channel selector (red), a button (green), two labels (blue) and a display (purple) were added to test the HVREMOTE class. ... 62 Figure 5.22. Code of the “Channel” class. The “change_state” method handles the event of checking/unchecking the checkboxes. The methods “check_box” and “uncheck_box” are used to change the boxes states internally. ... 62 Figure 5.23. Content of the “MainWindow” class. The constructor’s code is presented. The class’ methods definitions are also shown. ... 63 Figure 5.24. HV Remote’s digital control emulated in a breadboard connected to the Raspberry Pi. The cables coming from the Pi (highlighted by the blue circle) are: the SPI MOSI (green), the SPI MISO (orange), the SPI SCLK (yellow), the chip-select of the port expanders (white), the 3.3 V (red and purple) and ground (black). Signalled by the red circle are the port expanders, signalled with the blue arrow is the DAC, signalled by the pink arrow is the ADC and signalled the green is the MUX. ... 64 Figure 5.25. Block Diagram of the breadboard setup. The orange line is the MISO connection, the yellow is the SPI clock, the green is the MOSI line, the blue is the port expander’s shared chip-select, the purple represents the MUX’s address lines, the red is the DAC’s SYNC and the black is the DAC’s output. ... 65 Figure 5.26. Testing the enable/disable channels. We can see that the LEDs are turned on, meaning that at that moment, those channels were enabled. ... 66 Figure 5.27. Set of write and read tests. It shows the value inserted by the user (“Channel i set to:”) and the values read by the ADC (“Channel i reading:”). A difference of 20 to 30 counts can be seen between the value specified to the DAC and the one that is read by the ADC. ... 67 Figure 5.28. Write-Read test for all the DAC’s eight channels. This test was made with a step of 15 counts and spans the entire valid range of data (from 0 to 4095). This test was made with a SPI clock frequency of 976 kHz. ... 68 Figure 5.29. Zoom of the middle region of the plot of Figure 6.28. ... 68 Figure 5.30. Plot of the error of the ADC readings versus value inserted to the DAC. This test
Figure 5.31. Plot of the error of the ADC readings versus value inserted to the DAC, with capacitors connected. This test was made with the same steps and same clock frequency as the ones above. ... 69 Figure 5.32. Error of the ADC’s readings with capacitors connected and without the MUX. The DAC’s channels were directly connected to the ADC. ... 71 Figure 5.33. SPI clock frequency tests. The plot of the right side is a zoom of the middle region of the left plot. These tests were made with a step of 1 count for the clock frequencies of 15.2, 61, 244 and 976 kHz. ... 71 Figure 5.34. SPI clock frequency tests. The plot of the left side shows the tests results for the clock frequencies of 1.953, 3.1, 7.8 and 15.6 MHz, and the plot of the right side shows the same results but without the 15.6 MHz data. ... 71 Figure 5.35. SPI clock frequency tests for 3.1 MHz and 1.953 MHz. This plot is a zoom of the middle region of the original plot, that shows the distribution of points with more detail. ... 72
Tables Index
Table 5.1. Digital coding to select the SPI mode ... 24
Table 5.2. Address mappings of all the registers, in the case where BANK = ‘1’ and BANK = ‘0’. ... 27
Table 5.3. IOCON register bits. The bit 0 is unimplemented. At POR the bits are ‘0’. ... 27
Table 5.4. IODIR register bits. At POR the bits are ‘1’ ... 28
Table 5.5. GPIO register bits. At POR the bits are ‘0’. ... 28
Table 5.6. Summary of all the registers with IOCON.BANK = 0 ... 29
Table 6.1. SPI clock speeds. ... 45
Table 6.2. Summary of the mean error and mean error variation for a setup without capacitors and with capacitors. ... 70
Acronyms
ADC Analog to Digital Converter ALICE A Large Ion Collider Experiment ATLAS A Toroidal LHC ApparatuS
CLK ClocK
CMS Compact Muon Spectrometer
CPHA Clock PHAse
CPOL Clock POLarity CS Chip Select
DAC Digital to Analog Converter DCS Detector Control System GPIO General Purpose Input Output GUI Graphical User Interface HL High Luminosity
HL LHC High Luminosity Large Hadron Collider HV High Voltage
HVLV High Voltage and Low Voltage LHC Large Hadron Collider
LHCb Large Hadron Collider beauty LSB Least Significant Bit
LV Low Voltage
MISO Master Input Slave Output MOSI Master Output Slave Input MSB Most Significant Bit
MUX MUltipleXer
POR Power On Reset
SCLK Serial CLocK
SPI Serial Peripheral Interface SSH Secure Shell
1 Introduction
1.1 Scope of this work
The ATLAS detector of the LHC in CERN, is under an upgrade program which includes improvements to its hadronic barrel calorimeter, named TileCal. This calorimeter has a cylindrical geometry with a tile structure. The tiles are made of a material that interacts with particles resultant from the collisions in the LHC, releasing high-frequency radiation. This radiation is collected by wave-length shifting fibers that convert the absorbed radiation to visible light and send it to photomultipliers tubes that convert it into an electrical signal. These photomultiplier tubes need a high voltage supply, which an HV regulation system provides.
The High-Luminosity LHC project predicts an increase in radiation, due to its aim of increasing the luminosity, which is an important factor for the performance of accelerators. This will have an impact on the HV regulation system’s electronics that are already obsolete. So, it was proposed to develop a new radiation-hard system with newer components. Two solutions are on their way: one on-detector solution, based in the old system but with replaced components, and a remote system complemented by multi-conductor cables that distribute the HV to the PMTs.
The remote system constitutes 256 pairs of boards, each pair comprising one HV distribution board (HV Remote) and one power supply board (Power Supplies). The boards will be controlled by 16 FPGAs (each controlling 16 pairs of boards). The communication between the boards and the FPGAs will be made by an SPI interface and the communication with the ATLAS DCS (Detector Control System) is made via an ethernet protocol.
1.2 Objectives
This work focuses on developing a software to control the (HV Remote) boards of the remote HV regulation system, that is being developed with elements of LIP (Laboratory of Instrumentation and Experimental Particle Physics) and DF-FCUL (Physics Department of Faculty of Sciences of the Lisbon University), and on the development of a GUI (Graphical User Interface) that will be used to functionally test the boards before they are sent to CERN. Besides, the boards will also undergo some temperature tests in order to see how its components behave at different temperature regimes.
The development of the software interface required a good understanding of the digital control circuits of the HV Remote boards, which includes a good knowledge of how the communication with its components is established.
1.3 Software implementation
In this work, instead of an FPGA, a Raspberry Pi was chosen to be used, although in the future the software must be adapted to the FPGA. The Raspberry Pi tests the whole control hierarchy of the remote system, before using the FPGA. The Pi also offers a very versatile and friendly user set of GPIO ports, which allow us to control two or three boards at the same time and perform several tests to the software, while it is being developed. Because of this, the software will be implemented directly in the Pi, using version 3 of the Python language. The Pi’s Python libraries come with two very useful modules that will be used in this project: the SpiDev module, that
establishes communication with the SPI peripherals of the Pi, therefore allowing us to transfer data with the HV Remote boards, without accessing the Pi’s hardware, and a RPi.GPIO library (that will be imported as simply GPIO), that directly controls the state of the Pi’s GPIO ports. The GUI will be written in Python 3 as well, but using an additional module, specifically designed for graphical interface developments. This module is named PyQt5, which implements a C++ library, Qt5, in the Python language. This module offers several functionalities that will be useful to build a good test interface. It is based on a set of layout and widget objects, where the widgets can be buttons, checkboxes, lists, insert lines, and other kind of user interactive objects, and the layouts allow us to freely place the widgets in any part of our application’s window.
1.4 State of the HV Remote board
When this thesis was completed, one HV Remote board has arrived, which will be the target of several tests. The digital control will only be tested after the most important connections have been verified and after guaranteeing the good functioning of the power supplies. A picture of this board is shown below. The software that was developed in this thesis will then be tested in this board, to test the operations on all the 48 channels.
1.5 Document structure
This document consists of 6 chapters, including this introduction. In the second chapter a description of the ATLAS and its sub-detectors is made. The third chapter focus on the TileCal calorimeter, giving a brief explanation of its mechanical structure, the tiles, the wavelength shifting fibers and the old HV regulation system of the PMTs. Chapter 4 mentions the high-luminosity upgrade that is being made to the LHC and its impact on the HV regulation system, pointing the key points that must be improved. Chapter 5 gives a description of the HV Remote board, and of its digital components. Finally, chapter 6 offers a brief explanation of the Raspberry Pi and explains how the software interface and the GUI were developed. In the end some tests to the GUI’s features are presented. This work ends with a conclusion in chapter 7, that summarizes what was accomplished and predicts future work.
Figure 1.1. Frontal picture of the HV Remote board on the left, and of the backward part on the right.
2 The Atlas Experiment
2.1 Overview of the ATLAS detector
The LHC (Large Hadron Collider) in CERN, is the largest particle accelerator in the world. It is a circular accelerator with a length of 27 km and is located 175 m deep beneath the surface. Its purpose is to accelerate beams of particles, thus increasing their energies, and collide them in the regions of its detectors.
There are four locations where the collision may happen, which correspond to 4 different detectors: ATLAS, CMS, ALICE and LHCb. ATLAS is a general-purpose detector of the LHC, that investigates a wide range of physics, from the search for the Higgs boson to extra dimensions and particles that could constitute the dark matter [1]. It is the largest volume particle detector ever built, until this date. The collisions that occur at the centre of ATLAS can reach energies of the order of TeV for p-p (proton – proton) collisions and hundreds of TeV for A-A (heavy ion – heavy ion) collisions. These will result in the creation of numerous particles that will fly in all directions. Some of the properties of these particles, such as momentum and energy will be measured by six different subsystems that are arranged in layers around the collision point. The conditions present in the LHC’s experiments and the phenomena resultant from the particle collisions imposes some requirements on ATLAS [2] [3]:
▪ It requires fast, radiation-hard electronics and sensor elements.
▪ High detector granularity, which help to handle particle fluxes and to reduce the influence of overlapping events.
▪ Large pseudorapidity1 coverage for all azimuthal angles. The azimuthal angle is measured
around the beam axis.
▪ Good charged-particle momentum resolution.
▪ Good electromagnetic calorimetry, which is necessary for electron and photon identification. ▪ Full-coverage hadronic calorimetry for accurate jet and missing transverse energy
measurements.
▪ High-precision muon momentum measurements, with the capability to guarantee accurate measurements at the highest luminosity (highest particle flux).
▪ Highly efficient triggering, since there will be a great quantity of events, so the detector must be able to efficiently select the ones with the most interest for the experiment.
A view of the overall layout of ATLAS is shown in Figure 2.1. It is composed of several cylindrical sections surrounding the interaction point (the location where the particles collide).
1 Pseudorapidity, 𝜂, is a measurement that is related to the angle that particles, resultant from the collisions, make with respect to the beam axis. It is defined as 𝜂 = − ln(tan 𝜃/2), where 𝜃 is the polar angle from the
The detector is composed of 4 main sub-detectors: ▪ The inner detector.
▪ The electromagnetic and hadronic calorimeters. ▪ The muon spectrometer.
▪ The magnetic system.
The magnetic system comprises a thin superconducting solenoid that surrounds the inner detector cavity and three large superconducting air-core toroids encompassing the calorimeters.
The inner detector is contained in a cylindrical cavity, located at the central part of the detector. It is composed of a discrete combination of high-resolution semiconductor pixel and strip detectors in its inner part and straw-tube tracking detectors in its outer part.
The electromagnetic calorimetry is composed of three (one barrel and two end-caps) high granularity liquid-argon (LAr) sampling calorimeters. The hadronic calorimetry is composed of a barrel calorimeter with a scintillator-tile structure and two LAr hadronic calorimeters at the end-caps. There are also two LAr forward calorimeters that provide both electromagnetic and hadronic measurements and extend the pseudorapidity coverage.
The calorimeters are surrounded by the muon spectrometer, that defines the overall dimensions of ATLAS, that presents a chamber structure, each with its own purpose. The muon spectrometer uses the magnetic fields generated by the magnetic system to perform muon momentum measurements.
2.2 The Magnet System
The ATLAS superconducting magnet system’s main task is to create two different magnetic fields that will allow the identification of particles and the measurement of its momentum. It features four large superconducting magnets (see Figure 2.2): one solenoid and three air-core toroids.
Surrounding the inner detector and aligned with the beam axis is the central solenoid, Figure 2.3, that was positioned in between the two end-cap electromagnetic calorimeters. This demanded a minimisation of the material thickness, in order to achieve the desired calorimeter performance. The system of three large air-core toroids (one barrel and two end-caps) surround the central solenoid and generate the magnetic field for the muon spectrometer. The two end-cap toroids (Figure 2.5) are inserted at each end of the barrel toroid (Figure 2.4), lining up with the central solenoid. Each of the three toroids consist of eight coils assembled radially and symmetrically around the beam axis.
Figure 2.2. Geometry of the magnetic system. In the centre is the solenoid system and around it, is the barrel toroid. At the sides are the end-cap toroids.
Figure 2.3. Image of the central Solenoid in the factory.
Figure 2.4. Picture of the Barrel Toroid. Figure 2.5. Picture of one of the end-cap toroids.
2.3 The Inner Detector
The inner detector is placed within a cylindrical cavity in the innermost part of ATLAS. Surrounding it, is the magnetic field generated by the central solenoid (section 2.2). The inner detector’s main function is to provide momentum and tracking measurements with high resolution. It also provides electron identification over a wide range of energies. The inner detector consists of three independent but complementary sub-detectors (Figure 2.6). In the inner radii a pixel detector and a semiconductor tracker (SCT) detector provide high-precision measurements, and in the outer radii a transition radiation tracker (TRT) arranged in straw tubes, provides continuous tracking measurements.
The mechanical layout of the inner detector is divided into three parts: one barrel extending over and two identical end-caps covering the rest of the cylindrical cavity. In the barrel region, the layers of the sub-detectors are arranged on concentric cylinders around the beam, while in the two end-caps the tracking detectors (SCT and TRT) are arranged in disk layers perpendicular to the beam axis, as shown in Figure 2.7.
Figure 2.6. A representation of the sub-detectors of the inner detector and their radial positions inside the cylindrical cavity.
2.3.1 The Pixel Detector
The pixel detector, Figure 2.8, is designed to provide a set of very granularity and high-precision measurements close to the interaction point. Because of this, it will be exposed to great quantities of radiation, so it requires the use of radiation-hard materials for its construction. The whole sub-detector was placed in the central barrel of the inner detector. There, pixel layers segmented in 𝑅𝜙 and 𝑧 coordinates, are arranged in 3 concentric cylinders around the beam, as shown in Figure 2.6 and Figure 2.8 (barrel layers 0, 1 and 2). At the end-caps of this central barrel, 5 pixel-layers in a disk format are placed perpendicular to the beam axis, on each side.
2.3.2 The semiconductor tracker (SCT)
The SCT is designed to provide precision tracking measurements in the intermediate radial length of the inner detector, and momentum, impact parameter, vertex position and pattern recognition measurements. A section of it is placed in the central barrel and the others are at the end-cap regions of the inner detector.
The barrel SCT consists of eight layers of silicon microstrip detectors that provide precision points in the 𝑅𝜙 and 𝑧 coordinates, using small angle stereo strips. The layers are mounted in 4 carbon-fibre cylinders, concentric to the beam axis. The end-cap modules use tapered strips, with one set aligned radially, and are mounted up in three rings onto nine disks, as shown in Figure 2.9.
Figure 2.8. Scheme of the pixel detector. In the centre are the 3 pixel-layers, concentric to the beam axis and in the extremities are the 5
end-cap disk layers.
2.3.3 The Transition Radiation Tracker (TRT)
The TRT consists on a series of straw tube detectors, each having 4 mm in diameter, designed to give information in the 𝑅𝜙 coordinate. It also contributes significantly to momentum measurements and its straw tubes also contain xenon gas, that allows for electron identification.
In the barrel region, the straws are aligned parallel to the beam axis, while in the end-cap regions the straws are arranged radially in wheels (part of one is shown in Figure 2.10).
2.4 The Muon Spectrometer
The muon spectrometer forms the outer part of ATLAS and it is designed to detect charged particles exiting the barrel and end-cap calorimeters and to measure their momentum. It is based on the magnetic deflection of muon tracks in the superconducting air-core toroid magnets of the magnetic system of ATLAS (chapter 2.2).
Precision measurements of the muon tracks are provided by two main chambers: the monitored drifted tubes (MDTs), which perform measurements in the principal bending direction of the magnetic field, and the cathode strip chambers (CSCs) that have a higher granularity and are used at larger pseudorapidity and closer to the interaction point. A trigger system composed of two main chambers, the resistive plate chambers (RPCs) that are used in the barrel, and the thin gap chambers (TGPs) that are used in the end-cap sections, provide track information, thus complementing the precision measurement chambers In the barrel the chambers are arranged in concentric cylinders with the beam axis. At the end-cap regions they are arranged in four disks also concentric to the beam axis. A layout of the muon spectrometer’s chambers is shown in Figure 2.11.
Figure 2.10. Straw tubes mounted in one of the end-cap TRT wheels.
2.5 Calorimeters
The calorimetry of ATLAS, Figure 2.12, consists of two main sampling calorimeter systems: the electromagnetic that measure the energy of electrons and photons and the hadronic calorimeters that measure the energy of particles that interact via the strong force (hadrons). These are composed of smaller sections of sampling detectors with full φ-symmetry. The calorimeters closest to the beam axis are housed in three cryostats: one barrel that contains the electromagnetic barrel calorimeter, and two end-caps, each containing an electromagnetic end-cap calorimeter (EMEC), a hadronic end-cap calorimeter (HEC) located behind the EMEC and a forward calorimeter (FCal), placed in the region closest to the beam. These calorimeters use liquid argon (Lar) as the active medium. Finally, in the outer region there is a hadronic sampling calorimeter with a cylindrical geometry that uses a structure of plastic scintillator tiles as the active medium and steel as the absorber. This calorimeter is designated TileCal and it will be described in more detail in chapter 3.
3 The TileCal Calorimeter
3.1 Overview of the calorimeter
The TileCal [4][5] is a hadronic sampling calorimeter. Its main task is to provide precision measurements of hadrons and missing transverse energy and energy reconstruction of the jets produced in p-p interactions. Being a hadronic calorimeter means that it is designed to measure particles that interact via the strong nuclear force. The TileCal forms the inner shell of the ATLAS detector, encompassing all ATLAS sub-detectors, except for the muon spectrometer.
The TileCal uses a laminate of steel plates of various dimensions as the absorber material and a scintillator tile structure as the active medium. The highly periodic structure of the system allows the construction of the detector in a modularized way. The scintillating tiles are placed in a plane perpendicular to the beam and are radially staggered in depth. The scintillating tiles are readout by wavelength shifting (WLS) that deliver the light to photomultipliers (PMTs) which amplify and convert the optical signal into an electrical signal. Each scintillating cell is readout by 2 PMTs.
3.2 The Mechanical Structure of TileCal
The TileCal has a cylindric geometry concentric to the beam axis and is subdivided into a central barrel and two extended barrels. Each barrel consists of 64 wedge-shaped modules, shown in Figure 3.1, each covering the azimuthal angle, of 2𝜋/64 ≈ 0.1. The assembled modules form a periodic structure of alternating layers of iron plates and scintillating tiles, allowing for a good sampling frequency and a compact calorimetry.
At the rear of each module there is a support girder, where some units called drawers are inserted. These units are mechanically divided in two but work together as a single unit (the whole unit is usually called super-drawer). They contain the photomultiplier tubes, the wavelength shifting fibers and all the front-end electronics, which includes the readout electronics and the high voltage regulation system of the PMTs. The low-voltage power supplies which power the readout electronics are mounted in an external steel box, which has the cross-section of the support girder.
Figure 3.1. Schematic of a wedge-shaped module of the TileCal Calorimeter. At the rear of the module there is a
The TileCal’s structure is self-supporting through a bearing connection between modules at the inner radius and a bolted plate connection at the outer radius, where the girder is mounted. The calorimeter is assembled by mounting and bolting the modules to each other, and the modules are built by bolting the girders to the steel-scintillating structures. In the end, shims are inserted in the inner and outer radius load-bearing surfaces to control the overall geometry and yield a nominal module-module azimuthal gap of 1.5 mm. Figure 3.2 shows the mechanical sections of TileCal, its wedge-shaped modules and the drawer units.
3.3 Optical Elements: the scintillating tiles, the wavelength shifting fibres
and the photomultipliers.
The active medium of the TileCal calorimeter consists of several plastic scintillating tiles with eleven different trapezoidal sizes (one for each depth in radius) [6]. The scintillating tiles are radially staggered in depth and normal to the beam line and are made of polystyrene (a polymer with a light emission peak around the 420 nm). Particles crossing the tiles will induce the production of ultraviolet light, that will be collected and subsequently converted to visible light by the wavelength shifting fibres. Light is collected at the tiles’ edges and propagates along the fibres by total reflection, being transmitted to a PMT.
The TileCal’s wedge-shaped modules are segmented in depth and in η, thus forming three different layers (A, BC, D) with different interaction lengths (Figure 3.3). This results in a cell structure, where each cell has a transversal segmentation. The cells are obtained by grouping a specific bundle of fibres, that bring light from one side of a group of tiles, to the same PMT. Each cell is readout by two PMTs, with each PMT reading one of its two sides.
Figure 3.2. TileCal’s modules and the drawer units. The WLS fibres
are connected to the PMTs through a circular opening in the drawer units that contain the photomultipliers and all the front-end electronics.
After the insertion of tiles and fibres into a module, the bundles of fibres are glued together into a fibre-insertion tube (Figure 3.4). These tubes are then fixed into the girder holes as shown in Figure 3.1, where they will serve as the optical interface to the PMTs.
The PMTs and the front-end electronics are housed in the mechanical units named super-drawers, which are inserted in the rigid steel girder. The drawers can be extracted of the girder, in order to perform maintenance on the PMTs, or on the electronics, during shut-down periods. The high-voltage regulation system, needed to supply and regulate the high high-voltage of the PMTs’ anodes, is also included in the front-end electronics.
Each super-drawer is mechanically divided into two smaller drawers that contain 24 PMTs each, thus each super-drawer contains a maximum of 48 PMTs. One drawer is needed in order to perform a reading of an entire module of the extended barrels, while for reading one module of the long barrel, two drawers are needed. In total, the TileCal has approximately 10000 PMT channels.
Figure 3.3. Segmentation in depth and η of the TileCal’s barrel module (left) and extended barrel
(right).
3.4 The old HV distribution system of the PMTs: HV Opto and HV Micro
boards
The HV distribution system [7] is responsible for supplying, monitoring and setting the high voltage of approximately 104 PMTs. The voltage can be changed to any value in the interval from
-500 to -900 V. In order to have stable measurements, the high voltage supplied to the PMTs must also be very stable, because a small change in the HV will induce a large gain variation.
The hardware of this system is located inside the super-drawer units that are divided into an internal and an external drawer. These drawers can house at maximum 24 PMTs. In total there are 256 drawers that are inserted on the girder of the 64 wedge-shaped modules.
The specifications of the HV system are:
▪ To individually adjust the HV of the TileCal’s PMTs;
▪ The applied HV must be stable with a spread smaller than 0.5 V and a ripple smaller than 20 mV peak to peak, in order to ensure the stability of the PMT’s voltage.
▪ Sensitivity to temperature variations smaller than 0.2 V/oC and insensitivity to humidity
values up to 60%;
▪ Insensitive to magnetic field values up to 0.1 T;
▪ Able to deliver an electric current of 350 μA to each PMT;
▪ Able to switch off some PMT channels, in order to reduce the impact on cell energy measurements.
The system was designed to 48 PMTs and had one HV power supply, that can supply the PMTs with either -830 V or -950 V, per super-drawer. The old HV regulation system consisted on: two HV Opto boards, one for each drawer; one HV Micro board; two long HV bus boards, one in each drawer; one short bus, HV Flex, that links the internal and external HVBus boards; and seven temperature probes, integrated on all HV boards that are monitored by the HV Micro.
The HV Micro board, shown in Figure 3.5, is the one that processes all the information. It is responsible for commanding the HV Opto boards, monitoring the input HV and LV (low voltage) of the power supplies, converting the DAC and ADC’s (located in the HV Opto) words to physical values and to make the link of the entire system to the DCS. The communication with the ATLAS DCS is established using a CANBus network [8] that is connected to a CANBus interface, integrated in the HVMicro’s microcontroller, through an opto-isolated interface.
Figure 3.5. Picture of the HV Micro board. (1) is the Motorola microcontroller, (2) are 256 KB flash memories, (3) are the 256 KB RAM, (4) is the 2 KB EEPROM, (5) are opto-couplers of the CANBus interface
The HV Opto boards, Figure 3.6, are responsible for regulating the HV of 24 PMTs. To each channel is associated a regulation loop and a 12-bit DAC, as shown in Figure 3.7, that allow this the voltage regulation. The 24 PMTs are divided into two groups of 12, which are connected to emergency switches that enable/disable the groups’ channels. Besides adjusting the HV and enabling/disabling the PMTs, the HV Opto boards also perform voltage readings to the PMTs’ channels and temperature probes.
Each HV Opto board contains only one ADC. To perform the necessary operations on the 24 channels, 3 multiplexers are used. The HV Opto also has a reference voltage to monitor the stability of the ADC.
The HV Bus boards, Figure 3.8, purpose is to provide mechanical support for the HV Micro and HV Opto boards and to make the connections between these two boards and between the HV Opto and the PMTs. Each HV Bus board also contains one temperature probe.
All the HV systems of the TileCal’s partitions are controlled and monitored independently by the DCS that uses a WinCC OA application. So, the user uses the DCS to perform the required operations, like reading the voltage of a PMT channel or adjusting its voltage and enabling/disabling channels. The DCS sends this command to the respective HV Micro that interprets the information and commands the HV Opto boards to perform the specified operations.
Figure 3.7. Electric scheme of the regulation Loop.
Figure 3.6. Picture of part of the HV Opto. (1) is one of the regulation loops, (2) is a opto-coupler, (3) are two 100 MΩ HV resistor, (4) are 6 DACs, (5) are two -5 V regulators, (6) is the ADC, (7) is a voltage reference and
(8) is a 2 KB EEPROM.
Figure 3.8. Two partial views of the HV Bus. In the top, the low and high voltage input connector (1) and the connectors of the PMTs’ HV (2). In the bottom picture, the
connectors that link the HV Opto to the HV Bus (3) and the connector that links the two HV Bus boards (4).
3.5 The High-Luminosity Upgrade of the LHC
The LHC luminosity exceeded its target in 2017, producing up to 60 collisions at each bunch crossing, with each bunch containing around 100 billion particles [9]. It has provided its two main experiments (ATLAS and CMS) with approximately 50 fb-1 (unit used to measure integrated
luminosity) of data, which corresponds to 5 billion million collisions. Figure 3.9 shows a graph where the predicted and achieved integrated luminosity for the year of 2017 are compared. As we can see the performance of the LHC far exceeded the predicted one.
Since the LHC has had an excellent performance, providing its experiments (ATLAS for example) with a large amount of data, including the discovery of the Higgs boson, CERN scientists are convinced that they can push its performance even further. This upgrade of the LHC will result in an increase of luminosity (which is the main indicator of the performance of an accelerator) and the energies involved in the collisions. This will allow physicists to study mechanisms and physical theories in much greater detail, such as the recently discovered Higgs boson, as well as to discover new rare phenomena that might reveal itself in future experiments. The materials budget for the upgrade is set at 950 million Swiss francs between 2015 and 2026. The HL-LHC (High Luminosity LHC) should be operational around 2026 [10].
Figure 3.9. Graph showing the predicted (blue) and achieved (green) integrated luminosity along the year of 2017. Image taken from [9].
3.5.1 The ATLAS upgrade for the HL-LHC
ATLAS (chapter 2) was initially designed to study proton-proton collisions at the LHC with center of mass energies up to 14 TeV at a maximum luminosity of 1034 cm-2s-1. The LHC is
undergoing a series of upgrades with the intent of increasing its luminosity by 10-fold. This will be a great challenge for ATLAS, since it will require significant detector optimizations, changes and improvements, specially to the systems located at the inner radii, due to the increase of radiation resultant from collisions. Consequently the Inner Detector, the forward calorimeter and forward muon wheels will be the most affected systems, by the increasing radiation and particle fluxes, while the barrel calorimeters and muon chambers are expected to handle the conditions of the HL-LHC, so they will not be modified (although they will need upgrades).
The ATLAS upgrade is planned in three phases: phase 0, phase 1 and phase 2 [11]. In phase 0 a new cooling system for the Inner Detector and a new neutron shielding for the Muon Spectrometer will be implemented. On phase 1 the end-cap module of the muon spectrometer, will be replaced by a new one. On the new MSW will be installed new detector technologies, like the Micromegas chambers and the Small Strip Thin Gap Chambers [12]. Phase 2 will mainly see Inner Detector and calorimeter upgrades, as well as improvements on the trigger and data acquisition system. At the end of this phase the integrated luminosity should reach its target value for the HL-LHC. The upgrades of the ATLAS calorimeters will mainly consist on an upgrade or replacement of the readout electronics of all calorimeters and a replacement of the cold electronics inside the LAr Hadronic end-cap calorimeters. The upgrades to the TileCal will be described with more detail in the next section.
3.5.2 The upgrades to the Tile Calorimeter
One of the objectives of the upgrades to the TileCal is to have an easier maneuverability of the drawer units. To accomplish this, the super-drawer will comprise four independent mini-drawers that will host up to 12 PMTs (Figure 3.10). This change will improve the reliability of the cooling circuits and will grant an easier access to the front-end electronics. Other changes will consist on an upgrade of TileCal’s on- and off-detector electronics, due to its obsolescence and a need for an increase in reliability and radiation tolerance.
It is proposed that each mini-drawer will contain (besides the 12 PMTs): 3in1 front-end boards to shape and amplify the PMTs signals, one mainboard that will control, monitor and readout the front-end boards; one daughterboard that will receive digitized signals from the mainboard and
Figure 3.10. Super-drawer concept. In yellow new electronics, in green: adaptor boards for compatibility with the legacy system and in blue legacy system electronics.
will perform the communication between the front-end and off-detector electronics; and one low voltage power supply that supplies the on-detector electronics with +10 V. To evaluate this concept a demonstrator is being designed with a compatibility with the legacy system, in order to be tested in the current TileCal [14].
The high voltage power supply system has two possible solutions for its upgrade. One consists on an on-detector solution that uses one HV board per mini-drawer to provide high voltage control up to 12 channels. These boards must be radiation hard and must communicate with the off-detector side via the daughterboard. The other solution, which is the scope of this work, is based on a remote system, where the regulation boards would be placed away from the detector, and the high voltage would be distributed to the PMTs via long multi-conductor cables.
There will also be a change to the voltage dividers of PMTs. Whereas before there was passive dividers that shared the voltages between the photocathode, the 8 dynodes and the anodes of the PMTs, some will be replaced by active dividers (transistors and diodes). The usage of these transistors will increase the high voltage noise, and as such, both solutions must change the regulation loop of the HV boards. After some tests [15], it was concluded that the old electrical scheme of the regulation loop, Figure 3.7, should be changed to the one of Figure 3.11, where the transistor near the opto-coupler was removed.
The remote system project will take a different approach for the communication interface with the DCS. The main advantage of this solution is that the electronics won’t be vulnerable to radiation, which will increase its performance and durability. Instead of using a dedicated controller board, as was done in the old system with the HV Micro, this solution proposes the use of several pairs of boards, each comprising a HV distribution and regulation board (HV Remote) and a power supply board (Power Supplies), that provides the necessary low voltage for the HV Remote’s components and a fixed high voltage value (either -830 V or -950 V). These boards will be controlled by FPGA development boards via the SPI communication protocol and will communicate with the ATLAS DCS through an ethernet network.
Figure 3.11. Part of the electrical scheme of the new regulation loop.
4 The HV Remote system
Answering the demands of the High-Luminosity upgrade of ATLAS, a new system for the high voltage control of the TileCal’s PMTs was proposed. It consists of several HV Remote boards that regulate and monitor the high voltage of the TileCal’s PMTs, each associated with a power supplies board (Power Supplies), that provides a fixed HV value (-830 V or -950 V) and the LV necessary for the HV Remote. These boards are going to be controlled by FPGA development boards that will be connected to the DCS via an ethernet network. All the boards will be allocated in crates (each containing up to 16 HV Remote and their dedicated power supplies boards, and one FPGA Development board) in the ATLAS’s US15, a room at 100 m from the detector, where the amount of radiation does not become a critical problem for the electronics. This also has the advantage of having a permanent access to the room, in order to perform an easier maintenance to the system, during the LHC shutdowns. The regulated high voltage will be distributed along multi-conductor cables of approximately 100 m length. This new HV remote system will also have a different communication interface with the DCS, an on/off control for each PMT’s channel and the small change in the regulation loop that was described in section 3.5.2.
This chapter will start with an explanation of the control hierarchy, followed by an explanation of the interface that is going to be used to communicate with the boards. After this, the main digital components used in this project will be explained. Finally, the electric schematics of the HV Remote boards and the purpose of each digital component will be explained in the end of this chapter.
4.1 General Description of the HV Remote System
Each HV Remote board will be able to regulate the HV of 48 PMTs (unlike the old HV Opto boards that only regulated 24 PMTs each). In total, 256 boards will be needed for the control of all the TileCal’s 9852 photomultipliers. They will be stored in crates where each crate will contain 16 HV Remote boards. Unlike the HV Opto (section 3.4), the HV Remote will be able to individually enable/disable the PMTs’ channels. Although the HV Opto also had the enable/disable feature, it could only be used on the odd or even channels of each 24 PMTs. This improvement of the HV Remote boards will result in a lower consumption and will prevent the acquisition of incoherent data.
Initially the control hierarchy of the new system was as shown in Figure 4.1. The PC workstation, configured as a node of the DCS of ATLAS, would receive the commands from the DCS via ethernet, and would resend them to a tree of ethernet switch ports, that would select the board to which the command was intended to. Attached to the HV Remote boards was a Tibbo module [16], that would receive the commands from the PC Workstation, interpret them and perform the necessary operations on the digital components of the board.
The communication between the Tibbo module and the digital components was established via an SPI (Serial Peripheral Interface) protocol, which is a synchronous serial communication, based on a master-slave architecture. The Tibbo module acted as the master and the HV Remote’s chips acted as the slaves. The SPI will be explained in more detail in section 4.2.
In the initial design the HV Remote boards were still designed for 24 PMTs and the control circuit was composed of:
▪ Three 16-bit serial-parallel port expanders (MCP23S17 [17]) to implement the enable/disable feature and to control the other components.
▪ Three 12-bit DACs (DAC7568 [18]) to adjust the PMT’s high voltages. ▪ Two temperature probes (TMP17 [23]) to measure the board’s temperature. ▪ One test voltage of 1.2 V (AD589 [24]).
▪ One 12-bit ADC (TLV2541 [19]) and two 16-bit MUXs (MPC506 [25]) responsible of measuring the PMT’s voltages, the temperature probes and the test voltage.
▪ One embedded Tibbo EM1206 module, that established the communication between the PC and the board.
▪ One DC/DC converter (MAX 3002 [26]). This was necessary to convert the digital signals because the digital signals of the HV Remote were of 5 V and the Tibbo signals were of 3.3 V.
These were all new components that were chosen in order to replace the obsolete ones that were present in the HV Opto boards and to change the digital control. It was expected that these will guarantee a long period of operation for the HV regulation system, without a need of replacement. The analogue circuitry, which includes a read voltage loop, a regulation loop, and a switching channel loop, is designed for one channel. So, each HV Remote, initially had 48 of these analogue circuits, that are shown in Figure 4.2. There are three main signals: RD_CHi which are signals
Figure 4.1. Control tree of the first version of the HV Remote system.
