System Status & News

Computer science student Vilde Roland Arntzen uses the IDUN clusters in her master thesis study to detect offensive and hateful language in social media comments. Using the HPC clusters enables her to build complex transformer-based models that transfer knowledge from general language understanding learned from objective corpora such as Wikipedia, newspapers, and books to the specific task of separating neutral, offensive, and hateful social media comments. The model used is called BERT, which stands for Bidirectional Encoder Representation from Transformers and is a state-of-the-art language model.

The difference between using more complex transformer-based models is that additionally to learning word occurrences, it can learn more complex contexts within the text. This makes a massive difference for hate speech detection, which is as dependent on context as the types of words used.

For example, in some cultures, it is more common to use curse words than in others, such as in the northern parts of Norway compared to the south. Although curse words are often used in some hateful comment types generally, a comment is not hateful because it contains a curse word. Likewise, a hateful comment does not necessarily contain any indicative hateful or curse words but can be expressed implicitly (E.g., through metaphors or sarcasm). Therefore, it is necessary to analyze other aspects of the comments. For instance, if it contains a targeted insult and whether the insult targets a group or an individual.

Dear IDUN/EPIC GPU users,

The EPIC/IDUN is a fantastic research tool. Since 2017 more than 28 international publications within a wide range of fields from NTNU/SINTEF has involved the use of the IDUN/EPIC GPGPU resources. To keep IDUN/EPIC updated and hopefully add resources we, all users and NTNU IT, need to be able to document what a fantastic research tool we have.
To keep track of scientific use of the machine we (IDI and NTNU IT) have an arXiv paper that describes the research infrastructure and list the scientific contributions that included IDUN/EPIC. However, to be able to keep track we need you all to include the reference (in paper or acknowledgement).  Such a reference enables us to document, and there by argue for support for maintenance and new investments in our research tool.
Please help to ensure that a reference in papers, PhD thesis and  master projects is in included.

Reference to IDUN/EPIC:
@article{Epic2019,
  title = {{{EPIC}}: {{An Energy}}-{{Efficient}}, {{High}}-{{Performance GPGPU Computing Research Infrastructure}}},
  shorttitle = {{{EPIC}}},
  author = {Sj{\"a}lander, Magnus and Jahre, Magnus and Tufte, Gunnar and Reissmann, Nico},
  year = {2019},
  month = dec,
  archivePrefix = {arXiv},
  eprint = {1912.05848},
  eprinttype = {arxiv},
  file = {/home/joh/.zotero/zotero/ga2rhggk.default/zotero/storage/BFIBWT2D/Själander et al. - 2019 - EPIC An Energy-Efficient, High-Performance GPGPU .pdf;/home/joh/.zotero/zotero/ga2rhggk.default/zotero/storage/XTB8TZRF/1912.html},
  journal = {arXiv:1912.05848 [cs]},
  keywords = {Computer Science - Distributed; Parallel; and Cluster Computing},
  primaryClass = {cs}
}  

Link to paper: https://arxiv.org/abs/1912.05848

Thanks
Gunnar Tufte
(Project coordinator for  EPIC/IDUN at IE/IDI)

Dear Idun User.

We have started the work on and are well underway of transitioning Idun to a new platform.

The new system is available on:
idun-login4.hpc.ntnu.no.
This node does not have any GPUs, so compilation for GPUs is not possible on this node.

For you this means:
1: Upgrade of the OS to centos8. This operating system comes with significant improvements and will also give us a path forward as the older centos/ platform is seeing an "End of life" in the not so distant future.
2: upgrade from old slurm 16 to new slurm 20. This will give us better management of the queueing system and more features.
3: Change from Hierrarchical modules/software structure to flat structure. This might create som issues for your job scripts and you may have to change your setup, depending on what modules you are currently using.

Your files will stay the same.


We have already moved at least 25% of the nodes to new setup and some users have been testing the setup.
We encourage you to start using this login node and test your setup as well.

We hope to move the current login nodes; idun-login[1-3] in the following weeks, unless there are som major showstoppers.

epic.hpc.ntnu.no is currently pointing to a GPU enabled login node. Next tuesday, this node will be migrated to the new system.
idun.hpc.ntnu.no is currently pointing the old system. Next tuesday, this will start pointing to the new system.

The old idun login will continue to be available on: idun-login1.hpc.ntnu.no for a while after the transition, but will probably be transfered to new system some time before easter.

Please report any troubles to: help@hpc.ntnu.no



In other News:
* We have a new partner in Idun: Jan Torgersen of Department of Mechanical and Industrial Engineering. Welcome aboard
* Kavli Institute has decided to expand their share with 6 computenodes and to buy into a compute node
* IDI is currently considering expanding their share with FPGA node(s).
* We are planning to replace the current storage system on Idun later this year. Hopefully in late autumn.