gLite 3.1기반 Belle Grid 구축 보고서 머리말 본 책자는2008 년 노벨물리학상을 배출한 KEK 의 Belle 실험을 위한 그리드 인프라를 구축하는 과정을 기술하고 있습니다. Belle 실험의 그리드 인프라는 주로MC(Mote Carlo) Production 을 수행하고 있으며 최근 후속 실험인 Belle II 에서는 기하급수적으로 늘어나는 데이터량과 프로세스량을 처리하기 위해 적극 적으로 그리드 도입을 서두르고 있습니다. 이 책자를 통해 향후 국내 연구자 들을 위한Belle 및 Belle II 실험 지원에 도움이 될 수 있기를 기원합니다 . 이 책자를 만들어내기까지 도움을 주신 모든 분들께 감사드립니다. 2009년 11 월 15 일 작 성 자: 김 법 균 (KISTI) 유 진 승 (KISTI) 윤 희 준 (KISTI) 권 석 면 (KISTI) Christophe Bonnaud (KISTI) 박 형 우 (KISTI) 장 행 진 (KISTI) gLite 3.1기반 Belle Grid 구축 보고서 제목 차례 Glossary ······························································································································· 1 Ⅰ. 개요 ······························································································································· 4 1. KEK 소개 ······················································································································· 4 Ⅱ. Belle Grid 구축 ··········································································································· 9 1. Belle Grid 구축 개요 ··································································································· 9 2. CE (with lcg-CE) ·········································································································· 9 (1) VOMS인증서 설치 (Installation of VOMS Certificate) ································· 10 (2) Belle VO 가입 ······································································································ 10 (3) VO Parameter Setting ·························································································· 10 (4) Pool Account 생성 ······························································································· 11 (5) GIP (Generic Information Service) 설정 ··························································· 14 (6)로컬 큐 (Local Queue) 확인 및 Job Submission 설정 ··································· 15 3. WN (Worker Nodes) ····································································································· 16 4. 설치 테스트 ··················································································································· 16 (1) 로컬 스케쥴러 ······································································································ 16 (2) Information System ······························································································· 17 (3) 그리드 작업 수행 ································································································· 18 5. Belle Application Test ·································································································· 19 (1) Belle Application을 위한 NFS 요구사항 ·························································· 19 (2) NFS 서버 ·············································································································· 19 (3) WN및 CE ············································································································ 21 Appendix A. site-info.def for Belle VO ·········································································· 23 gLite 3.1기반 Belle Grid 구축 보고서 표 차례 <표 1> Current Belle Computing ····················································································· 8 gLite 3.1기반 Belle Grid 구축 보고서 그림 차례 그림1 KEK 에서 배출한 2008 년 노벨물리학상 ··························································· 5 그림2 Belle 실험 검출기 ································································································ 5 그림 3 KEK Organization Chart ······················································································ 6 그림4 KEKB 가속기 및 Belle 검출기 ·········································································· 7 그림 5 Belle Collaboration ······························································································· 7 그림6 2009 년 10 월 현재 Belle VO 지원 그리드 사이트 ·········································· 22 그림7 Belle Grid 활용 현황 1 ······················································································· 22 gLite 3.1기반 Belle Grid 구축 보고서 Glossary □ AFS : Andrew File System □ API : Application Programming Interface □ BDII : Berkeley Database Information Index □ CASTOR : CERN Advanced STORage manager □ CE : Computing Element □ CERN : European Laboratory for Particle Physics □ ClassAd : Classified advertisement (Condor) □ CLI : Command Line Interface □ CNAF : INFN's National Center for Telematics and Informatics □ CREAM : Computing Resource Execution And Management □ dcap : dCache Access Protocol □ DIT : Directory Information Tree (LDAP) □ DLI : Data Location Interface □ DN : Distinguished Name □ EDG : European DataGrid □ EDT : European DataTAG □ EGEE : Enabling Grids for E-sciencE □ ESM : Experiment Software Manager □ FCR : Freedom of Choice for Resources □ FNAL : Fermi National Accelerator Laboratory □ FTS : File Transfer Service □ GFAL : Grid File Access Library □ GG : Grid Gate (aka gatekeeper) □ GGF : Global Grid Forum (now called OGF) □ GGUS : Global Grid User Support □ GIIS : Grid Index Information Server □ GLUE : Grid Laboratory for a Uniform Environment □ GMA : Grid Monitoring Archictecture □ GOC : Grid Operations Centre □ GRAM : Grid Resource Allocation Manager - 1 - gLite 3.1기반 Belle Grid 구축 보고서 □ GRIS : Grid Resource Information Service □ GSI : Grid Security Infrastructure □ gsidcap : GSI-enabled version of the dCache Access Protocol □ gsirfio : GSI-enabled version of the Remote File Input/Output protocol □ GUI : Graphical User Interface □ GUID : Grid Unique ID □ HSM : Hierarchical Storage Manager □ ICE : Interface to CREAM Environment □ ID : Identifier □ INFN : Istituto Nazionale di Fisica Nucleare □ IS : Information Service □ JDL : Job Description Language □ kdcap : Kerberos-enabled version of the dCache Access Protocol □ LAN : Local Area Network □ LB : Logging and Bookkeeping Service □ LDAP : Lightweight Directory Access Protocol □ LFC : LCG File Catalogue □ LFN : Logical File Name □ LHC : Large Hadron Collider □ LCG : LHC Computing Grid □ LRC : Local Replica Catalogue □ LRMS : Local Resource Management System □ LSF : Load Sharing Facility □ MDS : Monitoring and Discovery Service □ MPI : Message Passing Interface □ MSS : Mass Storage System □ NS : Network Server □ OGF : Open Grid Forum (formerly called GGF) □ OS : Operating System □ PBS : Portable Batch System □ PFN : Physical File name □ PID : Process IDentifier □ POOL : Pool of Persistent Objects for LHC - 2 - gLite 3.1기반 Belle Grid 구축 보고서 □ PPS : Pre-Production Service □ RAL : Rutherford Appleton Laboratory □ RB : Resource Broker □ RFIO : Remote File Input/Output □ R-GMA : Relational Grid Monitoring Archictecture □ RLI : Replica Location Index □ RLS : Replica Location Service □ RM : Replica Manager □ RMC : Replica Metadata Catalogue □ RMS : Replica Management System □ ROC : Regional Operations Centre □ ROS : Replica Optimization Service □ SAM : Service Availability Monitoring □ SASL : Simple Authorization & Security Layer (LDAP) □ SE : Storage Element □ SFN : Site File Name □ SMP : Symmetric Multi Processor □ SN : Subject Name □ SRM : Storage Resource Manager □ SURL : Storage URL □ TURL : Transport URL □ UI : User Interface □ URI : Uniform Resource Identifier □ URL : Uniform Resource Locator □ UUID : Universal Unique ID □ VDT : Virtual Data Toolkit □ VO : Virtual Organization □ WLCG : Worldwide LHC Computing Grid □ WMS : Workload Management System □ WN : Worker Node □ WPn : Work Package #n - 3 - gLite 3.1기반 Belle Grid 구축 보고서 Ⅰ 개요 1. KEK 소개 □ KEK :高エネルギー 加速器研究機構 Kō Enerugī Kasokuki Kenkyū Kikō ○ High Energy Accelerator Research Organization ○ 일본쯔꾸바(Tsukuba) 에위치한고에너지물리연구기관 ○ 1955년설립된 INS(Institute of Nuclear Study) 를모태로하여 1971 년 KEK 가 설립되어 오늘에 이름 ○ 전자양전자충돌가속기인- KEKB 검출기를활용한 Belle 실험을진행 ○ Kobayashi-Maskawa가 1975 년에발표한이론을최근 Belle 실험에서증명하여 Kobayshi와 Maskawa 가 2008 년노벨물리학상을수상 ○ 2009년초까지 KEKB 검출기를통해데이터획득후 3 년간 shutdown 기간을 통해가속기를upgrade 하여 2012 년부터기존의가속기보다 50 배이상많은 양의데이터를생산하는Belle II 실험을수행할계획임 ○ KEK는 High Energy Physics 를연구하는기관으로가속기를활용하여생성되 는 데이터를 분석하는 연구를 하고 있음 ○ Belle실험은 KEK 의여러실험중의하나이며 KEK 내의 IPNS(Institute of Particle and Nuclear Studies)에서 취급하고 있음 □ KEK의 컴퓨팅 기능 구성 ○ 다양한실험을동시에지원하는CRC(Computing Research Center) 와각실험별 컴퓨팅 센터가 존재함 ○ Belle실험의 경우 별도의 컴퓨팅 자원을 보유하고 있으며 B-Factory Computer 또는B-Computer 라 불림 ○ Belle Grid는 CRC 에서구축ㆍ 담당하고있음 – Belle Grid와관련한협력은 CRC 를통해이루어짐 – 실험데이터확보는Belle Computing 담당자와의협의를통해이루어져야 함 ○ Belle Computing의 책임자는 Nobuhiko Katayama 임 - 4 - gLite 3.1기반 Belle Grid 구축 보고서 노벨 물리학상 2008 16 Citation: 5483 그림 1. KEK에서 배출한 2008 년 노벨물리학상 그림 2 Belle 실험 검출기 - 5 - gLite 3.1기반 Belle Grid 구축 보고서 그림 3 KEK Organization Chart - 6 - gLite 3.1기반 Belle Grid 구축 보고서 그림 4 KEKB가속기 및 Belle 검출기 그림 5 Belle Collaboration - 7 - gLite 3.1기반 Belle Grid 구축 보고서 1999~ 2001~ 2006~ 2009~ (4 years) (5 years) (6 years) (continued) ~ 100 ~ 1200 ~ 42500 ~ 115200 CPU (SI2K) (WS) (WS + PC) (PC) (PC) Disk (TB) 4 9 1000 1500 Tape (TB) 160 620 3500 3500 80 80 Workgroup 3 + 9 11 (+16 (+26 server (# hosts) file servers) file servers) User 23WS + Workstation (# 28WS + 68X 128PC 128PC 100PC hosts) 표 1 Current Belle Computing - 8 - gLite 3.1기반 Belle Grid 구축 보고서 Ⅱ Belle Grid 구축 1. Belle Grid 구축 개요 □ Belle Grid는 WLCG 인프라에서 동작하므로 기본적으로 gLite 미들웨어를 기반으 로 그리드가 구축된다. ○ 즉기존의, LCG-CE (gLite3.1) 를 Belle VO 가이용할수있도록설정해주어야 한다. □ Belle VO가 로컬 자원을 이용할 수 있도록 하기 위해 필요한 절차를 요약하면 다음과 같다. ○ CE (with lcg-CE) – Install VOMS Certificate – Join Belle VO – VO Parameter Setting – Create Pool accounts – Configure Information Service (GIP) – Check Local Queue & Configure Job Submission ○ WN – Install VOMS Certificate – VO