A Silicon Detector Measurements DB
Total Page：16
File Type：pdf, Size：1020Kb
A silicon detector measurements DB FeasibilityFeasibility studystudy && progressprogress reportreport MatthiasMatthias Bergholz,Bergholz, WolfgangWolfgang FriebelFriebel CERN, Nov 25.2009 Outline of the talk RequirementsRequirements andand designdesign principlesprinciples LayoutLayout ofof thethe databasedatabase ComponentsComponents ofof thethe frameworkframework TheThe useruser interfaceinterface StatusStatus ofof thethe projectproject NextNext stepssteps Nov 25 2 Requirements for the DB FirstFirst measurementsmeasurements comingcoming soonsoon DB needed with at least limited functionality early in 2010 Ambitious schedule for development AccessibilityAccessibility ofof thethe DBDB As simple as possible with few or no restricttions BeBe preparedprepared forfor designdesign changeschanges Due to the nature of an R&D project flexibility is important UserUser interfaceinterface shouldshould bebe easyeasy toto useuse andand powerfulpowerful Upload of data needs to be simple and efficient Complicated queries should be possible Nov 25 3 Design principles RapidRapid prototypingprototyping essentialessential duedue toto timetime constraintsconstraints Working solutions can be discussed, fast iterations, proof of concepts Alternatively reuse of existing solutions LayeredLayered conceptconcept toto hidehide implementationimplementation detailsdetails fromfrom useruser Allows for design changes without user visible effects Access to the DB possible but discouraged WebWeb applicationapplication insteadinstead ofof GUIGUI No need to install software for using the DB Web access almost everywhere possible (no restrictions) Nov 25 Established scalable solution with many options (auth, 4 proxy, ...) Design principles (2) FlexibilityFlexibility inin thethe selectionselection ofof aa databasedatabase Use SQL without advanced or DB specific constructs if possible Migration to a different DB should be easily possible AutomationAutomation ofof taskstasks shouldshould bebe possiblepossible As little language dependency as possible for easy tasks Access to the project internals should be possible (language and learning curve matters) MaintainabilityMaintainability Modularity is important to decrease bug hunting time Nov 25 Automated testing procedures need to ensure quality 5 Design choices SeveralSeveral solutionssolutions forfor rapidrapid prototypingprototyping forfor thethe webweb For many languages, all inspired by Ruby on Rails (2004) e.g. Django (Python) Implementation using a Model-View-Controller (MVC) design i.e. separation of the data model from the program logic and visualisation CatalystCatalyst (perl)(perl) hashas beenbeen chosenchosen Huge user base, vast number of perl modules worldwide known as comprehensive perl archive network, CPAN Nov 25 Familiarity with the language 6 Many design principles can be easily fulfilled (eg. DB choice) Highly automated generation of web user interfaces Design choices (2) StartingStarting withwith mysqlmysql asas DBDB backendbackend Also tried: migration to sqlite3 as proof of concept PutPut asas muchmuch infoinfo intointo thethe datadata modelmodel (constraints,(constraints, rules,rules, transformations)transformations) insteadinstead ofof writingwriting programprogram logiclogic UsingUsing thethe TemplateTemplate ToolkitToolkit 22 forfor webweb pagespages layoutlayout e.g. used in mailing list management software “sympa” WillWill needneed toto enhanceenhance webweb interfaceinterface byby AJAXAJAX (usability)(usability) Most promising candidate: prototype library Nov 25 7 Communication with the web server then using JSON AlternativesAlternatives toto HTMLHTML outputoutput possible:possible: e.g.e.g. XMLXML Layout of the database PrincipalPrincipal tablestables relatedrelated toto taskstasks toto bebe performedperformed withwith thethe DBDB history - contains all actions recorded in the DB meas_action – describes what measurements are available mod_action – actions that modify the detector (e.g. irradiation) ship_action – transfer of detectors between centers sim_action – simulation results for e.g. field maps etc. free_action – everything else person, center – tables related to places, people and accounts person_role, role – additional role based authorisation parameter, meas_value – measured parameters Nov 25 More tables for parameters, measurement methods and 8 related values existing, simplified here Person management person id name center prename phone id email person_role name center_id person_id country accountname role_id town password street created postcode updated expirydate role id role Nov 25 9 Structure management type id name substructure wafer structure id id name id name type_id name wafertype structure_id wafer_id technology status_id number status id name Nov 25 10 Action mgmnt, e.g. measurement parameter id name actiontype_id actiontype unit meas_action_value id required id name derivated meas_action_id parameter_id history meas_action value id id date uploaddate substructure_id datalocation annotation person_id filename id actiontype_id errorflag name annotation_id history_id Nov 25 11 Database flexibility AvoidAvoid schemaschema changeschanges ifif possiblepossible UseUse separateseparate tablestables holdingholding listslists ofof thingsthings thatthat maymay needneed toto bebe modifiedmodified later,later, e.g.e.g. parameter – just the name, the units and a pointer to a method in a table instead of fixed parameter columns meas_value –– a value, a pointer to the parameter (name), a pointer to the actual measurement DoDo notnot relyrely onon advancedadvanced SQLSQL functionalityfunctionality Can be compensated by writing perl code Can be compensated by more complicated templates Nov 25 12 Components of the framework DatabaseDatabase mysql,mysql, provideprovide aa schemaschema onlyonly ObjectObject relationalrelational mappermapper Tables become classes, columns become methods Additional functionality by enhancing methods WebWeb contentcontent defineddefined usingusing templatestemplates (TT2)(TT2) WebWeb layoutlayout (Colors,(Colors, TextText sizesize etc)etc) byby CSSCSS AdditionalAdditional classesclasses forfor programprogram logiclogic (controller)(controller) AJAXAJAX forfor easiereasier useruser interactioninteraction withwith thethe WebWeb GUIGUI AJAX = “Asynchronous JavaScript and XML” e.g. autocompletion of names, drag and drop etc. Nov 25 13 The object relational mapper package CecDB::Schema::Result::Role; use base 'DBIx::Class'; __PACKAGE__->load_components( "InflateColumn::DateTime", "TimeStamp", "EncodedColumn", "Core", ); __PACKAGE__->table("role"); __PACKAGE__->add_columns( "id", { generated by mapper data_type => "INTEGER", default_value => undef, is_nullable => 0, size => undef, }, "role", { data_type => "varchar", default_value => "NULL", relation between tables is_nullable => 1, size => 15, }, ); __PACKAGE__->set_primary_key("id"); # Created by DBIx::Class::Schema::Loader v0.04006 @ 2009-09-15 12:01:31 # You can replace this text with custom content, and it will be preserved on regeneration use overload '""' => sub {$_[0]->id}, fallback => 1; __PACKAGE__->utf8_columns(qw/id role/); __PACKAGE__->has_many(map_user_roles => 'CecDB::Schema::Result::PersonRole', 'role_id'); Nov 25 14 Enhancing the data model package CecDB::Schema::Result::Person; ... "password", { data_type => "VARCHAR", default_value => undef, size => 63, }, ... # overwrite some of the definitions above __PACKAGE__->add_columns( "password", instead of storing plain { data_type => "VARCHAR", passwords only an encrypted default_value => undef, version can enter the DB size => 31, encode_column => 1, encode_class => 'Digest', additional methods, e.g. encode_args => {salt_length => 10}, checks for min length and encode_check_method => 'check_password', content may be added }, ... Nov 25 15 Enhancing the data model (2) NewNew pseudopseudo columnscolumns cancan bebe createdcreated thatthat dodo notnot existexist inin thethe DBDB Note: columns become methods Additional methods can be defined InIn templatestemplates methodsmethods areare calledcalled Makes no difference whether method corresponds to a real table column or is a newly added method The “has_many” method does generate such new methods dynamically Nov 25 16 Flexibility by using roles PersonsPersons getget assignedassigned aa rolerole Can be used to restrict access rights Can be used to generate different views to the data example: edit functionality for tables may be added OtherOther informationinformation fromfrom thethe DBDB andand thethe GUIGUI useableuseable Location dependent display of information Location is obtained from REMOTE_ADDR, i.e. Calling host example: phone number 79454 displayed as 79454 if host from cern.ch 022/7679454 if host from ch +41-022/7679454 if host from outside ch Nov 25 ThereforeTherefore loginlogin withwith accountaccount namename usefuluseful 17 A sample template [% # Provide a title -%] [% META title = 'Detector List' -%] <table> <tr><th>Name</th><th>Detail</th><th>Type</th><th>Links</th></tr> [% # Display each detector in a table row %] [% FOREACH detector IN detectors -%] <tr> <td>[% detector.name %]</td> <td>[% detector.subname.name %]</td> <td>[% detector.type.name %]</td> <td> [% # Add a link to delete a detector %] <a href="[% c.uri_for(c.controller.action_for('delete'), [detector.id]) %]">Delete</a> </td> </tr> [% END -%] </table> <p> For any other action than "list detectors" you need to <a href="[% c.uri_for('/login') %]">Login</a><br> <p>Roles for [% c.user.email %]:<br> <ul> [% FOR role =c.user.roles