DEVELOPER GUIDE PIPELINE PILOT INTEGRATION COLLECTION 2016 Copyright Notice
©2015 Dassault Systèmes. All rights reserved. 3DEXPERIENCE, the Compass icon and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA and NETVIBES are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval.
Acknowledgments and References
To print photographs or files of computational results (figures and/or data) obtained using BIOVIA software, acknowledge the source in an appropriate format. For example: "Computational results obtained using software programs from Dassault Systèmes BIOVIA. The ab initio calculations were performed with the DMol3 program, and graphical displays generated with Pipeline Pilot."
BIOVIA may grant permission to republish or reprint its copyrighted materials. Requests should be submitted to BIOVIA Support, either through electronic mail to [email protected], or in writing to:
BIOVIA Support 5005 Wateridge Vista Drive, San Diego, CA 92121 USA Contents
Chapter 1: Introduction 1 Creating Subprotocol Components 32 Who Should Read this Guide 1 Data Flow 32 Requirements 1 Creating the Interface 32 Additional Information 1 Ports 33 Chapter 2: Creating Components 2 Global Properties 33 Review-o-Matic Component 2 Reusing Subprotocol Components 33 Standard Component Categories 2 Subprotocol Customization Checklist 34 Readers 3 Organizing your Saved Components 35 For Each Data Components 4 Publishing your Components 37 Writers 4 Chapter 3: Creating a Package 39 Viewers 5 Package Deployment Checklist 39 Calculators 5 Planning Data Record Types 40 Reporting 6 About the Type of a Data Record 40 Filters 6 Best Practices for Creating New Data Record Validator Filter Components 7 Types 40 Converters 7 Adding Components to Packages 41 Utilities 8 Adding Package-Specific Components 41 Utility Subfolders 8 Component Creation Checklist 41 Manipulators 9 Adding Protocol Examples 42 Learners 9 Naming and Organizing Example Protocols 42 Customizing the Component Interface 10 Protocol Examples Checklist 42 Configuring Parameters 11 Adding Regressions 44 Parameter Enabling 13 Regression Checklist 44 Parameter Customization Checklist 14 Documenting a Component Collection 44 Parameter Groups 15 Parameter Groups Checklist 16 Configuring Ports and Icons 17 Documenting Component Help 19 Summary Help 20 Guidelines for Writing Summary Help Content 20 Description Help Content 21 Guidelines for Writing Description Help Content 22 Formatting Conventions 23 Cross-Referencing with Links 24 Adding Links to Components 25 Component Creation Checklist 26 Manually Publishing a Component 27 Publishing a Component as Part of a Package 27 Data Selection Checklist 28 Functionality Checklist 29 Performance Checklist 29 Naming Convention Checklist 30 Chapter 1: Introduction
You can extend Pipeline Pilot by creating your own components and protocols. These new items may be for your own use, or they can be "published" to a shared area for others to use. You can customize Pipeline Pilot to provide domain-specific functionality or examples, or deliver ready-to-use applications. This guide begins by describing the process for creating your own components, presented in several chapters. It is often the case that more than one component is desired as the outcome of a development process. The last part of this guide covers how to develop and deliver larger projects as "component collections". Note: You can develop new components using different language-based tools, such as .NET, Perl, and Java. Information on developing components with these kinds of tools is outside the scope of this document. For further details, see the language-based integration publications in the Help Center Developers tab (Server-side Integration > Language-based Integration).
Who Should Read this Guide This guide provides information about how to create new components or design a new component collection. It assumes you have some familiarity with using Pipeline Pilot Client and that you already have an idea about what kind of component or collection you need to develop. Requirements To develop components, you need some experience working with components and protocols. You should have a basic understanding of how to use Pipeline Pilot and the Pipeline Pilot Client to build and run protocols. Step-by-step instructions for using the client are available in the online help. Additional Information For more information about Pipeline Pilot and other BIOVIA software products, visit https://community.3dsbiovia.com.
Introduction | Page 1 Chapter 2: Creating Components
You can design new components by customizing the interface of a standard component and then saving it with a new name. You can also create a completely new component implemented as a subprotocol. After you create your component in the workspace, you can save it to your "User Name" tab. When creating a new component, much of the work involves defining the look and feel (interface). You can customize the following component interface features: Component icons Input and output ports Component help text (including tooltip help) Parameters (values, order and grouping, exposed/hidden, internals) You frequently will also want to change the behavior (implementation). The most commonly used methods for altering implementation include: Defining scripts used in parameters (e.g., PilotScript or Perl) Invoking external code (e.g., Java libraries) Other components may be connected to define an implementation, and then wrapped into a "subprotocol" to hide complexity. This chapter covers tasks involved in creating custom components. For step-by-step instructions on using the Pipeline Pilot Client to customize components, see the online help (Help Center > Users tab).
Review-o-Matic Component It is recommended that you make frequent use of the Review-o-Matic component, which can evaluate all components in a folder for common issues. 1. From Pipeline Pilot Client, open the Components tab and search for and open the Component Review-o-Matic component. 2. Set the Component Folder to the folder of components to be validated and click Run.
Standard Component Categories First, find the component you wish to customize, and drag it into an empty workspace. You'll do all the work on the component here, then save the finished version on your user tab. But before you continue creating the component interface, you need to know what a "standard" component looks like. Trying to fit your new component into this convention will make it easier to understand and use. So, take a little time and review some of the standard component categories. When end users start working with a new component, they expect to find it analogous to a previously- seen "standard" component; then, they can use their experience with the pre-existing component to help them understand and use the new component. While none of these standard components features are "required", following these standards gives your end users a head-start in understanding using your new components. The component categories include: Readers Writers
Creating Components | Page 2 Viewers Calculators Reporting Filters Converters Utilities Manipulators Learners Tip: It is typical to save your components in a hierarchy reflecting these categories, a topic discussed later in this chapter.
Readers The most common standard component is a reader; indeed, often more than one reader is available in particular domains. Each data record type should have its own reader. If multiple formats are available, you have a choice: a single reader with a Format parameter, or separate readers for each format. If the parameters are similar, a single reader is appropriate. Multiple readers are more appropriate if the parameters for each format have unique elements (where a unified reader would have so many disabled parameters that it would be difficult to understand). The standard features of a reader include: Feature Description Component name The component name should be
Page 3 | Pipeline Pilot • Developer Guide Feature Description Parameters A reader component should include the following parameters: Source: A reader of files should have a Source parameter (with type URLType). A reader of directories should have a Directory Path parameter, (with type DirectoryPath). Maximum: A Maximum parameter (type LongType) specifies the maximum number of outputs. It should be nested under the Source parameter. SourceTag: A SourceTag parameter (type StringType) specifies if the property SourceTag is defined on the outgoing data records. The standard set of legal values is: None, Filename, FilenameAndExtension, FullFilename, ZipFilename, Number, Letter. It should be nested under Source parameter. Keep Properties: A Keep Properties parameter (type PropertyPreviewType) specifies a list of property names to keep on the data record (all other properties are removed). This is also useful for inspecting the source to see what properties are in it.) It should be nested under the Source parameter. Recurse Directories: For directory readers, a Recurse Directories parameter (type BoolType) declares if a single level of the directory is searched, or if the search is recursive. It should be nested under the Directory Path parameter. Folder location Publish your reader components in a "Readers" folder. For Each Data Components There is a special kind of reader (For Each Data) that accepts the name of the source from a property on the input data stream, then reads from that source until complete, repeating this process as long as there is input. For Each Data components differ from standard reader components because they have an input port, and instead of a Source parameter, they include a Source Property parameter. For clarity, component names of this type should end with the word "For Each Data" (e.g., URL Reader For Each Data). Writers Writers are the mirror image of readers. It is typical (though not required) to provide a writer for each format you can read. The standard features of a writer include: Feature Description Component name The component name should be
Creating Components | Page 4 Feature Description Ports There should be no output port. Parameters A writer component should include the following parameters: Destination: A writer should have a Destination parameter (type URLType). Maximum: A Maximum parameter (type LongType) specifies the maximum number of records that are written. It should be nested under the Destination parameter. ifFileExists: An IfFileExists parameter (type BoolType) specifies the behavior if the destination is a file and already exists. The standard set of legal values are: Overwrite (the default), Append, and Halt. Folder location Publish your writer components in a "Writers" folder.
Viewers A viewer displays the incoming data using some application on your client. You can create one or more data type-specific viewers (i.e., "XYZ Viewer", where
Calculators Calculators are a special type of component that allows for the generation of a specifically-named property on-the-fly (i.e., without having to explicitly drag in a particular component). Having a set of pre- defined calculators in your package allows end users to immediately understand a set of properties that they can easily generate whenever they have a particular data record type. The standard features of a calculator include: Feature Description Component name The name is typically the name of the property to calculate (e.g., AlogP). If a particular calculator bundles multiple calculations into a single component, the name should reflect the class of calculations that the component contains (e.g., Molecular Fingerprints). Prepending the data record type to the name is a good practice (i.e., "Molecular XYZ"). Icon Set the icon type to Calculator.
Page 5 | Pipeline Pilot • Developer Guide Feature Description Parameters A calculator component should have an Output parameter (type StringType). The names of the properties it can calculate are given as legal values. Do not use special characters or spaces in the names, only alphanumerics and underscores. Other parameters are allowed, but all must default to reasonable values. These are the values that are used when the property is calculated by name. Folder location Publish your calculator components in the "Calculators" folder.
Reporting Reporting components are used to create visually-appealing output of your single data record, some view of the data record, or a summary display of information about a set of data records. An example of reporting components are the renderers previously discussed; these components take individual data records and prepare them for display in the HTML Viewer and the Data Record Tree Viewer. Other example may be specific to your data record type. You can use any of the reporting components (pie charts, text, bar charts, tables, etc.) in the creation of new reporting components that make it easy for end users to display and inspect new data record types. The standard features of a reporting component include: Feature Description Component name The name suggests the type of display to perform. For example, for rendering components, names of the form "Render XYZ", where
Filters Filters are components that split the data stream between their Pass and Fail ports. Typically, some test is performed on each data record. Based on the test results, the record is said to "pass" (and is sent out the Pass port), or "fail" (and is sent out the Fail port). A well-behaved filter should pass each incoming data record out to either the Pass or Fail port, and generate no additional records. This means the count of records output by the Pass and Fail ports will equal the count of records input. The standard features of a filter include:
Creating Components | Page 6 Feature Description Component name The name is of the form "XYZ Filter", where
Page 7 | Pipeline Pilot • Developer Guide Utilities Utilities are components that perform special-purpose functions, but are typically not used by the casual end user. One utility component type we recommend are generators that create empty copies of each of your new data record type. This can be useful for testing and debugging. These types of components are modeled after the Generate Empty Data component. The standard features of a converter component include: Feature Description Component name The name is of the form "Generate Empty XYZ", where
Creating Components | Page 8 Best Practice: You do not need to append the word "Utility" to subfolders in Utilities; the context should make that clear.
Manipulators Manipulators are components that change the incoming data record in some way (e.g., by adding or removing properties). The standard features of a manipulator include: Feature Description Icon Set the icon type to Component, unless your manipulator performs a specific function where another icon would be more suggestive of its action (e.g., an operation that unmerges the data should use the Generate icon). Folder location Publish your manipulator components in the "Manipulators" folder. Often, this folder will contain much of your collection's functionality.
Learners Learners are components that take as input one or more data records, and then use that information to create a new component that is added to the component database (typically, in the user's own "User Name" tab). The standard features of a learner include: Feature Description Icon Set the icon type to Learner. If the new component is a model of the incoming data, use the Model icon for the new component. Ports Typically, a learner has no output ports. However, you can use the Output Pass port to output information about the learn operation for later use (or user inspection), and use the Output Fail port to send any incoming data records that were rejected for learning, for whatever reason.
Page 9 | Pipeline Pilot • Developer Guide Feature Description Parameters If users need to specify a set of properties for the learner, add a UseProperties parameter as a StringType. The values are: AllPropertiesOnFirstData, CalculablePropertiesOnFirstData, PredefinedSet, or UserSet. Set the default value to "PredefinedSet". You also need to add the following parameters (nested under UseProperties): PredefinedSet (StringType): Contains a legal value list of properties that would be typical to use, with some nice default set already selected. UserSet (StringType): Empty IgnoreProperties (StringType): Empty Note: There is a Calculate Properties component that implements the appropriate functionality of this set, and can be used (within a subprotocol) for convenience of implementation.
Folder location It is customary to have a particular folder name on the user's tab into which the new component is placed, but this is not required.
Customizing the Component Interface With some idea of how your new component should look, it's now time to cover details about the process of creating a component. The Edit dialog is where you change the interface. It includes tabs for help text, ports and icons, parameters, and implementation. (You also use this same dialog to customize a Subprotocol interface.) To open the Edit dialog: Right-click the component and select Edit. The Edit dialog opens. From here, you can do all your customizations. For example, here’s the Edit dialog for the Delimited Text Reader:
Creating Components | Page 10 Edit dialog for Delimited Text Reader component
Configuring Parameters Parameters are used to determine the properties and behavior of a component. Use the Parameters tab in the Edit dialog to configure your custom parameters. You can modify existing parameters and add/define new ones. Parameters are generally organized by group (explained later in this chapter).
Page 11 | Pipeline Pilot • Developer Guide Parameters for Top N Filter component For tips on using the Parameters tab, point over a toolbar button:
Toolbar button help for Parameters tab When the Parameters tab is displayed, you can open another dialog (Define Parameter) for customizing your parameters. To add a new parameter: To modify an existing parameter: Click Add Parameter . Select the parameter name and then click Edit Parameter .
Creating Components | Page 12 Define Parameter dialog The parameter attributes you can configure include: Parameter Attribute Details Parameter help The text you type will provide your end users with a description of what the parameter is intended to do. HTML markup is supported in this field, so you can apply formatting and separate information into multiple paragraphs and bulleted lists. Parameter name Provides a way to assign a unique name for the parameter that will help end users understand your interface. Parameter type All parameters need to be configured as a certain type. Parameter types have an associated range of possible values and uses. Numerous types are available and you can also define your own custom types. Required to run protocol If the parameter requires a value at run time, you need to select this checkbox. Required parameters are displayed in red, so end users know they must to provide a value to run the protocol. Array For parameter types that accept multiple values (an array), select Array. Then, type each value on a separate line in the Legal Values tab
Parameter Enabling You can configure your parameters so they only apply under certain conditions. This is known as "parameter enabling". If the condition applies, the parameter is enabled and its value is used by the component or protocol. When the condition does not apply, the parameter is ignored. To provide this
Page 13 | Pipeline Pilot • Developer Guide functionality, develop your own scripts. Use the Legal Values, Validation Script, Enable Script, and Legal Values Script tabs in the Define Parameters dialog for this purpose. Parameter Customization Checklist The following information offers suggestions for customizing components that are compatible with existing components and protocols from an operational and cosmetic standpoint. Although situations may arise where deviating from these guidelines is appropriate (and even necessary), end users can quickly work with your components if they have a familiar look and feel, and follow the recommended styles as closely as possible. Do this: Details: Done? Limit parameters The fewer parameters, the better. Each one causes additional cognitive overhead in the end user’s mind. Omit obscure, hard-to- understand, or unlikely-to-be-changed parameters. Use proper naming conventions Parameter names should use initial caps with spaces separating words. No underscores. Examples: Initial Expression, Keep Calculated Properties. One exception to this guideline is that the spaces should be omitted when using "standard" parameters, such as SourceTag, UseProperties, PredefinedSet, etc. Assign appropriate types to your For Boolean parameters, use the Boolean parameters parameter type, not a String parameter with Yes/No, On/Off, True/False, etc. as legal values. Assign list boxes to multiple choice For multiple related Boolean parameters, parameters consider using a multiple select list box (e.g., see the Output parameter for the Molecular Properties component). Enable/disable parameters with scripts If certain parameters apply in some situations but not in others, use the Enable Script tab in the Define Parameter dialog to enable and disable these parameters, according to the values of other parameters. This gives the end user visual clues about which parameters are applicable in a given context. Be thorough Your text should fully explain how to use the parameter. Include all relevant information that will help your end users understand how to set values for the parameter.
Creating Components | Page 14 Do this: Details: Done? Be precise Ensure your content is not too vague. Do not assume your end users will understand that you are implying anything. Be explicit instead. Remember that the end user does not know your context the way you do. Be consistent If you have a group of components that use the same parameter, be sure to reuse the help text in all the components. If you have a group of components that uses similar parameters, use consistent grammatical construction and phrases for each parameter. Define options For parameters that have multiple values that are available to select from a drop-down list, define the behavior of each option to help end users understand their options. For three or more options, use bulleted list items. Avoid wordiness Parameter help should be as brief as possible without omitting necessary information that helps end users make decisions. Avoid listing options and then saying "former" and "latter" in the next sentence to refer to such options. Instead, provide the details for each option directly after the option. Chunk your content, so it’s clear and to-the-point. Provide examples When appropriate, provide a brief example to enhance usability. For example, for a "range" parameter, add "1-2-3". Test your facts Double-check facts and test your component to ensure that the information you provide is 100 percent accurate.
Parameter Groups A group is a way to organize parameters into categories based on their purpose. Related and similar parameters are located in the same group so they are easy to find. This is especially useful for complex components that include a large number of parameters. It makes the component easier to maintain. Groups are expandable and collapsible by pressing [+] and [-], so you can hide or show parameters for a given group directly in the dialog.
Page 15 | Pipeline Pilot • Developer Guide Parameter groups for "Expression" and "Additional Options" Parameter Groups Checklist
Do this: Details: Done? Stay on top of your groups from the get-go After you add new parameters, be sure to group them in the Parameters tab. Consider GroupType parameters When you add a new parameter, it is added as a string type, which by default, allows you to specify a string value for the parameter. You can change this to any type that is appropriate for the parameter. When a set of parameters naturally work together, it may be beneficial to group them by creating a GroupType parameter and placing the parameters inside. A GroupType parameter does not accept a value. Group parameters only if it simplifies the If a component has less than five interface parameters, grouping usually isn’t necessary. Group parameters according to function You can add your own groups and you can add members (parameters) to a group. Put some thought into your parameter and All parameter members within a group group names require their own unique names, even if they are similar. Group consistently across components For example, if Maximum is grouped under Source in Component A, repeat this for all other components that use the same set of parameters.
Creating Components | Page 16 Do this: Details: Done? Organize parameters for usability Place the most important parameters, which are more commonly used, at the top of the list. Place required parameters, which need a value at run time, at the top level and visible by default. Place advanced parameters, which are less commonly used, inside the group. Reuse your work Whenever possible, select an existing parameter to become the group parameter instead of creating a new group parameter. Keep it simple Keep parameter hierarchy shallow; go two levels into the hierarchy if necessary.
Configuring Ports and Icons Use the Ports and Icons tab in the Edit dialog to configure ports and icons. These settings are included in your component documentation, and are displayed in the Help window, after your description help text.
Page 17 | Pipeline Pilot • Developer Guide Ports for Top N Filter Follow these guidelines for configuring your icons and data ports: Guideline Details Use an appropriate icon, since it provides a visual clue for Assign an icon that visually conveys your end users what your component is designed to do. Many options are available. Follow the guidelines provided in section Standard Component Categories. Fully configure all your ports For each type of port used with your component, specify if the supported data is generic and/or molecular. Check or uncheck the options accordingly. Remove ports not used by your component by clearing both checkboxes. Do not assume end users will understand the data flow on For the Fail port, be sure to explain their own under what conditions an object can fail.
Creating Components | Page 18 Guideline Details Add comments where necessary Comments are useful to explain requirements for the data, such as the presence of certain properties. Be specific about data records and their properties by describing why data records are sent to a Pass versus a Fail port (explain the precise criteria used to define this behavior).
Documenting Component Help When creating a new component or modifying an existing one, make sure that the component and all of its parameters are fully documented. The Pipeline Pilot Client makes this process relatively simple by providing text editors for your content and formats, and by storing this information in the component XML file for easy access.
Component help for Custom Filter (PilotScript)
Page 19 | Pipeline Pilot • Developer Guide Summary Help The summary help is a brief statement that explains what you can do with the component. The text you enter in this field is displayed in bold when end users view your information in the Help window. It is also used as fly-over help in the Explorer window (as a yellow shaded box).
Purpose statement for a component Guidelines for Writing Summary Help Content
Guideline Details Make it brief Summary text should be brief statement, since it is also used as fly-over help in the Explorer window. Brief means no more than one sentence in length (without a period), 10- 20 words maximum. Note that the flyover help may truncate after it reaches a maximum number of characters. If your statement is too wordy, there may not be enough room to display your text in the yellow popup box.
Creating Components | Page 20 Guideline Details Begin with an action word The statement should start with a verb. Correct: "Reads data records from a …" Incorrect: "Data records are read from a …" No period after brief statement for consistency No need to type period at the end of the statement (it is not a sentence). No blank lines before or after Be careful that your purpose statement does not include any blank lines after the text. Do not press ENTER in the Edit dialog to add any blank lines. Any extra lines included in the purpose text field are included in the fly-over help, making the yellow popup box display an extra blank line below the text. This increases the size of the rectangular box. If you are inconsistent about this, some of your fly-over help will have a larger yellow box, while other help will display a single line of text in the Explorer window. These little inconsistencies make a big difference in the overall quality of your user interface. Avoid wordiness and unnecessary words Do not use "This component" or "The component" in the purpose statement. Correct: "Reads data records from a …" Incorrect: "This component reads data records from a …" Do not substitute the component or protocol name Take the time to write something meaningful for the purpose text if possible: so users get an idea why they would want to work with your component: Correct: "Reads data records from a Microsoft Excel file (.xls) stored on the client" Incorrect: "Excel Reader (on Client)" Description Help Text"
Description Help Content The description help text contains detailed information about the component or protocol. It is displayed below the purpose statement in the Help window. It supports HTML markup for bold, italics, bulleted lists, and tables. It also supports hypertext links to other components and protocols, and to external URLs. All of your detailed information for end user assistance belongs in this area of the component reference help.
Page 21 | Pipeline Pilot • Developer Guide Guidelines for Writing Description Help Content
Tip Details Provide adequate details The description help should add detailed information that fully explains how to use the component or protocol. Include all information that helps your end users understand how to work with the component or protocol. Provide relevant details, so end users know what to expect. Important parameters should be described in the Description. Describe what they do, but don't go into detail about how they work (be sure to provide this level of detail in the parameter help instead). Avoid redundancy Do not repeat the same information in the description help that is used in the purpose statement, especially in the first paragraph. Redundancy takes up UI space, is time- consuming to maintain, and looks tacky. What's more, users often get frustrated when they see repetitive information that does not answer their questions ("How do I" and "What is this"). Prioritize information If a component can be used in multiple ways, describe the most common usage in the initial part of the Description. To explain less- common usages or behaviors of the component, add a Notes section. Include cross-references To cross reference related components, user guides, and external URLs, add a See Also list and include links to these other resources (see below for details). Cross-reference dependencies If a component is dependent on another upstream component to produce output or generate something, then the dependent component should always mention the other component it requires. Provide a link to this other component in a See Also section (see below for details). Use a minimalist editing style Be concise. Omit needless words. Avoid phrases such as "this component" to reduce wordiness. No need to say, "This component does this…". Instead, just say, "Does this…".
Creating Components | Page 22 Tip Details Avoid elegant variation Do not describe the same thing in two or three different ways, just to include extra text in your description. If you do not require a lengthy explanation, there is no need to fill up space with words. Talk to your reader if necessary as "you" Use First person singular ("you" instead of "the user"). Do not leave the description blank No component is too simple for a basic description. Formatting Conventions The Help Text tab supports HTML markup. Use the right-click menu to select HTML formatting. Use the following typographical and HTML/CSS conventions when formatting your text:
Item Format Example Boolean Unquoted, except for multi-word legal values True, False, PredefinedSet, StdDev. values and separated by spaces or values in all lowercase "Remove Uninformative Bins", pre-defined (should be quoted). "brlr", "FitSummary and FitPlot". list values Bulleted list Wrap the entire list with
- tags.
- Item items Separate each list item with
- tags. One
- Item Two
- Item list with
- tags. Separate each list item with One
- Item Two
- tags.
- Item list with
- tags. Separate each list item with One
- Item
- tags. Two
- For more information on X, see link text goes here.
- To get details about Y, see link text goes here.
Code Enclose the syntax with tags. sildenafil[ti] AND 2002..2004[date]
Component Italics Excel Viewer names Cross- Provide links to the help for these components Use right-click menu in Edit references dialog/Help Text tab to insert proper to other hyperlink markup syntax components or protocol examples Expressions Enclose the syntax with or as bulleted list items. Notes Enclose Notes heading in its own paragraph. Notes: tags Text Goes Here tags. sildenafil[ti] AND 2002..2004[date]
tags. Separate all lines of text with paragraph tags
Paragraphs Separate paragraphs with
Page 23 | Pipeline Pilot • Developer Guide Item Format Example Parameter Italics Maximum names Parameter Double-quotes for values that are entered by "100" values users. Values that are selected from a list are represented as they are given in the legal value list. Exceptions for this last rule occur when legal value lists are all lowercase or phrases. Do not use ‘single quotes’ unless you are adding programming code that requires it. Property Bold PropertyName names generated by the component Protocol Provide links to the help for these protocols Use right-click menu in Edit names dialog/Help Text tab to insert proper hyperlink markup syntax See Also lists Enclose See Also heading in its own paragraph.
See Also: Apply bold to the heading text. Wrap the entire
| Column tags, shown in the example. Heading | Column Heading |
|---|---|
| Table Cell Text | Table Cell Text |
Notes: You can test how the Help window will render your markup directly in the Edit dialog. From the Help Text tab, click the Preview tab. This is a good tool for testing the validity of your markup. Follow World Wide Web Consortium (W3C) standards for well-formed HTML markup. For special characters, be sure to insert the appropriate symbol. The proper syntax for copyright is © and the proper syntax for trademark is ™. Do not use tags to insert copyright or trademark symbols (this does not render properly in the browser). Cross-Referencing with Links If a component is dependent on another upstream component to produce output or generate something, then the dependent component should always cross-reference the required component. A See Also section is useful when you want to cross-reference related components, user guides, and external URLs. You can link to any kind of format supported in a Web browser (HTML, PDF, etc.). For a single item, you can add the link directly in the tag. For example: For further information about XYZ, see link text goes here.
Creating Components | Page 24 The HTML syntax for a See Also list is:
See Also:
Tip: Before each link, provide a brief phrase explaining why the user might want to click on the link. Adding Links to Components To add a link to another component: 1. Enter any text that should come before the link, (e.g., "For more information about X, see "). 2. From the right-click menu, select Create Link to Component. The Create Link to Component dialog opens. 3. Navigate to the path where the component is located and select (highlight) the target component to use in your link. 4. In Text to display as, enter the text you want exposed as the clickable link (or accept the default text which is based on the component name).
Cross-referencing the HTML Table Viewer component 5. Click OK. The component link is added to the description field, rendered with JavaScript code that makes the link work internally.
JavaScript code for component cross-reference IMPORTANT! If you move, rename or remove the target component, update the URL so it points to the correct file. To add a link to a URL:
Page 25 | Pipeline Pilot • Developer Guide 1. Enter your introductory statement if necessary. (For example, "For more information about X, go to"). 2. From the right-click menu, select Create Hyperlink (URL or File). The HTTP Link dialog opens. 3. In the first text box, enter the text to display as the clickable link. 4. In the second text box, enter the HTTP address for the URL or file.
Adding an external URL 5. Click OK. The link is added to the description field, rendered with HTML syntax that makes the link work internally.
JavaScript DoLink code for a URL Best Practices: Avoid inserting "raw" hyperlinks using the HTML tag. Instead, always use the right-click menu commands available from the Edit dialog/Help Text tab. Avoid creating links to files that are outside of your collection. For example, do not link to online help topics or user guides, as these may change in future releases, breaking your links.
Note: The DoLink() function is only relevant for the client's user interface, and the right-click menu provides this functionality (no hand-coding of links is required). When the Help text in the component XML is converted to HTML, the content for the
tag is pre-processed. The DoLink() function converts relative links to help topics to full URLs, ensures that external links open an external browser window (rather than appearing in the Help window), and ensures that a component GUIDs links to the appropriate component in the tree. After making these changes, you are ready to save and publish your new component. Below are some checklists to help you verify if you have performed the editing correctly. Component Creation ChecklistDo this: Details: Done? Decide what the In general, a component should perform a single, well-defined component will do task. (purpose)
Creating Components | Page 26 Do this: Details: Done? Decide how you want A component's unique set of parameters is the "component users to configure interface." For maximum clarity, determine how to group and parameters to control order the parameters. the component’s behavior Implement the new A new component may be implemented as a subprotocol, in a Perl component by using an script, and in a programming language such as Java. Suitable existing component as a templates for these cases are the Subprotocol, Perl (on Server), template Java (on Server), and No-Op components. Assign the component a It can be identical to your component name. short caption describing its function Customize the In the Edit and Define Parameters dialogs, create custom component as necessary parameters where necessary. From the Ports and Icon tab, select an appropriate icon based on the component’s function, and enable the ports used by the component. Document the Write help text for each parameter, each port that is used, and the component component as a whole. Test your component Test and refine your component as needed, until it behaves as intended. Save the component to The Pipeline Pilot Client assigns it a new globally unique identifier your "User Name" tab (GUID), which individually identifies your component.
Manually Publishing a Component
Do this: Details: Done? Right-click on the component in Publishing is moving the component from your private your "User Name" tab, and select "User Name" tab to a public tab, either Components or Move; browse to the destination Protocols. Using the Move operation preserves the folder and push OK. unique identifier (GUID) of the component.
Publishing a Component as Part of a Package
Do this: Details: Done? If the package does not yet For information on creating a new package, see the Application exist, create it. Packaging Guide.
Page 27 | Pipeline Pilot • Developer Guide Do this: Details: Done? Export the component to For example, if you had a reader component, the file-system the appropriate file-system location might be a folder of the form: location for the package "apps/vendor/package/xml/Components/domain/Readers" XML files. Apart from the .xml extension, it’s good policy to set the filename the same as the component name. Be careful you do not change any defaults unexpectedly, or point to any private components or data, otherwise it will run fine for you, but will not work for others when published. Remove the component Delete the version of the component on your "User Name" tab. from your user tab. Do not save future changes Make any further changes to the component by dragging it to the component; export from the Components tab and starting from there. After making instead. changes, export the component to the proper file-system location, e.g.: "apps/vendor/package/xml/Components/domain/Readers" IMPORTANT! Do not save a changed component to your "User Name" tab or anywhere else – it will change the unique identifier (GUID).
Re-install the package to You need to re-install the package to see your changes in the see your changes hierarchy.
Data Selection Checklist
Do this: Details: Done? Use actual data Components should have actual data, rather than references to data, as inputs and outputs. For example, they should accept and produce molecules, sequences, or generic records of numerical and text properties. They should not accept and produce file paths, database handles and the like, as this creates incompatibility with other standard components and can confuse end users. Group/ungroup data Grouping data into aggregates with the Group Data appropriately components is sometimes necessary, so each group can be processed separately from other groups (e.g., when finding the average value for each subset or group). However, end users should not be required to manually group or ungroup data; they should be able to simply work with your component. Implement your component as a subprotocol, do the grouping inside your component, and ungroup the results before they are output.
Creating Components | Page 28 Functionality Checklist
Do this: Details: Done? Separate When the same underlying functionality is used for two very different purposes tasks into (especially if different parameter choices are likely), it is good practice to different provide two versions of the same component, each with a different name, and components perhaps different defaults or help text. Consolidate Consider creating a utility component that encapsulates functionality used by functionality your other custom components. Store such utility components in a for reuse "Utilities/Internals" folder in the same package as the other components. Keep your Try not to build too much functionality into any one component. If an components operation can be split into smaller steps, it may be a good idea to build simple components for each operation, and deliver a subprotocol component that includes the steps linked together. This way, end users have the flexibility to recombine them when they want to do something you haven't envisioned.
Performance Checklist
Do this: Details: Done? Understand the different ways components handle Manipulator components such as data Sort Data and Merge Data must wait for all the data to enter before they can output any data. This may require a lot of memory to run; therefore, use these kinds of components judiciously. Frequently, there are other ways to achieve the desired result in a more efficient manner. For example, the Top N Filter is more efficient than Sort Data, since it does not have to sort the full data stream. Use RunToCompletion mode only when necessary Running subprotocols in RunToCompletion mode is sometimes inefficient; therefore, only use this option when absolutely necessary. Make sure that the number of times the RunToCompletion parameter gets invoked in your subprotocol is minimized. Run it for a group of data records, rather than for individual data records.
Page 29 | Pipeline Pilot • Developer Guide Naming Convention Checklist
Do this: Details: Done? Use descriptive names Use care when assigning names to protocols, components, and parameters. Assign a name that clearly conveys the purpose. Examples: Parameter names: Match Behavior, pH Step, Image Tags. Component names: Back Translate Sequence, Molecular Formula, Sort Data, QUANTUMm Calculation. Protocol names: Intensity Plot Comparison, Repacking Color, Chemical Trends in Arthritis. Use action words When possible, use an action word (i.e., verb) to convey the purpose. Use simple verbs that do not end in "ing", in the present tense at the start of the name. For example, Generate Empty Data, Identify Salts, Find GC Rich Regions. No combined words Avoid combining multiple words into a single name without spaces. Instead, use spaces between words for clarity. No underscores Do not use underscores to separate words. Use spaces instead. Use initial caps Use initial caps for each major word. Do not use all uppercase (or a combo of caps) unless a word is an acronym or special case where mixed case is the standard way to spell the word. Examples: Is Property Defined, XY Plot, Mol2 Reader, AlogP, logD, MegaBLAST.
Creating Components | Page 30 Do this: Details: Done? Use lowercase for minor words Use lowercase for prepositions of four or fewer letters (at, from, of, up, with, etc.), for conjunctions (and, as, but, for, nor, or, that, etc.), and for articles (a, an, the). For brevity, avoid using articles whenever possible. Examples: Add Terms to Text Query, Substructure Count from File, Group Data by Tag. Specify singular or plural Use singular or plural nouns, based on how many the action is performed on. Examples: Cluster Documents, Apply Function to Property, Join Data from Sequence File, Filter by Annotations, Center Molecule. No abbreviations For clarity, avoid abbreviating. Spell out words instead. No ampersands For consistency, avoid ampersands (&). Always use the word "and" instead. No hyphens For consistency, do not use hyphens unless the term is hyphenated as a standard convention. Hyphenate words that precede and modify a noun. Examples: No-op, R Remove Zero-Variance Properties. No punctuation Do not include punctuation (periods, colons, semicolons, quotations, apostrophes, etc.). Use numbers correctly Use numerals for 10 and above and for standard acronyms that use numerals. Spell out zero through nine, if the number does not precede a unit of measure or is not used as input. For round numbers of 1 million or more, use a numeral plus the word, even if the prefix number is less than 10. Examples: R 2D Plot, Align cDNA to Genomic DNA (Sim4), ADMET CYP2D6 Binding, R Two- Variables Tests, Compare Two Sequences (BLASTn).
Page 31 | Pipeline Pilot • Developer Guide Do this: Details: Done? No "and/or" Avoid using the "and/or" word phrases. Use one or the other. Spell check always Be sure to check the spelling of component, protocol, and parameter names, tooltips, sticky notes, etc.
Creating Subprotocol Components A "subprotocol" is a special kind of component that allows us to define the implementation of a component using other components. Subprotocols provide a way to bundle the functionality of several components into a single element that looks and acts like a single component. This gives the end user access to the parameters for multiple components as a single interface. By hiding the implementation, subprotocols provide a user-friendly way to reuse and distribute combinations of components. Data Flow Like standard components, subprotocols can interact with other components by receiving data records, passing data records, or both. By default, subprotocols do not alter Pipeline Pilot's model of data flow. Each incoming data record is handled in turn, carrying out as many steps of the pipeline as possible before the next record is handled. Component initialization, which includes processing parameters and opening files, happens once, when the first record reaches a component and is not repeated for subsequent records. For example, this means that a file writer component opens its Destination file once and writes all the records to the same file. A subprotocol's data flow is altered using the parameter RunToCompletion. When this parameter is set to "True", components within the subprotocol are reinitialized each time a new record enters the subprotocol. This is useful to emulate looping behavior, particularly on subsets of data. Within a RunToCompletion subprotocol, a file writer opens a new file for writing for each record, writing each record to a separate file. For more details, search on "RunToCompletion" in the Help Center.
Creating the Interface Creating a subprotocol that has the look and feel of a native component makes it easier to use. Here are some guidelines to follow: The icons you select provide visual clues about the functionality, inputs, and outputs of your subprotocol component. You can replace the generic subprotocol icon with another icon in the Ports tab of the Edit dialog. Expose configurable parameters to end users by promoting them to the subprotocol. Avoid hard- coding property names or filenames within the subprotocol. Add custom parameters to provide run-time options for different use cases, output formats, etc. If the interface to your component is extensive, consider splitting the component into separate modules that each performs a single task.
Creating Components | Page 32 Ports Data records flow in and out of subprotocols via the ports. The following rules govern the flow of subprotocol data: All data records entering a subprotocol start at the same component. The starting component is defined by the existence of an open input port. As a result, only one component in a subprotocol may have an open input port. Tip: You can process data records through multiple pipelines in a single subprotocol by starting with a No-Op component and immediately branching the pipeline. Data records emerging from an open pass port on any component within a subprotocol exit from the Pass port on the subprotocol. Data records emerging from an open fail port on any component within a subprotocol exit from the Fail port on the subprotocol.
Global Properties Using global variables in subprotocols can affect reusability. Adding your subprotocol to a new protocol may create problems if a global variable of the same name is used elsewhere in the protocol. Even if the name for your variable is unique, adding two copies of your subprotocol may create the same type of problems. The solution is to declare all globals used within your subprotocol as local to the subprotocol. Use the DeclareLocal parameter that is available on each subprotocol for this purpose. Even if there are multiple nested layers (subprotocols within subprotocols) each subprotocol has its own scope in which to declare global variables. Tip: If your subprotocol is sharing a global with its parent protocol, make this a parameter on your subprotocol to pass the value explicitly.
Reusing Subprotocol Components When you save a subprotocol component to a tab or install a subprotocol component from a package, it is saved in the component database. As a subprotocol is incorporated into a protocol, the saved subprotocol is represented as a subprotocol shortcut. Each one has a different impact on the run-time behavior. Subprotocol copy: The subprotocol component in your protocol is a copy of the original one in the component database. When you make changes to the original component in the database, the copy is not updated. The changes in the database do not affect the behavior of the protocol that contains the copy at run time. Subprotocol shortcut: The subprotocol component is a reference to the original one in the component database. When you make changes to the original component in the database, the shortcut is automatically updated. The changes in the database affect the behavior of the protocol that contains the shortcut at run time. By default, a subprotocol component is incorporated into a protocol as a shortcut. The shortcut can be converted to a copy, either explicitly using an option on the right-click menu or automatically by opening the subprotocol and making changes to the implementation.
Page 33 | Pipeline Pilot • Developer Guide Note: Protocols which incorporate subprotocol components as shortcuts are much easier to maintain than those that use copies. When using shortcuts, upgrades and bug fixes are automatically applied to all instances of the subprotocol component. You can force users to incorporate subprotocol components as shortcuts by setting the "Owner Access Only" option when you save your component. This prohibits users from copying, saving exporting, or sharing the file in the component database. Users are only allowed use it as a shortcut within a protocol. This feature provides a way to hide your implementation, passwords, and any other sensitive information that you do not want to expose to users. For more detailed information, search for "subprotocol" in the Help Center. Subprotocol Customization Checklist
Do this: Details: Done? Simplify your protocol Collapse groups of components that work together to perform a particular function into a subprotocol. Document each subprotocol Assign the subprotocol a unique caption, description, and purpose, so your end users do not have to open it to understand what it does. Expose essential parameters to end users Make all essential parameters available at the protocol level. End users should not have to open the subprotocol to set its most important parameters. Customize the subprotocol's look After creating a new subprotocol, select an appropriate icon to reflect its function (e.g., Reader/Writer, Viewer, Manipulator). Remove unused ports For example, if a component does not expect input, remove this port. Properly scope global variables If you use global variables (@var) inside a subprotocol, be sure to scope it properly by declaring it as local variable, so it does not interfere with other instances of the same variable name used elsewhere. Hide implementation parameters Move implementation parameters, such as tmpfiles, to the Implementation tab.
Creating Components | Page 34 Organizing your Saved Components Often, you will be creating not a single component for work in a particular domain, but several components. It's helpful to have them saved into the hierarchy in a standard way. The following organization is used to organize components and protocols that will be published as part of a package, but it can be useful to help you organize your components for manual delivery, also. To create a folder hierarchy: 1. Create a folder on your "User Name" tab that will contain the hierarchy to move moved or export later. Name it "apps". 2. Within your new "apps" folder, create two subfolders that reflect your corporate name. One is for delivery of the publish components, and the other one is for development-related components and protocols not intended for end users. For example, if your company is called "Acme", name your folders "acme" and "acmedev". (In the screen-shots below, use the names "scitegic" and "scidev".) 3. Within both these folders, create a folder with a name that suggests the collection identity in all lowercase, with no spaces (this is because this name will become a folder name in the file system when you export). For this example, use the name "spectraanalysis". a. Into the newly-created folder in "scitegic", create a folder "xml". Into the "xml" folder, create the following three folders: "Components", "Protocols", and "Objects". b. Into the newly-created folder in "scidev", create a folder "xml"; within that folder, create a folder "Protocols". Within that folder, create a folder "Regression". Your structure will look something like this:
The hierarchies created in the Components and Protocols subfolders determine the folder structure as delivered on the Components and Protocols tabs. A given collection typically fits into one of the top- level domains that already exist in Pipeline Pilot. For example, a screenshot showing the top-level domains in my current version of the Components tab is shown below:
The first five are general-purpose domains that are designed for use across multiple scientific areas. The remaining five are application-specific domains that contain components specialized to work on problems in focused scientific areas.
Page 35 | Pipeline Pilot • Developer Guide 1. Select which of these top-level domains is the natural place for your collection, or create your own. 2. After selecting your top-level domain, create that folder name as a subfolder of your Components folder. 3. Create a folder within the domain that describes your collection. As an example, the Laboratory folder currently contains two sub-domains:
4. If your collection was destined for the Laboratory domain, name and create an appropriate sub- domain folder within Laboratory, (e.g., "Spectra Analysis"):
Later, after exporting your components and examples and installing them as a package, the folder structure on the Components tab will look like this:
Note: It may be that your sub-domain naturally subdivides into further categories to provide separate areas of functionality. In this case, create new folders within your sub-domain folder for these subcategories. Once the appropriate subdivision is reached, it is time to start providing the structure for the presentation of the components. End users will most easily understand your package if you reuse standard folder names to contain these components. The standard folder names used in Pipeline Pilot include: Folder Provides components that: Analysis Perform complicated processing on either single records or on streams of records, providing summary results as property data or alterations on the data itself. Calculators Use the Output parameter to allow for automatic calculation of properties on individual data records. Converters Convert individual data records from one data record type to another data record type. Filters Partition the data stream depending on the results of some binary test.
Creating Components | Page 36 Folder Provides components that: Learners Create new components (Models) from the data stream. Manipulators Change the incoming data record in some fashion, such as by adding or removing properties. Readers Read particular data record types or formats from external sources, typically specified by a Source parameter. Reporting Generate Reporting Elements from your data record types for display in HTML, PDF, or other reports. Similarity Compare the records on the data stream for how similar they are to reference objects. Utilities Perform special-purpose functions and are often part of subprotocols (and hidden from the end user), but may be useful for development. This folder may contain an "Internal" subfolder, which contains private components that users cannot find when using the Search feature. Viewers Allow for the visual display of data record or your data stream. Writers Write particular data record types to an external destination, typically denoted by a Destination parameter.
Note: You can also provide your own, domain-specific, folder names. However, this should only be done when it results in greater clarity for end users, because they are already accustomed to the standard names. 5. After you finish adding components to your collection, remove any folders that have no contents. Not all collections will contain examples of each of these types of component.
Publishing your Components You should continue to develop your components on your "User Name" tab until they are ready to publish. There are two way to publish: manually, and as a package. To manually publish: 1. Right click on each component in the package and use the Move command to put it into the appropriate place on a Components or Protocols tab. 2. Once finished, delete the domain hierarchy.
Page 37 | Pipeline Pilot • Developer Guide Tip: In the future, edit the components from their new location, saving them there when finished. This will avoid any unwanted changes to the unique identifier (GUID). To publish as part of a package: 1. Export the entire hierarchy into the "apps" folder under scitegic root. 2. Delete the domain hierarchy. 3. Install the package to see them appear in the appropriate places on the Components and Protocols tab. Tip: In the future, edit the components from their new location, exporting the changed components back to the file system rather than saving them, then reinstall the package. Again, this avoids unwanted changes to the unique identifiers (GUIDs). This concludes the discussion of developing a single component. The remaining chapters in this guide discuss component collections. A "component collection" is a set of components and protocols that are published as a group to aid protocol developers in a particular scientific domain. Examples include data modeling, chemistry, sequence analysis, text analytics, and reporting. You develop the items in the component collection on your private user tab, then "publish" to a shared tab for other users to access and use. If your collection is a small number of components/protocols, you can manage the publication yourself. However, larger collection are best published by, creating a "package", allowing simple and automated management of the publication, removal, or upgrading of your new component collection. This document assumes that your component collection will be published as a package; however, most of the information contains in this guide remains applicable even if you plan to manually publish your component collection. The remaining chapters of this document will describe a series of steps that will help you to develop component collections that fit naturally into the framework.
Creating Components | Page 38 Chapter 3: Creating a Package
If you plan to manually publish your component or components, you may skip this chapter. This chapter is meant to be a quick overview of packaging; for more detail, see the Application Packaging Guide. After you complete your regression testing and resolve errors, you can deploy the first installation of your new collection. Packaging allows you to specify steps to take during installation and uninstallation. Package installation can configure global variables and web aliases. Along with the ability to bundle multiple types of server extensions in a single distribution, packages automate the installation and upgrade processes and make these steps more convenient for both you and your end users. You create a package by organizing the content (components, sample data, documentation, etc.) into a set of folders and including a configuration file ("package.conf") that lists the package contents and directs the installation. Package naming consists of two parts, conventionally the vendor name and product name. The combined name (e.g., "vendor/instrumentcontrol") must be a unique identifier for the package. To install or upgrade a package, end users can deploy the package content in the Pipeline Pilot install path on the server, and run the packaging utility program ("pkgutil.exe"). To prepare components for distribution, follow the instructions in the Application Packaging Guide. The alternative of simply sending components as XML files can give your end users problems when they upgrade to newer versions in the future. The packaging tools take care of these issues. To create a package: 1. Export your "apps" folder (and all its contents) into the Pipeline Pilot installation "apps" folder. This places the protocols and components into their final resting place. 2. Delete your own copies. This is best done by deleting the "apps" folder from your user tab in the Pipeline Pilot Client. (Otherwise, the GUIDs on your tab will collide when you try to import the package.) 3. Declare and install the package. (For detailed instructions on how to perform these tasks, see the Application Packaging Guide.) 4. After installing your package, make any further changes by editing and saving your components and protocols "in place" (i.e., using the instances that are on the Components and Protocols tab). 5. After you make the necessary changes, export the changed component or protocol back into the correct place in the "apps" folder.
Package Deployment Checklist
Do this: Details: Done? Check if your package Ensure that dependent third-party software is included in the requires any third-party package. The only exception is where the software is a pre- software condition for installation, (which should be covered in your collection documentation as a "Requirement"). Properly acknowledge all Ensure that you are in full compliance of legal and/or copyright third-party redistributables demands of all third-party software vendors.
Creating a Package | Page 39 Do this: Details: Done? Properly register/unregister Component libraries written in C++ should be DLLs (Windows) or install registered/unregistered via DLL registration/unregistration the appropriate shared during package install/uninstall (Windows) or appropriate object (SO) libraries (Linux) installation of Linux .so files. Deploy and test on all Successful deployment and testing is completed on a clean appropriate systems (non-development) environment for each supported platform
Planning Data Record Types The process of protocol development requires understanding "what" is flowing at any point in a protocol, and "which" components are applicable. The flowing data object (what) is called a "data record". A data record consists of a typeless data record wrapper (typeless in that there is only one implementation) around a typed root node; the root node may parent other nodes, and so on, to comprise a node hierarchy. The type of a data record is a declaration about the type of the root node, the presence of required properties, the structure of any node hierarchy, and most importantly, of which components are applicable for processing. As the number of domains has increased, so has the number of data types and components. User confusion as to what data record types are flowing at a particular point in a protocol, which components may be useful, and what their effects will be, makes learning and use of a component collection difficult. Clearly defining your data record types, and defining a simple set of components that applies to each type, is the best way to address this problem and gain user acceptance of a new collection. About the Type of a Data Record The type of a data record is defined as the type (name) of its root node. Note: This is different from the implementation type of the root node, (i.e., the C++ class that is used to construct the class instances). For most developer-defined types, you do not need to create a new C++ class. You can create a node of a current implementation and rename it to your own class. (The Generic node is the most commonly-used for this purpose.) You can use the PilotScript function setnodetype on the data root to set your own node type.
Best Practices for Creating New Data Record Types When is it appropriate to create a new data record type? There is no absolute right-or-wrong answer, but the best guide is this: Best Practice: Create a new data record type when there is a set of components requiring specific data record structure for input. Some judgment is required. For example, too many data record types could make your collection confusing, but too few data could cause end users to run the risk of misapplying components when they are inappropriate. A parallel rule that may help in deciding: Best Practice: Create a type when there is an appropriate, abstract, user-comprehensible concept that the type represents.
Page 40 | Pipeline Pilot • Developer Guide For example, a MoleculeQuery type would make immediate sense; but a Group of Three Molecules with a Reaction and a Spectrum is simply describing an implementation, and has no conceptual elegance. If you feel driven to create data record types such as the latter, it might be a sign your components are trying to do too much in one step; good, perhaps, for efficiency of your algorithms, but bad for ease of use. Consider breaking them down into separate, simpler, tasks. Your optimizations will do no good if end users are confused by complexity.
Adding Components to Packages Adding Package-Specific Components After you add the standard components for your new data record types and create the appropriate folder hierarchy, you can add new components (and new folders) that will make up the bulk of your collection's new functionality. Best Practice: While most of your components will likely be delivered in your subdomain folder (e.g., "Spectra Analysis"), you are not restricted to this structure. You can install new components anywhere in the hierarchy by creating a folder structure that mirrors the placement you desire. However, this can make future management of all the components in your package difficult, so we encourage you to rely on your own subdomain folder for delivery.
Component Creation Checklist
Do this: Details: Done? Appropriate display icons are set for all components Configuring Ports and Icons Component names are mixed case and match the component display name Naming and file name Convention Checklist Parameter names are mixed case, with spaces between words (unless Naming consistency considerations override this) Convention Checklist Parameter grouping is used where it helps to visually organize the component Parameter settings Groups Components and protocols are installed into appropriate application folders in Adding the protocol database (XMLDB) Components to Packages Adding Protocol Examples Subprotocol components in the package do not leak any global properties Creating Subprotocol Components
Creating a Package | Page 41 Adding Protocol Examples After you build the components you want to include with your collection, you should create a series of examples that illustrate the use of these components. You can start by adding an "Examples" folder beneath your Protocols folder. Example protocols should showcase the most important components (and functionality) in your collection. End users will rely on these examples to learn how to use your components. Best Practice: Design your example protocols so they can run out-of-the-box wherever possible. Where setting modifications are required, they should be defined at the protocol level. A prominent (red) sticky note should detail what required parameter values must be set to make the protocol run.
Naming and Organizing Example Protocols Select a name for your examples folder. The most obvious choice is the name of your subfolder on the Components tab (e.g., "Spectra Analysis"). You can organize your examples in this folder in any way you want, but the most common is to group them into a series of subfolders describing the different tasks you are demonstrating how to perform. Within a task folder, it is sometimes required that a series of different protocols run in a particular sequence. In this scenario, name the examples starting with a number (e.g., "01 Read Data", "02 Analyze Data", "03 Build Report", etc.) to convey that sequence to end users. After you finish designing and testing, you can manually publish them by moving them to the Protocols tab, or if using a package, you should export your protocols to the appropriate location (e.g., "apps/vendor/package/xml/Protocols/domain"), then delete the copies on your "User Name" tab. Best Practice: After exporting, delete the protocol versions on your "User Name" tab. Only work with the published versions. Make any further changes to the component by dragging it from the Protocols tab and starting from there. After making changes to the parameters, help, and implementation, save it back to the protocols tab (if manually publishing) or export the protocol into the package XML and re-install the package.
Protocol Examples Checklist
Do this: Details: Done? Decide what the protocol will do (purpose) In general, a protocol example should perform series of well-defined tasks that generate some type of output or results for end users to inspect. Protocol examples should also cover all typical component use cases. Document your protocols Write help content for your examples at the protocol level (the same way you document components). Explain what the protocol is designed to do.
Page 42 | Pipeline Pilot • Developer Guide Do this: Details: Done? Add sticky notes When used appropriately, sticky notes can greatly enhance the usability of your protocols. Use them to describe how specific components process data, so users better understand how a protocol operates. Highlight key areas of the pipeline by color-coding your notes. Indicate and explain points of interest throughout the pipelines. Document your data Add comments about how data flows throughout your pipelines. Update all components Ensure that your examples use the most up-to-date versions of all components. Test your protocols Test and refine your protocols as needed, until they behave as intended. Validate your protocols Protocol validation can alert you about configurations that could potentially cause errors on a different server from where the protocol was originally designed and tested. You can also use it to perform a security check to flag any hard-coded passwords. Follow proper naming conventions Be consistent and follow standard conventions. For details, see Naming Convention Checklist. Export protocols to the appropriate location and then "apps/vendor delete the ones in your "User Name" tab /package/xml/Protocols/domain" After exporting, do not save the changed protocol to your "User Name" tab (or anywhere else).
Creating a Package | Page 43 Do this: Details: Done? Only work with the published versions Make any further changes to the component by dragging it from the Protocols tab and starting from there. After making changes to the parameters, help, and implementation, export the protocol to its location in: "apps /vendor/package/ xml/Protocols/domain".
Adding Regressions A regression framework is an essential tool for maintaining a component collection. After you design your example protocols, you should perform some regression testing to ensure that they work properly. Place your newly-developed regression protocols in the "Regression" folder you previously created in "apps/vendor/package/xml/Protocols/Regression". Regression Checklist
Do this: Details: Done? Regression protocols and baselines exist for the For details on developing regression component collection protocols, see the Component Development Regression Test Guide. Each component is tested in at least one of the regression protocols Each component is tested for each significant mode of usage (including good coverage of possible parameter values)
Documenting a Component Collection In addition to the XML help for individual components, a collection should be distributed with supporting documentation such as a PDF user guide. This document should provide an overview of the package functionality and a broad discussion of its application. Ideally, the content should complement the help text authored with the components and parameters by highlighting essential components and illustrating example use cases. When distributed in PDF format as part of a package, this documentation is automatically installed with the documentation for the Pipeline Pilot collection. Best Practice: Be sure your documentation includes information about required third-party software that end users must install separately. For more details about packaging documentation, see "HelpDocs" in Application Packaging.
Page 44 | Pipeline Pilot • Developer Guide