Digital Imaging and Communications in Medicine (DICOM)

1 Digital Imaging and Communications in Medicine

2 (DICOM)

3 Sup nnn - JSON Representation of DICOM Structured Reports

4 DICOM Standards Committee - Working Group 23 - Artificial Intelligence/Application Hosting

5 1300 N. 17th Street Suite 1752 6 Rosslyn 7 VA 8 22209 9 USA 10

11 Status: Early Draft 12 Publication date 2019/09/08

1 Table of Contents

2 Document History ...... 5 3 ...... 5 4 To Do ...... 6 5 ...... 6 6 Open Issues ...... 7 7 ...... 7 8 Closed Issues ...... 9 9 ...... 9 10 Scope and Field of Application ...... 10 11 PS3.22 ...... 12 12 Notice and Disclaimer ...... 13 13 Foreword ...... 14 14 1. Scope and Field of Application ...... 15 15 2. Normative and Informative References ...... 16 16 3. Definitions ...... 17 17 3.1. Codes and Controlled Terminology Definitions: ...... 17 18 3.1. Representation Conversion Definitions: ...... 17 19 4. Symbols and Abbreviations ...... 18 20 5. Conventions ...... 19 21 A. XML Encoding ...... 43 22 A.1. Introduction to XML ...... 42 23 A.2. XML Encoding of DICOM Instances ...... 20 24 A.2.1. DICOM XML Encoding in Native Format ...... 20 25 A.2.1.1. Usage ...... 20 26 A.2.1.2. Identification ...... 20 27 A.2.1.3. Support ...... 20 28 A.2.1.4. Information Model ...... 20 29 A.2.1.5. Description ...... 20 30 A.2.1.6. Schema ...... 20 31 A.2.1.7. Examples ...... 20 32 B. JSON Encoding ...... 21 33 B.1. Introduction to JavaScript Object Notation (JSON) ...... 21 34 B.2. JSON Encoding of DICOM Instances and Messages ...... 21 35 B.2.1. Introduction ...... 21 36 B.2.2. DICOM JSON Encoding ...... 21 37 B.2.3. Transformation to and from other DICOM Encodings ...... 21 38 B.2.4. DICOM JSON Encoding Example ...... 21 39 B.3. JSON Encoding of Structured Reports ...... 21 40 B.3.1. Introduction ...... 21 41 B.3.2. DICOM JSON Structured Report Encoding ...... 21 42 B.3.2.1. Attribute Encoding ...... 22 43 B.3.2.2. Structured Report Content Tree Encoding ...... 23 44 B.3.2.2.1. Content Item Encoding ...... 23 45 B.3.2.2.2. Nested Content Encoding ...... 24 46 B.3.2.2.3. Content Item Annotations ...... 24 47 B.3.2.2.4. Encoding of By-Reference Relationships ...... 25 48 B.3.2.2.5. Encoding of Content Items of Specific Value Type ...... 26 49 B.3.2.2.5.1. Encoding of Content Items Without a Value ...... 26 50 B.3.2.2.5.2. Encoding of Content Items with a Single Value ...... 26 51 B.3.2.2.5.3. Encoding of Numeric Content Items ...... 27 52 B.3.2.2.5.4. Encoding of Content Items That Reference Storage SOP Instances ...... 28 53 B.3.2.2.5.5. Encoding of Coordinate Content Items ...... 29 54 B.3.2.2.6. Encoding of Business Names File ...... 30 55 B.3.3. Transformation to and from other DICOM Encodings ...... 30 56 B.3.4. DICOM JSON Structured Report Encoding Examples ...... 30 57 B.3.4. DICOM JSON Simple Single Linear Measurement Encoding Example ...... 30 58 F. DICOM JSON Model ...... 41

59 - Draft - 17 Sup nnn - JSON Representation of DICOM Structured Reports Page 3

1 F.1. Introduction to JavaScript Object Notation (JSON) ...... 41 2 F.2. DICOM JSON Model ...... 41 3 F.3. Transformation with other DICOM Formats ...... 41 4 F.4. DICOM JSON Model Example ...... 41 5 A. Data Exchange Models ...... 43 6 A.1. Native DICOM Model ...... 42 7 A.1.1. Usage ...... 42 8 A.1.2. Identification ...... 42 9 A.1.3. Support ...... 42 10 A.1.4. Information Model ...... 42 11 A.1.5. Description ...... 42 12 A.1.6. Schema ...... 42 13 A.1.7. Examples ...... 42 14 A. Structured Reporting Templates (Normative) ...... 43 15 TID 1500. Measurement Report ...... 43

16 - Draft - 4 Sup nnn - JSON Representation of DICOM Structured Reports Page 4

1 List of Tables

2 TID 1500. Measurement Report ...... 43

3 - Draft - 8 Sup nnn - JSON Representation of DICOM Structured Reports Page 5

1 Document History

2 Document Version Date Content 3 01 2019/07/09 First draft for review by WG 23 4 02 2019/07/19 Changes after review by WG 23 and other feedback received 5 03 2019/07/22 Changes after feedback received (KH, AF) 6 04 2019/09/08 For WG 6 first read

7 - Draft - 25 Sup nnn - JSON Representation of DICOM Structured Reports Page 6

1 To Do

2 1 Fill in hyperlinks to other parts, and especially add them to PS3.3 descriptions of each Value Type. 3 2 Example and test tool round trip - add attributes to business names file (need to address private creator 4 presence/absence) 5 3 Text, examples and test tool round trip - add NUM Content Item Annotations 6 4 Text, examples and test tool round trip - add IMAGE Content Items with frame numbers (esp. array of multiple) 7 and segment numbers, etc. 8 5 Text, examples and test tool round trip - add SCOORD other Attributes as Annotations or positionally, specifically 9 PixelOriginInterpretation (for WSI) and FiducialUID 10 6 Text, examples and test tool round trip - add SCOORD3D Content Items 11 7 Text, examples and test tool round trip - add WAVEFORM and TCOORD Content Items 12 8 Add description of business names file 13 9 Check Number versus String (DS NUM versus FD coordinates in current implementations as Andrey observed 14 - consistency with PS3.18 Annex F). 15 10 Consider relationship (if any) to JSON-LD (https://en.wikipedia.org/wiki/JSON-LD), esp. for business names file. 16 11 Add diagram illustrating merging AI output tree with Composite Context 17 12 Test using single String rather than Array of String as value when only a single value. 18 13 Define a JSON Schema for the syntax and rules. 19 14 SP: Consider using SOP Class UID names (business name or standard keywords) in place of UIDs. 20 15 Add media type for transformed JSON content ? one for snippet versus entire instance? should await web service 21 extensions in future work item? 22 16 Test KH's suggestions for collapsing value arrays if single value 23 17 Compare with GeoJSON, esp. re. coordinates (https://geojson.org/).

24 - Draft - 48 Sup nnn - JSON Representation of DICOM Structured Reports Page 7

1 Open Issues

2 1 A new Part seems to be needed, since there is no good home for this transformation. It is proposed that all the alternative 3 representations be gathered in this new Part, specifically PS3.19 A.1 and PS3.18 Annex F. The PS3.19 A.2 Abstract 4 Multi-Dimensional Image Model has not been moved. The previously named "Model" is renamed as "Encoding". Confirm. 5 Consider "Representation" or "Format" rather than "Encoding". 6 2 For private data element, should creator insertion be automatic/transparent or explicit? 7 3 Should the PERSONNAME Value Type JSON encoding be decomposed into its components and component groups as 8 is done for the DICOM Attribute PN VR? Probably. Right now it is just a simple single JSON String. 9 4 Do we need a generic solution for including unrecognized private Attributes attached to Content Items in the Content 10 Item Annotations description? 11 5 Is there a need to allow annotation with explicit Value Type and/or Relationship Type for cases where either there is no 12 Business Name for the Concept Name (anonymous Content items, often used for IMAGE), or there is ambiguity (e.g., 13 same Business Name used with two different Value Type and/or Relationship Type)? 14 6 In TCOORD, how do we distinguish ReferencedSamplePositions, ReferencedTimeOffsets or Referenced DateTime? 15 Another String for the type? 16 7 WG23 2019/07/09 feedback: should there be standard default business names (e.g., for every code used in PS3.16? 17 8 WG23 2019/07/09 feedback: should there be an "include" mechanism for business name files? 18 9 WG23 2019/07/09 feedback: should business name definitions be allowed in same file as report? 19 10 KH: I would have expected each DICOM attribute to be encoded as a direct child of the top-level array, along with the 20 Content Tree. Answer: The approach taken follows the example in PS3.18 Annex F.4, which shows multiple query results, 21 each of which is an object in the top level array. Could this Annex F complication be elided/ It would reduce the 22 interoperability of parsers/generators designed to handle both. 23 11 KH: Is it necessary to include the metadata associated with the SR content. Answer: The header is required eventually 24 to make a valid DICOM object that can be stored in the PACS. The example shows a two step process, first generate 25 the SR content tree, then add the header (a separate tool, not the AI result creator, can do this, given context and the 26 original DICOM image headers). In the absence of a "platform" (which we are not yet defining), this work must occur out 27 of band. The preamble to the document (and work item) describes such a platform as might be based on DICOMweb as 28 out of scope for now, but you could imagine a service that added the JSON content tree (only) to an existing DICOM 29 study and that filled in the header. 30 12 KH: Should business names be used for header codes. Answer: Business names are not used for Code Sequence Items 31 in Code Sequence Attributes in the top level Data Set; rather, they are encoded in the traditional manner, i.e., as individual 32 DICOM Attributes. The reasons for this are (a) to align the DICOM Attribute header as closely with PS3.18 Annex F as 33 possible, and (b) very few, if any, Code Sequence Items are used in the headers of DICOM Structured Reporting SOP 34 Classes. Should this decision be reversed and business names used? 35 13 TID 1500 requires a Language, Procedure Reported, an empty section (CONTAINER) for the Image Library, and both 36 Tracking Identifier and Unique Identifier, all of which complicate the "simplest" example - should TID 1500 be modified 37 to relax some of these requirements? E.g., it would be harmless to omit an empty Image Library CONTAINER). and 38 Procedure Reported may not be known and will often be a dummy value based on Modality only, and the Language is 39 irrelevant for measurements (though not for code meanings of coded concepts, but could be assumed or just left unstated). 40 The Tracking Identifier and Unique Identifier are probably importsnt to retain though. 41 14 AF: What about business name prefixes, name spaces and JSON-LD? Answer: business names may be any string, no 42 standard prefixes are defined (perhaps with the exception of forbidding some keywords for annotations beginning with 43 '@'), there is no standard namespace mechanism in JSON and nor do we need one. "Clashes" are not an issue because 44 the scope of uniqueness is the pair of encoded JSON file and business name JSON file, and so the same business name 45 can be used for another pair with a different meaning. An open question is whether to allow JSON-LD annotations to be 46 present and be ignored in the encoded JSON file and the business names JSON file.

47 - Draft - 47 Sup nnn - JSON Representation of DICOM Structured Reports Page 8

1 15 AF: "anonymous" content items are burdensome, especially when there is no Value Type (e.g., IMAGE, confusing and 2 error-prone). Answer: A little perhaps, but the standard for the underlying SR infrastructure allows for "anonymous" 3 content items (no concept name) and the templates (like TID 1500 sub-templates) allows for them too, so they are 4 something the JSON representation has to support. The proposed syntax is trivial (empty quoted string where the business 5 name would go). Forcing the presence of a Value Type in this (or other settings) when it is not needed to resolve ambiguity 6 violates the terseness requirement (and starts to look like XML). Disagree that it is any more confusing or error prone 7 than using JSON in the first place. 8 16 AF: What is the motivation for the value always being a JSON Array, even for a leaf node and even when the value is a 9 single JSON String representing a text value or a coded value Business Name? Why not allow a single value? Answer. 10 To simplify the parsers job and reduce ambiguity (esp. for annotations and children). That said, will experiment with this 11 proposed simplification. 12 17 AF: Need to formalize allowed syntax; JSON Schema perhaps? Answer. Yes. TBD when more mature. 13 18 AF: Would prefer more full names rather than abbreviations for Content Item Annotations. Answer. No. Terseness is a 14 priority over readability (otherwise we would use XML). 15 19 AF: Don't use line numbering in JSON examples to allow select/copy/paste. Answer. The tooling doesn't permit line 16 numbers to be turned off just for the examples, and they are need for reference by commentators. Suggest using the 17 XML DocBook source if you want to copy/paste. The final standard will have no line numbers. 18 20 AF: Not convinced that positional rather than named Content Item Annotations will necessarily facilitate simplicity. Answer. 19 Same terseness argument. 20 21 AF: Re. IMAGE positional argument: Is it spelled out somewhere that in the situations where multiple items can be 21 present, null will be used for those that are missing? Answer: Yes ("Trailing unused values may be elided but intervening 22 values are required to be null if there is no value, in order to preserve the positional order."). 23 22 AF: Example: Aren't Values missing here [in header attributes]? Answer. Intentionally. That's how PS3.18 Annex F 24 (existing JSON) encodes Type 2 (empty) Attributes. 25 23 AF: Example: Confused by the [PersonName syntax]. Answer. That's how PS3.18 Annex F (existing JSON) encodes 26 PersonName values to allow for phonetic and ideographic CJK names. 27 24 AF: Example: Image Library: Why all those brackets/parentheses? Answer. That's because of the level of nesting of child 28 content items and how they are encoded as objects in arrays and the additional level of arrays required to handle sibling 29 content items with non-unique concept names. 30 25 KH: Some of Javascript (and JSON)'s organising principles:

31 • Where order and/or duplication of elements is important, use arrays (as you do in the draft spec)

32 • Where element identity is important but order isn't, use objects

33 • Dynamic typing is supported and even expected

34 Based on this, suggest:

35 • Retain the original array-of-objects approach for sibling content items

36 • Single-item arrays, whatever their type, may optionally be collapsed to a single value

37 • Where a named content item has only a value (or an array of values), it may optionally be collapsed to a key/value 38 pair

39 • Where a named content item has one or more children, annotations, or value "qualifiers" (such as measurement units 40 or graphic type), it must be defined by an object with appropriate property names. (That is, the top-level list of children 41 in a given content item, along with its individual annotations, value and/or value qualifiers, form a set of unique named 42 elements whose identities are important but their order is not.)

43 Also, I wonder if it might be reasonable to let valueless, non-annotated container nodes to specify their children directly.

44 Answer: Will evaluate these suggestions and their impact on ambiguity for parsing, and preservation of content item 45 order.

46 - Draft - 4 Sup nnn - JSON Representation of DICOM Structured Reports Page 9

1 Closed Issues

2 1

3 - Draft - 52 Sup nnn - JSON Representation of DICOM Structured Reports Page 10

1 Scope and Field of Application

2 This Supplement describes a simplified JSON representation of DICOM Structured Reports, similar to the PS3.18 JSON representation, 3 to allow AI developers to encode image-derived results. Patterns are defined for transformation of measurement and annotation in- 4 formation for use-cases related to the reporting of artificial intelligence (AI), machine learning (ML) and quantitative imaging (QI) results. 5 The approach is applicable not only to export of AI/ML results but also to encoding of truth data for AI/ML training, testing and validation.

6 JSON has emerged as the preferred representation for simple results from machine learning algorithms amongst developers who 7 are not familiar with the DICOM Standard. Such results typically lack the composite context information required in a managed clinical 8 environment (such as patient and study identity information) as well as references to the DICOM images used as input needed to 9 store, distribute and render the results on an image viewing system. DICOM Structured Reports (SR) are the most common form of 10 semantically meaningful annotation created and distributed in an interoperable manner for clinical use. Accordingly, this supplement 11 describes a mapping between a simple JSON representation of the measurement and annotation result payload expected from an 12 AI system, and the traditional binary DICOM SR encoding of the same information.

13 DICOM PS3.16 defines templates for different applications. The TID 1500 Measurement Report template describes a generic pattern 14 that is suitable for encoding AI results as well as other quantitative and qualitative (categorical or descriptive) results. The JSON 15 representation is exemplified using TID 1500, but the conversion supports full fidelity round-trip encoding of any DICOM SR instance, 16 regardless of the template.

17 The intent is to allow a simple encoded output that can be transcoded to the traditional binary encoding using an appropriate tool. A 18 full-fidelity and compliant binary SR can be created automatically from the JSON, with the subtleties of DICOM encoding hidden from 19 the user. There are many other use cases for this approach beyond the primary AI application that motivated the work.

20 The JSON representation leverages the "business name" concept from HL7 Green CDA, such that simple meaningful strings can be 21 used in the JSON for coded tuples for concept names and values, as well as for DICOM Attributes in the top level Data Set. The 22 business names are defined in a separate, potentially reusable, JSON file, which may be user- or organization-supplied (or automat- 23 ically generated in the case of a forward conversion, e.g., heuristically from the code meaning). The business names for standard 24 DICOM Attributes are the PS3.6 Keywords. Private Attributes and private codes are supported.

25 DICOM SR traditionally makes very extensive use of coded terminology to maximize semantic interoperability and to avoid reinventing 26 existing content. However, choosing and encoding codes can be burdensome to developers. Similarly, the use of DICOM Data Element 27 tags rather than keywords can be similarly complicated. Accordingly, "business names" are used as a substitute for more arcane 28 codes and tags that are normally used. For example, "StudyDate" may be used in the JSON representation in place of (0008,0020), 29 and "Length" used as opposed to (410668003, SCT, "Length"). The business names to be used can either by supplied by the creator 30 of the JSON representation, some other organization or authority, or selected from a standard list of keywords from the DICOM Data 31 Dictionary (PS3.6) or business names for concepts provided in the DICOM Content Mapping Resource (DCMR) (PS3.16). The business 32 names are not qualified by any prefix or namespace in the interests of terseness and simplicity. The scope of uniqueness of the 33 business names only has to encompass the encoded JSON file and its accompanying business names JSON file. That said, business 34 names may be as complex a string as the user cares to create, and any current or future convention for pseudo-name space mech- 35 anisms can be utilized if desired. TBD. JSON-LD annotations are ignored in the encoded JSON file and the business names JSON 36 file.

37 In striving for simplicity and ease of use, a two step process for transformation is envisaged. First, the result payload itself may be 38 encoded in JSON, e.g., by the AI/ML algorithm developer; this is limited to the minimal necessary information to describe the result 39 itself. Then, the JSON is merged with the necessary JSON representation of the composite context and other mandatory or relevant 40 optional SR content (such as UIDs, image libraries, hierarchical identification and report status management information), which, 41 when transformed, would result in a valid SR IOD with a template-compliant content. Finally the JSON is transformed into the tradi- 42 tional binary DICOM SR representation for transport and storage in an interoperable form. The intent is that the transformation of the 43 DICOM SR to and from the full JSON representation should have bi-directional semantic fidelity.

44 It is not the intent to use the JSON representation as the persistent form, but rather to leverage the existing DICOM binary object 45 storage infrastructure. However, it is anticipated that a future Supplement will be defined to extend the DICOMweb services to support 46 transformation of JSON DICOM SR into binary DICOM SR, and to retrieve transformed content, e.g., by leveraging the existing STOW- 47 RS and WADO-RS RetrieveRendered mechanisms. The scope of this Supplement is limited to describing the representation and the 48 transformation (TBD., although a media type, e.g., application/dicom-sr+json or similar may be defined).

49 The choice of JSON representation leverages the existing PS3.18 Annex F JSON representation of ordinary DICOM objects at the 50 data element level, though with the use of keywords in place of data element tags for clarity and simplicity. The DICOM SR content

51 - Draft - 32 Sup nnn - JSON Representation of DICOM Structured Reports Page 11

1 is divided into two components, the data elements representing the SR content tree, which are encoded as if each node (content 2 item) of the content tree were a distinct entity, and the remainder of the DICOM data elements that are normally encoded in the top 3 level DICOM data set. When parsing the JSON representation, all DICOM keywords that are recognized and that are not part of the 4 SR content tree are extracted, then all remaining content is examined for matches with the defined business names. To the extent 5 possible, the relationships and value types that are defined for the DICOM binary SR representation are elided from the JSON rep- 6 resentation, and either defined with the business names, or inferred from context. For example, whether a coded concept has a 7 CONTAINS or HAS CONCEPT MOD relationship with its parent CONTAINER value type is not something an AI/ML developer is 8 likely to be concerned about, and requires a level of DICOM expertise that they are unlikely to have. Accordingly, a coded concept 9 that is always used as a concept modifier can have this declared in the business name descriptor rather than repeating the information 10 in every use in the JSON payload. Similarly, some concepts always have a predictable value type (e.g., of CODE or TEXT or NUM) 11 that can be declared in the business name file. Occasionally, SR content items have no required concept name (e.g., IMAGE references 12 within SCOORD) but such patterns can be detected and inferred. When ambiguity is possible, then the JSON representation can be 13 made explicit to resolve it (e.g., to distinguish TEXT from CODE value types when either may be used and there is ambiguity as to 14 whether the value should be used literally or looked up as a business name for the code value).

15 Again, in the interest of simplicity, the JSON representation chosen is largely positional rather than using named parameters; e.g., 16 the payload of a content item that is a TEXT or CODE value type is an array of strings, which in the degenerate form is a single value, 17 wherein the string is used as a TEXT value or a CODE business name depending on what the concept name is recognized to be, 18 rather than explicitly naming the type of value (i.e., "Kidney" rather than {"@code": "Kidney"}. Similarly, more complex content item 19 value types like NUM or SCOORD or IMAGE, for which there are multiple values with a specific purpose are defined positionally as 20 well, e.g., the position in the array determines whether the business name is for the units, or the UID is for the SOP Class or SOP 21 Instance, or whether the value is the graphic type or an array of coordinates (and for the latter, which are X and Y), etc. This places 22 greater demands on the parser/converter as well as the documentation, but provides simplicity (if not always clarity) for the user. Al- 23 ternatives such as fully qualifying every parameter with its type and purpose produced an undesirably cluttered representation, especially 24 for the simple use cases.

25 At this time, no similar standard XML representation is defined, thought the concepts are equally applicable, theoretically. The consensus 26 in the AI/ML community at this time seems to be to focus on JSON. Several toolkits have their own XML schemas and representations 27 for DICOM SR, but there has been no significant effort to harmonize or standardize these, and the outstanding work item to do so 28 (2012-11B) has been withdrawn after many years of inaction.

29 This Supplement defines a new Part, PS3.22, to contain the alternative representation, and includes the existing Attribute level XML 30 and JSON encodings factored out of PS3.19 and PS3.18.

31 - Draft - 6 Sup nnn - JSON Representation of DICOM Structured Reports Page 12

1 PS3.22

2 DICOM PS3.22 20xxz - Alternative Representations

3 DICOM Standards Committee

5 - Draft - 27 Sup nnn - JSON Representation of DICOM Structured Reports Page 13

1 Notice and Disclaimer

2 The information in this publication was considered technically sound by the consensus of persons engaged in the development and 3 approval of the document at the time it was developed. Consensus does not necessarily mean that there is unanimous agreement 4 among every person participating in the development of this document.

5 NEMA standards and guideline publications, of which the document contained herein is one, are developed through a voluntary 6 consensus standards development process. This process brings together volunteers and/or seeks out the views of persons who have 7 an interest in the topic covered by this publication. While NEMA administers the process and establishes rules to promote fairness 8 in the development of consensus, it does not write the document and it does not independently test, evaluate, or verify the accuracy 9 or completeness of any information or the soundness of any judgments contained in its standards and guideline publications.

10 NEMA disclaims liability for any personal injury, property, or other damages of any nature whatsoever, whether special, indirect, 11 consequential, or compensatory, directly or indirectly resulting from the publication, use of, application, or reliance on this document. 12 NEMA disclaims and makes no guaranty or warranty, expressed or implied, as to the accuracy or completeness of any information 13 published herein, and disclaims and makes no warranty that the information in this document will fulfill any of your particular purposes 14 or needs. NEMA does not undertake to guarantee the performance of any individual manufacturer or seller's products or services by 15 virtue of this standard or guide.

16 In publishing and making this document available, NEMA is not undertaking to render professional or other services for or on behalf 17 of any person or entity, nor is NEMA undertaking to perform any duty owed by any person or entity to someone else. Anyone using 18 this document should rely on his or her own independent judgment or, as appropriate, seek the advice of a competent professional 19 in determining the exercise of reasonable care in any given circumstances. Information and other standards on the topic covered by 20 this publication may be available from other sources, which the user may wish to consult for additional views or information not covered 21 by this publication.

22 NEMA has no power, nor does it undertake to police or enforce compliance with the contents of this document. NEMA does not cer- 23 tify, test, or inspect products, designs, or installations for safety or health purposes. Any certification or other statement of compliance 24 with any health or safety-related information in this document shall not be attributable to NEMA and is solely the responsibility of the 25 certifier or maker of the statement.

26 - Draft - 5 Sup nnn - JSON Representation of DICOM Structured Reports Page 14

1 Foreword

2 This DICOM Standard was developed according to the procedures of the DICOM Standards Committee.

3 The DICOM Standard is structured as a multi-part document using the guidelines established in [ISO/IEC Directives, Part 3].

4 - Draft - 9 Sup nnn - JSON Representation of DICOM Structured Reports Page 15

1 1 Scope and Field of Application

2 This part of the DICOM Standard specifies the alternative representations of DICOM encoded instances and messages to the tradi- 3 tional binary encoding defined in PS3.5.

4 It includes:

5 • Encoding of DICOM instances and messages at the Attribute level in XML

6 • Encoding of DICOM instances and messages at the Attribute level in JSON

7 • Encoding of DICOM Structured Reports at the Content Item level in JSON

8 - Draft - 15 Sup nnn - JSON Representation of DICOM Structured Reports Page 16

1 2 Normative and Informative References

2 The following standards contain provisions that, through reference in this text, constitute provisions of this Standard. At the time of 3 publication, the editions indicated were valid. All standards are subject to revision, and parties to agreements based on this Standard 4 are encouraged to investigate the possibilities of applying the most recent editions of the standards indicated below.

5 2.1 International Organization for Standardization (ISO) and International 6 Electrotechnical Commission (IEC)

7 [ISO/IEC Directives, Part 3] ISO/IEC. 1989. Drafting and presentation of International Standards.

8 2.2 Internet Engineering Task Force (IETF) and Internet Assigned Names 9 Authority (IANA)

10 [RFC4627] IETF. July 2006. The application/json Media Type for JavaScript Object Notation (JSON). http://tools.ietf.org/html/rfc4627 11 .

12 2.3 Other References

13 [XML] W3C. 2006/09/29. Extensible Markup Language (XML) 1.1. https://www.w3.org/TR/2006/REC-xml11-20060816/ .

14 - Draft - 40 Sup nnn - JSON Representation of DICOM Structured Reports Page 17

1 3 Definitions

2 For the purposes of this Standard the following definitions apply.

3 3.1 Codes and Controlled Terminology Definitions:

4 The following definitions are commonly used in this Part of the DICOM Standard:

65 Coding Schemes Dictionaries (lexicons) of concepts (terms) with assigned codes and well defined meanings.

87 Content Item A node in the Content Tree of a DICOM SR document, consisting of either a container with a coded Concept 9 Name, or a name-value pair with a coded Concept Name and a Concept Value.

1110 Content Tree The tree of Content Items of a DICOM SR document.

1312 Context Group A set of coded concepts defined by a Mapping Resource forming a set appropriate to use in a particular 14 context.

1615 Context ID (CID) Identifier of a Context Group.

1817 Template A pattern that describes the Content Items, Value Types, Relationship Types and Value Sets that may be 19 used in part of a Structured Report content tree, or in other Content Item constructs, such as Acquisition 20 Context or Protocol Context. Analogous to a Module of an Information Object Definition.

2221 Template ID (TID) Identifier of a Template.

23 3.1 Representation Conversion Definitions:

24 The following definitions are commonly used in this Part of the DICOM Standard:

2625 Business Name Identifier for a Concept or Attribute that corresponds to a business requirement for information exchange.

27 Note

28 See also a similar definition in PS3.20 in the context of HL7 CDA templates, and discussion in 29 PS3.20 Section 5.2.1, which includes the statement that "the use of readable and intuitive Business 30 Names provides a method of direct access to insert data that is specific to each clinical report in- 31 stance".

3332 Composite Context Those Attributes of higher-level Entities in the Information Model that provide the Context for newly created 34 lower-level Entities, and have the same values as other lower-level Entities have for the same higher-level 35 Entities.

36 Note

37 Typically the Patient and Study Composite Context are merged with (shared by) new Series and 38 Instance level information when a new Series of Instances is created on a new device.

39 - Draft - 24 Sup nnn - JSON Representation of DICOM Structured Reports Page 18

1 4 Symbols and Abbreviations

2 The following symbols and abbreviations are used in this Part of the Standard.

43 DICOM Digital Imaging and Communications in Medicine

65 IOD Information Object Definition

87 ISO International Standards Organization

109 JSON JavaScript Object Notation

1211 NEMA National Electrical Manufacturers Association

1413 SR Structured Reporting

1615 UCUM Unified Code for Units of Measure

1817 UID Unique Identifier

2019 XML Extensible Markup Language

2221 XSLT Extensible Stylesheet Language Transformations

23 - Draft - 4 Sup nnn - JSON Representation of DICOM Structured Reports Page 19

1 5 Conventions

2 Terms listed in Section 3 Definitions are capitalized throughout the document.

3 - Draft - 23 Sup nnn - JSON Representation of DICOM Structured Reports Page 20

1 A XML Encoding

2 A.1 Introduction to XML

3 XML ... It is .... It is described in detail by the W3C [XML].

4 A.2 XML Encoding of DICOM Instances

5 Include contents of PS3.19 Section A.1 Native DICOM Model, renaming "Native DICOM Model" to "XML Encoding of DICOM 6 Instances in Native Format" and renumbering sections appropriately.

7 A.2.1 DICOM XML Encoding in Native Format

8 A.2.1.1 Usage

9 ...

10 A.2.1.2 Identification

11 ...

12 A.2.1.3 Support

13 ...

14 A.2.1.4 Information Model

15 ...

16 A.2.1.5 Description

17 ...

18 A.2.1.6 Schema

19 ...

20 A.2.1.7 Examples

21 ...

22 - Draft - 35 Sup nnn - JSON Representation of DICOM Structured Reports Page 21

1 B JSON Encoding

2 B.1 Introduction to JavaScript Object Notation (JSON)

3 JSON is a text-based open standard, derived from JavaScript, for representing data structures and associated arrays. It is language- 4 independent, and primarily used for serializing and transmitting lightweight structured data over a network connection. It is described 5 in detail by the Internet Engineering Task Force (IETF) in [RFC4627].

6 B.2 JSON Encoding of DICOM Instances and Messages

7 B.2.1 Introduction

8 The JSON Encoding of DICOM Instances and Messages defines a representation of binary-encoded DICOM SOP Instances as JSON 9 that allows a sender or recipient of data to create or navigate through a DICOM Data Set using JSON-based tools instead of relying 10 on tool kits that understand the binary encoding of DICOM.

11 B.2.2 DICOM JSON Encoding

12 Include contents of PS3.18 Section F.2 DICOM JSON Model, renaming "DICOM JSON Model" to "JSON Encoding of DICOM 13 Instances and Messages" and renumbering sections appropriately:

14 B.2.3 Transformation to and from other DICOM Encodings

15 Include contents of PS3.18 Section F.3 Transformation with other DICOM Formats, renaming "DICOM JSON Model" to "JSON 16 Encoding of DICOM Instances and Messages" and renumbering sections appropriately:

17 B.2.4 DICOM JSON Encoding Example

18 Include contents of PS3.18 Section DICOM JSON Model Example, renaming "DICOM JSON Model" to "JSON Encoding of DICOM 19 Instances and Messages" and renumbering sections appropriately:

20 B.3 JSON Encoding of Structured Reports

21 B.3.1 Introduction

22 The JSON Encoding of DICOM Structured Reports defines a representation of binary-encoded DICOM Structured Report Instances 23 as JSON that allows a sender or recipient of data to create or navigate through a DICOM Structured Report using JSON-based tools 24 instead of relying on tool kits that understand the binary encoding of DICOM.

25 B.3.2 DICOM JSON Structured Report Encoding

26 The JSON SR encoding consists of two files:

27 • the JSON encoded SR Content File

28 • the JSON encoded Business Names File

29 The Content File consists of:

30 • a single top-level array containing a single JSON object (i.e., a single "result" in PS3.18 Annex F terminology),

31 • that single JSON object containing an unordered set of subordinate JSON objects, each of which is either:

32 • a JSON encoded DICOM Attribute of the top level Data Set, or

33 • the root node of a JSON encoded DICOM Structured Report Content Tree

34 - Draft - 39 Sup nnn - JSON Representation of DICOM Structured Reports Page 22

1 Note

2 In PS3.18 Annex F, the set of subordinate objects is defined to be ordered by their property name in ascending order. No 3 such order is required in the representation defined here, since the property names are Business Names, not hexadecimal 4 numeric representations of a DICOM Tag. There is no need to sort the property names alphabetically, and it would be unne- 5 cessarily burdensome to the author of the JSON representation to require them to be sorted in their binary DICOM Tag order, 6 though such sorting will be required when converting to binary DICOM encoding.

7 Business Names are used to identify:

8 • DICOM Attributes (rather than using DICOM Data Element Tags)

9 • Codes used in the DICOM Structured Report Content Tree

10 Standard DICOM Attributes used in the SR Content File are identified by the Keyword used in the PS3.6 Table 6-1 Registry of DICOM 11 Data Elements.

12 Note

13 It is not necessary to describe Standard DICOM Attributes used in the Business Names File but it is not prohibited.

14 Private DICOM Attributes used in the SR Content File shall be described in the Business Names File.

15 All codes used in the SR Content File shall be described in the Business Names File.

16 Note

17 Business names are not used for Code Sequence Items in Code Sequence Attributes in the top level Data Set; rather, they 18 are encoded in the traditional manner, i.e., as individual DICOM Attributes. The reasons for this are (a) to align the DICOM 19 Attribute header as closely with PS3.18 Annex F as possible, and (b) very few, if any, Code Sequence Items are used in 20 DICOM Structured Reporting SOP Classes.

21 The Business Names File also encodes other information related to the Content Items for which a Code is used as the Concept Name, 22 including:

23 • Value Type

24 • Relationship

25 B.3.2.1 Attribute Encoding

26 Each DICOM Attribute in the top level Data Set, and all of the DICOM Attributes nested within Sequence Attributes in the top level 27 Data Set, except the Attributes describing the root node of the Structured Report Content Tree, are encoded as follows:

28 • Each Attribute is encoded in the same manner as used for the JSON Encoding of DICOM Instances and Messages, as defined in 29 Section B.2.2 DICOM JSON Encoding, except that

30 • In place of the eight character uppercase hexadecimal representation of a DICOM Tag used as the name of each Attribute object, 31 either of the following may be used:

32 • a Standard DICOM Data Element Keyword from PS3.6 Table 6-1 Registry of DICOM Data Elements, or

33 • a Business Name defined in the Business Names File

34 • The Value Representation ("vr") may be omitted for Standard Data Elements (since a dictionary is expected to be available to the 35 parser)

36 For example, either of the following is a valid encoding of the same Attribute:

37 "00080020": { "vr": "DT", "Value": [ "20130409" ]

38 - Draft - 33 Sup nnn - JSON Representation of DICOM Structured Reports Page 23

1 "StudyDate": { "vr": "DT", "Value": [ "20130409" ]

2 "StudyDate": { "Value": [ "20130409" ]

3 The following would also be valid, if the Business Names "FechaDeEstudio" or "試験日" were defined in the Business Names File:

4 "FechaDeEstudio": { "vr": "DT", "Value": [ "20130409" ]

5 "試験日": { "Value": [ "20130409" ]

6 TBD. Private data element example with or without creator.

7 The Attributes in the top level data Set describing the root node of the Structured Report Content Tree that are not encoded are:

8 • ContentSequence

9 • ValueType

10 • ConceptNameCodeSequence

11 • ContinuityOfContent

12 • ContentTemplateSequence

13 • MappingResource

14 • TemplateIdentifier

15 B.3.2.2 Structured Report Content Tree Encoding

16 A DICOM Structured Report Content Tree consists of a nested set of Content Items of unlimited depths, beginning with a single root 17 Content Item.

18 B.3.2.2.1 Content Item Encoding

19 Each Content Item, including the root Content Item, is encoded as a single JSON object, consisting of:

20 • a JSON object name, that is a Business Name defined in the Business Names File for the DICOM Concept Name Code Sequence

21 a JSON object value, that is a JSON Array, whose encoding depends on:

22 • the DICOM Content Item Value Type

23 • whether the Content Item is a leaf node of the tree or has children

24 • whether children are by-value or by-reference

25 Neither the Value Type nor the Relationship Type are explicitly encoded in the JSON representation, since:

26 • appropriate default values are defined in the Business Name File

27 • for anonymous Content Items (those without a Concept Name), sufficient context allows the Value Type and the Relationship Type 28 to be deduced

29 The following is an example of a leaf Content Item:

30 { 31 "TrackingIdentifier": [

32 - Draft - 47 Sup nnn - JSON Representation of DICOM Structured Reports Page 24

1 "5b6eb4301d3175942d29985a3d0fbb00" 2 ] 3 }

4 Note

5 The value is always a JSON Array, even for a leaf node and even when the value is a single JSON String representing a 6 text value or a coded value Business Name.

7 B.3.2.2.2 Nested Content Encoding

8 For Content Items with children, the last entry of the JSON object value Array is itself an Array, which contains the ordered list of child 9 Content Items, each of which is a JSON object.

10 Note

11 1. Leaf Content Items have no such final Array, rather than an empty Array.

12 2. Use of an Array allows for preservation of the order of child Content Items, which may be significant.

13 3. Use of an Array allows for multiple children with the same Concept Name, since JSON does not permit multiple JSON 14 Objects with the same name.

15 The following is an example of a CODE Content Item with one child that is also a CODE Content Item:

16 { 17 "LanguageOfContentItemAndDescendants": [ 18 "English", 19 [ 20 { 21 "CountryOfLanguage": [ 22 "UnitedStates" 23 ] 24 } 25 ] 26 ] 27 }

28 B.3.2.2.3 Content Item Annotations

29 Though most of the Content Tree encoding is addressed positionally or implicitly, there are less commonly used Attributes of Content 30 Items that need to be encoded either to preserve the full fidelity of the content or to address structural concerns, such as by-reference 31 relationships.

32 Accordingly, the first item of the JSON object value Array may be a JSON Object, containing a set of JSON Objects each of which is 33 a Content Item Annotation.

34 The names of all Standard Content Item Annotations begin with the "@" symbol.

35 The Standard Content Item Annotations are:

3736 @label The label of a Content Item that is the target of a by-reference relationship

3938 @ref The reference to a labeled Content Item that is the target of a by-reference relationship

4140 @tid The TemplateIdentifier value of a CONTAINER Content Item

4342 @tmr The TemplateMappingResource value of a CONTAINER Content Item

4544 @cont The ContinuityOfContent value of a CONTAINER Content Item

46 - Draft - 48 Sup nnn - JSON Representation of DICOM Structured Reports Page 25

1 Other annotations than these are permitted in the Content Item Annotation object, as long as their names do not begin with the "@" 2 symbol.

3 Note

4 The intent of allowing other annotations is to allow preservation of private DICOM Attributes that may be associated with the 5 Content Item, but no standard representation for such private information is defined. I.e., this is not a generic solution for 6 including unrecognized private Attributes attached to Content Items.

7 This is an example of Content Item Annotations used to describe the template used for a root level CONTAINER (child Array illustrated 8 but children omitted):

9 "ImagingMeasurementReport": [ 10 { 11 "@tmr": "DCMR", 12 "@tid": "1500" 13 }, 14 [ ... ] 15 ]

16 B.3.2.2.4 Encoding of By-Reference Relationships

17 Most commonly the Content Tree is strictly hierarchical (i.e., a tree) and many templates and some SR Storage SOP Classes constrain 18 the encoding to that pattern. However, by-reference relationships are permitted by underlying mechanism (a directed acyclic graph) 19 and hence are supported in the JSON encoding by use of Content Item Annotations.

20 Both the referenced Content Item and the referencing Content Item need to be decorated with the Content Item Annotations as follows:

21 • The referenced Content Item must have a @label annotation, whose value may be any string unique within the instance.

22 • The referencing Content Item must have a @ref annotation, whose value shall correspond to the @label annotation of a Content 23 Item within the instance.

24 Note

25 1. The @label and @ref annotations are not required to be the numeric hierarchical position in the Content Tree. This 26 simplifies the creator's task, which is the priority.

27 2. In the traditional binary DICOM SR encoding, the references are made using ReferencedContentItemIdentifier (see 28 PS3.3 C.17.3.2.5), which is the numeric hierarchical position in the Content Tree, so the parse of the JSON represent- 29 ation is required to map the @ref values to the numeric hierarchical position by tracking the position of Content Items 30 with @label annotations. Since forward references are permitted, this may require two passes of the JSON file.

31 The following is a simplified example of a reference from a CAD finding SCOORD to an IMAGE Content Item in an Image Library:

32 [ 33 {"ImageLibrary": [[{"": [ 34 {"@label": "label1"}, 35 "1.2.840.10008.5.1.4.1.1.1.2", 36 "1.3.6.1.4.1.5962.99.1.993064428.2122236180.1358202762732.2.0" 37 ]}]]}, 38 {"CADProcessingAndFindingsSummary": [ 39 "AllAlgorithmsSucceededWithFindings", 40 [{"IndividualImpressionRecommendation": [[ 41 ..., 42 {"Center": [ 43 "POINT", 44 [165,2433], 45 [{"": [{"@ref": "label1"}]}] 46 ]},

47 - Draft - 44 Sup nnn - JSON Representation of DICOM Structured Reports Page 26

1 ... 2 ]]}] 3 ]} 4 ]}, 5 ... 6 ]

7 B.3.2.2.5 Encoding of Content Items of Specific Value Type

8 The encoding of Content Items depends on their Value Type. There is a specific JSON representation for each of the Value Types 9 defined in PS3.3 Table C.17-5 Document Content Macro Attributes.

10 Note

11 The pattern of encoding for each Value Type not only allows each Content Item to be encoded with full fidelity, but also allows 12 for recognition of the Value Type by the parser when it is not explicitly defined for the Concept Name in the Business Names 13 File, or is potentially ambiguous, such as for anonymous Content Items that do not have a Concept Name.

14 B.3.2.2.5.1 Encoding of Content Items Without a Value

15 The following Value Type never has a value, though may have children, and is encoded in the JSON value Array without a value:

16 • CONTAINER

17 The following is an example of a CONTAINER Content Item, where the code and Value Type for "ImageLibrary" are defined in the 18 Business Names File:

19 { 20 "ImageLibrary": [] 21 }

22 The following is an example of a CONTAINER Content Item, with one child that is a CONTAINER with no children of its own:

23 { 24 "ImageLibrary": [ 25 [ 26 { 27 "ImageLibraryGroup": [] 28 } 29 ] 30 ] 31 }

32 B.3.2.2.5.2 Encoding of Content Items with a Single Value

33 The following Value Types consist of a single value that is either textual or a code, and are all encoded in the JSON value Array using 34 a single JSON String:

35 • CODE

36 • DATE

37 • DATETIME

3938 • PERSONNAME ???? should be decomposed as for DICOM Attribute VR ???

40 • TEXT

41 • TIME

42 • UIDREF

43 - Draft - 45 Sup nnn - JSON Representation of DICOM Structured Reports Page 27

1 The JSON String value may represent a text value or a coded value Business Name, depending on the Value Type deduced from 2 the Concept Name.

3 The following is an example of a TEXT Content Item with no children, where the code and Value Type for "Comment" is defined in 4 the Business Names File:

5 { 6 "Comment": [ 7 "Kidney" 8 ] 9 }

10 The following is an example of a CODE Content Item with no children, where the code and Value Type for "FindingSite", as well as 11 the code for the "Kidney", are defined in the Business Names File:

12 { 13 "FindingSite": [ 14 "Kidney" 15 ] 16 }

17 The following is an example of a CODE Content Item with one child that is also a CODE Content Item, with no children of its own:

18 { 19 "LanguageOfContentItemAndDescendants": [ 20 "English", 21 [ 22 { 23 "CountryOfLanguage": [ 24 "UnitedStates" 25 ] 26 } 27 ] 28 ] 29 }

30 B.3.2.2.5.3 Encoding of Numeric Content Items

31 The following Value Type a single numeric value and its measurement units encoded in the JSON value Array as a pair of JSON 32 Strings:

33 • NUM

34 The first JSON String is the value of the DS VR NumericValue in MeasuredValueSequence.

35 The second JSON String is the Business Name of the code in the MeasurementUnitsCodeSequence in MeasuredValueSequence.

36 The following is an example of a NUM Content Item with no children, where the code and Value Type for "Length", as well as the 37 code for the "mm", are defined in the Business Names File:

38 { 39 "Length": [ 40 "66.43856134", 41 "mm" 42 ] 43 }

44 - Draft - 41 Sup nnn - JSON Representation of DICOM Structured Reports Page 28

1 The encoding of the other less frequently used Attributes within MeasuredValueSequence defined in PS3.3 Table C.18.1-1 Numeric 2 Measurement Macro Attributes is addressed by the use of Content Item Annotations as follows:

43 • TBD.

5 B.3.2.2.5.4 Encoding of Content Items That Reference Storage SOP Instances

6 The following Value Types reference Storage SOP Instances and encode in the JSON value Array the SOPClassUID and SOPIn- 7 stanceUID of the referenced instance as a pair of JSON Strings:

8 • COMPOSITE

9 • IMAGE

10 • WAVEFORM

11 The first JSON String is the value of the ReferencedSOPClassUID in ReferencedSOPSequence.

12 The second JSON String is the value of the ReferencedSOPInstanceUID in MeasuredValueSequence.

13 The following is an example of an IMAGE Content Item with no children, for which there is no Concept Name encoded (i.e., is an 14 anonymous Content Item):

15 { 16 "": [ 17 "1.2.840.10008.5.1.4.1.1.2", 18 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 19 ] 20 }

21 Note

22 A parser can detect that this is an IMAGE Content Item even in the absence of a Business Name for the Concept Name by 23 recognizing that the JSON value Array contains two JSON Strings, the first of which is a recognized Storage SOP Class 24 UID. Were the reference to be to an unrecognized SOP Class, such as a Private SOP Class, the risk of ambiguity can be 25 avoided by providing a Concept Name with the appropriate Value Type even if the Template does not require it. TBD. ???? 26 allow annotation with explicit Value Type for such cases ????

27 If an IMAGE Content Item has other Attributes than ReferencedSOPClassUID and ReferencedSOPInstanceUID, they are encoded 28 as additional values in the following order:

29 1. Array of ReferencedFrameNumber

30 2. ReferencedSegmentNumber

31 3. ReferencedSOPClassUID in ReferencedSOPSequence (to a Presentation State Instance)

32 4. ReferencedSOPInstanceUID in ReferencedSOPSequence (to a Presentation State Instance)

33 5. ReferencedSOPClassUID in ReferencedRealWorldValueMappingInstanceSequence

34 6. ReferencedSOPInstanceUID in ReferencedRealWorldValueMappingInstanceSequence

35 Trailing unused values may be elided but intervening values are required to be null if there is no value, in order to preserve the posi- 36 tional order.

37 Note

38 These additional Attributes are encoded positionally rather than as named Content Item Annotations for simplicity and 39 compactness.

40 - Draft - 47 Sup nnn - JSON Representation of DICOM Structured Reports Page 29

1 The following is an example of an IMAGE Content Item with no children, for which there is no Concept Name encoded (i.e., is an 2 anonymous Content Item) and with a ReferencedSegmentNumber:

3 { 4 "": [ 5 "1.2.840.10008.5.1.4.1.1.66.4", 6 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288813963226516023762", 7 null, 8 3 9 ] 10 }

11 If a WAVEFORM Content Item has other Attributes than ReferencedSOPClassUID and ReferencedSOPInstanceUID ...TBD.

12 B.3.2.2.5.5 Encoding of Coordinate Content Items

13 The following Value Types encode in the JSON value Array the type of coordinates as a JSON String followed by a JSON Array of 14 numeric coordinates:

15 • SCOORD

16 • SCOORD3D

17 • TCOORD

18 For SCOORD Content Items:

19 1. The JSON String is the value of the GraphicType.

20 2. The JSON Array contains the values of GraphicData

21 The following is an example of an SCOORD Content Item SELECTED FROM an IMAGE, for which there is no Concept Name encoded 22 for either (i.e., they are anonymous Content Items):

23 { 24 "": [ 25 "POLYLINE", 26 [ 27 172.83535766601562, 28 270.0640869140625, 29 133.79888916015625, 30 343.0453186035156 31 ], 32 [ 33 { 34 "": [ 35 "1.2.840.10008.5.1.4.1.1.2", 36 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 37 ] 38 } 39 ] 40 ] 41 }

42 Note

43 1. A parser can detect that this is an SCOORD Content Item even in the absence of a Business Name for the Concept 44 Name by recognizing that the JSON value Array contains a JSON String followed by a JSON Array of Numbers; further 45 the JSON String is a recognized GraphicType. The risk of ambiguity can be avoided by providing a Concept Name with

46 - Draft - 45 Sup nnn - JSON Representation of DICOM Structured Reports Page 30

1 the appropriate Value Type even if the Template does not require it. TBD. ???? allow annotation with explicit Value 2 Type for such cases ????

3 2. A parser can assume that a SELECTED FROM Relationship Type is needed between the parent SCOORD Content 4 Item and the child IMAGE Content Item, since that is the only relationship permitted between these two Value Types.

5 TBD. Add other Attributes as Annotations or positionally, specifically PixelOriginInterpretation (for WSI) and FiducialUID.

6 For SCOORD3D Content Items: TBD.

7 For WAVEFORM Content Items:

8 1. The JSON String is the value of TemporalRangeType.

109 2. The JSON Array contains the values of ReferencedSamplePositions, ReferencedTimeOffsets or Referenced DateTime. TBD 11 How to distinguish these????? Need another type String?????

12 B.3.2.2.6 Encoding of Business Names File

13 TBD. May or may not want to describe this before the content tree.

14 B.3.3 Transformation to and from other DICOM Encodings

15 B.3.4 DICOM JSON Structured Report Encoding Examples

16 B.3.4 DICOM JSON Simple Single Linear Measurement Encoding Example

17 The following is the simplest example of the content that TID 1500 allows for a single linear distance measurement.

18 Note that TID 1500 requires both a Tracking Identifier and Tracking Unique Identifier. These are minimal requirements of the Template, 19 not of the JSON transformation per se, and might not be needed by other Templates.

20 A compact representation of the semantic content of the transformed DICOM SR tree is shown here:

21 : CONTAINER: (126000,DCM,"Imaging Measurement Report") [SEPARATE] (DCMR,1500) 22 >CONTAINS: CONTAINER: (126010,DCM,"Imaging Measurements") [SEPARATE] 23 >>CONTAINS: CONTAINER: (125007,DCM,"Measurement Group") [SEPARATE] 24 >>>HAS OBS CONTEXT: TEXT: (112039,DCM,"Tracking Identifier") = "5b6eb4301d3175942d29985a3d0fbb00" 25 >>>HAS OBS CONTEXT: UIDREF: (112040,DCM,"Tracking Unique Identifier") = "1.3.6.1.4.1.5962.1.1.0.0.0.1535644357.22655.1" 26 >>>CONTAINS: NUM: (G-D7FE,SRT,"Length") = 66.43856134 (mm,UCUM,"mm") 27 >>>>INFERRED FROM: SCOORD: = POLYLINE {172.835357666016,270.064086914062,133.798889160156,343.045318603516} 28 >>>>>SELECTED FROM: IMAGE: = (1.2.840.10008.5.1.4.1.1.2,1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762)

29 This is the JSON File consisting of just the Content Item Tree, with the DICOM top level Data Set omitted for clarity, such as might 30 be produced by an AI tool before merging with the composite context:

31 [ 32 "ImagingMeasurementReport": [ 33 { 34 "@tmr": "DCMR", 35 "@tid": "1500" 36 }, 37 [ 38 { 39 "ImagingMeasurements": [ 40 [ 41 { 42 "MeasurementGroup": [ 43 [

44 - Draft - 58 Sup nnn - JSON Representation of DICOM Structured Reports Page 31

1 { 2 "TrackingIdentifier": [ 3 "5b6eb4301d3175942d29985a3d0fbb00" 4 ] 5 }, 6 { 7 "TrackingUniqueIdentifier": [ 8 "1.3.6.1.4.1.5962.1.1.0.0.0.1535644357.22655.1" 9 ] 10 }, 11 { 12 "Length": [ 13 "66.43856134", 14 "mm", 15 [ 16 { 17 "": [ 18 "POLYLINE", 19 [ 20 172.83535766601562, 21 270.0640869140625, 22 133.79888916015625, 23 343.0453186035156 24 ], 25 [ 26 { 27 "": [ 28 "1.2.840.10008.5.1.4.1.1.2", 29 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 30 ] 31 } 32 ] 33 ] 34 } 35 ] 36 ] 37 } 38 ] 39 ] 40 } 41 ] 42 ] 43 } 44 ] 45 ] 46 } 47 ]

48 The following is a more realistic example of a TID 1500 encoding of a single linear distance measurement, which adds Language and 49 Country, the Person Observer, the Procedure Reported, an Image Library entry, and a Finding Site.

50 A compact representation of the semantic content of the transformed DICOM SR tree is shown here:

51 : CONTAINER: (126000,DCM,"Imaging Measurement Report") [SEPARATE] (DCMR,1500) 52 >HAS CONCEPT MOD: CODE: (121049,DCM,"Language of Content Item and Descendants") = (eng,RFC5646,"English") 53 >>HAS CONCEPT MOD: CODE: (121046,DCM,"Country of Language") = (US,ISO3166_1,"United States") 54 >HAS OBS CONTEXT: PNAME: (121008,DCM,"Person Observer Name") = "accomplished_peafowl" 55 >HAS CONCEPT MOD: CODE: (121058,DCM,"Procedure reported") = (41806-1,LN,"CT Abdomen") 56 >CONTAINS: CONTAINER: (111028,DCM,"Image Library") [SEPARATE]

57 - Draft - 59 Sup nnn - JSON Representation of DICOM Structured Reports Page 32

1 >>CONTAINS: CONTAINER: (126200,DCM,"Image Library Group") [SEPARATE] 2 >>>CONTAINS: IMAGE: = (1.2.840.10008.5.1.4.1.1.2,1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762) 3 >>>>HAS ACQ CONTEXT: CODE: (121139,DCM,"Modality") = (CT,DCM,"Computed Tomography") 4 >>>>HAS ACQ CONTEXT: DATE: (111060,DCM,"Study Date") = "19870620" 5 >>>>HAS ACQ CONTEXT: TIME: (111061,DCM,"Study Time") = "135823" 6 >CONTAINS: CONTAINER: (126010,DCM,"Imaging Measurements") [SEPARATE] 7 >>CONTAINS: CONTAINER: (125007,DCM,"Measurement Group") [SEPARATE] 8 >>>HAS OBS CONTEXT: TEXT: (112039,DCM,"Tracking Identifier") = "5b6eb4301d3175942d29985a3d0fbb00" 9 >>>HAS OBS CONTEXT: UIDREF: (112040,DCM,"Tracking Unique Identifier") = "1.3.6.1.4.1.5962.1.1.0.0.0.1535644357.22655.1" 10 >>>HAS CONCEPT MOD: CODE: (G-C0E3,SRT,"Finding Site") = (T-71000,SRT,"Kidney") 11 >>>CONTAINS: NUM: (G-D7FE,SRT,"Length") = 66.43856134 (mm,UCUM,"mm") 12 >>>>INFERRED FROM: SCOORD: = POLYLINE {172.835357666016,270.064086914062,133.798889160156,343.045318603516} 13 >>>>>SELECTED FROM: IMAGE: = (1.2.840.10008.5.1.4.1.1.2,1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762)

14 This is the JSON File consisting of just the Content Item Tree, with the DICOM top level Data Set omitted for clarity, such as might 15 be produced by an AI tool before merging with the composite context:

16 [ 17 "ImagingMeasurementReport": [ 18 { 19 "@tmr": "DCMR", 20 "@tid": "1500" 21 }, 22 [ 23 { 24 "LanguageOfContentItemAndDescendants": [ 25 "English", 26 [ 27 { 28 "CountryOfLanguage": [ 29 "UnitedStates" 30 ] 31 } 32 ] 33 ] 34 }, 35 { 36 "PersonObserverName": [ 37 "accomplished_peafowl" 38 ] 39 }, 40 { 41 "ProcedureReported": [ 42 "CTAbdomen" 43 ] 44 }, 45 { 46 "ImageLibrary": [ 47 [ 48 { 49 "ImageLibraryGroup": [ 50 [ 51 { 52 "": [ 53 "1.2.840.10008.5.1.4.1.1.2", 54 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762", 55 [ 56 { 57 "Modality": [

58 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 33

1 "ComputedTomography" 2 ] 3 }, 4 { 5 "StudyDate": [ 6 "19870620" 7 ] 8 }, 9 { 10 "StudyTime": [ 11 "135823" 12 ] 13 } 14 ] 15 ] 16 } 17 ] 18 ] 19 } 20 ] 21 ] 22 }, 23 { 24 "ImagingMeasurements": [ 25 [ 26 { 27 "MeasurementGroup": [ 28 [ 29 { 30 "TrackingIdentifier": [ 31 "5b6eb4301d3175942d29985a3d0fbb00" 32 ] 33 }, 34 { 35 "TrackingUniqueIdentifier": [ 36 "1.3.6.1.4.1.5962.1.1.0.0.0.1535644357.22655.1" 37 ] 38 }, 39 { 40 "FindingSite": [ 41 "Kidney" 42 ] 43 }, 44 { 45 "Length": [ 46 "66.43856134", 47 "mm", 48 [ 49 { 50 "": [ 51 "POLYLINE", 52 [ 53 172.83535766601562, 54 270.0640869140625, 55 133.79888916015625, 56 343.0453186035156 57 ], 58 [ 59 { 60 "": [

61 - Draft - 59 Sup nnn - JSON Representation of DICOM Structured Reports Page 34

1 "1.2.840.10008.5.1.4.1.1.2", 2 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 3 ] 4 } 5 ] 6 ] 7 } 8 ] 9 ] 10 } 11 ] 12 ] 13 } 14 ] 15 ] 16 } 17 ] 18 ] 19 } 20 ]

21 This is the entire JSON File consisting of the DICOM top level Data Set and the Content Item Tree, such as might be produced after 22 merging the AI tool output with the DICOM composite context required to encode a valid SOP Instance:

23 [ 24 { 25 "SOPClassUID": { 26 "vr": "UI", 27 "Value": [ 28 "1.2.840.10008.5.1.4.1.1.88.22" 29 ] 30 }, 31 "SOPInstanceUID": { 32 "vr": "UI", 33 "Value": [ 34 "1.3.6.1.4.1.5962.1.1.0.0.0.1535642467.22502.1" 35 ] 36 }, 37 "StudyDate": { 38 "vr": "DA", 39 "Value": [ 40 "19870620" 41 ] 42 }, 43 "SeriesDate": { 44 "vr": "DA" 45 }, 46 "ContentDate": { 47 "vr": "DA", 48 "Value": [ 49 "20171126" 50 ] 51 }, 52 "StudyTime": { 53 "vr": "TM", 54 "Value": [ 55 "135823" 56 ] 57 },

58 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 35

1 "ContentTime": { 2 "vr": "TM", 3 "Value": [ 4 "224217" 5 ] 6 }, 7 "AccessionNumber": { 8 "vr": "SH" 9 }, 10 "Modality": { 11 "vr": "CS", 12 "Value": [ 13 "SR" 14 ] 15 }, 16 "Manufacturer": { 17 "vr": "LO", 18 "Value": [ 19 "PixelMed" 20 ] 21 }, 22 "InstitutionName": { 23 "vr": "LO" 24 }, 25 "ReferringPhysicianName": { 26 "vr": "PN" 27 }, 28 "StationName": { 29 "vr": "SH", 30 "Value": [ 31 "NONE" 32 ] 33 }, 34 "StudyDescription": { 35 "vr": "LO", 36 "Value": [ 37 "Renal" 38 ] 39 }, 40 "SeriesDescription": { 41 "vr": "LO", 42 "Value": [ 43 "Crowds Cure Cancer Annotation as Measurement Report" 44 ] 45 }, 46 "ManufacturerModelName": { 47 "vr": "LO", 48 "Value": [ 49 "XSLT from annotations_expanded.csv" 50 ] 51 }, 52 "ReferencedPerformedProcedureStepSequence": { 53 "vr": "SQ" 54 }, 55 "PatientName": { 56 "vr": "PN", 57 "Value": [ 58 { 59 "Alphabetic": "TCGA-BP-4343" 60 }

61 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 36

1 ] 2 }, 3 "PatientID": { 4 "vr": "LO", 5 "Value": [ 6 "TCGA-BP-4343" 7 ] 8 }, 9 "PatientBirthDate": { 10 "vr": "DA" 11 }, 12 "PatientSex": { 13 "vr": "CS" 14 }, 15 "DeviceSerialNumber": { 16 "vr": "LO", 17 "Value": [ 18 "9723613413261" 19 ] 20 }, 21 "SoftwareVersions": { 22 "vr": "LO", 23 "Value": [ 24 "0.1" 25 ] 26 }, 27 "StudyInstanceUID": { 28 "vr": "UI", 29 "Value": [ 30 "1.3.6.1.4.1.14519.5.2.1.9203.4004.100545948082587796019777812429" 31 ] 32 }, 33 "SeriesInstanceUID": { 34 "vr": "UI", 35 "Value": [ 36 "1.3.6.1.4.1.5962.1.3.0.0.1535642467.22502.1" 37 ] 38 }, 39 "StudyID": { 40 "vr": "SH" 41 }, 42 "SeriesNumber": { 43 "vr": "IS", 44 "Value": [ 45 4578 46 ] 47 }, 48 "InstanceNumber": { 49 "vr": "IS", 50 "Value": [ 51 1 52 ] 53 }, 54 "AuthorObserverSequence": { 55 "vr": "SQ", 56 "Value": [ 57 { 58 "InstitutionName": { 59 "vr": "LO" 60 },

61 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 37

1 "InstitutionCodeSequence": { 2 "vr": "SQ" 3 }, 4 "PersonIdentificationCodeSequence": { 5 "vr": "SQ" 6 }, 7 "ObserverType": { 8 "vr": "CS", 9 "Value": [ 10 "PSN" 11 ] 12 }, 13 "PersonName": { 14 "vr": "PN", 15 "Value": [ 16 { 17 "Alphabetic": "accomplished_peafowl" 18 } 19 ] 20 } 21 } 22 ] 23 }, 24 "PerformedProcedureCodeSequence": { 25 "vr": "SQ" 26 }, 27 "CurrentRequestedProcedureEvidenceSequence": { 28 "vr": "SQ", 29 "Value": [ 30 { 31 "ReferencedSeriesSequence": { 32 "vr": "SQ", 33 "Value": [ 34 { 35 "ReferencedSOPSequence": { 36 "vr": "SQ", 37 "Value": [ 38 { 39 "ReferencedSOPClassUID": { 40 "vr": "UI", 41 "Value": [ 42 "1.2.840.10008.5.1.4.1.1.2" 43 ] 44 }, 45 "ReferencedSOPInstanceUID": { 46 "vr": "UI", 47 "Value": [ 48 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 49 ] 50 } 51 } 52 ] 53 }, 54 "SeriesInstanceUID": { 55 "vr": "UI", 56 "Value": [ 57 "1.3.6.1.4.1.14519.5.2.1.9203.4004.178060436087844836988805217813" 58 ] 59 } 60 }

61 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 38

1 ] 2 }, 3 "StudyInstanceUID": { 4 "vr": "UI", 5 "Value": [ 6 "1.3.6.1.4.1.14519.5.2.1.9203.4004.100545948082587796019777812429" 7 ] 8 } 9 } 10 ] 11 }, 12 "CompletionFlag": { 13 "vr": "CS", 14 "Value": [ 15 "COMPLETE" 16 ] 17 }, 18 "VerificationFlag": { 19 "vr": "CS", 20 "Value": [ 21 "UNVERIFIED" 22 ] 23 }, 24 "ImagingMeasurementReport": [ 25 { 26 "@tmr": "DCMR", 27 "@tid": "1500" 28 }, 29 [ 30 { 31 "LanguageOfContentItemAndDescendants": [ 32 "English", 33 [ 34 { 35 "CountryOfLanguage": [ 36 "UnitedStates" 37 ] 38 } 39 ] 40 ] 41 }, 42 { 43 "PersonObserverName": [ 44 "accomplished_peafowl" 45 ] 46 }, 47 { 48 "ProcedureReported": [ 49 "CTAbdomen" 50 ] 51 }, 52 { 53 "ImageLibrary": [ 54 [ 55 { 56 "ImageLibraryGroup": [ 57 [ 58 { 59 "": [ 60 "1.2.840.10008.5.1.4.1.1.2",

61 - Draft - 62 Sup nnn - JSON Representation of DICOM Structured Reports Page 39

1 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762", 2 [ 3 { 4 "Modality": [ 5 "ComputedTomography" 6 ] 7 }, 8 { 9 "StudyDate": [ 10 "19870620" 11 ] 12 }, 13 { 14 "StudyTime": [ 15 "135823" 16 ] 17 } 18 ] 19 ] 20 } 21 ] 22 ] 23 } 24 ] 25 ] 26 }, 27 { 28 "ImagingMeasurements": [ 29 [ 30 { 31 "MeasurementGroup": [ 32 [ 33 { 34 "TrackingIdentifier": [ 35 "5b6eb4301d3175942d29985a3d0fbb00" 36 ] 37 }, 38 { 39 "TrackingUniqueIdentifier": [ 40 "1.3.6.1.4.1.5962.1.1.0.0.0.1535644357.22655.1" 41 ] 42 }, 43 { 44 "FindingSite": [ 45 "Kidney" 46 ] 47 }, 48 { 49 "Length": [ 50 "66.43856134", 51 "mm", 52 [ 53 { 54 "": [ 55 "POLYLINE", 56 [ 57 172.83535766601562, 58 270.0640869140625, 59 133.79888916015625, 60 343.0453186035156

61 - Draft - 26 Sup nnn - JSON Representation of DICOM Structured Reports Page 40

1 ], 2 [ 3 { 4 "": [ 5 "1.2.840.10008.5.1.4.1.1.2", 6 "1.3.6.1.4.1.14519.5.2.1.9203.4004.268018422288818573226516023762" 7 ] 8 } 9 ] 10 ] 11 } 12 ] 13 ] 14 } 15 ] 16 ] 17 } 18 ] 19 ] 20 } 21 ] 22 ] 23 } 24 ]

25 - Draft - 20 Sup nnn - JSON Representation of DICOM Structured Reports Page 41

1 F DICOM JSON Model

2 Amend PS3.18 as follows (changes to existing text are bold and underlined for additions and struckthrough for removals):

3 F.1 Introduction to JavaScript Object Notation (JSON)

4 JSON is a text-based open standard, derived from JavaScript, for representing data structures and associated arrays. It is 5 language-independent, and primarily used for serializing and transmitting lightweight structured data over a network con- 6 nection. It is described in detail by the Internet Engineering Task Force (IETF) in [RFC4627], available at http://www.ietf.org/ 7 rfc/rfc4627.txt.

8 See Section B.2.1 “Introduction”.

9 The DICOM JSON Model complements the XML-based Native DICOM Model, by providing a lightweight representation of data returned 10 by DICOM web services. While this representation can be used to encode any type of DICOM Data Set it is expected to be used by 11 client applications, especially mobile clients, such as described in the QIDO-RS use cases (see Annex HHH “Transition from WADO 12 to RESTful Services (Informative)” in PS3.17).

13 F.2 DICOM JSON Model

14 Retired. See Section B.2.2 “DICOM JSON Encoding”.

15 F.3 Transformation with other DICOM Formats

16 Retired. See Section B.2.3 “Transformation to and from other DICOM Encodings”????.

17 F.4 DICOM JSON Model Example

18 Retired. See Section B.2.4 “DICOM JSON Encoding Example”????.

19 - Draft - 19 Sup nnn - JSON Representation of DICOM Structured Reports Page 42

1 A Data Exchange Models

2 Amend PS3.19 as follows (changes to existing text are bold and underlined for additions and struckthrough for removals):

3 A.1 Native DICOM Model

4 A.1.1 Usage

5 Retired. See Section A.2.1.1 “Usage”????.

6 A.1.2 Identification

7 Retired. See Section A.2.1.2 “Identification”????.

8 A.1.3 Support

9 Retired. See Section A.2.1.3 “Support”????.

10 A.1.4 Information Model

11 Retired. See Section A.2.1.4 “Information Model”????.

12 A.1.5 Description

13 Retired. See Section A.2.1.5 “Description”????.

14 A.1.6 Schema

15 Retired. See Section A.2.1.6 “Schema”????.

16 A.1.7 Examples

17 Retired. See Section A.2.1.7 “Examples”????.

18 - Draft - 31 Sup nnn - JSON Representation of DICOM Structured Reports Page 43

1 A Structured Reporting Templates

2 (Normative)

3 Amend PS3.16 as follows (changes to existing text are bold and underlined for additions and struckthrough for removals):

4 TID 1500 Measurement Report

65 Type: Extensible 87 Order: Non-Significant 109 Root: Yes

11 Table TID 1500. Measurement Report

12 NL Rel with Parent VT Concept Name VM Req Type Condition Value Set 13 Constraint 14 1 CONTAINER DCID 7021 "Measurement 1 M Root node 15 Report Document Titles" 16 2 > HAS CONCEPT INCLUDE DTID 1204 "Language of 1 MU 17 MOD Content Item and 18 Descendants" 19 3 > HAS OBS INCLUDE DTID 1001 "Observation 1 M 20 CONTEXT Context" 21 4 > HAS CONCEPT CODE EV (121058, DCM, "Procedure 1-n MU BCID 100 22 MOD reported") "Quantitative 23 Diagnostic 24 Imaging 25 Procedures" 26 5 > CONTAINS INCLUDE DTID 1600 "Image Library" 1 MU 27 6 > CONTAINS CONTAINER EV (126010, DCM, "Imaging 1 C IF row 10 28 Measurements") and 12 are 29 absent

30 - Draft -