Supplemental Information for: Creating augmented reality USDZ files to visualize 3D objects in the classroom on student phones Babak Sanii* Keck Science Department of Claremont McKenna, Pitzer and Scripps colleges. 925 N. Mills Ave, Claremont, CA, USA. Detailed steps for producing and distributing AR files ...................................................................2 Downloading Software ...............................................................................................................2 USDPython (needed for all three methods) ...........................................................................2 PyMol (needed for method 1) ................................................................................................2 MeshLab (needed for method 1)............................................................................................2 Blender (needed for method 2)..............................................................................................2 Octave (needed for method 3) ...............................................................................................2 Steps to make Augmented Reality Objects ................................................................................2 Method 1: Importing molecules via PyMol, e.g., from the protein database ........................3 Method 2: Composing scenes via Blender..............................................................................3 Method 3: Scripting mathematical representations via Octave.............................................4 All 3 methods: Making the USDZ file ......................................................................................4 Distributing USDZ files ............................................................................................................5 Brief Laboratory Materials..............................................................................................................6 Detailed steps for producing and distributing AR files Downloading Software USDPython (needed for all three methods) The last step in all the object-generation methods outlined in the manuscript is to convert an\ .OBJ file to a .USDZ file. There are several available ways to do this, including some online paid options, and Python libraries. Our current preferred method is to use Apple’s free USDPython libraries provided by Apple. They can be downloaded from: https://developer.apple.com/download/more/?=USDPython For reference we used version 0.61. Once decompressed note the location of the folder. PyMol (needed for method 1) Pymol can be downloaded for free at: https://pymol.org/ But after a trial period it will require a license. For educational purposes free licensing can be found here: https://pymol.org/edu/?q=educational MeshLab (needed for method 1) MeshLab can be downloaded for free from http://www.meshlab.net . Blender (needed for method 2) Blender can be downloaded for free from https://www.blender.org Octave (needed for method 3) Octave can be downloaded for free from: https://www.gnu.org/software/octave/download.html Steps to make Augmented Reality Objects Method 1: Importing molecules via PyMol, e.g., from the protein database 1. Start the program PyMol 2. From the “File” menu select “Get PDB” 3. Enter your desired PDB ID code (e.g., 1gfl) and click on “Download” 4. From the “File” menu select “Export Image As…” then “VRML 2” 5. Choose a location and enter a filename (e.g., “1gfl”) and press “save” 6. Start the program MeshLab (you may close or minimize PyMol) 7. From the “File” menu select “Import Mesh…” 8. Select the file you just downloaded (it should have a .wrl extension) and press “Open” 9. Note: for large files like proteins you will have to wait a moment for it to load. 10. From the “File” menu select “Export Mesh As…” 11. Give it a filename (e.g., “1gfl.wrl.obj”)and the pull-down menu select the format “Alias Wavefront Object (*.obj)”, note where the file will be saved and press “Save”. Click “Ok” on the panel of saving options. 12. Follow the steps below titled “Making the USDZ file” Method 2: Composing scenes via Blender 1. Start the program Blender 2. Compose your scene (there are many tutorials here: https://www.blender.org/support/tutorials/). Alternatively, open up a scene like the one included in the supplementary information by going to the “File” menu and selecting “Open…” and selecting the file (e.g., “nucleus_size.blend”). 3. In the “File” menu select “Export” and then “Wavefront (.obj)” 4. Choose a location (note where the file is) and give it a filename (e.g., “nucleus_size.obj”) and click the small “Export OBJ” button in the top-right of the window. 5. Follow the steps below titled “Making the USDZ file” Method 3: Scripting mathematical representations via Octave 1. Start the program Octave 2. Generate your 3D object (this will require scripting). Alternatively you can run the script we included in the supplemental information (makeorbital.m), which is an amalgam of scripts written by Peter van Alem to generate electron orbitals, and Anders Sandberg to generate a .obj file. Run the script by being in the same directory as the script (via the “cd” command) and typing the name of the script (e.g., “makeorbital”). 3. Follow the steps below titled “Making the USDZ file” All 3 methods: Making the USDZ file 1. On a mac, go to the folder of the USDPython tool you downloaded from Apple. Double-click on “USD.command” and give it permission to run (you can also right- click on it and select “open”). It will open a terminal window you can type in. You may close or minimize MeshLab. 2. In this terminal type “usdzconvert “ but do not press enter. 3. From the Finder (i.e. the File folders) find the .obj file you generated earlier, and drag that file into the Terminal window. This should add the path to the file to your command. 4. Back to the Terminal, press the space bar and type your desired name of a file, adding a “.usdz” extension. (e.g., 1gfl.usdz). 5. Add a space bar and add the following text “-iOS12 -diffuseColor 1,1,1” to make the resulting file compatible with more phones, and give the object a white color. 6. Your total command should look something like this (with your filenames/paths): usdzconvert /Users/babaksanii/Documents/1gfl.obj 1gfl.usdz -iOS12 - diffuseColor 1,1,1 Press enter. 7. In the terminal type “open .” to produce a Finder window where the usdz file you created is saved. You can now open it in Preview to confirm it is a 3D object. 8. Email/Airdrop/Message the file to someone with an iPhone with a recent operating system (iOS 12). 9. On the phone, open the file and view it as a finger-manipulatable object. Press the AR button on the top and slowly pan the camera of your phone over the edge of a table until the AR object appears. Distributing USDZ files Distributing the files to students can be achieved multiple ways. Per the instructions above they can be sent directly (e.g., via email). Alternatively they may be hosted on a website. However, this may require whoever is hosting your website to add a line for the file to be recognized. There are detailed instructions here: https://webkit.org/blog/8421/viewing-augmented-reality-assets-in-safari-for-ios/ The configuration for a common web serving system, Apache, would be: AddType model/vnd.usdz+zip usdz # All files ending in .usdz served as USD. 1 The HTML code on the webpage itself should include rel=”ar” as follows: <a rel="ar" href="model.usdz"> <img src="model-preview.jpg">. </a> 1 Brief Laboratory Materials Prelab: o Download and install PyMol: https://pymol.org/2/ . Also register for the PyMol educational license: https://pymol.org/edu/?q=educational/ . When you run pymol it will prompt you to find the license file o Download and install MeshLab: http://www.meshlab.net/#download o Ensure you can successfully start both programs o If you can, bring a 3-button mouse. If not be sure to learn how to right-click with your computer. If your computer does not have a USB-A (the "standard" USB port) and you have the appropriate adapter please bring it. Learning goal: passing familiarity with tools to visualize complex biomolecules (and perhaps connect their structures to their functions). In lab you will have three tasks in the Pymol program: 1. Build a simple molecule 2. View a molecular simulation made by advanced lab students [omitted] 3. Study a DNA-recognition protein from the Protein Data Base Much of this will be learning the software by experimentation, and can be frustrating. Instructors often won't have "answers" and the software is complex enough that there are usually several ways to achieve any task. It's ok to search online for guidance, and if you ever want to "start fresh" go to the menu File > Reinitialize > Everything. Another useful menu command is Display > Zoom > All, which will let you see everything at once. 1. Build and save a molecule o Start PyMol o Top-right quadrant: click on the "Builder" button o You can choose any of the three types (chemical, protein, nucleic acid) and experiment as you like, but I recommend following this tutorial first: https://pymolwiki.org/index.php/Builder . Note that on many computers you may select a point by right-clicking on that part of the molecule (on some computers that corresponds to clicking while holding down "control."). o Once you've made a molecule of your liking, explore it from all sides in Pymol. The bottom-right gives you a shortcut for which mouse buttons with which keys control what aspects of the viewing, but experiment with