COMS W4172 Case Study: 3D Windows/Desktops 2
Steven Feiner Department of Computer Science Columbia University New York, NY 10027
www.cs.columbia.edu/graphics/courses/csw4172
February 4, 2021
1
The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Active task is on stage (far wall) . User can . Move tasks to desired locations on room surfaces . Select a task to make it the active task . Study shows users . Prefer tasks on walls, not floor/ceiling . Remember near–far order better than left–right order
2
Feiner, COMS W4172, Spring 2021 The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Navigation . Designed to avoid confusion of unconstrained 3D travel . User can move only . forward or backward . one room at a time . always facing forward 3
The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Navigation . On-screen Only user icon is visible until mouse controls hovers over it . Keyboard accelerators
4
Feiner, COMS W4172, Spring 2021 The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Glancing to left reveals palette of apps and docs . Add to existing task or create new task . Designed to exploit spatial memory 5
The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Glancing to right reveals file browser
6
Feiner, COMS W4172, Spring 2021 The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Task windows organized into . Selected set . Loose stack . Can be moved anywhere on stage, scaling based on distance . On stands . Ordered stack . Inactive windows . On podium
7
The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
. Selection . Replace or add to selected set . Automated layout . Move selected window to loose stack or ordered stack
8
Feiner, COMS W4172, Spring 2021 The Task Gallery Microsoft https://www.microsoft.com/en-us/research/publication/the-task-gallery-a-3d-window-manager
1:38-3:36 https://www.youtube.com/watch?v=4uCt2criawQ
9
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Originally internal research testbed, 2003 . Then java.net open source project for Linux/Solaris
https://www.youtube.com/watch?v=JXv8VlpoK_g 2:10
10
Feiner, COMS W4172, Spring 2021 Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Avoids 3D navigation by supporting only rotation among a surrounding set of desktops
. Switching virtual desktops by panning
11
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Avoids 3D navigation by supporting only rotation among a surrounding set of desktops
. Switching virtual desktops by panning
12
Feiner, COMS W4172, Spring 2021 Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Avoids 3D navigation by supporting only rotation among a surrounding set of desktops
. Switching virtual desktops by panning
13
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Browsing a 3D jukebox
14
Feiner, COMS W4172, Spring 2021 Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Rotating a window
15
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Rotating a window
16
Feiner, COMS W4172, Spring 2021 Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Rotating a window
17
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Rotating a window to interact with info on back
18
Feiner, COMS W4172, Spring 2021 Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Adding a note on the back of a window
19
Project Looking Glass Sun https://web.archive.org/web/20130609081034/https://java.net/projects/lg3d
. Viewing windows at an angle with labeled edges
20
Feiner, COMS W4172, Spring 2021 BumpTop A. Agarawala and R. Balakrishnan www.bumptop.com
. Originally research project (2006), later product, then acquired by Google (2010), now open source . Piles . Selective use of physics . Pen-based interaction
22
BumpTop A. Agarawala and R. Balakrishnan www.bumptop.com
http://www.youtube.com/watch?v=M0ODskdEPnQ . Piles . Selective use of physics . Pen-based interaction
23
Feiner, COMS W4172, Spring 2021 SpaceTop J. Lee, A. Olwal, H. Ishii, C. Boulanger, CHI 2013
. Transparent display: luminous (OLED) vs. occlusive with true black (LCD) . Depth cameras track head and hands . Main window appears in front . When user raises hands from kbd, main window slides down to allow user to select another window from “virtual cabinet” behind it to take its place
http://www.leejinha.com/spacetop.html https://www.youtube.com/watch?v=b8MQxD0ozlc http://dx.doi.org/10.1145/2470654.2470680 https://vimeo.com/130447706
25
SpaceTop J. Lee, A. Olwal, H. Ishii, C. Boulanger, CHI 2013
. Transparent display: luminous (OLED) vs. occlusive with true black (LCD) . Depth cameras track head and hands . Main window appears in front . When user raises hands from kbd, main window slides down to allow user to select another window from “virtual cabinet” behind it to take its place
http://www.leejinha.com/spacetop.html https://www.youtube.com/watch?v=b8MQxD0ozlc http://dx.doi.org/10.1145/2470654.2470680 https://vimeo.com/130447706
26
Feiner, COMS W4172, Spring 2021 Personal Cockpit B. Ens, R. Finnegan, P. Irangi, CHI 2014
. Exploration of layout and interaction with arrays of windows in AR . Accounting for . Asymmetric interaction around the body . Transfer between body-fixed and world-fixed (1:14–1:31) . Limited FOV (2:08–2:50)
http://hci.cs.umanitoba.ca/publications/details/personal-cockpit https://www.youtube.com/watch?v=L0ZjmEP-c1E 27
Microsoft, Apple, Linux, Android
. Compositing window managers (e.g., Microsoft “Desktop Window Manager”) appear 2001–2007 . Each window is rendered to a separate off-screen buffer Flip 3D [Windows . Window manager 7, but not 8…] composites buffers to create final desktop, so user can 3D dock [OS X . View one window’s Leopard,…] pixels through another window (and process the pixels; e.g., to blur) . 3D possibilities . Texture-map 3D polygon(s) with a window’s texture Cover Flow . Transform window’s [was in iP{hone,od}, Os X] polygon(s) in 3D
30
Feiner, COMS W4172, Spring 2021 Full 3D vs. Desktop 3D
. Will there be an advantage for “3D windows” in a “full 3D” environment (large FOV, immersive,…)?
32
2D Windows in 3D Space Redux
. Industrial research reinvigorated by commercial AR/VR HWD dev kits and products
Microsoft HoloLens (composited image)
Meta 2
33
Feiner, COMS W4172, Spring 2021 2D Windows in 3D Space Redux
vSpatial https://www.vspatial.com/ 34
2D Windows in 3D Space Redux
Facebook Infinite Office https://www.youtube.com/watch?v=5_bVkbG1ZCo 35
Feiner, COMS W4172, Spring 2021 2D Windows in 3D Space Redux
Facebook Infinite Office https://www.youtube.com/watch?v=5_bVkbG1ZCo 36
2D Windows in 3D Space Redux
Facebook Infinite Office https://www.youtube.com/watch?v=5_bVkbG1ZCo 37
Feiner, COMS W4172, Spring 2021