DOCSLIB.ORG

Smart Bookmarks: Automatic Retroactive Macro Recording on the Web

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Smart Bookmarks: Automatic Retroactive Macro Recording on the Web Smart Bookmarks: Automatic Retroactive Macro Recording on the Web by Darris Hupp Submitted to the Department of Electrical Engineering and Computer Science in Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science at the Massachusetts Institute of Technology May 11, 2007 © 2007 Darris Hupp. All rights reserved. The author hereby grants to M.I.T. permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole and in part in any medium now known or hereafter created. A utho r ...................................... ...... Department of Electrical Engineering and Computer Science May 11, 2007 Certified by ................................. Robert C. Miller Associate Professor 'lesis Supervisor Accepted by............. Arthfr-Gmith Professor of Electrical Engineering Chairman, Department Committee on Graduate Theses MASSACHUSETTS INS'r E OF TECHNOLOGY OCT 0 3 2007 BARKER LIBRARIES 2 Smart Bookmarks: Automatic Retroactive Macro Recording on the Web by Darris Hupp Submitted to the Department of Electrical Engineering and Computer Science May 11, 2007 In Partial Fulfillment of the Requirements for the Degree of Master of Engineering in Electrical Engineering and Computer Science ABSTRACT New technology has made the Web more dynamic and personalized, while at the same time interaction with the Web has become more complicated and involved. This thesis presents Smart Bookmarks, a web automation system that allows users to automate complex tasks or easily return to otherwise hard-to-reach dynamic web pages by creating smart bookmarks. A smart bookmark consists of an automatically generated script of recorded browsing commands that returns to a particular web page or web application state. Smart bookmarks can be created retroactively, meaning that the user does not need to explicitly initiate recording before performing a task, but can instead request a bookmark after visiting the destination page; the appropriate sequence of commands need to return to a page is selected automatically from a history of the user's browsing interactions. Smart Bookmarks provides a rich, visual representation of recorded bookmarks in order to clearly illustrate the actions that a bookmark performs, and includes textual descriptions, screenshots, and animated previews of each command. Finally, the system allows users to easily and intuitively edit bookmarks after they have been created, and to share smart bookmarks with other users. Thesis Supervisor: Robert C. Miller Title: Associate Professor 3 4 Acknowledgements I would like to thank Rob Miller for his invaluable advice and support. He was a constant source of encouragement and inspiration throughout this process, and I am grateful to have had him as my research advisor. I would also like to thank all the other members of the User Interface Design group for their technical advice and valuable feedback, and especially Kevin Su, Vikki Chou, and Greg Little for their insight and support. Finally, I would especially like to thank my parents, Russ and Regina, and my sister, Chelsie. They have been a constant source of motivation and support throughout this project and my entire MIT career. None of this would have been possible without them and their unwavering encouragement and belief in my abilities. 5 6 Contents 1 Introduction......................................................................................................................14 2 Related W ork....................................................................................................................18 2.1 W eb Autom ation and Custom ization ..................................................................... 18 2.2 W eb-based M acro Recording ................................................................................. 19 2.2.1 Proactive vs. Retroactive Recording ................................................................. 19 2.2.2 Architectures for Recording User Actions........................................................ 20 2.2.3 Representing Recorded Com mands ................................................................. 21 2.2.4 Variable and Private Com mands ..................................................................... 22 2.3 Other Web-based Progra mming-by-Demonstration Systems ................................. 23 2.4 Visual M acro Recording ......................................................................................... 23 3 Usage Scenarios ................................................................................................................ 25 3.1 Accessing a Hard-to-Reach Bank Portfolio Page ..................................................... 25 3.2 Searching for a Flight on a Travel W ebsite............................................................... 27 3.3 Sharing a Custom Laptop Configuration ................................................................. 28 4 User Interface Design.................................................................................................... 30 4.1 Overview of the Layout .......................................................................................... 30 4.2 Creating a Bookmark............................................................................................... 32 4.3 Graphical Representation of a Bookmark ............................................................... 33 4.4 Replaying a Bookmark............................................................................................. 35 4.5 Editing a Bookm ark ................................................................................................. 37 4.5.1 M anually Editing a Command .......................................................................... 37 4.5.2 Copying, M oving, and Deleting Com mands ...................................................... 38 4.5.3 Live Edit ............................................................................................................... 40 4.6 Param eterizing a Bookmark ................................................................................... 41 4.7 Graphical Browsing History ..................................................................................... 42 4.8 Sharing Bookm arks ................................................................................................. 44 5 Im plem entation ................................................................................................................ 45 7 5.1 Overview of the Design .......................................................................................... 45 5.2 Internal and External Representation of Bookmarks .............................................. 48 5.2.1 Internal Representation ................................................................................... 48 5.2.2 External Representation....................................................................................49 5.3 Recording the User's Browsing Actions ................................................................... 50 5.3.1 Recording Com mands ..................................................................................... 50 5.3.2 Recording W eb Page DOM s ............................................................................ 54 5.3.3 Recording Im ages............................................................................................. 54 5.4 Autom atically Generating a Bookmark .................................................................... 56 5.4.1 Determ ining W hether a W eb Page is Bookmarkable ........................................ 59 5.4.2 Removing Unnecessary Commands from a Generated Bookmark .................... 60 5.5 Playing a Bookmark................................................................................................. 61 6 Discussion of Challenges............................................................................................... 63 6.1 Dealing with Side Effects ........................................................................................ 63 6.1.1 Side Effect Detection User Interface..................................................................64 6.1.2 Side Effect Detection Im plementation............................................................. 66 6.2 Security and Privacy ............................................................................................... 68 7 Evaluation ......................................................................................................................... 70 7.1 Autom atic Bookmark Generation ............................................................................. 70 7.2 Robustness of Bookmarks ........................................................................................ 72 7.3 W eb Page Com parison Algorithm ........................................................................... 73 8 Conclusion ........................................................................................................................ 77 8.1 Future W ork................................................................................................................78 8 List of Figures Figure 1.1 A smart bookmark script that accesses my MIT Webmail inbox ............................ 14 Figure 1.2 Smart Bookmarks is implemented as a sidebar in the Firefox browser..................16 Figure 1.3 An example bookmark displayed in the Smart Bookmarks sidebar........................16 Figure
Load more

Recommended publications
  • What Makes a Webpage the Most Effective, Attractive and Useful Business Tool?
    ISSN 2394-7314 International Journal of Novel Research in Computer Science and Software Engineering Vol. 3, Issue 1, pp: (62-81), Month: January-April 2016, Available at: www.noveltyjournals.com What Makes A Webpage the Most Effective, Attractive and Useful Business Tool? Zobair Ullah Sam Higginbottom Institute of Agriculture, Technology & Sciences, Allahabad, India Abstract: This paper is intended to describe and discuss the major developments of a website design and internet that makes a website much more attractive, useful, an effective business tool and much more powerful than ever before. The paper is basically designed and prepared for the aspiring web designers and developers with a need to understand the HTML as well as website design in enough detail. Keywords: SGML, HTML, CSS, Javascript, JQuery, XML, PHP, SQL, ASP, ASP. Net, AJAX, JQuery Mobile, App ML, Angular JS, Json, HTML Graphics, Open source, web browser, web server, homepage, website, web pages, URL, domain name, hyperlink, hypertext, www, internet, intranet, HTTP, SSL, TLS, firewall, browser plug-ins, spyware and adware, Active X, DNS, IP address, dynamic IP address, static IP address, Flash, Javascript, Java, secure site, caching, proxy server, the top level domains, search engine, world wide web accessibility, HTML templates, blog, RSS, Gopher, MME types, phishing, bookmarks and favourites. 1. INTRODUCTION The HTML is a simple mark up language used to create hypertext documents that are platform independent. Here, platform independent means HTML documents work in the same way on different platforms and browsers. Historically, HTML documents are SGML documents with generic semantics that are appropriate for representing information from a wide range of domains.
    [Show full text]
  • Distributed Application Architecture
    Mobile Agents as a Distributed Application Architecture Remco Slotboom Master thesis for completion of a degree in Business Informatics at the Erasmus University Rotterdam. Supervision by Dr. Ir. Jan van den Berg and Drs. Mark Polman. Supported by Cambridge Technology Partners. Copyright statement Copyright © 2000 Remco Slotboom. All rights reserved. This document is provided “as is”, without warranty of any kind. Company and product names mentioned in this document may be claimed as trademarks by their respective companies. Contact information Remco Slotboom Ellemare 193 3085 JR Rotterdam, The Netherlands. Email: [email protected] Student number: 126801 Table of Contents Preface................................................................................VII How this thesis came to be...................................................... VII A word of thanks.................................................................... VIII CHAPTER 1 Introduction............................................................1 1.1 Executive summary .................................................... 1 1.2 Motivation.................................................................. 2 1.3 Goals......................................................................... 3 1.4 Chapter outline........................................................... 3 CHAPTER 2 Agents.....................................................................5 2.1 Agent according to the dictionary................................. 5 2.2 Why agents are important ..........................................
    [Show full text]
  • Automating Web Navigation with the Webvcr
    Automating Web Navigation with the WebVCR Vinod Anupam, Juliana Freire, Bharat Kumar, Daniel Lieuwen Bell Laboratories, 600 Mountain Ave., Murray Hill, NJ 07974, USA anupam,juliana,bharat,lieuwen ¡ @research.bell-labs.com Abstract Recent developments in Web technology such as the inclusion of scripting languages, frames, and the growth of dynamic content, have made the process of retrieving Web con- tent more complicated, and sometimes tedious. For example, Web browsers do not provide a method for a user to bookmark a frame-based Web site once the user navigates within the initial frameset. Also, some sites, such as travel sites and online classifieds, require users to go through a sequence of steps and fill out a sequence of forms in order to access their data. Using the bookmark facilities implemented in all popular browsers, often it is not possible to create a shortcut to access such data, and these steps must be manually repeated every time the data is needed. However, hard-to-reach pages are often the best candidates for a shortcut, because significantly more effort is required to reach them than to reach a standard page with a well-defined URL. The WebVCR system addresses this problem by letting users record and replay a series of browsing steps in smart bookmarks— shortcuts to Web content that require multiple steps to be retrieved. It provides a VCR-style interface to transparently record and replay users’ actions. Creating and updating smart bookmarks is a simple process involving only the usual browsing actions and requiring no programming by the user. In addition to sav- ing users time by providing shortcuts to hard-to-reach Web content, smart bookmarks can be used as building blocks for many interesting Web applications and new e-commerce services.
    [Show full text]
  • Sharing and Reusing Web Activity with Actionshot
    Here’s What I Did: Sharing and Reusing Web Activity with ActionShot Ian Li Jeffrey Nichols, Tessa Lau, Human Computer Interaction Institute Clemens Drews, Allen Cypher Carnegie Mellon University IBM Research – Almaden 5000 Forbes Avenue 650 Harry Road Pittsburgh, PA 15217 San Jose, CA 95120 [email protected] {jwnichols,tessalau,cdrews,acypher}@us.ibm.com ABSTRACT users to share bookmarks; Digg and Reddit allow users to ActionShot is an integrated web browser tool that creates a share interesting web pages that they found. However, fine-grained history of users’ browsing activities by conti- these web sites only allow people to share the URLs of nually recording their browsing actions at the level of inte- individual pages. If people want to share what they did on a ractions, such as button clicks and entries into form fields. web site, they have to write it down manually, which can ActionShot provides interfaces to facilitate browsing and be so tedious that they forego sharing the information. searching through this history, sharing portions of the his- Social scripting services such as CoScripter [8] allow users tory through established social networking tools such as to record and share interactions with websites, but these Facebook, and creating scripts that can be used to repeat tools require forethought and planning to enable recording previous interactions at a later time. ActionShot can also at the right time to capture a reusable script. Moreover, create short textual summaries for sequences of interac- CoScripter's one-to-all sharing model was found to deter tions. In this paper, we describe the ActionShot and our many users [8], who asked for finer grained control over initial explorations of the tool through field deployments with whom they shared their scripts.
    [Show full text]
  • Architectural Frameworks for Automated Content Adaptation to Mobile Devices Based on Open-Source Technologies
    Architectural Frameworks for Automated Content Adaptation to Mobile Devices Based on Open-Source Technologies Thesis submitted to the Faculty of Economics of the European University Viadrina (Frankfurt/Oder) in fulfillment of the requirements for the degree of Dr. rer. pol. Author: Bożena Jankowska First Advisor: Prof. Dr. Eberhard Stickel Second Advisor: Prof. Dr. Karl Kurbel Submitted: 03.11.2006 Thesis defense: 06.09.2007 Abstract The Web and enterprise information systems are gradually increasing their reach to a wide range of mobile devices. Although analysts hope for a breakthrough in the popularity of mobile solutions, field studies show that, except for Japan and South Korea, there is still a large gap between the technical capabilities of wireless devices/networks and the adoption of mobile services for business and private use. This paradox can be attributed to a high extent to low quality of existing mobile solutions and to their insufficient usability, represented particularly by two attributes: simplicity of use and content relevance. Additionally, network providers are afraid that mobile Internet could cannibalize their revenues from SMS and entertainment services and do not want to cooperate with service providers to improve the quality of services offered. Wireless applications depend on device-specific features such as input/output mechanisms, screen sizes, computing resources, and support for various multimedia formats and languages. This leads to the need for multi-source authoring - the creation of separate presentations for each device type or, at least, for each class of devices. Multi-source authoring is not a cost-efficient and feasible solution, especially for mobile services consisting of numerous pages.
    [Show full text]
  • GNOME User Guide GNOME User Guide SUSE Linux Enterprise Desktop 15 SP3
    SUSE Linux Enterprise Desktop 15 SP3 GNOME User Guide GNOME User Guide SUSE Linux Enterprise Desktop 15 SP3 This guide introduces the GNOME desktop of SUSE Linux Enterprise Desktop. Publication Date: September 24, 2021 SUSE LLC 1800 South Novell Place Provo, UT 84606 USA https://documentation.suse.com Copyright © 2006– 2021 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see https://www.suse.com/company/legal/ . All other third-party trademarks are the property of their respective owners. Trademark symbols (®, ™ etc.) denote trademarks of SUSE and its aliates. Asterisks (*) denote third-party trademarks. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its aliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents Preface xii 1 Available documentation xii 2 Improving the documentation xii 3 Documentation conventions xiii 4 Support xiv Support statement for SUSE Linux Enterprise Desktop xv • Technology previews xvi I INTRODUCTION 1 1 Getting started with the GNOME desktop 2 1.1 Logging in 2 Switching the session type
    [Show full text]
  • Save As Pdf, Use Your Keyboard Ctrl + P Keys (PC) Or Command P Keys (Mac) to Open the Printer Window
    CPD Diary - about and frequently asked questions Education & Professional Development team Your CPD Diary is in RCPCH ePortfolio on the Kaizen platform. RCPCH ePortfolio provides a life-long learning platform for paediatricians, from training and throughout a doctor’s working life. This page provides you with FAQs and user guidance on how CPD Diary works in ePortfolio. Last modified 4 June 2020 Post date 22 February 2019 Table of contents How do I access CPD Diary in RCPCH ePortfolio? How do I create, edit and print activities and reports? Why are my CPD entries all 'shared' and CPD documents all 'private' / Who can see my CPD records? Can I add my CPD activities retrospectively? Why do all my newly added activities look like they took place on the same day? Does RCPCH ePortfolio have a web app, or offline access? As a trainee due to CCT, what should I do to join the CPD Diary? I don't have an ePortfolio account/RCPCH website account/RCPCH number. What should I do? Why did CPD Diary move into RCPCH ePortfolio? Where can I found more information about the privacy of my data? Which email address are you using for me in RCPCH ePortfolio? If I have questions or can't login, who can I contact? Downloads How do I access CPD Diary in RCPCH ePortfolio? You can log in using your RCPCH number (not GMC number) and RCPCH website password. You can find it in the Quick Links menu of this site - select CPD Diary or ePortfolio and follow the prompts to log in.
    [Show full text]
  • Desktop Paradise Had It with Human? Bored with Blue? Let Graham Morrison, Our Resident Aesthete, Guide You to Form and Function on the Linux Desktop
    Get the perfect desktop Desktop paradise Had it with Human? Bored with blue? Let Graham Morrison, our resident aesthete, guide you to form and function on the Linux desktop. e all spend a great deal of time at the Linux the applications that let you change everything in the first place. desktop. It’s our office, living room and games Some changes are purely for decoration – a background image or room rolled into one. And like those rooms, the colour scheme, for instance. But there are others that reflect your Wexperience is a working routine, such as lot more pleasant when application icons in the toolbar you’ve made your “We’re going to cover and the keyboard shortcuts for environment more the areas where small launching them, and even the comfortable and more tuned font sizes and window position. to the way you work. It’s the changes yield big results.” Over the next few pages, virtual equivalent to moving we’re going to cover the main the sofa in front of the telly, putting beer in the fridge and areas on the desktop where small changes yield big results, and lighting a few candles. cover the main desktops most of us enjoy using. We hope you’ll be It might sound strange, but your desktop is an expression of inspired enough to make a few changes yourself. who you are, and how we use computers. That’s why the internet is littered with people who post images of their desktop, even going to the trouble of designing their own icon sets and building Under the microscope: Xfce KDE It’s often overlooked, but this is Forget blue, change the colours a desktop environment you can around and banish all thoughts truly make your mark on.
    [Show full text]
  • Unanswered Questions and Policy Challenges
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by The University of Utah: J. Willard Marriott Digital Library Personalizing the Web Using Site Descriptions Vinod Anupam Yuri Breitbart Juliana Freire Bharat Kumar {anupam,yuri,juliana,bharat}@research.bell-labs.com Bell Laboratories, Lucent Technologies Murray Hill, NJ 07974 " A b s tra c t mation when signing up, and the sites can track the interests of the user. The information overload on the Web has created a In this paper we propose a new approach to person­ great need fo r efficient filtering mechanisms. Many sites alization that allows a user to create personalized pages (e.g., CNN and Quicken) address this problem by allow­ that contain information from any Web site. The user has ing a user to create personalized pages that contain only full control over which information is retrieved, and thus information that is o f interest to the user. We propose a she need not sign-up for any special service. In addition, new approach fo r personalization that improves on exist­ personalized pages may include information from multiple ing services in three significant ways: the user can create sites (e.g., one’s favorite news categories from her preferred personalized pages with information from any site (with­ news source, weather information for her city, and the traf­ out being restricted to sites that offer personalization); fic report for her afternoon commute). This service can personalized pages may contain information from multiple also be extended to customize change notifications based Web sites (e.g., a user can create a personalized page that on conditions that may span multiple sites.
    [Show full text]
  • Mozilla Services Documentation
    Mozilla Services Documentation Tarek Ziade Aug 13, 2021 Contents 1 How To.. 3 2 Services 13 3 Client Development 45 4 Miscellaneous 69 Index 73 i ii Mozilla Services Documentation Welcome to the Mozilla Services Documentation front page. This site contains technical information about the services and products provided by the Mozilla Services Team. Right now that means Firefox Sync. (There were additional services documented here in the past, but they’ve been reprecated). To contribute to this site, see About this Website. Contents 1 Mozilla Services Documentation 2 Contents CHAPTER 1 How To. 1.1 Run your own Sync-1.5 Server Mozilla does not provide any pre-packaged release of the Firefox Sync server. The easiest way to install a Sync Server is to checkout our repository and run a build in-place. Once this is done, Sync can be run behind any Web Server that supports the WSGI protocol. 1.1.1 Important Notes The sync service uses Firefox Accounts for user authentication, which is a separate service and is not covered by this guide. Note: By default, a server set up using this guide will defer authentication to the Mozilla-hosted accounts server at https://accounts.firefox.com. You can safely use the Mozilla-hosted Firefox Accounts server in combination with a self-hosted sync storage server. The authentication and encryption protocols are designed so that the account server does not know the user’s plaintext password, and therefore cannot access their stored sync data. Alternatively, you can also Run your own Firefox Accounts Server to control all aspects of the system.
    [Show full text]
  • GNOME User Guide Opensuse Leap 42.1 GNOME User Guide Opensuse Leap 42.1
    GNOME User Guide openSUSE Leap 42.1 GNOME User Guide openSUSE Leap 42.1 Introduces the GNOME desktop of openSUSE Leap. It guides you through using and configuring the desktop and helps you perform key tasks. It is intended mainly for end users who want to make efficient use of GNOME as their default desktop. Publication Date: November 05, 2018 SUSE LLC 10 Canal Park Drive Suite 200 Cambridge MA 02141 USA https://www.suse.com/documentation Copyright © 2006–2018 SUSE LLC and contributors. All rights reserved. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Docu- mentation License, Version 1.2 or (at your option) version 1.3; with the Invariant Section being this copyright notice and license. A copy of the license version 1.2 is included in the section entitled “GNU Free Documentation License”. For SUSE trademarks, see http://www.suse.com/company/legal/ . All other third party trademarks are the prop- erty of their respective owners. A trademark symbol (®, ™ etc.) denotes a SUSE or Novell trademark; an asterisk (*) denotes a third party trademark. All information found in this book has been compiled with utmost attention to detail. However, this does not guarantee complete accuracy. Neither SUSE LLC, its affiliates, the authors nor the translators shall be held liable for possible errors or the consequences thereof. Contents About This Guide xii I INTRODUCTION 1 1 Getting Started with the GNOME Desktop 2 1.1 Logging In 2 Switching the Session Type Before Logging In 3 • Assistive Tools
    [Show full text]
  • Clip, Connect, Clone: Combining Application Elements to Build Custom Interfaces for Information Access
    Clip, Connect, Clone: Combining Application Elements to Build Custom Interfaces for Information Access Jun Fujima 1∗ Aran Lunzer 1 Kasper Hornbæk 2 Yuzuru Tanaka 1 1Meme Media Laboratory 2Natural Sciences ICT Competence Centre Hokkaido University University of Copenhagen Sapporo 060-8628, Japan 2100 Copenhagen Ø, Denmark {fujima, aran, tanaka}@meme.hokudai.ac.jp [email protected] ABSTRACT The motivation for this work arises from three kinds of chal- Many applications provide a form-like interface for request- lenge that are commonly observed in the use of applications ing information: the user fills in some fields, submits the for information access. form, and the application presents corresponding results. Such a procedure becomes burdensome if (1) the user must First, it can be laborious for users to find the interface ele- submit many different requests, for example in pursuing a ments that are of interest. For example, the input fields for trial-and-error search, (2) results from one application are to a Web application may be on a page that also includes many be used as inputs for another, requiring the user to transfer other items; similarly, the page of results delivered by that them by hand, or (3) the user wants to compare results, but application may include details that the user does not need. only the results from one request can be seen at a time. We If the application is used repeatedly, even the effort of lo- describe how users can reduce this burden by creating custom cating the relevant input fields and the relevant results can interfaces using three mechanisms: clipping of input and re- become a burden.
    [Show full text]