2/22/2011 Apple Takes Huge Steps towards a Gr… Patently Apple

Home Archives Contact Subscribe February 03, 2011 Apple Takes Huge Steps towards a Graphics for iPad

Just yesterday the news broke about HP's new dubbed the Digital Sketch coming to market with a three- button and today you'll read all about Apple's latest research and development in this same area. Last July Apple was granted a patent for a graphics tablet and since then several other specialty pen and display patents have come to light. Today's patent report covers a new stylus-friendly touch display designed to work with a new kind of stylus that works with a conductive disk and unique pivoting head. While the proposed pen system could be used with standard iPhone apps, the patent clearly points to the system being more graphics oriented as it mentions working with brush tools and the like. And lastly, the new pen even has a gaming angle to it that could be fun.

Facts vs. Myth

Although Apple's CEO scoffed at a stylus being used on the iPhone during his 2007 keynote, the fact remains that Apple continues to this day to bankroll expensive engineers to the task of developing exotic , , displays and digital engines. I'd follow the facts more than the myth that Apple will never introduce a stylus or digital pen. Expanding the iPad's usage into the field of graphics only makes sense. And with future iterations of HP's Digital Sketch likely to use WebOS, Apple can't ignore this market any longer.

Patent Background

Apple provides us with a basic overview of why this invention is needed by discussing the need to advance touch screens to accept stylus inputs and what it will take to accomplish this. Apple states that touch screens could allow a user to perform various functions by touching the touch sensor panel using a finger, stylus or other object at a location dictated by a user interface (UI) being displayed by the display device.

A low resolution array of row and column elements is usually sufficient for finger detection. This is because the width of the patentlyapple.com/…/apple-takes-hug… 1/7 2/22/2011 Apple Takes Huge Steps towards a Gr… typical human finger is relatively large (roughly 10 mm) in relation to at least one dimension of a capacitive element. Therefore, if it is known in advance that the touch sensor panel will primarily be driven by finger input – then fewer capacitive elements could be built into the touch sensor panel. Additionally, the rows and columns could be separated at a greater distance.

However, when a stylus is subsequently employed on a touch sensor panel optimized for finger input, the stylus's small tip could often contact a region of the touch surface that is between adjacent capacitive elements (e.g., as between adjacent column sensors). Since the tip of the stylus is not sufficiently wide so as to guarantee the level of electrical interaction necessary for it to be sensed by at least one capacitive element, many situations exist where the touch sensor panel will not be able to identify an input even if the stylus is making contact with the touch surface.

Apple's Solution

In many conventional touch sensor panels, capacitive elements are arranged into a plurality of rows and columns so as to service an entire region of a touch surface. By analyzing the state of each column sensor after a particular row has been driven, a centroid could be calculated indicating the approximate position of a contacting entity upon the touch surface.

In many cases, however, the small tip of a stylus will contact a region of the touch surface that is between adjacent sensors (for example, as in certain low resolution touch sensor panels that are adapted for finger input). Without sufficient electrical interaction with at least one sensory element, a centroid may not be properly identified, and hence the input will not be recognized. Various embodiments of the present invention therefore ensure that contact from the stylus will be detected on a low resolution touch sensor panel irrespective of the location of the region of contact upon the touch surface.

Engineering a Unique Conductive Disk into a Stylus

In some embodiments, a metallic or otherwise conductive disk may be attached to one end of the stylus. The disk may be sized so as to guarantee sufficient electrical interaction with at least one sensory element of the touch sensor panel. In some embodiments, the disk may be attached to one end of the stylus via a pivotal connector. This increases the likelihood that the disk will remain flush with the touch surface as the user applies different combinations of directional forces to the stylus.

Introducing Force and/or Angle Sensors

In some embodiments, the stylus may be powered so as to provide a stimulus signal to the capacitive elements. In this manner, the capacitive elements do not need to be driven continuously within a host device. Optionally, one or more force and/or angle sensors disposed within the stylus can supply additional data to the touch panel. This additional data could be used for selecting various features in an application executing on the host device (e.g., selecting various colors, brushes, shading, line widths, etc.).

Next Gen Pen for Future iOS Devices

Apple's patent FIG. 1 illustrates an exemplary stylus 200 adapted for use with a host device 100 which includes a touch surface that is serviced by a plurality of capacitive elements (104) arranged into a plurality of rows and columns.

patentlyapple.com/…/apple-takes-hug… 2/7 2/22/2011 Apple Takes Huge Steps towards a Gr…

When the stylus makes contact with the touch surface, one or more capacitive elements undergo a change in capacitance that could be detected by charge amplifier circuitry. These sensors define a crude two-dimensional "patch" which represents the "image" of the touch provided by the stylus. From the shape and dimensions of the patch, a centroid could be calculated which represents an approximate center of the touch area. Once the centroid has been calculated, its position could then be transmitted to an application resident on the host device for input processing.

As shown by FIG. 1, the stylus includes a conductive disk with a diameter large enough to ensure sufficient electrical interaction with a minimum number of capacitive elements for the purposes of centroid calculation. In this manner, a centroid may be calculated irrespective of the position of the conductive disk upon the touch surface.

patentlyapple.com/…/apple-takes-hug… 3/7 2/22/2011 Apple Takes Huge Steps towards a Gr…

Replacement Stylus Tips

Apple's patent FIG. 2 is a diagram illustrating components of an exemplary stylus that includes a shaft, a replacement tip, and a conductive disk attached to a disk pivot that is connected to the replacement tip.

In one embodiment, the shaft of the stylus 200 has a length of approximately 130 millimeters (5.1 inches) and a diameter of approximately 8 millimeters (0.31 inches).

Conductive Materials and Grounding

In some embodiments, the shaft contains a conductive material such as a metal or a metal alloy (e.g., aluminum or copper). The conductive material in the shaft allows the user's body to extend the conductor upon contact with the shaft, thus facilitating current flow from the user's body to the conductive disk and providing a ground path for charge coupled onto the conductive disk from the touch sensor panel. In some embodiments, this allows for stronger signal detection at the touch sensor panel. A recent Apple patent was dedicated to the issue of quality grounding .

In other embodiments, the shaft 202 contains an insulating material such as plastic or glass. In some embodiments, the insulating material in the shaft serves to prevent electrical noise picked up by the user's body from being transmitted to the touch surface. This electrical noise could interfere with the input detection mechanism of the touch sensor panel.

In some embodiments, a detachable replacement tip may be attached to one end of the shaft and includes a disk pivot and a conductive disk.

The Disk Pivot

patentlyapple.com/…/apple-takes-hug… 4/7 2/22/2011 Apple Takes Huge Steps towards a Gr… Apple's patent FIGS. 4A to 4C are diagrams illustrating an exemplary disk pivot that increases the likelihood that the conductive disk will remain flush with the touch surface. As shown by the figure, the replacement tip 204 rotates about the disk pivot as the angle of application 400 changes from 400(1) to 400(2) and 400(3). In this manner, amount of charge is greatest at the electrodes situated closest to the center of the disk, thus ensuring proper centroid calculation.

In Apple's patent FIG. 3, we see a diagram illustrating how an exemplary stylus including a rigid tip could yield a non-uniform signal. FIG. 5 is a diagram illustrating an exemplary stylus including a conductive disk emanating a set of fringe fields.

Embedded Accelerometer & More

In some embodiments, the stylus could include one or more embedded accelerometers adapted to transmit positional information to the touch sensor panel. Positional information generated by the capacitive elements 300 may be synthesized with the accelerometer data by a processor in the host device in order to derive the precise region of contact upon the touch surface 102.

In some embodiments, stylus functionality may also include sensory modules including motion or pressure sensors and other similar devices. In certain embodiments, the stylus could include one or more squeeze (force) sensors, switches, buttons and/or other toggles adapted to allow a user to quickly select among various types of associated functionality (for example, selecting colors, brush sizes, shading, line width, eraser functionality, etc.).

Used for Gaming Too

According to Apple, Alternatively, the stylus contacting a touch surface may be adapted to navigate among a plurality of selections upon a display screen, thus functioning in a manner similar to that of a joystick.

patentlyapple.com/…/apple-takes-hug… 5/7 2/22/2011 Apple Takes Huge Steps towards a Gr…

And lastly, Apple's patent presents us with a stylishly designed old fashioned pen type of stylus which makes you think of an or should I say Apple's inkwell . That would fit in nicely with Apple's Ink Engine technology that we covered in one of our 2009 reports.

Apple credits John Elias, as the sole inventor of this patent application originally filed in 2008 or more than a year after the iPhone was released. Also see Apple's patent on the potential of adding a unique pivotal stand to a future iPad, iPhone and other devices.

Notice: Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments: Patently Apple reserves the right to post, dismiss or edit comments.

Also see our Tablet and Touch Display Archives for more subject-related information.

Update Feb 16, 2011: HTC's new Flyer Tablet provides stylus option. Also see Engadget's photo gallery of this feature.

Community Sites Covering our Original Report:

MacSurfer , Techmeme , The New York Times (Bit Blog), Apple Investor News, Apple Enthusiast, Google Reader, Technologizer, MacDailyNews , Cult of Mac , iPhone World Canada, MacPlus France, Mobile Whack , AppAdvice , TechnoUp Indonesia, TUAW , Telefonino Italy, Mac Life Germany, Media Bistro , Macmagazine Brazil, igeneration France, Know Your Mobile UK, ZDNet , Technaute (French, Canada), Le Blog iPhone France, Macgasm , PCWorld , ModMyi , GottaBeMobile , Fast Company , TechWave Japan, il PC Tablet Italico (Italy), Stuff TV , VIPad France, PC Mag , T3 , Phones Review , A5 Device Russia, Gadgets and Gizmos , PDA Club Poland, ITProPortal , Rant Rave , TFTS , EFY Times India, Korrespondent Russia, Glazberg (Patent Attorneys) Israel, Digital Arts UK, Tom's Guide France, The Consumerist , Digital & Midia Rio Brazil, MacLovers South America, and more.

patentlyapple.com/…/apple-takes-hug… 6/7 2/22/2011 Apple Takes Huge Steps towards a Gr…

Posted by Jack Purcher in Displays: LCD/Touch , Tech: Tablet | Permalink Tweet This!

Comments Excellent report and it's nice to see that y ou cover foreign sites in their native languages. Posted by: George R. | February 05, 2011 at 02:09 AM Whiney Appleinsider puts out a skimpy little report in January with no detail and now that this site puts out a detailed report that we could sink our teeth into and enjoy , the insider whines that they were first. First with what? They should be called AppleWhiners! Posted by: Ricky W. | February 05, 2011 at 02:04 AM A serious animation director, artist and toy designer named Steve Talowski over at Sketchbot tweets: "Would lov e to see this come to fruition!"

You'll also notice that his artist buddies Joenis, Oddernod, chipocaly pse, Atomos and NonToy also giv e thumbs up for this inv ention by tweeting: "Let's hope Apple is listening! Great to hear that Artists want this. So Apple, are y ou Listening? :) http://twitter.com/Stev eTalkowski? utm_campaign=newfollow20100823&utm_content=profile&utm_medium=email&utm_source=follow

Posted by: Jack Purcher | February 03, 2011 at 09:56 PM With HP trademarking DuoPad, TouchSlate and TouchCanvas recently, it's clear that next Wednesday's unv eiling of HP's true iPad competitors will take on Apple in earnst. HP will have Apple in a tail spin, especially with Jobs out of the picture. It may take a y ear to pinch Apple, but their days are numbered. HP will rule!

Posted by: H. Patterson. | February 03, 2011 at 02:53 PM Wouldn't Iv e's ... [take a fit at the thought of scratching the iPad's precious screen?] (I'm sure that Apple engineers discuss these matter with Iv e. If it launches, Tom, I'm sure Iv e's will sing its praises, as usual. JP.)

Posted by : Tom | February 03, 2011 at 02:06 PM Apple needs to show leadership by reinventing the stylus and graphics pens of yesteryear. Especially the stylus in general. I like touch but it's not the answer for every thing. Nice to see Apple hard at work to find the right solution worthy of their brand. Posted by: Michael Grey | February 03, 2011 at 01:28 PM Cool. Lets hope they put an improved from Newton into a future iPad. Posted by: veggiedude | February 03, 2011 at 01:24 PM What I don't understand is why Wacom has y et to release a bluetooth pen that supports pressure. This is a no brainer to do for iPad. The value of their more expensive display tablets is about to hit rock bottom. Now if Apple makes the density of the surface more refined, ev en better graphics art could be done. I am betting iPad3 is going to shake up digital art. look for Apple to file more patents on this in the future. Posted by : Andrew | February 03, 2011 at 12:04 PM

The comments to this entry are closed. Patently Apple

patentlyapple.com/…/apple-takes-hug… 7/7