Europaisches Patentamt (19) European Patent Office Office europeeneen des brevets EP 0 845 894 A2
(12) EUROPEAN PATENT APPLICATION
(43) Date of publication: (51) |nt CI H04M 3/50, H04M 7/00, 03.06.1998 Bulletin 1998/23 H04M 11/06
(21) Application number: 97308910.5
(22) Date of filing: 05.11.1997
(84) Designated Contracting States: • Root, Thomas Lyman AT BE CH DE DK ES Fl FR GB GR IE IT LI LU MC No. Andover, Massachusetts 01845 (US) NL PT SE • Schuleter, Jeffrey John Designated Extension States: Wakefield, Massachusetts 01880 (US) AL LT LV MK RO SI (74) Representative: Skone James, Robert Edmund (30) Priority: 05.11.1996 US 743793 GILL JENNINGS & EVERY Broadgate House (71) Applicant: BOSTON TECHNOLOGY INC. 7 Eldon Street Wakefield, Massachusetts 01880 (US) London EC2M 7LH (GB)
(72) Inventors: • Picard, Donald F. Medford, Massachusetts 02155 (US)
(54) A system for accessing multimedia mailboxes and messages over the internet and via telephone
(57) A unified messaging system that provides a speech to the telephone user. The system allows a per- multimedia mailbox. The system allows a subscriber to sonal computer user to obtain the data network access access scored multimedia messages, such as voicemail using an Internet browser. The browser is used to ac- messages, facsimile messages, combined voice and cess a home page of the system and get information facsimile messages and video messages, not only about the messages stored, and is used to download through a public switched telephone network using a tel- (get) and play the messages at the personal computer ephone but also over a data network, such as the Inter- via data streaming in the case of a voice or video mes- net or an intranet, using a personal computer. The sys- sages or view the messages in the case of text type tern provides voicemail access over the telephone net- messages, such as facsimile and e-mail. The user can work, indicating message number, etc. with the ability to also perform the other typical messaging functions over play messages to the telephone user as desired. For the data network connection that are provided for tele- text type messages, such as facsimile and e-mail, the phone access, such as viewing a message list, saving system converts the text into speech and plays the and deleting messages, group list administration and other administration tasks.
FIG. 1
a. LU
Printed by Jouve, 75001 PARIS (FR) 1 EP 0 845 894 A2 2
Description SUMMARY OF THE INVENTION
REFERENCE TO MICROFICHE APPENDIX It is an object of the present invention to provide a system that allows a subscriber to access stored mes- A microfiche appendix having 5 microfiche and 490 5 sages over not only a telephone network but also over frames is included herewith that includes source code a network such as the Internet or an intranet. in the C+7HTML languages. It is an additional object of the present invention to provide a system that unifies message storage allowing BACKGROUND OF THE INVENTION different types of messages or electronic communica- 10 tions such as voicemail, facsimile, e-mail and video mail Field of the invention to be stored on a single system in a single unified mul- timedia mailbox, and accessed via different pathways, The present invention is directed to a system for ac- such as via a telephone network or the Internet/Intranet. cessing stored messages over a network and, more par- It is an object of the present invention to provide a ticularly, is directed to a system for providing unified ac- is system that will allow multimedia messaging via a mul- cess to stored messages, such as multimedia mail mes- timedia mail box. sages, in a unified multimedia mailbox through multiple It is another object of the present invention to pro- access pathways such as over a telephone network us- vide a system that is easy to use and which uses access ing a telephone and over the Internet using a browser, tools that are familiar to telephone and Internet users. 20 It is a further object of the present invention to pro- Description of the Related Art vide a simple visual interface to a message storage sys- tem that simplifies the tasks associated with message Communication systems currently exist that allow access and administration. different types of messages, such as voice mail mes- It is also an object of the present invention to provide sages and facsimile messages, to be stored for later re- 25 a platform that allows services for a variety of message trieval by a subscriber to such systems. These types of types such as voice mail, video mail and facsimile mail, systems are described in U.S. patents 5,029,199; as well as other network services such as Internet and 5,193,110; 5,260,990; 5,263,080; 5,475,748; intranet services. 5,493,607; 5,524,139; 5,519,766 and 5,008,926, all in- It is another object of the present invention to pro- corporated by reference herein. These systems allow a 30 vide a system architecture that is easily scaleable, has caller or sender to leave a message, such as a voice a high availability and which provides a fast response. mail message, for a subscriber whenever the subscriber It is an object of the present invention to provide a is not available. When a voice mail message is to be standards based system that will support mailbox ac- retrieved the subscriber typically connects with the sys- cess to a multimedia mailbox using conventional web tem over a conventional telephone line via a telephone 35 browser software. call and plays the message by using the touchtones pro- It is another object of the present invention to pro- duced by the telephone to control playback, as well as vide a system in which the message service provider other functions. I n these systems the access by the sub- does not need to supply the user with any client appli- scriber is typically only through a telephone line connec- cation software. tion. Today, there is a need to allow access to such sys- 40 It is a further object of the present invention to pro- tems through other means such as the Internet or In- vide message waiting/urgent notifiers when new or ur- tranet. gent messages are deposited in the mailbox or the mes- Several different types of messaging systems, such sage status changes by a simultaneous different con- as voice mail and e-mail, are also available to users. nection into the mailbox such as when a mailbox is ac- Users of the variety of today's messaging systems typ- 45 cessed by computer and while the computer is logged ically have to use several different systems and/or ter- into the mailbox an access via a telephone interface de- minals to get their messages. A typical business user letes a message. may have several voice mailboxes, several e-mail mail- The above objects can be attained by a system that boxes, and perhaps some mailbox-like facsimile servic- allows a subscriber to access stored messages, such es. Each of these mailboxes requires separate opera- so50 as voicemail messages, facsimile messages, e-mail tions and different types of terminals (DTMF telephone messages and video messages, that are stored in a uni- for voice mail, personal computer (PC) for e-mail ac- fied multimedia mailbox not only through a public cess, facsimile machine/PC for facsimile messages). switched telephone network using a telephone but also The mailboxes have different names (addresses) and over a data network, such as the Internet or an intranet. cannot usually interwork. Notification mechanisms are 55 The system provides voicemail access over the tele- either non-existent, or tied to one of the mailboxes. What phone network, indicating message number, etc. with is needed is a mailbox system that integrates all of these the ability to play messages to the telephone user. For message types and access methods. text type messages, such as facsimile and e-mail, the
2 3 EP 0 845 894 A2 4 system converts the text into speech and plays the Integrated mailboxes have certain desirable and speech to the telephone user. The system allows a per- preferable characteristics. A fully integrated mailbox, in sonal computer user to obtain the data network access accordance with the present invention, includes the fol- using an Internet browser. The browser is used to ac- lowing major capabilities that are not present in a single- cess information about the messages stored and is used s media mailbox: to download and play the messages via data streaming in the case of a voice or video messages or view the a. The ability to deal with messages of different da- messages in the case of text type messages, such as ta/information types, or having multiple parts (mul- facsimile and e-mail. The user can also perform the oth- timedia mailbox). er typical messaging functions over the data network 10 b. A single inventory (message list), listing all mes- connection that are provided for telephone access, such sages of all data types, with the ability to control as saving and deleting messages, group list administra- presentation of the inventory (e.g., sort the invento- tion and other administration tasks. ry according to message type, priority, or time of de- These together with other objects and advantages posit, regardless of the type of message), with con- which will be subsequently apparent, reside in the de- 15 ceptually similar user interface actions for equiva- tails of construction and operation as more fully herein- lent operations on any message type, and with the after described and claimed, reference being had to the ability to randomly select messages for retrieval. accompanying drawings forming a part hereof, wherein c. Notification mechanism(s) which can be used to like numerals refer to like parts throughout. alert the user of the deposit of any type of message. 20 d. The ability to access the mailbox through a vari- BRIEF DESCRIPTION OF THE DRAWINGS ety of commonly-available mailbox access termi- nals (PC, DTMF phone, etc.), without special equip- Figure 1 illustrates a virtual unified system unified ment, and with, as far as practicable, logically the by the actions of a personal computer 60; same capabilities for all terminal types Figure 2 depicts a unified virtual system unified by 25 e. The ability to perform data type conversions au- director 80; tomatically, in support of transparent multi-terminal Figure 3 depicts a voice mail system 66 performing user access, or upon subscriber explicit request unification; f. The ability to receive and send messages to sub- Figure 4 depicts real integrated system 100; scribers of existing messaging systems, using a va- Figure 5 depicts the functional architecture of a sys- 30 riety of widely-implemented messaging protocols. tern according to the present invention; Figure 6 depicts a distributed architecture system Note that there are degrees of integration in today's 130 according to the present invention; single-media mailboxes, both with respect of allowed Figure 7 illustrates the flow of control during a re- message types and the access terminal types which can fresh operation; 35 be used. For instance, integrated facsimile/voice mail- Figures 8 and 9 depict message list and group list boxes are common today, and e-mail can be used to templates, respectively; transfer non-text information. Similarly, e-mail mailbox- Figure 1 0 illustrates the flow of control in a retrieval es cannot be accessed using telephones, and voice/fac- operation; and simile mailboxes cannot be accessed using a PC. Figure 11 depicts a message retrieval template. 40 Although it is possible to have a mailbox which is integrated with respect to multiple message types but DESCRIPTION OF THE PREFERRED which can only be accessed through a single type of ter- EMBODIMENTS minal (e.g., e-mail systems using MIME), a fully integrat- ed mailbox is preferably accessible from several types The present invention provides an integrated multi- 45 of terminals and pathways, to maximize the subscriber's media mailbox and unified messaging. The term "mail- ability to access his messages in various circumstanc- box" is used to mean an entity visible to the subscriber. es. The following terminal types are provided by the This is the entity the subscriber logs into and appears present invention: a. Conventional DTMF telephone to operate on when the subscriber performs mail-related handset; and b. Personal Computer (PC). operations. This subscriber-visible entity may not corre- so However, otherterminal types such as personal dig- spond directly with a single implementation entity, but ital assistants, cellular telephones, two way pagers, etc, may exist only through the cooperation of several dis- can be used. tinct messaging systems, each wich it's own message It should be emphasized that an integrated mailbox storage capability. To avoid confusion, the term "mail- subscriber using the present invention is able to dynam- box" is used to mean only the subscriber visible entity, 55 ically change the terminal used, from session to session. and, where necessary, the term "message endpoint" is The integrated messaging system (IMS) of the used to denote the implementation entity or entities present invention is preferably interfaced to external which underlie the integrated mailbox. systems. This allows the subscriber to exchange mes-
3 5 EP 0 845 894 A2 6 sages wich external subscribers and can be used to in- ceived by the subscriber, of an integrated mailbox, when tegrate several existing messaging accounts on differ- a personal computer 60 (PC), as illustrated in figure 1, ent systems so that the user accesses a single (virtual) is used for message endpoint access, there are several integrated mailbox. The following types of external sys- non-integrated messaging systems (MS) such as e-mail tems can be included: 5 system (EMS) 66 and voicemail (VMS) 68, and the in- tegration of the mailbox may not occur for non-PC ac- a. The Internet. cess terminals; in other words, special-purpose PC soft- b. Commercial subscription mail systems (usually ware "does" the integration of the mailboxes; the MSs X.400). need no ability to handle non-native data or communi- c. Private mail systems (e.g., MS Mail, cc:Mail). 10 cate with each other. One approach to this integration d. CFE voicemail systems and other foreign net- is to use a conventional browser with a local home page work-based voicemail systems (e.g., the subscrib- that includes linkto the various messaging systems. An- er's cellular phone voice mailbox). other simple DTI approach is to use separate TCP/IP connections 62 and 64 to each MS 66 and 68 on a single There are several ways, in accordance with the is dialup point-to-point protocol (PPP) connection to a present invention, in which integrated messaging can router 70 providing hardware dedicated to routing IP be realized. However, the preferred approaches are dis- packets over various physical hardware interfaces, as cussed below. shown in figure 1 . The integration of the mailbox can be real or virtual. DTI of this type is useful when no close integration "Real" mailbox integration means that the messages of 20 of the MSs is possible for political or administrative rea- all types are located in a single messaging system (MS), sons. and that subscriber and administrative control facilities In another approach to virtual mailbox integration, for messages and mailbox configuration parameters are a front-end director 80, as illustrated in figure 2 directly provided at a single user interface point and do not in- interfaces with the customer, thus avoiding changes in volve cooperation or interaction with any other MS. "Vir- 25 any of the integrated MSs 66 and 68. The director 80 tual" integrated mailboxes provide the same subscriber- communicates on the back end with the separate MSs visible functionality, and appear the same to the sub- 66 and 68 that need to be integrated. Two major variants scriber as a real integrated mailbox. However, in the vir- of this approach are possible: the director 80 simply tual integrated mailbox, the subscriber's messages are passes requests through in real-time, and thus stores stored in at least two different MSs, whose configuration 30 no messages itself; or, the MSs 66 and 68 forward mes- can be (but need not be) performed separately. The dif- sages to the director 80 when they are deposited, and ferent messaging systems cooperate to provide the the director 80 stores them until the subscriber logs in. complete functionality. The term "associated MS" is In the latter case, the director 80 effectively becomes a used to denote and MS that is in a special relationship full IMS with external MS interfaces. with another MS for the purposes of synthesizing a vir- 35 The director 80, if it does not store messages, ac- tual IMS, and the term "external MS" is used to denote cepts subscriber requests, interprets them, and then an MS which is not so closely associated, but whith still communicates with the various MSs as needed to either has an interface to the IMS. retrieve or send messages and inventories. The director The distinction between real and virtual integrated 80 in this embodiment is essentially a transport-level or mailboxes is invisible to the subscriber. Real messaging 40 application-level relay, with some firewall-like function- systems can comprise multiple subsystems, such as the ality for security. The back end network is typically a lo- preferred distributed system described herein, with the cal area network (LAN) or high speed wide area network "mailbox" spread across several pieces of hardware. (WAN), onto which subscriber requests are multiplexed. Both types of integration need interfaced to external However, to provide DTMF access, the front-end di- MSs, even if they are not part of a virtual IMS. The re- 45 rector 80 has the necessary voice, facsimile, data and lationship between the MSs that are being integrated in- voice/data telephone interfaces of a system such as that to a virtual IMS ("associated MSs") is much closer than described in U.S. Patent 5, 193, 110. In practice the voice that between MSs that just happen to interwork ("exter- user interface is a superset of the interface provided by nal MSs"). While an integrated mailbox system could be the VMS 68 alone. totally self-contained (allowing messaging only between so Pass-through integration is another approach its subscribers, like most voicemail systems today), it is where the front-end director 80 functionality is tightly in- preferable to be able to send and receive mail from other corporated into one of the MSs, such as VMS 68. The systems. Real mailbox integration is preferred and de- MS deals with the messages of its native type, but acts scribed in detail herein. as a real-time proxy for subscriber requests for other Several approaches are available to create a virtual 55 messages types. As discussed previously, the need for IMS and are described here: a. desktop integration, b. both voice and facsimile as well as PC access, the dif- front-end director, and c. pass-through integration. ficulty of interfacing in real-time to an external VMS, the Desktop integration (DTI) is the emulation, as per- difficulty of augmenting an EMS to handle voice, and the
4 7 EP 0 845 894 A2 8 need for VMS-like notification mechanisms results in a he logs into his mailbox, and provide the ability to select preference that the director be added to a VMS 68 rather messages for retrieval. In addition, some of the mes- than to an EMS 66 as illustrated in figure 3. sage envelope information can be presented in the in- The foregoing discussion indicates that the prefer- ventory. The amount of information presented in the in- able approaches to an integrated multimedia mailbox 5 ventory, and the format of presentation are determined with both DTMF and PC access are either a full, real IM largely by the human aspects of the access terminal; system, or enhancement of a VMS so that it can provide when the voice interface is used, the presentation is pass-though real-time access to other messaging sys- preferably limited to simple spoken message counts tems. As a result, there are two preferred system-level ("You have three new voice messages, one newfacsim- architectures: a. An enhanced VMS (i.e., the IMS) which 10 ile message, and two new E-mail messages and one provides all message storage and all user interfaces for new video message. One voice message is urgent."), all types of message. All other MSs interface to the IMS otherwise the subscriber may quickly get confused. For as external, non-integrated systems, b. An IMS which the same reason, complex inventory sorting, message provides permanent storage for voice, video, text, e-mail selection or folder capabilities are preferably not provid- and facsimile messages and exchanges other message es ed through the voice interfate, even though they can be types on demand with one or more closely associated if desired. However, a PC interface preferably shows systems such as an associated EMS (in addition to any much more information to the user without overloading interfaces to other external MSs). The IMS as all user the subscriber, and allows sophisticated operations interfaces and passes through user commamds related such as organizing messages into folders. to the associated EMS(s). The IMS exchanges deposit 20 Generation of integrated message inventories oc- notifications with the associated EMS(s). curs naturally on a real IMS. A virtual IMS needs to get Figure 4 illustrates the system/network architecture the individual inventories from the external systems for either the real or virtual integrated mailbox and inte- specified for the subscriber. This is considered below in grated message system (IMS). As illustrated, the sys- the discussion of interfaces to other MSs. tem architecture 1 00 allows access by a telephone 1 02 25 The voice interface of the present invention through a public switched telephone network (PSTN) presents a spoken message inventory which gives 104 to the integrated messaging system 106 either di- counts of messages per type, and additional salient in- rectly or through a modem 108. Access by a personal formation such as whether any are urgent. There are computer 109 can also be accomplished through the essentially two folders: new and saved messages, plus PSTN 1 04. Access by a personal computer 1 1 0 through 30 a virtual "wastebasket" which may be emptied (or not) the Internet 112 using a modem 114 is also provided. at the end of the session. Selection of a message to play Note the associated e-mail system (EMS) 115 is shown is predetermined by the system (play only voice and to depict the architecture of a virtual system. The IMS text-to-speech type messages), with some limited ad- 1 06 is also coupled to other systems 116, 119 and 1 20. ministrator controls (e.g., play new or saved messages The content of the IMS 106 will be discussed in detail 35 first) ; and the user cannot chose to select a specific with respect to figure 6. message, other than by skipping forwards or backwards The system 1 00 can also be provided with a multiple through the messages. integrations unit (MIU) subsystem (not shown) such as A PC interface according to the present invention described in U.S. Patent 5,260,990. The MIU can host provides an inventory much like the message list of e- small numbers of PC sessions. This is, however, not a 40 mail systems. Typically, it includes, for each message: large-scale architecture. type of message (voice, video, e-mail, facsimile), sub- The present invention has the ability to receive, ject (if any), sender, time of deposit, size of message send, and store messages of several data types includ- (bytes, pages, seconds, as appropriate), and status ing voice, facsimile, multipart (i.e., voice-annotated fac- (new/read, urgent, replied to, forwarded, etc.). simile), video, text and e-mail messages. The platform 45 In a PC interface the user clicks an inventory entry 1 32 (see figure 6) is designed to accommodate the mas- to select the message to be retrieved; the system re- sive amounts of storage required for video data. trieves the appropriate message and provides it to the All messages have certain information (the mes- user in the proper format. The PC can support folders, sage envelope) that goes along with them, such as so the subscriber can organize his messages. This can sender, date/time of deposit, length, etc. The informa- so be done locally at the desktop, by moving messages to tion varies with message type and, to some degree, with local file system directories or files. However, for the pre- the means by which the message was received. The en- ferred type of PC interface (e.g., using a standard web velope information is preferably stored with the messag- browser), the folders can be implemented by the IMS es carried along with the message if it is to be delivered 106. to an external system, and be presented to the subscrib- 55 Message headers preferably include important de- er. The IMS 106, as previously discussed, is able to tails from the message envelope for non-voice/facsimile present a single list or inventory, containing all messag- messages. The envelope of messages received via e- es of all types (sorted into types), to the subscriber when mail can have a lot more detail than these of voice/fac-
5 9 EP 0 845 894 A2 10 simile messages, and are also in text format. These type format for output over same other interface and is not envelope elements are parsed by the system 106, and preferred. spoken in the same way that sender mailbox number, For output to a conventional telephone, non-voice date/time, urgency and other envelope elements are messages of the text data type are converted to voice, handled (i.e., by concatenating pre-recorded prompt 5 or otherwise represented by voice. However, there are fragments). Preferably, the envelope data is stored in some structures (such as the MIME multipart/parallell the same way, regardless of message type, to output that may be too complicated, or not useful, or impractical the non-voice/facsimile envelope in spoken format. to output to a voice telephone. In such a situation, the One problem area is in identifying the message system should indicate to the subscriber the type of the sender to the subscriber. Ideally, the system should 10 message and other envelope information, and explain speak the sender's name, preferably in the sender's own that it cannot be retrieved from a DTMF telephone using voice. For messages received from a sender on the a standard voice message. same system, the system can access the sender's name However, a picture-telephone can receive image announcement (or speak the mailbox number if there is type mail messages. no name announcement). The same is true of messag- is Recording of a voice, facsimile or voice-annotated es from other systems connected through digital net- facsimile message over the voice interface is performed working using the Digital Messaging Network Version II in a conventional manner. To generate other data types (DMN II)™ system available from Boston Technology DTMF key entry is used to select from a menu of simple Inc. of Massachusetts. Digital Messaging Network Ver- parameterized pre-canned messages. For instance, a sion II (DMN II)™ conforms to the AMIS-D voice mes- 20 caller can send text messages containing a callback saging protocol, which passes the name announcement phone number to a subscriber's cellular handset or pag- of the sender along with the message, using X.400 and er. The present invention provides these capabilities for a real time DMN II protocol to retrieve remote name an- video, text, e-mail messages, as well as for other data nouncements during message addressing between types (facsimile back services are an example of similar systems. 25 usage). For non-voice messages received from other ven- The limitations of the DTMF keypad do restrict the dor's platforms (including X.400 and MIME/SMTP mes- ability to address the message to non-numeric address- sages from the Internet), the sender address informa- es. In the present invention, however, a group address tion is a character string, and there is no explicitly iden- containing non-numeric addresses (defined using the tified name announcement. Of course, an X.400 or 30 PC interface) can be specified through the DTMF inter- MIME voice body part can be included with any mes- face. The present invention also allows a reply with a sage, and so the sending system can send a name an- voice message to any received message from any send- nouncement. However, the receiving system will not ing address. The system 106 records the voice and de- know it is the name announcement and just play it as posits it in the local sender's mailbox, or converts it to part of the message body. An example of such an ad- 35 an appropriate MIME, AMIS-D or X.400 body part using dress is "[email protected]". The receiving system 106 DMN II or the functions of a client e-mail reader such as parses this string and speaks in the same way as a mail- MSExchange, cc:Mail or Netscape Internet Mail, and box number, character by character. Text-to-speech sends it to the original message sender's address, if ex- could also be used, but there are problems for today's ternal. text-to-speech technology in handling an infinite variety 40 Speech-to-text, or voice recognition, is also a of strange words (such as "abc"). In any case, it is often means to send text messages from a conventional tel- clearer to spell out the address when spoken. A pre- ephone and also a way of addressing non-numeric ad- ferred solution is to use the prompt voice fragment ap- dresses (by spelling them out). proach with additional prompt fragments for common Forwarding is the copying of a message and the de- address elements, to output (for example) the voice 45 posit of the copy in a different mailbox, or transmission prompts "b" "i" "I" "I" "at" "a" "b" "c" "dot" "com" for the to a third party. Forwarding of non-voice messages is address noted above. This does not require text-to- provided by the present invention through both the voice speech capability. interface and the PC interface because when the mes- The need to handle messages from the external sage does not need to be converted to voice, such as MSs 116-120 means that some voice messages will be so for facsimile messages, the message can be sent either received in encodings other than the native format of the to another mailbox, or to a subscriber-entered facsimile platform. Examples would be MIME messages contain- telephone number, without actually retrieving the mes- ing audio/wav body parts, or WAV files. The present in- sage. This approach is also used by the present inven- vention can output this data to the telephone line just as tion for other data types (video), as is the turnaround of easily as the native format using the freeware software 55 lies for alternate facsimile/voice use. When the destina- package SOX. An alternative is to convert everything to tion is not the same IMS the message is format convert- the native format, but this can cause quality loss if the ed as needed. Forwarding to another mailbox on the data subsequently needs to be converted to some other same IMS is implemented in the same way as for voice
6 11 EP 0 845 894 A2 12 and facsimile, and operates independently of data type. provide the "web" of connected objects, specially-for- Forwarding to atelephone number, when done automat- matted text scripts or templates, using the Hypertext ically by the system, is considered a form of message Markup Language (HTML), are used. These "web pag- deposit notification. The same mechanisms are used for es" can have embedded links to other objects systems/ forwarding, so we address it below. 5 hosts), as well as presentation control capabilities. The Facsimile machines may also be used to access the browser follows these links by using the linked object's system 106 via the dialup voice interface; often, the fac- URI to send a GET request to the host identified in the simile machine may actually be a PC running facsimile URI, when the user selects the URI. By putting links to software. The system of the present invention has fac- further pages (HTML static documents, or dynamically simile modem capabilities, and handles facsimile mes- 10 synthesized documents) in a web page, a hierarchical sages in a variety of ways, often in conjunction with a organization of information can be established. HTML DTMF telephone for control and listening to voice anno- and HTTP also support entry of data via forms, access tations. The system stores facsimile messages in their to files via FTP, and interfaces to other information sys- native form (tiff) and no conversion is necessary. The tems, such as e-mail, GOPHER, and News. additional complexities for the IMS relate to the output is To provide the PC 1 09 or 1 1 0 (see figure 4) with web of non-facsimile messages to facsimile machines. To do browser access, the IMS 106 of the present invention this the system converts text messages and non-facsim- provides an HTTP server (IPU 146) to handle the re- ile image data to facsimile format using a function avail- quests from the browser, and provides an organization able from Natural Microsystems or the public domain of the information in the IMS 1 06 into a logical structure, software pbmpius can be used to convert to/from vari- 20 using HTML pages as will be discussed in more detail ous picture forms. Exchange of facsimile data with ex- later. ternal systems is performed using both MIME and X.400 In general, HTTP servers are widely available, both for G3 facsimile body parts. as public domain source code, and as commercial prod- For PC access, two physical interface types are pro- ucts. Toolkits are available to simplify page creation. HT- vided: dialup to the IMS telephone ports, and via the In- 25 TP servers are highly configurable, including the map- ternet (or another TCP/IP network). In addition, several ping of universal resource locators (URLs) to internal (or ways to handle voice messages are provided; purely external) data or operations (executable scripts). digital, where the voice data is simply transferred like A simplified example of an HTML page hierarchy any other type of data (such as by using a browser as for the messages stored in the IMS and the subscriber previously discussed), and the PC turns it into audio; the 30 operations used to access messages is given below. use of a voice/data line-sharing scheme, such as pro- Sample URLs are given to aid understanding. vided with the VoiceView system available from Radish (the latter would only be available through the IMS dial- a. Subscriber (or caller) uses a browser 144 (see up ports) and transfer of the voice data via an e-mail figure 6) on a PC 142 to open subscriber's IMS attachment with the conversion to audio occurring in the 35 home page (e.g., "http://www.mail.somerboc.comy PC. JoeQUser/", or "http://www.mail.somerboc.com/ Accommodating access from a PC (see figure 6) 6 1 7-246-9000/") or http://www. mai I. some rboc. com/ can be provided through an APU 150 andlPU 146 (NIU) awscripts/btv.dll?REFRESH. A "bookmark" could hosting SMTP/MIME and POP protocols. Because of also be used to remember the URL. the rapid growth in Internet connectivity and on-line 40 b. The PC 142 dials directly to the IMS 132 or via services, message user agents (MUAs) (or client mail an Internet service provider (ISP, 1 40, and connects readers) using these protocols are becoming universal to the IMS host 132. The PC 142 sends an HTTP for subscribers using dialup access. However, the Hy- GET method for the home page URI. pertext Transfer Protocol (HTTP) is the preferred proto- c. The I MS HTTP server 1 40 sends the home page col since it is the one used to interface with standard 45 in the response to the GET, and the PC's browser web browsers. It is a simple request/response protocol 144 displays it. The home page has welcome text which uses TCP/IP. Requests (called methods) are pro- and/or graphics and/or voice announcement, in- vided to get and create objects (real or synthesized da- cluding a password entry field. For users wishing to ta), and to do other operations in support of navigating leave a message for the subscriber instead of log- a global, interconnected set of information. The subjects so ging in, there is preferably a "button" on the sub- of the methods are identified by a Universal Resource scriber's home page to "leave a message." The call- Identifier (URI) or Locator (URL) which specifies the lo- ing line identifier (CLI) can be used to verify that the cation (including the Internet name of the host where the calling number matches the subscriber name, or for information is stored) and the means to access the ob- using the authentication capabilities of HTTP. A se- ject. Responses are returned to a requester in MIME- 55 cure socket layer (SSL) is used to provide an initial compatible format, so the MIME content-type and con- secure connection for authentication. A menu of ad- tent-encoding can be determined by the requester, and ditional services besides integrated messaging can the object presented in the appropriate way. To actually also be provided.
7 13 EP 0 845 894 A2 14
d. Using the browser 1 44, the subscriber enters the preferably stored either in native voice format as de- password in the login information form, the browser scribed in U.S. Patents 5,029, 199 and 5, 193, 110, native 144 sends it to the HTTP server 146. The server facsimile format (preferably conventional tiff), or in a 146 validates the password and synthesizes the MIME format. For PC access, or delivery to a PC via subscriber's main messaging page ("http://www. 5 outdial, handling of data other than native voice data is mail.somerboc.com/JoeQUser/ inbox") from the preferably by sending it unconverted using the stored contents of the subscriber's message store, and re- MIME format. Facsimile information is preferably sent turns the page to the browser 1 44. The subscriber's in the MIME image/tiff format. Voice data is preferably messaging page contains links to each of his stored sent as the MIME audio/wav type. messages ("http://www.mail.somerboc.com/aw- 10 Notifications are used primarily to alert a subscriber scripts/btv.dll?REFRESH) and whatever inventory that he has messages, so the subscriber need only ac- information ("http://www.mail.somerboc.com/aw- cess the system when messages exist. Notification scripts/btv.dll?DRTR&Unique Msgld) is desired for mechanisms vary from the communication of one bit of display, plus buttons for sending, deleting, forward- information (messages present or not), to delivery of the ing or other message actions. is actual messages themselves. e. If the user clicks a message link, the message is The present invention, using Access NP™, pro- retrieved by the HTTP server 145 from wherever it vides comprehensive notification delivery capabilities is stored. The message is formatted and encoded including: paging via outdial, TAP and TNPP; message as a MIME message, and is presented by the waiting indication (MWI), via SMDI, and using various browser 144 according to the MIME type (text, im- 20 SS7 or ISDN capabilities; special delivery outdial (deliv- age, video, voice, etc.). Voice and facsimile mes- ery of voice and facsimile messages by outdialing the sages stored need to be converted to MIME format subscriber at a specified telephone number); and cellu- first. lar short messages, containing mailbox message f. Buttons are also links to URIs. If the subscriber counts or callback numbers. clicks a button, the HTTP server 146 may simply 25 Most of the above do not actually deliver messages, use the URI as a command name and perform the and so require no message datatype conversions. Spe- action, or it may return an HTML form to allow fur- cial delivery, however, is more complicated. The desti- ther user input, or to give more details about some nation terminal must be capable of receiving the data entity. type, or the system must be capable of recognizing the 30 terminal type and converting the message accordingly. In the present invention, an HTTP server is provided The data conversions discussed (e.g., text-to-facsimile in each application processing unit (APU) 150 (see fig- or text-to-speech) should, however, handle most notifi- ure 6) that is configured to handle data calls. In addition, cations. Special delivery via outdial to a PC is also pro- an HTTP server (IPU 146) is provided for subscribers vided. accessing the IMS 1 32 via an ISP 1 46 and the Internet 35 In addition to data conversion for special delivery, it 1 36. For small numbers of concurrent PC sessions, the is necessary to consider how other message types af- MIU previously mentioned, with modem banks, can also fect the notification algorithms. One approach is to han- be used to provide PC dial-up access. Note that all HT- dle them in exactly the same way, so that any e-mail TP servers are configured identically, and use the dis- message causes the MWI light of the subscribers tele- tributed subscriber database access mechanisms (in 40 phone to come on, or causes a page to a pager if it is the same way as a voice application) to locate and re- marked urgent. trieve subscriber data and messages, even when they A real IMS is self-contained for notifications, in that are not on the same APU. The HTTP servers use the any message which is to be notified is stored in the IMS. existing platform operating system TCP/IP and PPP However, in a virtual IMS, one MS does not store all protocol capabilities. 45 messages. Further, some of the integrated message Two situations for data type conversion can arise: systems may have no notification capability. Thus, there when a subscriber's terminal type will not accept the needs to be a way to communicate the presence of mes- stored message format, and upon user request. An in- sages between MSs ("cross-notification"). This may re- termediate situation is when a subscriber requests de- quire non-standard mechanisms, since most MSs are livery or forwarding of the message to a system or ter- so not designed to accept notifications. This, in turn, may minal, other than the one he is using, which does not require MS modifications, which defeats much of the support the data type. Most conversions are implicit purpose of a virtual IMS. Below we discuss how to pro- from the message type and the destination, but it may vide VMS exchange notifications with an associated be preferable for a subscriber to explicitly request con- EMS, without major EMS modifications. version (e.g.) of a facsimile message to text for forward- 55 An associated e-mail system (EMS) will preferably ing to an Internet address, even though the message have one of SMTP/MIME and POP protocol capabilities could have been sent as a MIME facsimile message. (or equivalent X.400 capabilities). The EMS can initiate For the IMS of the present invention, messages are SMTP sessions, but the VMS must initiate POP ses-
8 15 EP 0 845 894 A2 16 sions. The EMS is configured to automatically create a point. Interfaces and operation for the "pass-through di- copy of each message deposited, or an additional mes- rector" approach to virtual I MSs will be discussed in sage with inventory information, give the message a more detail (notifications have been addressed above special VMS recipient address, and send it via SMTP to already). As for notifications, an EMS with SMTP and the VMS (IMS 106). This has desired asynchronous no- 5 POP capabilities, connected as shown in figure 3, is as- tification characteristics. The VMS (IMS 106) receives sumed. the message, parses it (or merely notices it), and uses The general handling of a virtual IMS subscriber's the information to control the notification mechanisms operations in a virtual IMS will be described below for (but not score it). Alternatively and less preferably, the each of the typical phases of a session. VMS (IMS 106) uses POP to poll the subscriber's mes- 10 When the subscriber logs into the VMS 68, the VMS sage inventory. This is a periodic event, likely to cause 68 generates an integrated message inventory. This is a lot of traffic, since it needs to be fairly frequent and done through the use of a POP session between the there could be large numbers of subscribers. In either VMS 68 and the EMS(s) 66. The VMS 68 logs into the situation, the VMS (IMS 106) needs to be provided with POP server on behalf of the subscriber (POP USER and the subscriber's e-mail address and password. is PASS commands), queries the EMS inventory (POP There is also a hybrid of the above-discussed ap- LIST command) and merges it with the local inventory. proaches where the VMS is an IMS, and it uses POP to The VMS 68 maintains a map from the POP message retrieve EMS messages but also offers POP to the sub- IDs to an identification (place in sequence, ID number scriber so that client e-mail SW connects to the VMS or synthesized URI) used between the VMS 66 and the (IMS) to retrieve e-mail messages. 20 subscriber. Locally stored messages are likewise given For notifications from the VMS to the EMS, the VMS identifiers. sends an e-mail text message containing the voice/fac- To select a message for retrieval, the subscriber in- simile message inventory using SMTP. terface may use POP (for MUA interfaces), an HTTP For both real and virtual I MSs, there is a need to GET method (for web browser interface access), or en- interface with external systems 116-120 using the reg- 25 ter DTMF commands (for voice access). In any event, ular store-and-forward e-mail paradigm, in order to the VMS 68 maps the subscriber-requested message make the IMS subscribers part of a single world-wide ID or URI to the EMS message ID, or to an internal mes- messaging community. For the virtual IMS, special con- sage ID. If the message is on the EMS 66, the EMS 66 sideration must be given to the "associated MSs", since uses POP to retrieve the message from the VMS 68 that non-standard (for e-mail systems) but conventional 30 (POP RETR
9 17 EP 0 845 894 A2 18
d. If the message is any MIME type, and the recip- The PC application 140 provides the structure of the ient address is not a phone number, the message user interface for HTTP access, and the necessary is sent unchanged to the EMS 66. "glue" to interface with the internal system mechanisms e. When messages are sent to the EMS 66 for han- (in this case, just the message storage). dling, the sender address is set to the e-mail user 5 Microsoft WinSock 1 42 is used to provide the TCP/ name of the subscriber, so it appears that it origi- IP protocol support, both externally and internally. Win- nated in the EMS 66. Sock is an integral part of Windows NT, and includes support for VoiceView data transfers. For Boston Tech- The approach discussed above is a straight forward nology, Inc. platforms using the UnixWare OS, the TCP/ way of operating. There are much more complicated 10 IP support is also part of the OS, but VoiceView is not ways of doing it. The advantage of the simple approach supported. is that the VMS involvement in handling e-mail messag- The "real" IMS system 130 (106) of the present in- es is minimized. vention is preferably implemented using a distributed ar- Figure 5 illustrates a general functional architecture chitecture such as illustrated in figure 6. The system 1 30 for a real IMS which provides the capabilities discussed is includes a message platform 132 that is connected to above. A more detailed description will be provided with both the public switched telephone network 134 (via a respect to figure 1 . digital matrix switch 135) and the Internet 136. A sub- Various elements of the archictecture will be dis- scriber can access the platform 132 using a telephone cussed. Note that figure 5 is not internded to be a defin- 138 to perform message access functions such as re- itive and complete representation of the actual software 20 trieving and listening to voice mail messages, forward- architecture of the IMS which will be described in more ing messages, recording messages, and converting and detail with respect to figure 6. playing facsimile messages. A detailed description of The dialup interfaces 130 in the APUs 150 have this type of access can be found in the U.S. patents pre- drivers for access to conventional DSP hardware, pro- viously described and is available in the CO Access® viding voice interface functions (V), facsimile modem 25 and Access NP™ systemsfrom Boston Technolohy, Inc. capabilities (F), data modem capabilities (D) and The subscriber can also access the platform 1 32 for ac- VoiceView voice/data modem capabilities (V/D). cessing voice mail messages and other types of mes- For DTMF access, the voice application 131 is the sages such as facsimile and video messages over the application described in the related patent previously Internet 1 36 through a conventional Internet service pro- mentioned, which accommodates multimedia messag- 30 vider system (ISP) 1 40 using a personal computer 1 42. es and data conversions, for example text-to-speech The personal computer 142 is atypical convention- and text-to-facsimile. al multimedia personal computer capable of running or The multimedia message store 132 is distributed executing an Internet browser 144, such as the pre- across all the APUs 150, with most of a subscriber's ferred Netscape 3.0 browser available from Netscape messages being stored on a "home" APU. Standard 35 Communications or Internet Explorer 3.0 available from methods are used by applications in any subsystem to Microsoft, and capable of connecting to an Internet serv- access the distributed message store in a location-inde- ice provider 140. The PC 142 can also, of course, be pendent way. connected to the Internet through a company local area The Boston Technology, Inc. Digital Messaging Net- network (LAN) via a high speed connection. The com- work Version II (DMN II)™ product 133 provides store- 40 puter 1 42 preferably includes a modem with a speed of and-forward communication with other MSs, including at least 14.4 Kbps and preferably at least 28.8 Kbps other Boston Technology I MSs or VMSs, other vendor's when the user intends to access video images. The VMSs, and external EMSs. Either the AMIS-D protocol computer 142 also includes a conventional sound card 1 34 or SMTP 1 35 is used for this. DMN 1 1 is automatically and associated audio speaker and audio software such activated when a message requires delivery to an ex- 45 as the preferred TrueSpeech available from the DSP ternal address. Group. For the recording of voice messages that will be The subscriber PC access functions reside in the transmitted to other mailboxes the computer 1 42 needs APU 150 for dialup access, and in the NIU (IPU 146) for a microphone half duplex recording board such as network access. Both instances of this set of functions SoundBlaster from Creative, Inc. and software such as are essentially the same, except that the datalink proto- so MediaPlayerfrom Microsoft. For displaying still images, cols are different. such as facsimile mail, the computer 142 needs to in- An HTTP server 136 is provided in the IPU 166 to clude a tiff reader such as Microsoft Facsimile available allow access using conventional web browsers and as- from Microsoft Corp. These still image components can sociated web pages 137, and SMTP and POP servers also be used to record facsimile messages for transmis- 138 are provided to allow access from MUAs. All other 55 sion to other mail boxes. For the playing of motion video services POP3/IMAP4/Voice/PPP/X.400 are utilized on images, the computer 142 uses ActiveMovie from Mi- other components and in turn utilize the IPU 166 for HT- crosoft. If video images for broadcast are to be recorded TP if necessary. the computer 1 42 needs a conventional camera and as-
10 19 EP 0 845 894 A2 20 sociated software such as Connectix from Connectix, or selecting the message and clicking on an appropriate Inc. icon such as "Play" in the display of the browser 144. The browser 144 is used in a conventional method The unit 146 responds by obtaining the selected mes- to access an Internet web page hosted by the platform sage from the application processing unit 150 that 132. The browser 144 allows the user to perform func- 5 stores the message, converting the message into the tions such as viewing a message list, view addresses of proper format and transmitting it. In the case of voice voice/video mail recipients and senders as well as e- messages the voice data is converted from the encoding mail addresses, playing and saving messages, forward- for storage into a file in the encoding for playing used ing messages to others, replying to messages, creating, by a conventional audio application executable by the listening to and modifying personal greetings, name an- 10 browser 1 44 such as the preferred ActiveMovie from Mi- nouncements and prompts, etc. using the graphical user crosoft and streamed to the browser 144 where it is interface and the video/sound capabilities of the com- played as it is received. Forfacsimile and othertext mes- puter 142. sages a tiff file is created and transmitted to the browser The user accesses the platform 1 32 through one or 144. For video messages the video data, if necessary, more Internet processing units (IPUs) 146 over a con- is is conversed into the avi, mpg, mpeg, cu, etc. file formats ventional digital communication path typically used for that allow the data to be streamed to the browser and high speed access (10-1 00 Mb/s of a home page server. displayed in a pop-up window in real-time as it arrives. The unit 146 is preferably a pentium based computer That is, video as well as audio messages are played or that operates at 133/166 MHz with 32 MB of RAM and displayed as received by the computer 1 42. During play 4GB hard disk drives that are mirrored. A conventional 20 back the user can perform the conventional functions of Internet router (not shown), such as available from Cis- rewinding, pausing, fast-forwarding, skipping, etc. The co Systems or Bay Networks which performs packet fil- user can also perform operations associated with saving tering, can also be provided between the Internet 136 the message, deleting it or forwarding it to others. The and the unit 146. The router can provide a "bridge" be- processes performed by the Internet processing unit tween the ethernet 149 and the network backbone. In 25 146 as well as those performed by the master control such a configuration the router and unit 146 would be unit 148 and application processing units 50 and other coupled preferably using a 100Mb Ethernet connection units 52, will be discussed in more detail later and are and perform the primary function of moving packets to set forth in the source code appendix which code can and from the IPU 146. The unit 146 provides firewall pro- be stored on/in various types of media, such as various tection from hackers, acts as a web server for the mes- 30 types of disks and various type of computer memories, saging application, performs hypertext markup lan- in the platform 132. guage (HTML) generation and performs the voice en- The operations or entry points available to a user coding and image encoding for the messages streamed interacting with the browser 144 include the following to the computer 142. functions: During a typical session a user will access the plat- 35 form 132 over the Internet 136 using a standard web QUIT - Exits out or logoff of the system and de- browser to obtain a service provider home page where lete all session information. the user will log into the Internet service provided by the DELE - Removes one or more messages from platform 1 32. During this process the user is required to the system. enter a mailbox identifier and a pass code which are 40 SAVE - Saves a message. checked to ensure that the user is authorized. Once au- DRTR/RMSG - Retrieves data such as an audio mes- thorization is confirmed a service session is initiated and sage. the user is presented a page that includes a menu of RECORD - Sets and starts record/send process. service options such as viewing a message list, admin- REFRESH - Gathers data and instantiates to brows- istering mail box options, other network service fea- es er in HTML. tures, etc. USE - Selects a specific template. When the users e-mail is stored and supported by GRTR - Retrieves particular group list data. another platform the message list can include a cross GDEL- Deletes a group list. notification of the existence of an e-mail message in the GINS - Adds a single entry to group list. mailbox message list. so GUSE - Get group and use template. During a typical message retrieval function, the unit GPUT - Modify current group lists. 1 46 accesses a master control unit 1 48 over an internal GNEW - Make new list. dual channel ethernet 1 49 to locate the storage location MBOXADM - Administer user mailbox features such of the various types of messages stored for a subscriber as change the password. and generates a web page which is transmitted to the 55 personal computer 142 and which includes a list of the Each of these operations has a routine of a correspond- messages. The user selects a message in a convention- ing name in the source code appendix. The prccesses al fashion by double clicking on a message descriptor for the above-listed functions perform many of the same
11 21 EP 0 845 894 A2 22 operations such as authenticating a request, however, template from the template file 34 is instantiated 19 to for simplicity of explanation three representative proc- match the data in the session cache 175. The data of esses, the process that refreshes (REFRESH) the the template is passed 20 to the IIS routine 170 where browser display, the processes (DRTR/RMSG) that the HTML template for the page is returned 21 to the download messages for playing/displaying and the 5 browser 144 which displays the refreshed page to the process records (RECORD) messages to send to oth- user. The templates preferably use the Microsoft. htx file ers, will be discussed. The other processes are de- syntax and the templates include standardized varia- scribed in detail within the source code appendix. bles for the various data, such as <%AccountNum- The refresh operation steps, with the data, ac- ber%>, <%From%>, <%Media%>, <%Length%>, etc. cessed, are depicted in figure 7. The refresh operation 10 The transmission of the HTML for a template allows is integral to the display update operation of the inven- the service provider to customize the home page (for tion because each of the web pages that are transmitted example, by adding advertisements) and deliver pages to the computer 42 is affixed with an expiration date, based on the domain or class of service of the subscrib- such as yesterday, that causes the browser 144 to re- er. quest a reload of the page each time the page is ac- is A refreshed active page that is designed to list cessed or additional information is requested. This re- stored messages can be formatted as illustrated in fig- sults in the message list being updated to include mes- ure 8. The messages can be accessed (played/dis- sages that have arrived since the current session start- played in real time), saved, deleted from the platform ed. The refresh operation is also used to provide web database and in saved to storage local to the personal pages that the user has selected during browsing where 20 computer 1 42 using the conventional point an click par- the refreshed page has never been transmitted. The re- adigm. fresh operation starts with a refresh request (a "GET" The group list administration functions can be facil- with a URI) being sent 1 to the Internet processing unit itated using an active page such as depicted in figure 9 146. The IIS routine 170 denies access if the authenti- where custom distributiin lists can be easily created and cation for the request is not present. If the authentication 25 edited to add, delete and modify addresses of recipients is present the request is passed 2 to the security filter of the various type of messages that are supported such routine 172. The security filter 172 sends 3 the request as voice, video, facsimile, telex, e-mail, etc with ad- to the support 174 for authentication by the platform 132. dresses such as telephone numbers, e-mail addresses, The routine 174 checks 4 the session information data universal resource location addresses, cable address- 175 to see if a "session key" currently exists for the re- 30 es, etc. quest and if so the flow skips to step 9 discussed below. The processing of a message retrieve or play re- If the session key does not exist the authentication re- quest (DRTR/RMSG), like the refresh request, starts quest is sent to UNIX support routine 176 of the MCU with the transmission of a request 1 by the browser 1 44 1 48 and/or APU 1 50 for authentication. The authentica- to the unit 1 46 as depicted in figure 1 0. The request can tion involves accessing 6 the user authentication data 35 be made by double clicking on a message in the list of 1 78 and returning 7 an indicator of success or failure at figure 8. The request is processed in the same way as the authentication task. The authentication successful the refresh request for steps 2-10. In step 11 the name result is returned 8 to the security filter 172 otherwise of the requested file which has been made using a cod- processing proceeds to step 21 where a message con- ed name may also be converted to the real file name, cerning the authentication failure is returned to and dis- 40 which is a security issue which will be discussed in more played by the browser 1 44. The security filter 1 72 trans- detail later. The file request is passed 1 2 to the gateway fers 9 control to the validation routine 1 80 where account routine 182. The routine 182 passes the request for the and file systems permission validation is performed and file to the support routine 1 74 which obtains 1 4 a unique which returns 10 the result of validation. If validation is identifier from the session information 175. The request not successful, control transfers to step 21 where a fail- 45 is then provided 15 to the application processing unit ure message is sent to the browser 1 44. If the validation 150. The processing unit 150 accesses 16 the list of is successful control transfers 11 and 12 through the Mi- messages 177 and obtains 16.5 the message data 186 crosoft Internet Information Server (IIS) routine 170 to (voice, text, video, etc.) from storage. The message data the template update routine 182. Routine 182 transfers passed to the support routine 1 74 and is converted from 1 3 the control to routine 1 74 to access 1 4 and check the so the native storage format (from Oki 24 when the mes- state of the session cache 1 75. Once this check is com- sage is a voice message) into and stored 1 9.5 in cache pleted a request 1 5 for an update of the message/group or temporary storage 188. The cache storage allows lat- information is presented to the routine 176. The routine er requests for the same message to be processed with- 176 obtains 16 the mailbox list information or group list out again decompressing the data. As the data is tem- information 177 stored in the MCU 148 and returns 17 55 porarily stored it is retrieved as needed to perform a real- the information to the unit 1 46 where the session cache time conversion into streaming mode data and into the is updated. The updated data is forwarded 1 9 to the desired format, such as TrueSpeech. This conversion HTML template update routine 182 where the current will convert voice data into data compatible with Net-
12 23 EP 0 845 894 A2 24 scape type "plug-in" play back systems that Include The present invention enhances the security of TrueSpeech, Voxware, Real Audio and WAV (which is each session with a number of different features. standard for Windows 95/NT). If the data is text data it Unit 126 is preferably limited to the HTTP (Hyper- can be converted into a tiff file compatible with a con- text Transfer Protocol) and SSL (Secure Sockets Layer) ventional tiff viewer. A facsimile message is typically 5 type Internet service to help eliminate problems associ- stored in a tiff file. If the data is video data it is converted ated with unsecure protocols. The present invention al- into MediaPlayer data (an .avi file) compatible with the so has certain authentication features. A request is first MediaPlayer system available from Microsoft. The data sent from the browser 144 to the server 146 that does can also be converted into JPEG or MPEG or still and not include authentication. The Microsoft Internet Infor- motion video players available for conventional brows- 10 mation Server (IIS-170) software running in the server ers. The real-time streamed data, which includes the 146 checks this request. The filter 170 also checks to content type, is sent 18 to the IIS routine 170 and im- see if the user is valid and since there is no authentica- mediately forwarded 19 on to the browser 144 with an tion the check fails and the browser 144 sends the re- option to store the data in the cache local to the unit 146. quest again this time with authentication. The request When streamed the browser 1 44 does not know the con- is either passes or fails based on the authentication. When tent type of the message until the message is received the request fails the browser 144 is notified. When the from the server 146 and the invention relies on the de- request passes the request is forwarded for processing. fault mode of the plug-in that handles the data type. At In an initial log-in situation the request is forwarded to the browser 144 the transmitted data, if it is Voxware the users home APU 150 where the validity of the user data, causes a window to open and it is immediately 20 is checked again to determine whether the user is a sub- played or displayed as the case may be. Other types of scriber. This double validation helps prevent nonsub- data such as RealAudio require that a play button in a scribers from obtaining access to the system. When the pop-up window be activated. user is a subscriber the user gets a message list sent to Once a message is received the subscriber, using the browser 144 where it is displayed. Once a session the browser 144, can save it locally, play it again, re- 25 is established the user it essentially communicating with verse it, skip forward and backward, copy it into other the home APU 1 50 for this subscriber which controls fur- files and other types of operations that can be performed ther transactions. All further requests by the browser with multimedia files. 144 to the server 146 go through the first level of au- During a record process, a media player, delivered thentication. with a conventional operating system such as windows 30 The system preferably uses secure socket layer 95/NT, records the message or the subscriber activates (SSL) packet encryption. a plug-in of the browser 144 to record the message The present invention also is preferably implement- (voice, video, text, etc.). Using the application, the sub- ed using dual horned-host Internet processing units that scriber edits, rerecords, etc. the message until satisfied. prevents packet sniffing on the internal ethernet. A dual When finished the user provides a file name to the re- 35 horned-host is a host that has two I P addresses that cor- cording and stores it locally. The browser 144 is then respond to one or more physical addresses allowing it used to POST the message to the server. Alternatively, to be configured differently based on the I P address. For the browser 144 can be used to request that the server example, one IP could be configured only to work with 146 record the message and send it. The server re- SSL active and the other IP is used in the clear, i.e. with- sponds by performing the steps 1-21 for refreshing a 40 out SSL. page previously discussed and forwards a template for The provision of a router to perform packet filtering a page, such as depicted in figure 11. The subscriber prevents source address spoofing. completes the template by providing the file name of the The present invention also assigns session num- recording, addresses (such as telephone numbers, e- bers and specially created file names to files that are mail addresses, etc.) of the recipient along with indica- 45 transferred to the browser 44. In this filter operation the tors for privacy, etc. The browser 144 when the submit process removes all correlation to any data set internal message" button is pressed creates a message header, to the platform 1 32 from data sent over the network 1 36. attaches the file and sends the file to the server 146. For example, a message identifier that is sent to the The server 146, when the file appears In the incoming browser includes a session identifier and a randomly as- message directory, converts the message into an ap- 50 signed file identifier (which can be the current time of propriate storage format (compresses it if necessary) day). The server 1 46 creates a session information entry and stores the message. The header is reviewed to as- that identifies the file for the session identifier and the certain the recipients addresses and the message is re- randomly assigned file identifier. When the browser 1 44 trieved and sent to the recipients. For example, a voice requests the file the session identifier and previously as- message to a particular telephone number would result 55 signed file identifier are included with the request. The in an outdial process being performed. If the recipient is server 146 uses the session information to convert the a subscriber the message is copied to the recipients file name into a real file name to retrieve the file. When mailbox. the server 146 sends the requested file to the browser
13 25 EP 0 845 894 A2 26
144 the server 146, if it is not a streamed data file, the The administration features of the system allows a server 146 assigns a pseudo file name that includes a verified system administrator to access a system admin- session number and a randomly created file name. This istration home page showing variables such as alarms, pseudo file name and the real file name are also stored number of users logged in, etc., and to perform functions in the session information so that the file can be request- 5 such as releasing an access block on a subscriber mail- ed again. The session mumber is part of the create iden- box. tifier to allow communication that use the same random The present invention has been described with re- number to be distinguished. If the file is a streamed data spect to handling text messages such as e-mail and fac- file, the data of the file with content type is sent without simile. The text messages could have other formats any file name identification. 10 such as a word processing format, a spread sheet for- The present invention preferably uses a buffer of mat, a database format, etc. and could be other MIME 81 92 bytes to improve transfer efficiency even though a type information than can be retrieved without conver- buffer size of from 512 bytes to 8192, except for 4096 sion. The applications that are performed by the appli- for certain audio formats when the data is audio, will cation processing units could include personal informa- work. 15 tion managers, appointments, address bocks, etc. The present invention, through the network inter- The many features and advantages of the invention face provides two methods of message deletion: imme- are apparent from the detailed specification and, thus, diate or marking messages to be deleted with a deletion it is intended by the appended claims to cover all such "Commit" (see figure 8) at the end of a session. The sec- features and advantages of the inventicn which fall with- ond option is like the deletion feature of the audio (tele- 20 in the true spirit and scope of the invention. Further, phone) interface where messages to be deleted can be since numerous modifications and changes will readily listened to again thereby removing the delete flag and occur to those skilled in the art, it is not desired to limit only those messages flagged for deletion are deleted the invention to the exact construction and operation il- when the audio session is ended. lustrated and described, and accordingly all suitable The present invention also allows different types of 25 modifications and equivalents may be resorted to, falling messages to be bundled together into a multimedia type within the scope of the invention. message with multiple body parts. The state information for a message includes body part information which in- dicates the content type of the body part. Claims The present invention is also suitable for providing 30 electronic data interchange (EDI) services where EDI 1. A message storage system, comprising: forms, such as purchase order forms, are provided to a user for the purchase of goods, etc. Other types of data a multimedia mailbox storing voice messages such as weather data can also be stored and transmit- and text messages; ted. 35 a voice interface providing access tco the mes- The present invention, using the administration fea- sages via a telephone; and tures, can be configured through the network interface a network interface providing acce3ss to the to perform operations such as sending standard text or messages over a network via a perssonal com- voice messages to doctor's patients. puter. The administration of mailbox features, such as the 40 password, telephone ring count, etc., is performed using 2. A system as recited in claim 1 , wherein said network HTML templates that can be customized for each serv- interface comprises an Internet server and said ice provider. voice interface comprises a voicemail server. The present invention provides priority of access to a mailbox by one of the owners to accesses that are 45 3. A system as recited in claim 2, wherein said network made through the telephone interface. interface provides a home page and an access by The browser 144, if it automatically requests a re- a user of the home page by an Internet browser gets fresh of a currently displayed page, allows a page to be an audio voice message when a the user selects a created that includes a message list icon that can be voice message and gets an image when the user updated to reflect that a new message has arrived dur- so selects a text message. ing the session. The present invention also includes an automatic 4. A system as recited in claim 3, wherein said home log-off feature that will log a subscriber out of the system page is refreshed each time it is accessed. when there has been no activity for a period of time. This allows subscribers to inadvertently leave their PC 55 5. A system as recited in claim 4, wherein said home logged in to the system, such as when going home from page includes an expired expiration date. work, and prevent others from accessing the system during the absence. 6. a system as recited in claim 1 , wherein said mailbox
14 27 EP 0 845 894 A2 28
stores still and motion video messages. sage list including message identifiers compris- ing a session number and a randomly assigned 7. A system as recited in claim 4, wherein said network file identifier responsive to a home page re- interface streams the messages to the user. quest by the browser, retrieving the messages 5 from the processing unit and streaming the 8. A system as recited in claim 7, further comprising a messages to the computer responsive to personal computer including a prowser which plays/ browser message play requests to the home displays messages streamed to the user. page having an expired expiration date, the computer playing an audio of the voice mes- 9. A system as recited in claim 1 , wherein said voice 10 sages in real-time as the voice messages are interface outputs the voice and text messages as received, displaying an image of the text mes- audio to said telephone. sages in real-time as the text messages are re- ceived and displaying images of video messag- 1 0. A system as recited in claim 1 , wherein the network es in real-time as the video messages are re- interface includes a data conversion cache for stor- 15 ceived, and said computer recording a mes- ing converted messages. sage and forwarding a message to said net- work unit for storage in said processing unit. 11. A system as recited in claim 1 , wherein said network interface provides for recording messages. 15. A storage media, comprising: 20 a multimedia mailbox storing voice messages 12. A system as recited in claim 1, wherein message and text messages, a process providing access to identifiers include a session number and a random- the messages via a telephone, and a process pro- ly assigned identifier. viding access to the messages via a personal com- puter. 13. An apparatus, comprising: 25 a message storage system including a multi- 16. A method, comprising: media mailbox storing voice messages and text messages, providing access to the messages via a providing access to messages in a multimedia telephone, and providing access to the messages mailbox storing voice messages and text mes- via a personal computer. 30 sages via a telephone; and providing access to the messages via a person- 14. A message storage and retrieval system, compris- al computer. ing:
a telephone; 35 a telephone network coupled to said telephone; a computer including a digital network browser; a digital network couplable to the computer; and a distributed architecture message system cou- 40 pled to sad telephone network and said digital network, said message system comprising: a digital switch coupled to said telephone net- work; a control unit storing addresses of multimedia 45 messages including voice, text, and video mes- sages, and controlling switching of said digital switch; a processing unit coupled to said digital switch, storing and retrieving the multimedia messag- so es, and outputing the voice and text messages to said telephone as audio over said telephone network responsive to telephone commands; a local network coupled to said control unit and said processing unit; and 55 a network unit coupled to the digital network and said local network including a data conver- sion cache, providing a home page with a mes-
15 EP 0 845 894 A2
FIG. 1
PC
FIG. 2
c66 •68 E-MAIL VOICEMAIL SYSTEM SYSTEM
DATA NETWORK^ • DIAL-UP FROM PC • LAN OR WAN TO MSs
MDM 60
PC
16 2?
|o LU UJ O— I Fi9 c L 2 &\ ?
CO S
1 PA? \ 1
< I Jk I00 J
- liT ku H a. >- o O^Pgw£ CM 1 1 O — U my- m rn 2 ^ An / Wh \ 5 C [CO 3 > rr ^ CD V3/ 5 2 2 ICO « 3 3 LU ' CD ••■/. ™Bl.vmm-:- >> CO CO 0CD mfl V." gig »CD J= I -2 % §en -zz o o Q(—1 TI CO cu o co .2_CD __ S 9:: o TJ 1 < X°X 55 '(D'(1) CD oO 55 oO coCD coCO Q ... \ :•• -OX3 552 £JH cnCD CD v. (8t- co co p ■./ ^y.- - ^ °. ^ II| v. I — ooCO CO* 5=£ A oo Io oQ ' CM O • ' •'•^_ vV.' o o M■' ■,^^t - ' ° c ° ^v-v-WA o -i— i— .■.•..'•^'■v/.'.v'-.'-. "ss siiSii S $ 5 go ^- ^~ >- >- -v'V;-V/<-:<.^V.V;-V/.V..V<- < < •/•>V-V;'.V;.•/•>V-V;'.v--:- '.v.- _J _J Q_ Q_ V;V;;.Vv'v>. — — ,, Jj 22 X 55 rv I ^ < cc _7 / CO 1— \ /Si i Q_ ■ Z> i O \ .1 l.l «l I." UJ I S O CO oo CO cd LU CL 2 □ cr CD L- ' O Q_ o n □ DC < ~- en l