US 20140098849A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2014/0098849 A1 Panich et al. (43) Pub. Date: Apr. 10, 2014

(54) METHOD AND DEVICE FOR CREATING AND Publication Classification MANTAINING SYNCHRONIZATION BETWEEN VIDEO SIGNALS (51) Int. Cl. H04N II/02 (2006.01) (71) Applicant: ATI TECHNOLOGIES, ULC, (52) U.S. Cl. Markham (CA) USPC ...... 375/240.01; 375/E07.026 (72) Inventors: Alexander Panich, Richmond Hill (57) ABSTRACT (CA); Syed Athar Hussain, A method and device for providing synchronized data output Scarborough (CA) is provided. The method includes generating two data streams sending data to be presented in Synchronization. Both streams (73) Assignee: ATI Technologies, ULC, Markham are generated by the same processor-based device. The first (CA) data stream follows a first protocol and the second data stream follows a second (different) protocol. The processor of the (21) Appl. No.: 13/645,020 processor-based device adjusts a data rate of the second data stream to cause a reduction in any timing offset between the (22) Filed: Oct. 4, 2012 StreamS.

3. ? 2

2. 15 1 4. 2 3. He

ri Corgia's Eiggiie Pixei Rate Register 8

16 *::s: Ragic: 2

A y V w isiay isplay isplay iisplay {articlef Coliidier fitoe? gic:

-, SA

Patent Application Publication Apr. 10, 2014 Sheet 1 of 2 US 2014/0098849 A1

N - sa

re

8 ii.

c) 3. --

- s ^ s

s . $ s M s is -- -- S t e 2. --- ^. s .. --- X. s - -- — s s res al c i.

i. - as S. - is a - H i - . Y - - t wner -

* M

w ia. M wa

----- S . to ear Patent Application Publication Apr. 10, 2014 Sheet 2 of 2 US 2014/0098849 A1

Start Generate multiple Sapie Outgoing data streas Stears 310 400 Deterine Offset - between streams

320 410 Deterie fate of change in offset betweer Stears 330 420 Aiie Siave Stears to Deternie if the approximate the offset will surpass waster Streat tress 430 340 END ) Adjust stream rate to reduce Offset FIG. 2 440 END FIG. 3

35

Reduce memory clock speed 360 FIG. 4 / END ) US 2014/0098849 A1 Apr. 10, 2014

METHOD AND DEVICE FOR CREATING AND other processes can use a lower clock speed, thereby saving MANTAINING SYNCHRONIZATION power. Again using the above example, when streams are BETWEEN VIDEO SIGNALS synchronized, it can be determined when active regions are present in the streams and when blank regions are present in FIELD OF THE DISCLOSURE the streams. Furthermore, the active and blank regions for the streams are likewise synchronized. Thus, during blank 0001. The present disclosure is related to methods and regions, a lower clock speed can be applied. devices for providing video streams to multiple panels. The 0006 Synchronization of different streams is currently present disclosure is related more specifically to methods and possible through the use of additional hardware modules. devices for ensuring synchronization between multiple pan Such modules include the S400 synchronization module and els where the panels utilize different protocols. the Quadro G-Synch. However, these synchroniza tion modules are provided for synchronizing signals from BACKGROUND different stream sources (GPUs, etc). 0002. When video output from a computer is provided to 0007 Accordingly, there exists a need for a device and multiple display panels, multiple video streams are created method to provide different video streams from a common between one or more graphics display boards and the display Source that operate so as to appear synchronous. panels. If the multiple panels are the same, the properties of the generated video streams that Supply them can be the same BRIEF DESCRIPTION OF THE DRAWINGS (resolution, refresh rate, etc.). However, if one wishes to 0008 FIG. 1 is a diagram of a computing device providing provide outputs to panels that are different from each other, synchronization between output video streams according to the video streams are often different. Furthermore, different an embodiment of the present disclosure; interfaces and protocols can be used for the different display 0009 FIG. 2 is a flow chart showing exemplary operation panels which also cause differences in the video streams. For example, if one display is being Supplied data using HDMI. of the computing device of FIG. 1 as it establishes synchro one display is being Supplied using DVI and another display nization between video streams in accordance with an is being Supplied using DisplayPort, the hardware driving the embodiment of the present disclosure; outputs to these displays may need to use different hardware 0010 FIG. 3 is a flow chart showing exemplary operation blocks to set up the various video streams. Similarly, two of the computing device of FIG. 1 as it maintains synchroni streams of the same protocol that are being sent to two dif Zation between video streams in accordance with an embodi ferent displays that Support different resolutions also require ment of the present disclosure; divergent streams. Also, two streams of different protocols 0011 FIG. 4 is a flow chart showing exemplary operation that are being sent to similar displays (or both to a single of the computer of FIG. 1 as it initiates synchronized video display) also require different streams. By way of further streams in accordance with an embodiment of the present example, AMD Eyefinity technology Supports up to six disclosure. simultaneous displays off of a single graphics card. Eyefinity supports display outputs of DVI, DisplayPort, miniDisplay DETAILED DESCRIPTION OF EMBODIMENTS Port, HDMI, and VGA. 0012. In an exemplary and non-limited embodiment, 0003. The differences in the video streams result in a dif aspects of the invention are embodied in a method of provid ference in bit rates of the data being output to the panels. This ing data output. The method includes generating two data difference in bit rates provides a lack of synchronicity. While streams sending data to be presented in Synchronization. Both the difference may be small, as the displays continue to run, streams are generated by the same processor-based device. the small differences begin to accumulate until the displayed The first data stream follows a first protocol and the second images are out of synchronization by Such a margin that it data stream follows a second (different) protocol. The pro becomes noticeable. Specifically, the lack of synchronicity cessor of the processor-based adjusts a data rate of the second presents problems for users who can see both streams (such as data stream to cause a reduction in any timing offset between in multi-screen or single screen 3D applications) or where the streams. multiple users need to perceive the video simultaneously 0013. In another exemplary embodiment, a graphics con (such as when there is a common audio stream projected to troller is provided including a processor operable to generate the viewers of the multiple screens). a first data stream sending first data and a second data stream 0004. When there is a lack of synchronization between sending the first data, the first data stream following a first Video streams, clock pulses that drive the video streams are protocol and the second data stream following a second pro each firing at different times. Accordingly, other processes tocol that is different from the first protocol. A second display that look to operate on the streams or use the stream data (Such controller is operable to adjust a data rate of the second data as memory controllers) need to run relatively frequently to be stream to causea reduction in any temporal offset between the Sure and pick up current data. By way of example, Video StreamS. streams contain blank and active regions. The active regions 0014. In another exemplary embodiment, a computer are the portions that need to be fetched. If streams are not readable medium containing non-transitory instructions synchronized, then blank regions of one stream can occur thereon is provided, that when interpreted by at least one while an active region is present for another stream. Thus, processor cause the at least one processor to: compare a first clock speeds must be maintained at all times in that an active data stream sending first data to a second data stream sending region can be present at any time. the first data to determine the degree to which transmission of 0005. If the streams are synchronized, then other pro the first data in each of the streams is temporally offset, the cesses can run at a similar clock pulse and know that the first data stream following a first protocol and the second data process is aligned with the stream data. In this manner, the stream following a second protocol that is different from the US 2014/0098849 A1 Apr. 10, 2014

first protocol; and adjust a data rate of the second data stream example, the synchronization described herein is used to Syn to cause a reduction in any temporal offset between the chronize signals that are originally within 0.25% of each StreamS. other. (Thus, if one signal is 60 Hz, other signals between 0015 FIG. 1 shows architecture for providing digital 59.85 Hz and 60.15 Hz are suitable candidates for synchro video information from a device (PC 10) to multiple display nization.) It should be appreciated that 0.25% is just an exem panels 12. When Supporting multiple display panels 12, a plary value. In fact, any two signals with differing refresh Video stream is generated for each panel 12 via a display rates can be synchronized so long as it can be assured that the controller 16, block 310 of FIG. 2. In the creation of the video synchronization will not cause the altered signal to violate its streams, the transport protocol (DVI, HDMI, DP, VGA) as specification (DVI, HDMI, DP etc.) This concept creates a well as the properties of the panel 12 are factored into the “master stream and then one or more “slave' streams. The creation of the video stream. Each protocol and panel has master stream remains unmodified, if possible. The slave specifications that define how the video stream should oper streams are modified (adjusted) to fit the characteristics of the ate. The display controllers 16 utilize a rate generator (using master stream, block 340 of FIG. 2. phase-locked loop, data transfer object, or otherwise as 0019 While one embodiment uses the above master/slave appropriate). It should be appreciated that while a single implementation, embodiments are also envisioned where all Video stream is shown for each display panel 12, embodi streams are modified and forced to conform to a specification. ments are envisioned where multiple video streams are pro One such example would be for all streams to be set at 60 Hz. vided to a single display panel 12. For purposes of this dis Thus, one stream set at 60.14 Hz and one stream set at 59.86 closure, the multiple streams would operate on different HZ would both be set to 60 Hz. Individually, the two streams stream specifications. differ by just less than /2 of a percent. However, each stream 0016 While the stream specifications state things such as would be moved less than 4 of a percent. Again, it should be refresh rate and resolution, the protocols and panels have a appreciated that the limit on how far a signal will be allowed tolerance around the prescribed specification values that will to be moved is dictated by the ability to ensure that such still allow operation of panel 12 in a way that is acceptable to movement will not violate the underlying specification for the a user. Accordingly, for signals with a 60 HZ rate speci signal. Additionally, embodiments are envisioned where Syn fication, it is not uncommon to find signals actually operating chronization is performed between streams having refresh at rates such as at 59.94 Hz or 59.95 Hz. As discussed previ rates operating at multiples of each other (signals at 60Hz, and ously, when viewed alone, panels 12 being Supplied by sig 120 Hz). nals operating at a 59.94 Hz (or 59.95 Hz) refresh rate are not 0020. Once the refresh rates of the signals are harmonized, distinguishable and do not provide a noticeable reduction in CPU 13 requires that the controllers 16 all start transmission quality compared to the prescribed 60 Hz. However, when at the same time, block 350 of FIG. 4. Thus, for a set of viewed simultaneously, after continuous operation over time, streams, it is known that they started at the same time and that a signal at 59.94 Hz, shows a noticeable lack of synchronicity they are progressing at the same rate. While the above causes with a signal at 59.95 Hz. all eligible controllers 16 to run at approximately the same 0017 Device 10 includes CPU13 that, during video setup, rate, the rate may not be identical. Also, it is still possible that via a driver for a graphics board 15, finds the display path events that introduce occurrences of dis-synchronicity will (including display controller 16) that exhibits a signal that happen. most closely approximates a desired specification. This dis 0021. As a display is rasterized to write a frame, such play path is chosen and designated as the “master' or “refer occurrences of dis-synchronicity (and the original dis-syn ence' path. Alternatively, CPU 13 chooses the display path chronicity) result in different being drawn at the same having the most in common with other display paths as the time on respective panels 12. For so long as the dis-synchro master path. (For example, if there are four display paths, with nicity is Small, it is not very noticeable. Thus, in addition to one operating at 60 Hz, two operating at 59.95 Hz, and one providing that the signals start offin a synchronous fashion, operating at 59.94 Hz, one of the paths operating at 59.95 Hz hardware and software is provided that continue to check and may be used as the master.) In yet another alternative, choice ensure that the synchronicities are maintained. of the master path is determined by the protocol being 0022. Thus, while the signals are active and running, the applied. For example, HDMI is most favored. Thus, if an signals are monitored as indicated by block 400 of FIG. 3. In HDMI protocol is to be used on one of the signal paths, the one embodiment, this monitoring takes the form of a thread. HDMI path is chosen as the master. Thus, for protocols, there At a given point in time, the position of the master stream is is an established hierarchy to progress in the determination of determined along with the position of any slave streams. With the master path. An exemplary hierarchy is HDMI, Display respect to "position, one way of determining this includes Port, DVI, VGA. Regardless of the method, data streams are determining which pixel is being output or writtento a display analyzed and compared via compare module 20 (or other at the given instant. The GPU monitors the position of each wise), block 320 of FIG. 2, and a master stream is established, stream at multiple instances and determines differences (del block 330 of FIG. 2. tas) between each slave stream and the master stream. Once 0018 For each display path other than the designated mas multiple stream positions overtime are recorded, the data can ter path, the associated rate generators of display controllers be analyzed to determine any difference in position (in pixels) 16 are reprogrammed to approximately the rate of the master between streams, block 410 of FIG.3, the speed of the change signal, block 340 of FIG. 2. This is achieved by CPU 13 in the position difference (whether the difference is getting writing to pixel rate register 30 of GPU14 to modify the pixel larger or smaller, and how fast), block 420 of FIG. 3, and an rate. It should be appreciated that some signals may operate at average acceleration of the change in the difference. rates that are very different. The functionality and “synchro 0023 Given the computed information regarding the delta nization” described herein is operable to harmonize signals between streams, the controller of the slave stream operates to that operate reasonably close to one another. By way of alter the slave stream to keep the difference between the slave US 2014/0098849 A1 Apr. 10, 2014

stream and the master stream to be less than one line worth of executable code stored in non-transitory memory such as pixels, block 430 of FIG. 3. If the slave stream gets ahead of RAM, ROM or other suitable memory inhardware descriptor the master stream, the rate of the slave stream is slightly languages such as, but not limited to, RTL and VHDL or any lowered to slowly pull the slave stream back to the master other suitable format. The executable code when executed stream, block 440 of FIG. 3. Likewise, if the slave stream gets may cause an integrated fabrication system to fabricate an IC behind the master stream, the rate of the slave stream is with the operations described herein. slightly raised to slowly push the slave stream back towards 0029. Also, design systems/integrated the master stream, block 440 of FIG.3. Once the slave stream fabrication systems (e.g., work stations including, as known is pushed back into synchronicity (or apparent synchronicity) in the art, one or more processors, associated memory in with the master stream, controller 16 for the slave stream communication via one or more buses or other Suitable inter again puts the stream at the approximately same rate as the connect and other known peripherals) are known that create master Stream. wafers with integrated circuits based on executable instruc 0024. The delta between streams can also be determined a tions stored on a computer readable medium Such as, but not different way. If the display is to be refreshed 60 times per limited to, CDROM, RAM, otherforms of ROM, hard drives, second, and there are 960 lines in a frame, then it should take distributed memory, etc. The instructions may be represented approximately 1.7361e–5 seconds to write a line. For any by any Suitable language Such as, but not limited to, hardware given pixel, CPU 13 and display controller 16 alter the descriptor language (HDL), Verilog or other Suitable lan streams to ensure that the pixel is delivered to screens (or at guage. As such, the logic, Software and circuits described least graphics card outputs) of master and slave within herein may also be produced as integrated circuits by Such 1.7361e–5 seconds of each other. Regardless of how it is systems using the computer readable medium with instruc calculated, the concept is the same. Furthermore, the use of tions stored therein. For example, an integrated circuit with “one line' as the boundary can be adjusted as desired if it is the aforedescribed software, logic and structure may be cre found that greater tolerances are permissible. ated using Such integrated circuit fabrication systems. In Such 0025. Additionally, if the signals are synchronized, then a system, the computer readable medium stores instructions the rate of sampling can likewise be lowered, block 360 of executable by one or more integrated circuit design systems FIG. 4. Video signals have active periods and blank periods. that causes the one or more integrated circuit design systems Samplings need to read from the active periods, but not the to produce an integrated circuit. blank periods. With synchronized signals, the blank periods What is claimed is: are also synchronized such that clock speeds during blank 1. A method of providing data output including: periods can be lowered without concern that needed data will generating a first data stream and a second data stream, the be missed. Without synchronized signals, the sampling rate streams being generated by a first device having a pro must be maintained at the high rate because at least one of the cessor, the first data stream following a first protocol and signals is likely to be in an active region at any given time. By the second data stream following a second protocol that lowering the sampling rate (memory clock), this process is different from the first protocol; and requires less resources from CPU 13 and PC 10 generally. The adjusting a data rate of the second data stream to cause a embodiments herein thus provide energy savings for mobile reduction in any temporal offset between the first data in devices, battery powered devices, and otherwise. the first stream and the first data in the second stream. 0026. The above provides synchronization regardless of 2. The method of claim 1, wherein both the first and second the size of the timing frame, pixel clock, spread spectrum data streams transmit the same data. settings and color depth. It should be further appreciated that the above system is self-stabilizing. It should also be appre 3. The method of claim 1, further including: ciated that if synchronization of all streams is not possible, obtaining the first data stream in a first format having a first embodiments are envisioned where Sub-groups are created in clock speed; which sets of synchronize-able streams are synchronized obtaining the second data streamina second format having while the Sub-groups are not synchronized with each other. a second clock speed that is different than the first clock 0027. The above detailed description and the examples speed; and described herein have been presented for the purposes of adjusting the second clock speed to approximate the first illustration and description only and not for limitation. For clock speed. example, the operations described may be done in any Suit 4. The method of claim3, further including causing the first able manner. The method may be done in any suitable order data stream and second data stream to begin transmission still providing the described operation and results. It is there simultaneously. fore contemplated that the present embodiments cover any 5. The method of claim3, wherein the first and second data and all modifications, variations or equivalents that fall within streams are video streams. the spirit and scope of the basic underlying principles dis 6. The method of claim 5, wherein the first and second closed above and claimed herein. Furthermore, while the streams each follow different protocols selected from the above description describes hardware in the form of a pro group of HDMI, DVI, DisplayPort, and VGA. cessor executing code, hardware in the form of a state 7. The method of claim3, further including determining if machine, or dedicated logic capable of producing the same the first clock speed is within a predetermined range of the effect are also contemplated. second clock speed and only adjusting the rate of the second 0028. The software operations described herein can be data stream if the first clock speed is within a predetermined implemented inhardware such as discrete logic fixed function range of the second clock speed. circuits including but not limited to state machines, field 8. The method of claim 1, further including: programmable gate arrays, application-specific circuits or determining if the temporal offset is of a magnitude that other suitable hardware. The hardware may be represented in causes one of the first and second streams to output both US 2014/0098849 A1 Apr. 10, 2014

a first line and second line before the other of the first and 19. The apparatus of claim 16, wherein the processor is second streams outputs the first line. further operable to determine if the temporal offset equates to 9. The method of claim 1, further determining if the tem an offset of a number of pixels greater than the number of poral offset equates to an offset of a number of pixels greater pixels in a row of a frame of the first data. than the number of pixels in a row of a frame of the first data. 20. The apparatus of claim 16, wherein the processor and 10. The method of claim 1, further including receiving the second display controller are part of battery powered device. data stream, receiving the second data stream, displaying the 21. The apparatus of claim 16, wherein the processor is first data stream, and displaying the second data stream. further operable to compare the first data stream to the second 11. The method of claim 10, wherein the first data stream is displayed on a first display and the second data stream is data stream to determine the degree to which transmission of displayed on a second display. the first data in each of the streams is temporally offset. 12. The method of claim 1, further including comparing, by 22. The apparatus of claim 21, wherein the processor is the processor, the first data stream to the second data stream to further operable to compare the first data stream to the second determine the degree to which transmission of the streams is data stream to obtain multiple samplings of the first and temporally offset. second data streams, the samplings providing data indicating 13. The method of claim 14, wherein comparing the first whether the temporal offset between the first and second data stream to the second data stream includes multiple sam streams is increasing or decreasing. plings that provide data indicating whether the temporal off 23. A computer readable medium containing non-transi set between the first and second streams is increasing or tory instructions thereon, that when interpreted by at least one decreasing. processor cause the at least one processor to: 14. The method of claim 13, further including determining compare a first data stream sending first data to a second that the first and second data streams, given their current data stream sending the first data to determine the degree settings, are within a threshold time of having a first condition to which transmission of the first data in each of the where one of the first and second streams outputs both a first streams is temporally offset, the first data stream follow line and second line before the other of the first and second ing a first protocol and the second data stream following streams outputs the first line. a second protocol that is different from the first protocol; 15. The method of claim 14, wherein the second data rate is and adjusted in response to determining that the first and second adjust a data rate of the second data stream to cause a data streams are within the threshold time of having the first reduction in any temporal offset between the streams. condition. 16. An apparatus including: 24. The computer readable medium of claim 23, wherein a processor operable to generate a first data stream sending the instructions further cause the processor to: first data and to generate a second data stream sending generate a first data stream sending first data and a second the first data, the first data stream following a first pro data stream sending the first data; tocol and the second data stream following a second compare the first data stream to the second data stream to protocol that is different from the first protocol; and determine the degree to which transmission of the first a second display controller operable to adjust a data rate of data in each of the streams is temporally offset; and the second data stream to cause a reduction in any tem adjust a data rate of the second data stream to cause a poral offset between the streams. reduction in any temporal offset between the streams. 17. The apparatus of claim 16, further including: 25. The computer readable medium of claim 23, wherein a first display controller operable to provide the first data the instructions further cause the processor to: stream in a first format having a first clock speed; the determine whether the temporal offset between the first second display controller operable to provide the second and second streams is increasing or decreasing. data stream in a second format having a second clock speed that is different than the first clock speed. 26. The computer readable medium of claim 23, wherein 18. The apparatus of claim 17, wherein the first and second the instructions are embodied in hardware description lan display controllers each output data streams that follow dif guage Suitable for one or more of describing, designing, orga ferent protocols selected from the group of HDMI, DVI, nizing, fabricating or verifying hardware. DisplayPort, and VGA. k k k k k