DOCSLIB.ORG

Semantic Equivalence Checking for HHVM Bytecode

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Semantic Equivalence Checking for HHVM Bytecode Semantic Equivalence Checking for HHVM Bytecode Nick Benton Facebook London [email protected] ABSTRACT But what do we mean by ‘the same’? Just running diff on the We describe a semantic differencing tool used to compare the byte- output of the two compilers requires literal textual equality, which codes generated by two different compilers for Hack/PHP at Face- is too strict a requirement. Two compilers for the same language book. The tool is a prover for a simple relational Hoare logic for will naturally produce code that varies in inessential details, such low-level code and is used in testing, allowing the developers to as the use of local variables, ordering of basic blocks, etc. Trying to focus on semantically significant differences between the outputs make a new code generator (written in OCaml) agree on all such of the two compilers. details with an existing one (written in C++) would both be unnec- essary work and create technical debt in the form of contorted code. CCS CONCEPTS The unappealing alternative is manually checking huge diff out- puts and attempting to classify individual differences as significant • Theory of computation → Logic and verification; Hoare or unimportant. What we really wanted was a mechanical way to logic; Invariants; Pre- and post-conditions; • Software and check that the two compilers produce code that is behaviourally its engineering → Software verification; Automated static equivalent, i.e. produces the same observable behaviour in all con- analysis; Compilers; Software testing and debugging; Object texts. (We also do not want performance regressions, but ignore oriented languages; that aspect of testing here.) This paper describes an automated ACM Reference Format: tool, semdiff, that we built to check HHAS files for behavioural Nick Benton. 2018. Semantic Equivalence Checking for HHVM Bytecode. equivalence. In The 20th International Symposium on Principles and Practice of Declarative Behavioural equivalence is obviously undecidable, and writing a Programming (PPDP ’18), September 3–5, 2018, Frankfurt am Main, Germany. ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3236950.3236975 sound-but-incomplete fully automatic analysis that can cope with significant equivalence-preserving differences, such as those be- 1 INTRODUCTION tween optimized and unoptimized code, is generally extremely hard. However, in our case, the two compilers already produced (or were HHVM, the HipHop Virtual Machine [14], is a high performance, supposed to produce) ‘essentially’ the same code. Thus we were JIT-compiled implementation of PHP. HHVM generates the Face- able to get good results with a fairly simple, fast analysis, that book site, as well as being used by Wikipedia, Slack, and others. concentrates on ‘trivial’ differences. Most of the code running on HHVM at Facebook is now written semdiff is essentially a prover for a restricted version of Rela- in Hack, a new language based on PHP, whose features include an tional Hoare Logic (RHL) for low-level code. It was integrated into expressive gradual type system, improved collections, lambdas, and the test process for HackC, running over the compiled code from support for asynchronous programming [16]. Hack was originally millions of lines of Hack and PHP, and freed the HackC team to implemented using a combination of an independent typechecker concentrate on eliminating significant differences in the output of plus some extensions to HHVM’s PHP front-end. We have now the two compilers. built HackC, a new compiler for Hack. HackC emits HHAS, which is a textual form of HHBC, the HipHop ByteCode already used as an intermediate language inside HHVM. 1.1 semdiff architecture Moving to an entirely new compiler for Hack code is not some- semdiff takes two hhas files as input. These are each parsed in thing to be undertaken lightly. The existing HHVM implementation an unsurprising way into the representation of hhbc that is used is mature and well-tested, and we needed to have a similar level by HackC. There are around 20 OCaml record types defining the of confidence in the new compiler. A conservative decision was nested structure of an bytecode file. For example, a program has a therefore taken that, in the first instance, the new compiler should list of literal array constants, a list of top-level function definitions, a produce the same target code as the old one for our existing code- list of class definitions, a list of type definitions, and a distinguished base and tests. main function. A function has a list of attributes, a name, a body, Permission to make digital or hard copies of all or part of this work for personal or and a collection of boolean flags saying if it is async, a generator, classroom use is granted without fee provided that copies are not made or distributed and so on. Some of the record types have a non-trivial number of for profit or commercial advantage and that copies bear this notice and the full citation fields (e.g. a class currently has 24) and for nearly all of them, we on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, just want to check for literal equality, set equality, or map equality. to post on servers or to redistribute to lists, requires prior specific permission and/or a We reduce the pain of writing the associated boilerplate by using fee. Request permissions from [email protected]. combinators: PPDP ’18, September 3–5, 2018, Frankfurt am Main, Germany © 2018 Association for Computing Machinery. type 'a compare = { ACM ISBN 978-1-4503-6441-6/18/09...$15.00 https://doi.org/10.1145/3236950.3236975 comparer : 'a -> 'a -> int * (int * edit_sequence); PPDP ’18, September 3–5, 2018, Frankfurt am Main, Germany Nick Benton size_of : 'a -> int; iterators, and memo tables, two kinds of exception handler, and string_of : 'a -> string instructions for dynamically loading classes and functions. } As well as the stacks, hhas allows the use of named locals, such $x _3 An 'a compare comprises a comparison function for values of as , and ‘unnamed’ numbered temporaries, such as . If v is a SetL type 'a, a function that computes a rough indication of the size variable, then the instruction v copies the value on the top PopC of values of type 'a, and a string_of function. The comparison of the stack into v (without popping), discards the value on CGetL function returns an edit distance (which is zero if the two values the top of the stack, and v pushes the contents of v onto are appropriately equal), a size, and an edit sequence, which can the evaluation stack. (We will describe other instructions as we be thought of as a string representation of the delta between the encounter them.) two values. Values of type 'a compare are built compositionally Figure 1 shows an example of some rather artificial Hack source according to the structure of 'a and the desired notion of equality. code. Two different bytecode sequences that might correspond to For example, there is a generic comparer for 'a lists that takes a that source are shown in Figure 2. comparer for 'a as argument, and computes the Levenshtein edit One can see that the two target files differ in their control flow, distance between the two lists. There is also a join combinator with simpler exception handling on the right, and different labels $pairs that takes two (independent) comparers for the same type and and jumps. For example, if is empty then the iterator created L0 L2 combines them into one that does both comparisons, and so on. The on the left on line 12 will jump to and then to before returning, L4 combinators are used to build up a value of type program compare, whereas the corresponding one on the right will just jump to _6 and it is the result of the associated comparer that is returned by and return. The uses of local variables are also different. Local on semdiff. the left, which stores the current value of the iterator, corresponds _8 _7 The interesting comparer is that for the lists of instructions that to local on the right. More subtly, local on the left, holding $f _8 are part of the bodies of functions and methods. Here we do not the result of the call to , also corresponds to local on the right. want to use our generic list comparer, as that would just yield a syn- Other differences include the reversal of the tests on line 50 onthe UnsetL tactic, rather than a semantic, comparison. Instead, we implement left and line 44 on the right, and the way instructions on a comparer that tries to prove that two sequences of instructions the left are not matched on the right. have the same observable behaviour under the assumption that Nevertheless, most of the instructions do correspond in a fairly they are started in equivalent contexts (comprising the rest of the simple way, and the two implementations are semantically equiva- program, the parameters passed in, etc.). The intention is that this lent. comparison is sound: if it reports that two bodies are equivalent then they really are according to the (intended) semantics of the bytecode. If the comparison fails, then it might be that there is an 3 RELATIONAL HOARE LOGIC FOR observable difference, or it might be that our analysis is too weak LOW-LEVEL CODE to establish equivalence. In such a case, we report the state of the Traditional Hoare logic for structured programs works with judge- prover when it got stuck and the syntactic difference between the ments ` fPg C fQg, meaning that if command C is started in a state two sequences, and hand over to a human being to decide whether satisfying the precondition P then if it terminates, it does so in a this is a false positive or not.1 state satisfying postcondition Q.
Load more

Recommended publications
  • 16 Inspiring Women Engineers to Watch
    Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year. https://hackbrightacademy.com 16 Inspiring Women Engineers To Watch Women's engineering school Hackbright Academy is excited to share some updates from graduates of the software engineering fellowship. Check out what these 16 women are doing now at their companies - and what languages, frameworks, databases and other technologies these engineers use on the job! Software Engineer, Aclima Tiffany Williams is a software engineer at Aclima, where she builds software tools to ingest, process and manage city-scale environmental data sets enabled by Aclima’s sensor networks. Follow her on Twitter at @twilliamsphd. Technologies: Python, SQL, Cassandra, MariaDB, Docker, Kubernetes, Google Cloud Software Engineer, Eventbrite 1 / 16 Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year. https://hackbrightacademy.com Maggie Shine works on backend and frontend application development to make buying a ticket on Eventbrite a great experience. In 2014, she helped build a WiFi-enabled basal body temperature fertility tracking device at a hardware hackathon. Follow her on Twitter at @magksh. Technologies: Python, Django, Celery, MySQL, Redis, Backbone, Marionette, React, Sass User Experience Engineer, GoDaddy 2 / 16 Hackbright Academy Hackbright Academy is the leading software engineering school for women founded in San Francisco in 2012. The academy graduates more female engineers than UC Berkeley and Stanford each year.
    [Show full text]
  • Memory Diagrams and Memory Debugging
    CSCI-1200 Data Structures | Spring 2020 Lab 4 | Memory Diagrams and Memory Debugging Checkpoint 1 (focusing on Memory Diagrams) will be available at the start of Wednesday's lab. Checkpoints 2 and 3 focus on using a memory debugger. It is highly recommended that you thoroughly read the instructions for Checkpoint 2 and Checkpoint 3 before starting. Memory debuggers will be a useful tool in many of the assignments this semester, and in C++ development if you work with the language outside of this course. While a traditional debugger lets you step through your code and examine variables, a memory debugger instead reports memory-related errors during execution and can help find memory leaks after your program has terminated. The next time you see a \segmentation fault", or it works on your machine but not on Submitty, try running a memory debugger! Please download the following 4 files needed for this lab: http://www.cs.rpi.edu/academics/courses/spring20/csci1200/labs/04_memory_debugging/buggy_lab4. cpp http://www.cs.rpi.edu/academics/courses/spring20/csci1200/labs/04_memory_debugging/first.txt http://www.cs.rpi.edu/academics/courses/spring20/csci1200/labs/04_memory_debugging/middle. txt http://www.cs.rpi.edu/academics/courses/spring20/csci1200/labs/04_memory_debugging/last.txt Checkpoint 2 estimate: 20-40 minutes For Checkpoint 2 of this lab, we will revisit the final checkpoint of the first lab of this course; only this time, we will heavily rely on dynamic memory to find the average and smallest number for a set of data from an input file. You will use a memory debugging tool such as DrMemory or Valgrind to fix memory errors and leaks in buggy lab4.cpp.
    [Show full text]
  • Automatic Detection of Uninitialized Variables
    Automatic Detection of Uninitialized Variables Thi Viet Nga Nguyen, Fran¸cois Irigoin, Corinne Ancourt, and Fabien Coelho Ecole des Mines de Paris, 77305 Fontainebleau, France {nguyen,irigoin,ancourt,coelho}@cri.ensmp.fr Abstract. One of the most common programming errors is the use of a variable before its definition. This undefined value may produce incorrect results, memory violations, unpredictable behaviors and program failure. To detect this kind of error, two approaches can be used: compile-time analysis and run-time checking. However, compile-time analysis is far from perfect because of complicated data and control flows as well as arrays with non-linear, indirection subscripts, etc. On the other hand, dynamic checking, although supported by hardware and compiler tech- niques, is costly due to heavy code instrumentation while information available at compile-time is not taken into account. This paper presents a combination of an efficient compile-time analysis and a source code instrumentation for run-time checking. All kinds of variables are checked by PIPS, a Fortran research compiler for program analyses, transformation, parallelization and verification. Uninitialized array elements are detected by using imported array region, an efficient inter-procedural array data flow analysis. If exact array regions cannot be computed and compile-time information is not sufficient, array elements are initialized to a special value and their utilization is accompanied by a value test to assert the legality of the access. In comparison to the dynamic instrumentation, our method greatly reduces the number of variables to be initialized and to be checked. Code instrumentation is only needed for some array sections, not for the whole array.
    [Show full text]
  • Memory Debugging
    Center for Information Services and High Performance Computing (ZIH) Memory Debugging HRSK Practical on Debugging, 03.04.2009 Zellescher Weg 12 Willers-Bau A106 Tel. +49 351 - 463 - 31945 Matthias Lieber ([email protected]) Tobias Hilbrich ([email protected]) Content Introduction Tools – Valgrind – DUMA Demo Exercise Memory Debugging 2 Memory Debugging Segmentation faults sometimes happen far behind the incorrect code Memory debuggers help to find the real cause of memory bugs Detect memory management bugs – Access non-allocated memory – Access memory out off allocated bounds – Memory leaks – when pointers to allocated areas get lost forever – etc. Different approaches – Valgrind: Simulation of the program run in a virtual machine which accurately observes memory operations – Libraries like ElectricFence, DMalloc, and DUMA: Replace memory management functions through own versions Memory Debugging 3 Memory Debugging with Valgrind Valgrind detects: – Use of uninitialized memory – Access free’d memory – Access memory out off allocated bounds – Access inappropriate areas on the stack – Memory leaks – Mismatched use of malloc and free (C, Fortran), new and delete (C++) – Wrong use of memcpy() and related functions Available on Deimos via – module load valgrind Simply run program under Valgrind: – valgrind ./myprog More Information: http://www.valgrind.org Memory Debugging 4 Memory Debugging with Valgrind Memory Debugging 5 Memory Debugging with DUMA DUMA detects: – Access memory out off allocated bounds – Using a
    [Show full text]
  • Three Architectural Models for Compiler-Controlled Speculative
    Three Architectural Mo dels for Compiler-Controlled Sp eculative Execution Pohua P. Chang Nancy J. Warter Scott A. Mahlke Wil liam Y. Chen Wen-mei W. Hwu Abstract To e ectively exploit instruction level parallelism, the compiler must move instructions across branches. When an instruction is moved ab ove a branch that it is control dep endent on, it is considered to b e sp eculatively executed since it is executed b efore it is known whether or not its result is needed. There are p otential hazards when sp eculatively executing instructions. If these hazards can b e eliminated, the compiler can more aggressively schedule the co de. The hazards of sp eculative execution are outlined in this pap er. Three architectural mo dels: re- stricted, general and b o osting, whichhave increasing amounts of supp ort for removing these hazards are discussed. The p erformance gained by each level of additional hardware supp ort is analyzed using the IMPACT C compiler which p erforms sup erblo ckscheduling for sup erscalar and sup erpip elined pro cessors. Index terms - Conditional branches, exception handling, sp eculative execution, static co de scheduling, sup erblo ck, sup erpip elining, sup erscalar. The authors are with the Center for Reliable and High-Performance Computing, University of Illinois, Urbana- Champaign, Illinoi s, 61801. 1 1 Intro duction For non-numeric programs, there is insucient instruction level parallelism available within a basic blo ck to exploit sup erscalar and sup erpip eli ned pro cessors [1][2][3]. Toschedule instructions b eyond the basic blo ck b oundary, instructions havetobemoved across conditional branches.
    [Show full text]
  • Magento on HHVM Speeding up Your Webshop with a Drop-In PHP Replacement
    Magento on HHVM Speeding up your webshop with a drop-in PHP replacement. Daniel Sloof [email protected] What is HHVM? ● HipHop Virtual Machine ● Created by engineers at Facebook ● Essentially a reimplementation of PHP ● Originally translated PHP to C++, now translates PHP to bytecode ● Just-in-time compiler, turning generated bytecode into machine code ● In some cases 5 to 10 times faster than regular PHP So what’s the problem? ● HHVM not entirely compatible with PHP ● Magento’s PHP triggering many of these incompatibilities ● Choosing between ○ Forking Magento to work around HHVM ○ Fixing issues within the extensive HHVM C++ codebase Resulted in... fixing HHVM ● Already over 100 commits fixing Magento related HHVM bugs; ○ SimpleXML (majority of bugfixes) ○ sessions ○ number_format ○ __get and __set ○ many more... ● Most of these fixes already merged back into the official (github) repository ● Community Edition running (relatively) stable! Benchmarks Before we go to the results... ● Magento 1.8 with sample data ● Standard Apache2 / php-fpm / MySQL stack (with APC opcode cache). ● Standard HHVM configuration (repo-authoritative mode disabled, JIT enabled) ● Repo-authoritative mode has potential to increase performance by a large margin ● Tool of choice: siege Benchmarks: Response time Average across 50 requests Benchmarks: Transaction rate While increasing siege concurrency until avg. response time ~2 seconds What about <insert caching mechanism here>? ● HHVM does not get in the way ● Dynamic content still needs to be generated ● Replaces PHP - not Varnish, Redis, FPC, Block Cache, etc. ● As long as you are burning CPU cycles (always), you will benefit from HHVM ● Think about speeding up indexing, order placement, routing, etc.
    [Show full text]
  • GC Assertions: Using the Garbage Collector to Check Heap Properties
    GC Assertions: Using the Garbage Collector to Check Heap Properties Edward Aftandilian Samuel Z. Guyer Tufts University Tufts University 161 College Ave 161 College Ave Medford MA Medford MA [email protected] [email protected] ABSTRACT The goal of this work is to develop a general mechanism, which GC assertion ex- This paper introduces GC assertions, a system interface that pro- we call a , that allows the programmer to express pected grammers can use to check for errors, such as data structure in- properties of data structures and convey this information to variant violations, and to diagnose performance problems, such as the garbage collector. The key is that many useful, non-trivial prop- memory leaks. GC assertions are checked by the garbage collec- erties can be easily checked by the garbage collector at GC time, tor, which is in a unique position to gather information and answer imposing little or no run-time overhead. The collector triggers the questions about the lifetime and connectivity of objects in the heap. assertion if it finds that the expected properties have been violated. We introduce several kinds of GC assertions, and we describe how While GC assertions require extra programmer effort, they cap- they are implemented in the collector. We also describe our report- ture application-specific properties that programmers want to check ing mechanism, which provides a complete path through the heap and can easily express. For example, we provide a GC assertion to to the offending objects. We show results for one type of asser- verify that a data structure is reclaimed at the next collection when tion that allows the programmer to indicate that an object should the programmer expects it to be garbage.
    [Show full text]
  • Automated Program Transformation for Improving Software Quality
    Automated Program Transformation for Improving Software Quality Rijnard van Tonder CMU-ISR-19-101 October 2019 Institute for Software Research School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Claire Le Goues, Chair Christian Kästner Jan Hoffmann Manuel Fähndrich, Facebook, Inc. Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Software Engineering. Copyright 2019 Rijnard van Tonder This work is partially supported under National Science Foundation grant numbers CCF-1750116 and CCF-1563797, and a Facebook Testing and Verification research award. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring corporation, institution, the U.S. government, or any other entity. Keywords: syntax, transformation, parsers, rewriting, crash bucketing, fuzzing, bug triage, program transformation, automated bug fixing, automated program repair, separation logic, static analysis, program analysis Abstract Software bugs are not going away. Millions of dollars and thousands of developer-hours are spent finding bugs, debugging the root cause, writing a patch, and reviewing fixes. Automated techniques like static analysis and dynamic fuzz testing have a proven track record for cutting costs and improving software quality. More recently, advances in automated program repair have matured and see nascent adoption in industry. Despite the value of these approaches, automated techniques do not come for free: they must approximate, both theoretically and in the interest of practicality. For example, static analyzers suffer false positives, and automatically produced patches may be insufficiently precise to fix a bug.
    [Show full text]
  • Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK
    1 From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis Mark Harman∗, Peter O’Hearn∗ ∗Facebook London and University College London, UK Abstract—This paper1 describes some of the challenges and research questions that target the most productive intersection opportunities when deploying static and dynamic analysis at we have yet witnessed: that between exciting, intellectually scale, drawing on the authors’ experience with the Infer and challenging science, and real-world deployment impact. Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, Many industrialists have perhaps tended to regard it unlikely impacting billions of people worldwide. that much academic work will prove relevant to their most The paper identifies open problems that have yet to receive pressing industrial concerns. On the other hand, it is not significant attention from the scientific community, yet which uncommon for academic and scientific researchers to believe have potential for profound real world impact, formulating these that most of the problems faced by industrialists are either as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. boring, tedious or scientifically uninteresting. This sociological phenomenon has led to a great deal of miscommunication between the academic and industrial sectors. I. INTRODUCTION We hope that we can make a small contribution by focusing on the intersection of challenging and interesting scientific How do we transition research on static and dynamic problems with pressing industrial deployment needs. Our aim analysis techniques from the testing and verification research is to move the debate beyond relatively unhelpful observations communities to industrial practice? Many have asked this we have typically encountered in, for example, conference question, and others related to it.
    [Show full text]
  • Heapviz: Interactive Heap Visualization for Program Understanding and Debugging
    Heapviz: Interactive Heap Visualization for Program Understanding and Debugging 1 Abstract Understanding the data structures in a program is crucial to understanding how the program works, or why it doesn’t work. Inspecting the code that implements the data structures, however, is an arduous task and often fails to yield insights into the global organization of a program’s data. Inspecting the actual contents of the heap solves these problems but presents a significant challenge of its own: finding an effective way to present the enormous number of objects it contains. In this paper we present Heapviz, a tool for visualizing and exploring snapshots of the heap obtained from a running Java program. Unlike existing tools, such as tra- ditional debuggers, Heapviz presents a global view of the program state as a graph, together with powerful interactive capabilities for navigating it. Our tool employs sev- eral key techniques that help manage the scale of the data. First, we reduce the size and complexity of the graph by using algorithms inspired by static shape analysis to aggregate the nodes that make up a data structure. Second, we implement a power- ful visualization component whose interactive interface provides extensive support for exploring the graph. The user can search for objects based on type, connectivity, and field values; group objects; and color or hide and show each group. The user may also inspect individual objects to see their field values and neighbors in the graph. These interactive abilities help the user manage the complexity of these huge graphs. By applying Heapviz to both constructed and real-world examples, we show that Heapviz provides programmers with a powerful and intuitive tool for exploring program behavior.
    [Show full text]
  • A Parallel Program Execution Model Supporting Modular Software Construction
    A Parallel Program Execution Model Supporting Modular Software Construction Jack B. Dennis Laboratory for Computer Science Massachusetts Institute of Technology Cambridge, MA 02139 U.S.A. [email protected] Abstract as a guide for computer system design—follows from basic requirements for supporting modular software construction. A watershed is near in the architecture of computer sys- The fundamental theme of this paper is: tems. There is overwhelming demand for systems that sup- port a universal format for computer programs and software The architecture of computer systems should components so users may benefit from their use on a wide reflect the requirements of the structure of pro- variety of computing platforms. At present this demand is grams. The programming interface provided being met by commodity microprocessors together with stan- should address software engineering issues, in dard operating system interfaces. However, current systems particular, the ability to practice the modular do not offer a standard API (application program interface) construction of software. for parallel programming, and the popular interfaces for parallel computing violate essential principles of modular The positions taken in this presentation are contrary to or component-based software construction. Moreover, mi- much conventional wisdom, so I have included a ques- croprocessor architecture is reaching the limit of what can tion/answer dialog at appropriate places to highlight points be done usefully within the framework of superscalar and of debate. We start with a discussion of the nature and VLIW processor models. The next step is to put several purpose of a program execution model. Our Parallelism processors (or the equivalent) on a single chip.
    [Show full text]
  • Nástroje Pro Sjednocení Datových Zdrojů Projektu Gloffer Tools for Unification of Data Sources Project Gloffer
    VŠB – Technická univerzita Ostrava Fakulta elektrotechniky a informatiky Katedra informatiky Nástroje pro sjednocení datových zdrojů projektu Gloffer Tools for unification of data sources project Gloffer 2018 Bc. Jakub Malchárek Rád bych poděkoval panu Ing. Radoslavu Fasugovi, Ph.D. za odbornou pomoc a konzultaci při zpracování této diplomové práce a cenné rady v průběhu implementace. Abstrakt V této diplomové práci se zabývám analýzou dostupných technologií pro implementaci webo- vého portálu Gloffer. Jsou zde popsány databáze (MySQL, Redis, MongoDB, Aerospike, Apache HBase, Apache Cassandra, Google Bigtable, Memcached), vyhledávače (Solr, Lucene, Elastic Search), webové servery (Apache HTTP server, Apache Tomcat), zprostředkovatelé zpráv (Rab- bit MQ), distribuované výpočetní technologie (Apache Hadoop) a vývojové technologie (PHP 7, Nette Framework, Java, Spring Framework). Cílem je nejen popis těchto technologií, ale také ná- vrh a implementace rozhraní pro sjednocení datových zdrojů projektu Gloffer v programovacím jazyce Java s využitím Spring Frameworku. Výstupem práce je inteligentní nástroj zpřístupňující data z více datových zdrojů. Závěr práce obsahuje výkonové testování vyvinutého nástroje. Klíčová slova: Aerospike, Apache Cassandra, Apache Hadoop, Apache HBase, Apache HTTP server, Apache Tomcat, aplikační rozhraní, datové zdroje, Elastic Search, fulltext, Google Bi- gtable, index, Java, Lucene, Memcached, MongoDB, MySQL, Nette Framework, PHP, Rabbit MQ, Redis, REST, Solr, Spring Framework Abstract In this diploma thesis I deal with analysis of the available technologies for implementation of the Gloffer web portal. There are described databases (MySQL, Redis, MongoDB, Aerospike, Apache HBase, Apache Cassandra, Google Bigtable, Memcached), search engines (Solr, Lucene, Elastic Search), web servers (Apache HTTP server, Apache Tomcat), message brokers (Rabbit MQ), distributed computing technologies (Apache Hadoop) and develop technologies (PHP 7, Nette Framework, Java, Spring Framework).
    [Show full text]