Nosql 的飞跃— 使用mysql 更加安全可靠便利地管理JSON 数据
Total Page:16
File Type:pdf, Size:1020Kb
NoSQL的飞跃 使用MySQL 更加安全可靠便利地管理JSON 数据 马楚成 (Ivan Ma) 周彦伟 MySQL 亚太区资深方案工程师 极数云舟CEO [email protected] Oracle MySQL ACE Director 2019-04-16 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 2 Outline 1 MySQL作为文档存储 (Ivan) 2 使用 X DevAPI / Connector (Ivan) 3 接管NoSQL,MySQL 管理平台设计和案例 (周彦伟) 4 总结 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 3 2018年4月19日 2018年7月27日 2018年10月22日 8.0.11 GA 8.0.12 发布 8.0.13 发布 2019年2月1日 8.0.15 发布 8.0.16 …… 8.0 The world's most popular open source database Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | MySQL作为文档存储 使用Document Store and X-DevAPI Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 5 NoSQL的魅力 • 越来越多 用户/公司 开始尝试NoSQL 对 RDBMS 的疑惑 后来发现NoSQL 的不足, 限制和问题 RDBMS扩展性低 缺乏“参照完整性””Referential Integrity” 开发人员不喜欢SQL ,等等 缺乏ACID支持 • 结果:SQL + NoSQL - 对传统数据库上开发NoSQL接口(例如 Uber使用Schemaless) - 开发存储引擎,实现NoSQL(例如 Facebook 使用MyRocks /RocksDB) - 使用一些支持JSON的 RDBMS Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Relational Databases vs Document Store Systems Relational Model Table Document Model customer id name email city_id { _id: 3412, Document 3412 John Smith [email protected] 45 ROW name: "John Smith", {date: "2017-10-02", total: 24.95} city ] } CO id city country_id LL 45 San Francisco US ROW ECT I { _id: 3412, O { _id: 3412, Document N na{ me:_id: "Joh 341n2 S, mith", Document shop_order {dnaate:{ me:_i "2d: "Joh0 31471-n102 S, -mi02th",",total: 24.95} {dnatame:{ e:_i "20d "Jo: 3147h1-n102S, -mith"02", to, tal: 24.95} id id_customer date total ] } {dnate:ame: "2 "Jo017h-n10S-mi02th",",total: 24.95} ] } {dnaate:me: "2 "Joh017-n10 S-mi02th",",to tal: 24.95} 381 3412 2017-08-24 312.20 ROW ] } ] }{date: "2017-10-02", total: 24.95} 412 3412 2017-10-02 24.95 ROW ] } Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 7 主要组件 App 应用 MySQL X DevAPI [Connector] Router MySQL 连接器 路由 X Plugin X 插件 MySQL Shell Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 8 文档 【Document 】和集合 【Collections】 • 文档在文档型数据库( Docstore )里以 JSON的形式展现 - MySQL完美支持 • 多个文档存储在一个集合里 – 数据库 - InnoDB 表为 Collection Table Collection - JSON 类型为常规列 Row [GenCol, JSON, …] Document [ _id, doc ] - 以虚拟列建索引 Row [ GenCol, JSON, …] Document [ _id, doc ] - 应用 - X-DevAPI - 管理 - MySQL Shell Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 9 Document Store with MySQL JSON Datatype CREATE TABLE employees (data JSON); INSERT INTO employees VALUES ('{"id": 1, "name": "Jane"}'), ('{"id": 2, "name": "Joe"}'); SELECT * FROM employees; +-------------------------------------+ | data | +-------------------------------------+ | {"id": 1, "name": "Jane"} | | {"id": 2, "name": "Joe"} | +-------------------------------------+ Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 10 MySQL Connectors include X DevAPI • Use SQL, CRUD APIs – Document (NoSQL) and Relational (SQL), or “All of the Above” – All of this is in addition to the Classic APIs Operation Document Relational Create Collection.add() Table.insert() Read Collection.find() Table.select() Update Collection.modify() Table.update() Delete Collection.remove() Table.delete() Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | . COMMIT和 SAVEPOINT 的支持 • 在会话范围内,提供原子性操作 – 在会话范围内创建, try { 提交或回滚事务 session.startTransaction() // run some operations (1) – 在这些事务中创建, session.createSavepoint('foo') 释放或回滚到中间保存点 // run more operations (2) session.releaseSavepoint('foo') session.commit() } catch (err) { try { session.rollbackTo('foo') // go to (2) } catch (err) { session.rollback() // revert the entire thing } } Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 12 NoSQL + 执行 SQL • 对于ETL ,结构化分析和报表,SQL始终是一种选择 • 允许使用尚未成为X DevAPI 一部分的功能,例如: - 关系表的DDL操作 - 管理键约束 session.sql( ‘< SQL Statement> ‘).execute - 使用JOIN // create a table session.sql('CREATE TABLE foo (bar VARCHAR(3))').execute() // add a unique constraint session.sql('ALTER TABLE foo ADD COLUMN bar VARCHAR(3) GENERATED ALWAYS AS doc->>"$.bar" VIRTUAL UNIQUE KEY NOT NULL').execute() // execute a JOIN query session.sql('SELECT DISTINCT t1.bar FROM foo t1 JOIN baz t2 ON t1.bar = t2.qux WHERE t1.qux = t2.quux').execute() Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 13 集合 [Collection]的使用 NodeJS // Java collection .find("name = 'foo' AND age > 42") const mysqlx = require('@mysql/xdevapi'); .fields("name", "age") .groupBy("name", "age") (async function () { .sort("name ASC", "age DESC") try { const session = await mysqlx.getSession({ user: 'root' }); .limit(4) const schema = session.getSchema('products') .offset(2) .execute() const collection = await schema.createCollection('servers', { ReuseExistingObject : true }); console.log(collection); // { schema: 'products', collection: 'servers' } const collections = await schema.getCollections(); console.log(collections); // [{ schema: 'products', collection: 'servers' }] # Python await session.close(); collection \ .find("name = 'foo' AND age > 42") \ process.exit(); .fields("name", "age") \ } catch (err) { console.error(err.message); .group_by("name", "age") \ process.exit(1); .sort("name ASC", "age DESC") \ } .limit(4) \ })(); .offset(2) \ .execute() Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 14 跨集合的原子更新 const mysqlx = require('@mysql/xdevapi'); (async function () { let session, release; try { session = await mysqlx.getSession({ user: 'root' }); const db = session.getSchema('products'); await db.createCollection('clients', { ReuseExistingObject: true }); await session.startTransaction(); await db.getCollection('servers').find('version = 8.0 AND revision = 11').lockExclusive(mysqlx.LockContention.DEFAULT) .execute(doc => { release = doc.release; }); await db.getCollection('clients').add({ name: 'c-nodejs' }).execute(); await db.getCollection('clients').modify('true').set('serverType', release).execute(); await session.commit(); await session.close(); process.exit(); } catch (err) { if (session) { await session.rollback(); } console.error(err.message); process.exit(1); } })(); Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 15 接管NoSQL MySQL管理平台设计和案例 周彦伟 极数云舟CEO Oracle MySQL ACE Director Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 17 MySQL管理JSON的优势 • 可靠的MySQL 整体技术框架 • MySQL Server+InnoDB • 完备的关系数据库核心功能 • REDO、 UNDO、 MVCC 、Transaction、 Isolation… • 丰富的技术资源和技术人才 • ACMUG /《 MySQL 运维内参》《MySQL 8 Cookbook 》 • 成熟的管理平台和运维方案 • Arkcontrol/Arkgate/Arkproxy/Arksentinel Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 18 MySQL管理平台功能 • 安装部署 • 监控告警 • 备份恢复 • 操作变更 • 审核审计 • 资源管理 高可用切换 • Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 19 实现案例:Arkcontrol • Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 20 Title • Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 21 总结 • 简化 并 保护数据库 的 安装 • 利用广泛和成熟MySQL技术 • 达到双赢 - RDBMS 和NoSQL 的好处 • InnoDB 集群为 高可用 的第一级别方案 • 标准化和简单的Node.js工作流程( async,npm) • 减小对ORM和其他无关的中间件的依赖 • Node.js 优越的部署场景(容器,微服务,无服务器和FaaS ,桌面,物联网等) • 通过利用标准API,避免平台锁定 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 22 链接和资源 • https://github.com/mysql/mysql-connector-nodejs • https://www.npmjs.com/package/@mysql/xdevapi • https://dev.mysql.com/doc/refman/8.0/en/document-store.html • https://dev.mysql.com/doc/x-devapi-userguide/en/ • https://dev.mysql.com/doc/dev/connector-nodejs/8.0/ • https://ruiquelhas.github.io/percona-live-europe-2017/ • https://www.mysql.com/news-and-events/web-seminars/mysql-document- store-and-node-js/ • https://insidemysql.com/mysql-document-store-crud-quick-start/ Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 23 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | 24 .