Administrivia
• Exams should be returned by next week Database Application – Grades might be on-line by tomorrow Development • Homework 3 should be available next week R&G Chapter 6 – Due date will be adjusted accordingly Lecture 16
Overview: Review How to access DBMSs from programs
• Last time: How DBMSs are used in Web Applications • Static approaches, i.e. SQL in application code – Internet basics: URIs, HTTP – Embedded SQL – Internet file formats: HTML, XML/DTD –SQLJ – Three-Tier Architectures: client, app server, database • Cursors, an important concept • client technologies: HTML, Javascript, Java, Style Sheets • Dynamic Approaches • app server tech: CGI, Servelets, JSPs, Cookies –ODBC •backend: DBMS –JDBC • Extending DBMSs • Today: How DBMSs are used from other programs – Stored procedures – Also, how DBMSs can be extended by other programs – External functions in Postgres
Embedded SQL SQL in Application Code • Approach: Embed SQL in the host language. • SQL commands can be called from within a host – Preprocessor converts SQL statements into API calls. language (e.g., C++ or Java) program. – Then regular compiler compiles the code. – SQL statements can refer to host variables (including special variables used to return status). – Must include a statement to connect to the right database. • Many different versions for different vendors –Oracle Pro*C • Main integration approaches: – Sybase Embedded SQL – Embed SQL in the host language (Embedded SQL, SQLJ) – DB2 Embedded SQL (for COBOL) – General APIs to call SQL commands (ODBC, JDBC) – ecpg for Postgres (see Postgres programmer’s guide) – Product-specific APIs to access DBMS – And many many more... • (every vendor has one) Embedded SQL – In a nutshell Embedded SQL: Variables
• In your program, put ‘EXEC SQL’ before special EXEC SQL BEGIN DECLARE SECTION database commands. char c_sname[20]; long c_sid; • A preprocessor changes them to regular C. short c_rating; • Example commands: float c_age; – Connecting to a database: EXEC SQL END DECLARE SECTION EXEC SQL CONNECT • Two special “error” variables: – Declaring variables: –SQLCODE (long, is negative if an error has occurred) EXEC SQL BEGIN (END) DECLARE SECTION –SQLSTATE (char[6], predefined codes for common errors) – Statements: EXEC SQL Statement;
Embedding SQL in C: An Example Embedded SQL Example
char SQLSTATE[6]; main() EXEC SQL BEGIN DECLARE SECTION { char c_sname[20]; short c_minrating; float c_age; EXEC SQL INCLUDE SQLCA; EXEC SQL BEGIN DECLARE SECTION; EXEC SQL END DECLARE SECTION int OrderID; /* Employee ID (from user) */ int CustID; /* Retrieved customer ID */ c_minrating = random(); char SalesPerson[10] /* Retrieved salesperson name */ EXEC SQL DECLARE sinfo CURSOR FOR char Status[6] /* Retrieved order status */ SELECT S.sname, S.age FROM Sailors S EXEC SQL END DECLARE SECTION; /* Set up error processing */ WHERE S.rating > :c_minrating printf ("Enter order number: "); ORDER BY S.sname; scanf ("%d", &OrderID); /* Execute the SQL query */ do { EXEC SQL SELECT CustID, SalesPerson, Status FROM Orders WHERE EXEC SQL FETCH sinfo INTO :c_sname, :c_age; OrderID = :OrderID INTO :CustID, :SalesPerson, :Status; printf(“%s is %d years old\n”, c_sname, c_age); printf ("Customer number: %d\n", CustID); } while (SQLSTATE != ‘02000’); printf ("Salesperson: %s\n", SalesPerson); EXEC SQL CLOSE sinfo; printf ("Status: %s\n", Status); exit();
SQLJ – Another way to embed SQL SQLJ Code
Int sid; String name; Int rating; SQLJ – Standard for embedding SQL in Java // named iterator #sql iterator Sailors(Int sid, String name, Int rating); Based on JDBC (more on this later) Sailors sailors;
Complements JDBC with a (semi-)static query model: Compiler // assume that the application sets rating can perform syntax checks, strong type checks, consistency of #sailors = { the query with the schema SELECT sid, sname INTO :sid, :name – All arguments always bound to the same variable: FROM Sailors WHERE rating = :rating #sql = { }; SELECT name, rating INTO :name, :rating FROM Books WHERE sid = :sid }; // retrieve results while (sailors.next()) { SQLJ (part of the SQL standard) versus embedded SQL (vendor- System.out.println(sailors.sid + “ “ + specific) sailors.sname)); } sailors.close(); Cursors Impedance Mismatch • Can declare a cursor on a relation or query statement • SQL relations are (multi-) sets of records, with no a (which generates a relation). priori bound on the number of records.
• Can open a cursor, and repeatedly fetch a tuple then • No such data structures exist in many languages such as C. move the cursor, until all tuples have been retrieved. – (Though C++ has STL, java has Multiset) – Can use ORDER BY clause in cursor queries to control the order in which tuples are returned. • Even for languages with Sets, may not want entire • Fields in ORDER BY clause must also appear in SELECT clause. query result in memory at one time! – The ORDER BY clause, which orders answer tuples, is only allowed in the context of a cursor. • SQL supports a mechanism called a cursor to handle this. • Can also modify/delete tuple pointed to by a cursor.
Cursor that gets names of sailors who’ve reserved a red boat, in alphabetical order Static vs. Dynamic Approaches
• Embedded SQL and SQLJ are Static – Text of queries is specified in program code
EXEC SQL DECLARE sinfo CURSOR FOR SELECT S.sname • Database APIs also exist FROM Sailors S, Boats B, Reserves R – every DBMS provides a vendor-specific API WHERE S.sid=R.sid AND R.bid=B.bid AND B.color=‘red’ – standard APIs also exist: JDBC, ODBC ORDER BY S.sname • APIs more flexible, powerful, harder to use
Database APIs: Alternative to embedding JDBC: Architecture
Rather than modify compiler, add library with database calls (API) • Four architectural components: • Special standardized interface: procedures/objects • Pass SQL strings from language, presents result sets in a – Application (initiates and terminates connections, language-friendly way submits SQL statements) • Sun’s JDBC: Java API – Driver manager (load JDBC driver) • Supposedly DBMS-neutral – Driver (connects to data source, transmits requests – a “driver” traps the calls and translates them into DBMS-specific code and returns/translates results and error codes) – database can be across a network – Data source (processes SQL statements) – most databases provide ODBC or JDBC drivers JDBC Architecture (Contd.) Connections in JDBC Four types of drivers: Bridge: We interact with a data source through sessions. Each – Translates SQL commands into non-native API. connection identifies a logical session. Example: JDBC-ODBC bridge. Code for ODBC and JDBC • JDBC URL: driver needs to be available on each client. jdbc:
Connection Class Interface Executing SQL Statements • public int getTransactionIsolation() and void setTransactionIsolation(int level) • Three different ways of executing SQL statements: Sets isolation level for the current connection. – Statement (both static and dynamic SQL statements) (semi-static SQL statements) • public boolean getReadOnly() and – PreparedStatement void setReadOnly(boolean b) – CallableStatment (stored procedures) Specifies whether transactions in this connection are read- only • PreparedStatement class: • public boolean getAutoCommit() and Precompiled, parametrized SQL statements: void setAutoCommit(boolean b) If autocommit is set, then each SQL statement is considered – Structure is fixed its own transaction. Otherwise, a transaction is committed – Values of parameters are determined at run-time using commit(), or aborted using rollback().
• public boolean isClosed() Checks whether connection is still open.
Executing SQL Statements (Contd.) ResultSets String sql=“INSERT INTO Sailors VALUES(?,?,?,?)”; PreparedStatment pstmt=con.prepareStatement(sql); • PreparedStatement.executeUpdate only returns the pstmt.clearParameters(); number of affected records pstmt.setInt(1,sid); • PreparedStatement.executeQuery returns data, pstmt.setString(2,sname); encapsulated in a ResultSet object (a cursor) pstmt.setInt(3, rating); pstmt.setFloat(4,age); ResultSet rs=pstmt.executeQuery(sql); // rs is now a cursor // we know that no rows are returned, thus we use While (rs.next()) { executeUpdate() // process the data int numRows = pstmt.executeUpdate(); } ResultSets (Contd.) Matching Java and SQL Data Types SQL Type Java class ResultSet get method A ResultSet is a very powerful cursor: BIT Boolean getBoolean() • previous(): moves one row back CHAR String getString() • absolute(int num): moves to the row with the VARCHAR String getString() specified number DOUBLE Double getDouble() • relative (int num): moves forward or backward FLOAT Double getDouble() • first() and last() INTEGER Integer getInt() REAL Double getFloat() DATE java.sql.Date getDate() TIME java.sql.Time getTime() TIMESTAMP java.sql.TimeStamp getTimestamp()
Warning and Exceptions (Contd.)
JDBC: Exceptions and Warnings try { stmt=con.createStatement(); warning=con.getWarnings(); • Most of java.sql can throw and SQLException if while(warning != null) { an error occurs. // handle SQLWarnings; warning = warning.getNextWarning(); } • SQLWarning is a subclass of SQLException; not con.clearWarnings(); as severe (they are not thrown and their stmt.executeUpdate(queryString); existence has to be explicitly tested) warning = con.getWarnings(); … } //end try catch( SQLException SQLe) { // handle the exception }
Database Metadata (Contd.) Examining Database Metadata
DatabaseMetaData md=con.getMetaData(); DatabaseMetaData object gives information ResultSet trs=md.getTables(null,null,null,null); about the database system and the catalog. String tableName;
While(trs.next()) { DatabaseMetaData md = con.getMetaData(); tableName = trs.getString(“TABLE_NAME”); // print information about the driver: System.out.println(“Table: “ + tableName);
System.out.println( //print all attributes “Name:” + md.getDriverName() + ResultSet crs = md.getColumns(null,null,tableName, null); “version: ” + md.getDriverVersion()); while (crs.next()) { System.out.println(crs.getString(“COLUMN_NAME” + “, “); } } A (Semi-)Complete Example Extending Database Functionality Connection con = // connect DriverManager.getConnection(url, ”login", ”pass"); Statement stmt = con.createStatement(); // set up stmt String query = "SELECT name, rating FROM Sailors"; • So far, discussed adding DBMS functions to programs ResultSet rs = stmt.executeQuery(query); • What about extending the functionality of a DBMS? try { // handle exceptions // loop through result tuples while (rs.next()) { • Two ideas: String s = rs.getString(“name"); Int n = rs.getFloat(“rating"); – Stored Procedures: writing programs inside the DBMS System.out.println(s + " " + n); – External Functions: adding functions in other languages }
} catch(SQLException ex) { System.out.println(ex.getMessage () + ex.getSQLState () + ex.getErrorCode ()); }
Stored Procedures: Examples Stored Procedures CREATE PROCEDURE ShowNumReservations SELECT S.sid, S.sname, COUNT(*) FROM Sailors S, Reserves R • What is a stored procedure: WHERE S.sid = R.sid – Program executed through a single SQL statement GROUP BY S.sid, S.sname – Executed in the process space of the server Stored procedures can have parameters: • Three different modes: IN, OUT, INOUT • Advantages: – Can encapsulate application logic while staying “close” to CREATE PROCEDURE IncreaseRating( IN sailor_sid INTEGER, IN increase INTEGER) the data UPDATE Sailors – Reuse of application logic by different users SET rating = rating + increase – Avoid tuple-at-a-time return of records through cursors WHERE sid = sailor_sid
Stored Procedures: Examples (Contd.) SQL/PSM Most DBMSs allow users to write stored procedures in a Stored procedure do not have to be written in SQL: simple, general-purpose language (close to SQL) Æ SQL/PSM standard is a representative
CREATE PROCEDURE TopSailors(IN num INTEGER) Declare a stored procedure: LANGUAGE JAVA CREATE PROCEDURE name(p1, p2, …, pn) EXTERNAL NAME “file:///c:/storedProcs/rank.jar” local variable declarations procedure code; Declare a function: CREATE FUNCTION name (p1, …, pn) RETURNS sqlDataType local variable declarations function code; Main SQL/PSM Constructs Main SQL/PSM Constructs (Contd.)
CREATE FUNCTION rate Sailor (IN sailorId INTEGER) RETURNS INTEGER • Local variables (DECLARE) DECLARE rating INTEGER • RETURN values for FUNCTION DECLARE numRes INTEGER • Assign variables with SET SET numRes = (SELECT COUNT(*) • Branches and loops: FROM Reserves R – IF (condition) THEN statements; WHERE R.sid = sailorId) ELSEIF (condition) statements; IF (numRes > 10) THEN rating =1; … ELSE statements; END IF; ELSE rating = 0; – LOOP statements; END LOOP END IF; • Queries can be parts of expressions RETURN rating; • Can use cursors naturally without “EXEC SQL”
Extending Postgres in C Extending Postgres (useful for HW3) • Postgres also permits C functions: • Postgres permits creating functions using SQL CREATE FUNCTION add_one(int4) RETURNS CREATE FUNCTION add_em(integer, integer) RETURNS int4 AS 'PGROOT/tutorial/funcs' integer AS ' SELECT $1 + $2; ' LANGUAGE SQL; LANGUAGE C WITH (isStrict); SELECT add_em(1, 2) AS answer;
SELECT add_em(1, 2) AS answer; PG_FUNCTION_INFO_V1(add_one); answer Datum add_one(PG_FUNCTION_ARGS) ------{ 3 int32 arg = PG_GETARG_INT32(0); PG_RETURN_INT32(arg + 1); }
Summary Extending Postgres Aggregates • Embedded SQL & SQLJ allow execution of parametrized static queries within a host language • Postgres even permits extending aggregates:
• Cursor mechanism allows retrieval of one record at CREATE AGGREGATE complex_sum ( a time and bridges impedance mismatch between sfunc = complex_add, host language and SQL basetype = complex, stype = complex, • APIs such as JDBC & ODBC introduce a layer of initcond = '(0,0)‘ abstraction between application and DBMS ); Summary (Contd.)
• SQLJ: Static model, queries checked a compile-time.
• Stored procedures execute application logic directly at the server
• SQL/PSM standard for writing stored procedures
• Postgres allows you to add new functions written in SQL, or an external language like C. – You can even use these functions to extend aggregates.