Coordination and Support Action
European Commission Seventh Framework Project (IST-257822)
Mediabase Ready and First Analysis Report
Deliverable D4.3
Editor: Michael Derntl (RWTH Aachen University) Contributors: Adam Cooper, Manh Cuong Pham, Ralf Klamma, Dominik Renzel Dissemination level: Public Delivery date: 2011-09-30
Work Package WP4: Weak Signals Analysis – Emerging Reality Dissemination Level Public Status Version 1.0 — Final Date September 30, 2011
Amendment History
Vers ion Date Ed itor Description/Comments 1.0 30 Sept. 2011 Michael Derntl Final version
Contributors
Name Institution Role Michael Derntl RWTH Aachen University Editor /Author Adam Cooper University of Bolton (CETIS) Author Ralf Klamma RWTH Aachen University Author Manh Cuong Pham RWTH Aachen University Author Dominik Renzel RWTH Aachen University Author Paul Lefrere The Open University (OU) Reviewer Lampros Stergioulas Brunel University Reviewer Christian Voigt Zentrum für Soziale Innovation (ZSI) Reviewer
Deliverable description in the DoW:
The deliverable will describe the continuation of the established PROLEARN Mediabase equipped with new tools combining the existing social network analysis with topic mining. This will realize a structural-semantic analysis of signals from the Web 2.0 strongly related to technology enhanced learning. Results from the analysis will be reported here but can be obtained continuously from the Web interfaces of the Mediabase afterwards.
Contents
1 Introduction ...... 1
2 The TEL-Map Mediabase ...... 2 2.1 Conceptual Model of the TEL-Map Mediabase ...... 3 2.2 Components Overview ...... 4 2.3 Analysis Approach ...... 7 2.4 Potential Questions ...... 10
3 Analysis of the European TEL Project Landscape ...... 12 3.1 Data Set ...... 12 3.2 TEL Projects as Social Networks ...... 14 3.3 Project Consortium Progression ...... 15 3.3.1 FP7 Projects ...... 15 3.3.2 All TEL Projects – FP6, FP7, and eContentplus ...... 16 3.3.3 Identifying Project Clusters ...... 17 3.4 Organizational Collaborations ...... 19 3.4.1 Collaborations in FP7 projects ...... 19 3.4.2 Collaborations in all TEL Projects: FP6, FP7, and eContentplus ...... 21 3.4.3 Dynamic SNA of the TEL Project Landscape ...... 25 3.5 Geo-Mapping TEL Projects ...... 28
4 Analysis of TEL Publication Outlets ...... 29 4.1 Data Set ...... 29 4.2 Social Network Analysis of TEL Venues and Papers ...... 31 4.3 Co-Authorship Network Analysis ...... 32 4.3.1 Formal Foundations ...... 32 4.3.2 Overview ...... 32 4.3.3 Dynamic SNA ...... 34 4.3.4 Most Prolific Authors and Their Topics ...... 35 4.3.5 Overall TEL Co-authorship Network ...... 37 4.3.6 Central Authors in the Co-Authorship Network...... 38 4.4 Structural-Semantic Analysis: SNA and Topic Mining Combined ...... 39 4.5 Citation Network Analysis ...... 43
5 Analysis of the TEL Social Web ...... 44 5.1 Social Web Data Set ...... 45 5.2 Formal Foundations ...... 46 5.3 Results ...... 47 5.3.1 TEL Blog Network and Most Central Blogs ...... 47
5.3.2 TEL Blog Clusters ...... 49 5.3.3 Bursts ...... 51
6 Embeddable Interactive Visualizations and Queries ...... 52
7 Key Findings for Weak Signals ...... 55 7.1 TEL Projects...... 55 7.2 TEL Papers ...... 56 7.3 TEL Social Web...... 57
8 Conclusion ...... 57
References ...... 58
Appendix A: TEL Projects — Timeline ...... 60
Appendix B: TEL Projects — SNA Metrics ...... 61
Figures Figure 1: Concept map underlying of the TEL-Map Mediabase metamodel...... 4 Figure 2: TEL-Map Mediabase components overview model...... 5 Figure 3: Data model of TEL projects...... 12 Figure 4: Word clouds of project descriptions...... 14 Figure 5: FP7 TEL projects graph visualization...... 15 Figure 6: Project consortium progression between FP6, FP7, and eContentplus projects...... 17 Figure 7: Visualization of the FP7 collaboration graph...... 19 Figure 8: Center region cut-out of the FP7 collaboration graph...... 20 Figure 9: Word cloud of the 20 word stems with highest frequency in the FP7 project descriptions .... 21 Figure 10: Partner collaborations spanning FP6, FP7, and eContentplus projects...... 22 Figure 11: Local clustering of organizations plotted against (a) PageRank and (b) degree...... 24 Figure 12: Overall development of collaboration network since 2004...... 26 Figure 13: Impact of newly launched projects the collaboration network...... 26 Figure 14: Impact of organizations on collaboration...... 27 Figure 15: Development of the ratio of projects coordinated by novice organizations ...... 27 Figure 16: Google Map overlay with organizations involved in TEL projects...... 28 Figure 17: Data model for TEL papers and events...... 30 Figure 18: Word cloud of most frequent terms in TEL conference paper titles...... 31 Figure 19: Development model for conference communities...... 32 Figure 20: Cumulative annual (co-)author figures of selected TEL conferences over the last 10 years. 33 Figure 21: Co-authorship network visualization for the TEL conferences...... 33 Figure 22: Co-authorship network measures of five conferences in TEL...... 35 Figure 23: Most frequent terms in papers of top TEL authors in 2010...... 37 Figure 24: Complete co-authorship network in the core TEL venues...... 37 Figure 25: Co-authorship network of the “inner circle” of authors in the core TEL venues...... 38 Figure 26: Citation network measures of five conferences in TEL...... 44 Figure 27: Relational model of the TEL blogosphere...... 45 Figure 28: Number of blogs added to and blog entries indexed in the TEL-Map Mediabase...... 46 Figure 29: TEL blogs link network visualization, excluding self-references...... 47 Figure 30: Top 100 word stems appearing in 2011 blog entries of the top 20 blogs...... 49 Figure 31: Colored TEL blog clusters...... 50 Figure 32: Bursty terms appearing only in 2011...... 51 Figure 33: Bursty terms with rising frequency over the last three years...... 52 Figure 34: Visualization of the same SQL query as a table (left) and as a graph (right)...... 53 Figure 35: SQL query visualization as an annotated timeline...... 54
Tables
Table 1: Uses of social network analysis and topic mining in the TEL-Map Mediabase...... 10 Table 2: Overview of the 77 TEL Projects in the TEL-Map Mediabase...... 13 Table 3: TEL project clusters in FP6, FP7, and eContentplus (ECP) and the word clouds of their project descriptions...... 18 Table 4: Top 30 organizations involved in TEL projects by PageRank. The numbers in square brackets next to the values represent the rank of that value among all 604 organizations...... 23 Table 5: Strongest partnership bonds over all TEL projects in FP6, FP7 and eContentplus...... 25 Table 6: Selection of conferences relevant to the TEL community...... 31 Table 7: Fifteen most prolific authors at conferences and journals with a broad TEL scope. Names marked with an asterisk (*) indicate authors currently based in Europe...... 36 Table 8: Top 15 TEL authors by betweenness centrality...... 39 Table 9: Top ten co-author pairs in core TEL venues...... 39 Table 10: Betweenness centrality of authors of papers identified in D4.1...... 40 Table 11: Summary of structural-semantic analysis: themes and matching papers...... 41 Table 12: Top twenty blog sources by PageRank. The number in square bracket indicates the blog’s overall rank for the respective metric...... 48 Table 13: Clusters of TEL blogs indexed in Mediabase...... 49
1 Introduction
The European Framework Programmes (FP) for Research and Technological Development are a key pillar of the European research area and act as the primary vehicle for the European Union to create and sustain growth, employment and global competitiveness [3]. FPs are complex frameworks defining the specific research programmes and challenges to be tackled over a seven-year period with a multi-billion Euro budget. In FP7, the Cooperation programme, which also hosts the Technology Enhanced Learning (TEL) thematic area, received the largest share of the total FP7 funds. For the twenty-six partly completed and partly running TEL projects in FP7 the European Commission has provided or will provide a total funding of more than one hundred million Euro. People and organizations with a stake in TEL research and development are likely to be interested in knowing where this enormous amount of money went and what impact it has generated and is generating on the TEL landscape. First and foremost, the European Commission itself is interested in what impact the spending has generated over the years. In addition, there are many organizations and individuals in Europe that have a stake in TEL, e.g. technology providers, technology adopters, and higher education institutes, to name a few (see [16] for a more comprehensive list of TEL-Map stakeholders). To address the issue of generating such information based on strong and weak signals in a variety of (web) sources, one core threads pursued in TEL-Map includes the application of social network analysis and visualization as well as topic mining.
This deliverable reports on social network analysis and topic mining work performed in WP4, “Weak Signal Analysis—Emerging Reality,” to support weak signal analysis and the mapping of the current TEL landscape. To achieve this, the deliverable outlines the conceptual foundations of the TEL-Map Mediabase, where all underlying data sources were stored, and presents first results of the analyses. The main task underlying the work reported in this deliverable is Task 3 in WP4, which comprises the following objectives:
• TEL-Map Mediabase : Based on the PROLEARN Mediabase the aim was to develop a TEL- Map Mediabase, which shall contain social media artifacts and related resources to support the mapping of the TEL landscape and complement the Delphi-based weak signal analysis approach reported in D4.1 [23]. The focus in the TEL-Map Mediabase shall be on issues, topics, and structures of relevance to TEL. This required a filtering of existing Mediabase content, an extension of the sources fed into Mediabase with TEL-related content and development of new tools to support analyses of these extended sources. The TEL-Map Mediabase is presented in Section 2.
• Social Network Analysis : One of the pillars of the analysis methodology in WP4 is social network analysis (SNA) of actors involved in TEL and their relationships. “Actor” is meant as an abstract concept in this context, which can refer to various kinds of entities like people, conferences, projects, publications, and so forth. SNA offers highly effective methods for obtaining interactive visualizations and network metrics for these social networks, allowing the identification of the most important actors from a wide range of perspectives. In this deliverable the focus is on analysis of TEL projects and organizations involved in those projects (Section 3); TEL papers, authors and publication outlets (Section 4); and TEL social media sources (Section 5).
• Topic Mining : In addition to the network-metrics and structural analysis approach taken in SNA, the analysis methodology shall be complemented with a topic mining approach. The objective is to identify bursty topics, shifts in topics, emerging/declining topics from a variety of sources in TEL, achieving a structural-semantic analysis of signals. This is tackled in combination with SNA in Sections 3 through 5.
– 1 –
• Continuous analysis : As indicated in the title, this deliverable was conceived to present a first analysis report, while TEL stakeholders shall be supported in continuously obtaining up- to-date analysis results from the Mediabase web interfaces. This requires web-based tools for continuous analysis of TEL sources (dealt with in Section 6) and an online resource page where data sets and data processing components can be accessed and/or downloaded. For the latter goal, a resource page was set up on the TEL-Map homepage. This D4.3 resource page is available at http://telmap.org/?q=content/d4.3 and will be continuously updated with pointers to results obtained, tools developed, and analyses performed in WP4—Task 3, which will continue to run until the end of the project.
In regard to the embedding of this work into TEL-Map’s overall WP structure, the WP4’s mission—i.e. the identification of weak signals that can inform the overall road-mapping process—also requires us to propel the convergence of different analytical methods. For instance, this can be achieved in WP4 by feeding results from one analytical method into another one in order to cross-validate and enrich existing findings, but it also needs to happen between WPs, e.g. by informing WP5’s gap analysis and WP3’s scenario building. Gap analysis aims to explore why some technologies seem to be much more prominent in TEL research than in TEL practice (e.g. consider the uptake of 3D worlds) and other technologies are slowly becoming mainstream with no matching amount of research available (e.g. laptops in schools or social media at the workplace). Here, weak signals can inform an in-depth analysis of specific technologies by considering the spread of awareness of that technology across various communities as well as the use of synonyms referring to the same set of issues but under different labels. Likewise, scenario building events (WP3) can be informed through weak signals as they are early indicators of change that have the potential to alter the future of TEL adopters and TEL providers. In this context, scenarios that consciously consider weak signals increase their robustness, leading to better strategic planning processes.
This deliverable is structured as follows. In Section 2 we introduce the TEL-Map Mediabase, containing data relevant to TEL in terms of projects, publications, and social media. Each of the subsequent sections presents first analyses performed and results obtained in the TEL-Map Mediabase sources, i.e. TEL projects in Section 3, TEL publications in Section 4, and TEL blogosphere in Section 5. An embeddable, widget-based toolkit for enabling stakeholders to query and visually interact with the data contained in the TEL-Map Mediabase is presented in Section 6. Section 7 draws key findings from the analysis for weak signals collection from the core analysis sections, and Section 8 wraps up the deliverable with a discussion of limitations and an outlook on upcoming work in WP4.
2 The TEL-Map Mediabase
TEL-Map Mediabase is an evolution of the established PROLEARN Mediabase. In this section we first describe the original idea and concept of Mediabase and continue with detailing the structure, content, and meta-model of the enhanced TEL-Map Mediabase.
In the PROLEARN project 1, a TEL project funded by the European Commission under FP6, one core effort was the creation and maintenance of a media base for TEL in Europe, providing different target audiences like scientists, policy makers, and communities of practice with digital information obtained from mailing lists, newsletters, blogs, RSS/Atom feeds, websites, and so forth [10]. In addition to collecting large amounts of data, one key objective was the provision of easy-to-use end-user tools for extracting and presenting relevant information contained in the Mediabase, e.g. for cross-media social network analysis, self-observation and self-modeling of communities [18], collaborative administration and retrieval of media artifacts, etc. The key concepts in the metamodel of the PROLEARN Mediabase are (cf. [10], p. 248-9):
1 http://www.prolearn-project.org
– 2 –
• Community as a sub-network of the whole network, representing trustful relations among its members; • Process as a value-adding set of activities performed by community members, e.g. acquisition, retrieval, monitoring; • Actor as humans, users or groups of humans/users performing and being affected by processes; • Medium as an artifact produced or consumed by processes.
For the development of the TEL-Map Mediabase, particular emphasis was put on the TEL blogosphere, which is being observed and continuously retrieved using special-purpose crawlers (cf. [9]); the blogosphere sources in the Mediabase were extended by the TEL-Map members. In addition, the artifacts stored and indexed in the Mediabase were extended with digital information on European TEL projects as well as publications in TEL-related conferences and journals.
2.1 Conceptual Model of the TEL-Map Mediabase TEL-Map aims to empower stakeholders to find relevant projects and useful outputs as well as new collaborators for TEL projects; it also aims at giving a rich overview of different types of actors involved in the TEL domain (see DoW, p. 17-18). WP4 in particular focuses on analyses and visualizations from social media items gathered and automatically crawled from relevant sources. To realize these ambitious objectives, we have enhanced and extended the metamodel and the content of the existing PROLEARN Mediabase. This enhanced TEL-Map Mediabase additionally includes information on TEL projects and participants funded by the European Commission, as well as authors and their papers published in TEL-related conferences and journals.
The conceptual model of TEL-Map Mediabase is displayed in Figure 1. It exposes three main areas:
• TEL Social Media: blogs, feeds, and blog entries; currently focusing on the blogosphere that includes TEL-related blog sources.
• TEL Projects: information on projects funded by the European Commission under FP6, FP7, and eContentplus, including information on participating organizations.
• TEL Papers: information on papers published in TEL-related journals, conferences, and workshops.
For each of these three areas there is a dedicated database schema. These schemas are described in detail in the relevant sections. There are several components (crawlers, importers, exporters, and end- user tools) which were developed to obtain the relevant data, to feed the data into the database, as well as to extract and interact with the data. These are described in Section 2.2.
Limitations. While the TEL-Map Mediabase databases contain an enormous amount of data, there are several concepts and their links in Figure 1 which are currently not or only partly represented in the data. These include:
• Meeting and Project Meeting: While we have data on conference and workshop events in the TEL Papers database, we do not yet have data on project meetings (some of which are collocated with other events). This information is missing since there we do not yet have mechanisms of automatically obtaining these data.
• Deliverable: Project deliverables are also not yet included. This can be done in the future by crawling the web pages of the projects stored in our TEL Projects database. However, we expect that manual editing will be required, since the deliverable pages are not uniform across different projects. For some projects, the deliverables cannot be found at all on the project website.
– 3 –
TEL Social Media
Blogosphere
part of
has Blog Comment has TEL Papers ref's ref's published at Entry Publication Paper Venue ref's post is a is a has has author of Person Author
associated is a with
TEL Projects Journal
Organization take Deliverable part in consortium member Conference Workshop
produce Project
Project meeting organize
Meeting is a
collocated with
Figure 1: Concept map underlying of the TEL-Map Mediabase metamodel.
• Person: The concept “person” is actually the glue between the three different databases, since a person can be an author of a paper in the TEL Papers data, the owner of a blog in the TEL Media data, and a member of an organization participating in a project indexed in the TEL Projects data. We do currently not have an automated procedure that is capable of matching and obtaining data related to persons, mostly because the data is not readily available (e.g. some blogs do not contain personal information on their author, and most projects do not provide detailed information on the persons involved). We aim to work toward this integration in upcoming WP4 work.
2.2 Components Overview The components of TEL-Map Mediabase are conceptually arranged in different groups or layers (see Figure 2): the information to be used for weak signal analysis in the context of Mediabase is contained in many different web data sources . To collect and filter the relevant information in structured format, a set of importers and crawlers were deployed, which ingest the relevant data into different databases (or database schemas). To process the data for analysis, visualization or any other kind of interaction, a set of exporters enables end-user applications to obtain and present the data. The layers and their components are described in detail below.
Importers. This layer includes services and processes that obtain relevant data from web sources and transform these data into a structured, relational database format.
• Blog Crawler : The blog crawler is deployed as a cron job, which runs every night. It crawls the RSS/Atom feeds and the websites of indexed sources and extracts new entries and ingests
– 4 –
Web Data Sources
European Community DBLP Publisher Blogosphere Information Pages Bibliography Pages
LearningFrontiers Importers Portal
Projects DBLP Abstracts Blog Feed Feed Crawler Importer Crawler Crawler Importer Aggregator
Mediabase Databases Commander
TEL Projects TEL Papers TEL Media
Exporters
CSV Data GraphML Visualization Exporter Exporter Widget Creator Legend
Service / Data Processing Apps Graph Visualization Query Widgets Process and Analysis Apps Database R Excel Query Query yEd Graphviz Visualizer Explorer End-User Application Matlab ... Gephi ... Data Flow
Figure 2: TEL-Map Mediabase components overview model.
those into the database. Upon ingestion it not only stores the raw HTML of the entries; it also extracts a plain-text, non-markup version of the content, the comments associated with each blog entry, the URLs it references, and it computes burstiness of terms occurring in blog entries. The blogs scheduled for indexing are entered in two ways: (1) directly through the Mediabase Commander on the Learning Frontiers portal, or (2) indirectly through the Feed Aggregator, which is installed on the Learning Frontiers portal to collect links to relevant RSS or Atom feeds. These feeds are automatically ingested into the TEL Media database by the Feed Importer.
• Abstracts Crawler : The TEL Papers database contains data like title, authors and citations on TEL-related papers. Since DBLP, the data source of the TEL papers database, does not contain abstracts and keywords, the goal of this crawler is to enhance the basic paper information with abstracts and keywords. The following conferences were crawled: ECTEL, ICWL, ICALT, ITS, DIGITEL and WMTE. Since the crawler supports the abstract pages of springerlink.com (Springer Verlag), computer.org and IEEExplore, the crawler can be used to crawl many more conferences. The crawler is written in Ruby using the Mechanize Library for extracting the information from the HTML pages. The crawler does not directly interact with the TEL papers database. Instead, desired information from the database has to be exported and imported as CSV data.
• Feed Importer : One objective of TEL-Map is to analyze the voices in TEL to detect weak signals. This required enriching the Mediabase with TEL-related social media artifacts 2. On the Learning Frontiers portal, we installed the aggregator module, which allows registered
2 See task 3 in the description of WP4 in the DoW, p. 39: “ We will integrate current RSS aggregators to enhance the contents of the Mediabase. ”
– 5 –
users to provide links to their favorite TEL-related feeds, either RSS or Atom feeds. This module offers several forms of access to the aggregated feeds, e.g., directly through Drupal’s mysql relational database or through a machine-processible OPML file that contains all RSS or Atom feed sources, or through the Learning Frontiers portal front-end, which will display the recent feed entries to the user as an HTML page. To integrate the aggregated feeds into Mediabase, we developed a module that fetches all feeds from the feed aggregator that were not yet ingested into Mediabase; for each matching feed, the module then creates a blogwatcher project entry (including the feed’s tag associations) in Mediabase. Once a day, a blog crawler processes the blogs and adds all blog entries to Mediabase (including older entries that do not show up in the current RSS/Atom feed).
• DBLP Importer : The records in the papers database were obtained from DBLP, a free and open bibliography mainly for computer science and its sub-disciplines. DBLP data is valuable since it includes information on conference series and journals, authors, and the papers published in the conferences and journals. Importing the data is done via an XML file that includes all DBLP records. The DBLP importer extracts these records and stores them in a relational database schema. In addition it is capable of extracting citation information on the imported papers using the CiteSeerX database.
• Projects Crawler : In order to collect information about the running (or completed) TEL projects, we developed a crawler that automatically scrapes data from the project factsheets on the CORDIS website (for FP6 and FP7 projects), as well as from the eContentplus pages. All projects funded under TEL-related calls were scraped. The extracted information contains data like project description, start and end dates, project participants, funding and cost, project coordinator, etc. The data from these fact sheets were in a first step transformed to an XML-based format, which can be used by XML-processing applications like the project landscape story on the Learning Frontiers portal3. In a second step, the data was fed into a relational database schema to be used e.g. by the Drupal installation that is hosting the Learning Frontiers portal 4. Analyses performed using the projects data obtained by this crawler are reported in Section 3.
Databases. The TEL-Map Mediabase database consists of a collection of three relational database schemas, which are used to store and index TEL-related projects, papers, and social media artifacts (currently mainly blogs).
• TEL Projects : This database includes details on TEL projects funded under FP6, FP7, and eContentplus programmes. It includes detailed information on the projects like start and end dates, cost, EC funding, coordinator, and consortium members. The TEL projects database is fed by the Projects Crawler. Details on the project data set are given in Section 3.1.
• TEL Papers : This database includes information on TEL-related conference series, conference events, journals, authors, and papers published in the conferences, workshops and journals. It is fed by the DBLP Importer. Details on the papers data set are given in Section 4.1.
• TEL (Social) Media : This database includes TEL-related blogs, including the blog entries, comments and analytical information like length, words occurrences, and word burst for certain entries. Details on the blogosphere data set are given in Section 5.1.
Exporters. To enable analysis of the TEL-Map Mediabase data, the data are accessible either natively via clients that connect to the database(s) using the database drivers, or via exporters. The exporters ease the process of obtaining data for analysis by providing a set of predefined export formats.
3 http://learningfrontiers.eu/?q=story/tel-project-landscape 4 http://learningfrontiers.eu/?q=project_space
– 6 –
• CSV Data Exporter : Includes a set of scripts that export data contained in the databases into CSV format (CSV = comma separated values). These CSV files are supported by most data processing applications like Excel, R, SPSS, and so forth.
• GraphML Exporter : Data can also be exported as graphs for social network analysis. The data is exported in the most common graph exchange format, i.e. the XML-based GraphML language. These GraphML files can be imported, visualized, and analyzed in graph visualization and analysis applications like yEd, Gephi, or the igraph library for R. For many other graph visualization and analysis software packages, there are conversion tools from and to GraphML.
• Query Visualizer and Query Explorer : interacting with social network visualizations reaches its limits when it comes to specific queries that focus on selected aspects of the data set or the network graphs. To enable efficient end-user interaction with the data, we implemented a set of query visualization widgets. These widgets can be embedded on any web page (e.g. in iGoogle) and allow direct querying of the databases using SQL. The unique feature of these widgets is that they can be used to visualize the query results in different formats (e.g. table, pie chart, timeline, or graph) and that they can export the visualization of any given query as a widget. Additionally, CSV and GraphML export (see above) of query results is supported by the explorer widget. More details in Section 6.
Applications. End-users will mostly interact with the data through applications like Excel, R, and the Learning Frontiers portal. While Figure 2 includes many example applications, the following list only focuses on those that were developed for TEL-Map:
• Learning Frontiers Portal : The Learning Frontiers portal is the single-access-point portal to results generated in the TEL-Map project. It includes two apps that can be used to contribute to content generation in the TEL Media database: The Mediabase Commander enables adding blogs directly to the database, and the Feed Aggregator is a Drupal module that we installed to allow users to collect relevant feeds. The feeds are ingested into the database at regular intervals by the Feed Importer. Note that Mediabase Commander (MBC) is also available as a Firefox add-on.
• Query Widgets : We developed a set of widgets that can be used to (a) query the TEL-Map Mediabase databases using SQL, (b) to automatically visualize the query results in different formats, (c) export the query result in different formats, and (d) to export a query visualization as a self-contained widget that can be embedded into any web site.
2.3 Analysis Approach This deliverable reports on first results of using social network analysis (SNA) and topic mining on the data stored in the TEL-Map Mediabase. SNA contributes to the structural analysis of actors and their relationships and topic mining contributes to the semantic analysis of actors and relationships between actors. The combination of SNA and topic mining thus enables the structural-semantic analysis of TEL sources.
Social Network Analysis (SNA) is one of the work threads pursued in WP4 of TEL-Map to detect weak signals [23, 6] indicating future directions and insight into collaboration and communication networks in different types of media and settings. SNA constitutes a rather new field of research and its application to digital libraries is very promising in terms of knowledge discovery [19, 20]. SNA defines techniques used to compute metrics of different actors in a social network. These metrics typically represent the importance of actors within their network or neighborhood, e.g. their centrality, connectedness, etc.
– 7 –
To enable the calculation of SNA metrics for the data in TEL-Map Mediabase, the entities stored in the Mediabase need to be modeled as a social network. A social network is modeled as a graph = , with being the set of vertices (or nodes) and being the set of edges connecting the vertices with one another [2]. Any “actor” entity in the Mediabase can be modeled as a vertex, if it is connected to other actors through any relationship of interest (modeled as edges) that can be obtained from the Mediabase data. For instance, consider the following social network graphs:
• TEL projects can be modeled as nodes and overlaps in the consortia of any two projects can be modeled as edges;
• Organizations can be modeled as nodes, while projects in which organizations collaborated can be modeled as edges;
• Persons can be modeled as nodes, while co-authorships on papers relevant to TEL can be modeled as edges;
• Papers can be modeled as nodes, while citations between papers can be modeled as edges;
• Blogs can be modeled as nodes, while links between the blogs’ entries can be modeled as edges.
There are several different, yet complementary methods of gaining insight into the modeled social network graphs:
(1) Visual interaction : The graph can be visualized using graph visualization software (like yEd, Graphviz, or Gephi). Similar to maps software like Google Maps, graph visualization software typically allows the user to zoom (vertical filter) into the visualization and to pan the visualized graph (horizontal filter). In addition these tools often offer graph layout algorithms, which can be used to align the vertices in a predefined shape (e.g. circular, organic, hierarchical, etc.). Graph visualization generally provides a holistic, condensed view on the overall network.
(2) Data querying : Interacting with graph visualizations will typically spawn more specific questions and exploratory tasks [5]. Some of these explorations cannot be performed using the visualization alone, e.g. the number of shortest paths through the network that lead through a particular node. Such results can be obtained by enabling querying into the graph data. We developed a web-based toolkit for enabling this (see Section 6).
(3) SNA Metrics : SNA allows the computation of different metrics for the graph, its nodes and its edges. In the SNA reported in this deliverable, we mainly focus on the following metrics:
• Avg. shortest path length: this is a graph metric that represents the average length of all shortest paths through the network. Over time this metric will grow quickly initially, but slows down or may even shrink in “mature” graphs.
• Diameter: This represents the length of the longest shortest path through the network. In isolation this value will not be very informative; it is useful however for comparing network development over time (see e.g. Section 3.4.3).
• Largest connected component: This measure represents the number (or the share) of nodes that are connected with each other in the largest sub-network of the graph. The lower this value, the higher the fragmentation in the network.
• Density: This metric represents the ratio between the number of existing connections in the graph and the number of possible connections. The higher this value, the higher the connectedness of the nodes. One observation of interest is the development of density over
– 8 –
time, when new nodes join the graph, to see whether these new nodes inter-connect tightly with the existing ones.
• Betweenness centrality: The betweenness centrality of a node represents the share of shortest paths through the network that pass through that node. The betweenness centrality is typically higher for nodes that connect (“bridge”) two or more sub-networks (also called “connected components”) in the network. For instance, an author who works in the intersection of artificial intelligence and technology-enhanced learning is likely to have a higher betweenness centrality in a co-authorship network than a person in the same network who only publishes with members of the core artificial intelligence community.
• Degree centrality: The degree of a node is represented by the number of its direct ties with other nodes, i.e. edges coming in and leading out of that node. Typically this value is normalized into a value between 0 and 1 by dividing the degree of a node by the number of other nodes in the graph. This is the simplest centrality measure for network analysis
• Closeness centrality: This measure is used to determine how close a node is to all other nodes that are reachable via edges. The closeness centrality is obtained by computing the mean length of these (shortest) paths. Nodes with a favorable closeness centrality are important nodes in the sense that they can easily reach other nodes for collaboration, information, or influence.
• PageRank: This measure became widely known through Google’s use of it for ranking web sites by importance [17]. The PageRank of a node depends on the PageRank of nodes connected to it. So a node being connected to another node that is important makes the source node more important, too. With increasing distance between nodes this “diffusion” of importance to other nodes is gradually reduced by a damping factor.
• Clustering coefficient: The clustering of a node (local clustering) measures how strongly the neighborhood of the node tends towards forming a clique, where every two nodes are connected by an edge. The clustering coefficient of the whole network is obtained by computing the average local clustering coefficient of its nodes.
• Authorities and Hubs: authorities refer to nodes that represent authoritative sources of information in the network that are being pointed to by good hubs; a good hub is a node that point to many good authorities [12]. Thus there is a circular dependency between these two metrics.
Topic Mining is an approach for discovering knowledge from text sources. Typically topics are described by word distributions and sometimes also time distributions (cf. [24]). In the context of this deliverable we use a simplified approach to topic mining that mainly focuses on term stems and their frequency of appearance in the content entities stored in the Mediabase (e.g. blog text, paper abstracts, project descriptions) at a particular point in time or in a particular time window. For the first structural-semantic analyses reported in this deliverable, we focused on a “big picture” approach to complementing social network metrics with content analysis for different sources and actors in the TEL-Map Mediabase. This includes:
• For illustrating topic distribution in large sources we filtered the sources by identifying sources that are linked to key actors in the community (e.g. central organizations in projects, entries of central blogs). Following this, we present the core topics represented in these sources either through word clouds or through analysis of rising and falling frequency of topic occurrence in the sources.
– 9 –
• Building on the topic mining approach of selected TEL conferences in D4.1, we filtered the results for sources that were contributed by key authors in these conferences’ co-authorship networks and extracted weak signals there.
2.4 Potential Questions The combined results of SNA and topic mining can give rich insight into the available data and be used to detect and explore potential signals (both strong and weak ones) in the data. The matrix in Table 1 gives a brief overview of questions addressed by using SNA and topic mining on the different data sources in the TEL-Map Mediabase.
Table 1: Uses of social network analysis and topic mining in the TEL-Map Mediabase.
Social Network Analysis Topic Mining
TEL Papers • Most central authors in TEL • Rising and falling terms in TEL paper • Most frequent collaborations on TEL abstracts and keywords papers • Topics addressed by most important • Most important TEL conferences and TEL authors/papers journals • Development characteristics of authorship networks in TEL conferences.
TEL Projects • Consortium progression between • Topic distribution and shifts in TEL projects project foci over time • Partner collaborations across TEL • Funding and partners related to topics projects in TEL projects • Most central organizations in TEL projects • Most central TEL projects • Development of SNA metrics in project collaboration network over time
TEL Media • Citation network in TEL blogs • Topic bursts in TEL blogs over time • Most central web sources referenced in • Recently appearing topics TEL blogs • Topics with a rising frequency over the • Authorities and hubs in the TEL last years blogosphere • Co-occurrence of words/bursts in blog entries
In the following, we elaborate more on the objectives and potential signals that can be identified by tackling the questions outlined in Table 1.
TEL Papers Social Network Analysis and Topic Mining:
• Most central authors in (European) TEL: identifies authors that have a central position in the co-authorship and citation network of TEL papers; these authors are likely to have authority regarding the focus of current TEL research and directions for future TEL research, which can be analyzed using topic mining.
• Most frequent collaborations in TEL: Since TEL research is collaborative work, the identification of most important authors is complemented with collaboration frequency to identify strong ties between authors and communities.
– 10 –
• Most important TEL conferences and journals: identifying the most important outlets for publishing TEL research results will indicate venues where TEL key people meet for exchange and collaboration. Knowing the core TEL conferences will facilitate researchers in finding relevant collaborators.
• Development characteristics of TEL conferences: identifies patterns of development of authorship networks, which will reveal several insightful network characteristics, e.g. whether the TEL community is a fragmented community, whether TEL conferences develop like conferences in other disciplines, etc.
• Rising and falling terms in TEL papers: analysis of these terms will reveal topics and topic shifts in published TEL research. Of course, published TEL research is only a fraction of the research actually performed, and typically conference papers are up to one year behind the actual research work. For journal papers this lag is even worse, since journal papers often appear only 2-3 years after submission of the manuscript.
• Topics addressed by prolific authors: Prolific or otherwise central authors identified in the co- authorship networks of different (sets of) publication outlets can be used for revealing topics that likely have impact on current and future work.
TEL Projects Social Network Analysis and Topic Mining:
• Consortium progression between projects and partner collaborations across TEL projects: this will identify organizational collaboration between different (consecutive and concurrent) projects that sustain beyond the lifetime of one project’s consortium. Strong partnership ties between organizations on the one hand, and new project funding for participants of a project may indicate fruitful and successful collaboration in that project and can thus be considered as an indicator of project success.
• Most central TEL projects: analysis of consortium progression will also identify the most central projects in terms of having the largest consortium overlap with other projects, connecting different succeeding and preceding projects, and similar centrality measures.
• Most central organizations in TEL projects: SNA can be used to identify the most central organizations in the TEL collaboration network in terms of number of connections, closeness to other organizations in the network, and connections between different organizational clusters or sub-networks.
• Development of SNA metrics in project collaboration network over time: dynamic analysis of the collaboration network in projects over different funding calls or years will identify several characteristics of development patterns in the European TEL “market”, including development of collaboration network characteristics over time, impact of new projects on the collaboration network (e.g. introduction new organizations introduced by new projects) over time, and impact of new organizations on the creation of new collaboration ties between organizations.
• Topic distribution in projects can be analyzed using the descriptions of projects or project clusters which were previously identified by SNA.
TEL Media Social Network Analysis and Topic Mining:
• Citation network in TEL blogs: identifies the most central blogs and blog entries in the TEL blogosphere and can be used in combination with topic mining on those blogs to identify trending, upcoming, and declining topics.
– 11 –
• Most authoritative web sources referenced in TEL blogs: in addition to citing sources in the blogosphere, bloggers reference all sorts of sources on the web; analyzing these can help to identify the most authoritative (type of) sources on the web for TEL bloggers (this will be tackled in upcoming WP4 work)
• Topic bursts in TEL blogs over time: based on frequently occurring words in social media sources we are able to identify newly emerging terms and topics as well as topics with rising or falling frequency. This analysis is enhanced by filtering for those blogs that have a central position in the blogosphere.
3 Analysis of the European TEL Project Landscape
There currently exists no readily available, structured data set on TEL projects funded in recent programmes, with the exception of HTML factsheets offered on the web by the European Commission as well as a load of project websites and deliverables produced by the project consortia. Turning information overload into an opportunity is the driving vision of visual analytics [7], and this section aims to achieve this vision in the context of TEL projects funded under FP6, FP7 and eContentplus programmes by applying SNA and information visualization methods on projects and collaborations within project consortia.
3.1 Data Set Data Model. The database used for the analyses in this paper was scraped from publicly available project information pages on CORDIS [4], i.e. the Community Research and Development Information Service offered by the European Commission, and other European Community project information pages. The scraped data was captured according to the data model presented in Figure 3 and fed into a relational database. The data scraping was focused on TEL-related projects funded under FP6, FP7 and eContentplus.
ROLE participate Organization has_location N 1
ID N 1
NAME
COUNTRY Project Geolocation
ID ID CONTRACT_NO TITLE ACRONYM
LATITUDE DESCRIPTION DATE_START DATE_END TYPE
LONGITUDE PROGRAMME CALL COST FUNDING
PRECISION WEBSITE_URL FACTSHEET_URL RCN
Figure 3: Data model of TEL projects.
– 12 –
Information that was not available in CORDIS includes the geographical coordinates of project members. These locations were semi-automatically obtained by invoking the Google Maps API and Yahoo Maps API using the partner names and countries provided in the factsheets. Since some of the partner names produced ambiguous geographical results, the geographical coordinates will not be correct for some institutions. Also, the spelling of organization names and country names was inconsistent in the project fact sheets in many cases; this was corrected manually (which still does not guarantee correctness). Additionally, organizational name changes are not accounted for. For instance, Giunti Labs S.R.L. was rebranded to eXact Learning Solutions in 2010. In the data set, these—and all organizations with similar rebrandings—are represented as separate entities. Likewise, organizational mergers are not accounted for, e.g. ATOS Origin and Siemens Learning, which merged in 2011.
Selection of TEL Projects. Table 2 includes the details on the 77 TEL projects used in the following analyses, and a visual timeline of these projects can be found in Appendix A.
Table 2: Overview of the 77 TEL Projects in the TEL-Map Mediabase.
Programme Call # Projects (acronyms) Call 2005 4 CITER, JEM, MACE, MELT COSMOS, EdReNe, EUROGENE, eVip, Intergeo, KeyToNature, Call 2006 7 eConten plus 5 Organic.Edunet Call 2007 3 ASPECT, iCOPER, EduTubePlus Call 2008 5 LiLa, Math -Bridg e, mEducator, OpenScienceResources, OpenScout CONNECT, E -LEGI, ICLASS, KALEIDOSCOPE, LEACTIVEMATH, IST-2002-2.3.1.12 a 8 PROLEARN, TELCERT, UNFOLD APOSDLE, ARGUNAUT, ATGENTIVE, COOPER, ECIRCUS, ELEKTRA, FP6 IST-2004-2.4.10 b 14 I-MAESTRO, KP-LAB, L2C, LEAD, PALETTE, PROLIX, RE.MATH, TENCOMPETENCE ARISE, CALIBRATE, ELU, EMAPPS.COM, ICAMP, LOGOS, LT4EL, IST-2004-2.4.13 c 10 MGBL, UNITE, VEMUS ICT -2007.4.1 d 6 80DAYS, GRAPPLE, IDSPACE, LTFLL, MATURE, SCY COSPATIAL, DYNALEARN, INTEL LEO, ROLE, STELLAR, TARGET, ICT-2007.4.3 d 7 FP7 XDELIA ALICE, ARISTOTELE, ECUTE, GALA, IMREAL, ITEC, METAFORA, ICT-2009.4.2 b 13 MIROR, MIRROR, NEXT-TELL, SIREN, TEL-MAP, TERENCE Total: 77 a … Technology-enhanced learning and access to cultural heritage b … Technology-Enhanced Learning c … Strengthening the Integration of the ICT research effort in an Enlarged Europe d … Digital libraries and technology-enhanced learning
Topics and topic shifts. To give an indication of the topic focuses in these projects, Figure 4 presents for FP6, FP7, and eContentplus a word cloud of the funded projects’ descriptions. It reveals an interesting difference between FP6 and FP7 projects. In FP6, we find many meta-concepts in the descriptions like project , development , research , European , while descriptions of TEL projects in FP7 expose some concrete research and learning related topics like adaptive , social , design , process , activities , and so forth. It could be argued that during FP6 the TEL landscape was gradually beginning to take form, while in FP7 the research agenda already included several hot topics.
5 For each eContentplus call, only projects funded under the “Educational content” category were considered. The project SHARE-TEC (call 2007) was excluded from the data, since there was no official fact sheet available.
– 13 –
Looking at eContentplus in comparison to FP6 and FP7, there is a strong emphasis on content and metadata , while still including heavy use of educational and learning as terms. Content is a term found also in FP6 with some frequency, but it is missing in the top term list of FP7, probably showing that the eContentplus participants and the European Commission were targeting different foci.
FP6 FP7
eContentplus All TEL projects Figure 4: Word clouds of project descriptions.
3.2 TEL Projects as Social Networks A TEL project—like any other collaborative type of project—can be modeled as a social network where a number of partner organizations collaborate under coordination of a coordinating organization. A social network is modeled as a graph = with being the set of vertices (or nodes) and being the set of edges connecting the vertices with one, another [2].
Let be the set of projects, and let be the set of organizations involved in these projects. Function represents the membership of any organization in the consortium of any project and is defined as follows: ∈ ∈ if participated or particiaptes in , ∈ ∈ ∶ → otherwise . , The data model and these formal foundations enable powerful analyses and visualizations including the project network, the organizational partnership network, temporal relationships between project consortia, and the geographical mapping of organizations involved in projects. A selection of these analyses is presented in the following sub-sections, focusing on these objectives:
• Visualizing and analyzing project consortium progression . By progression we mean partnerships within project consortia that sustain beyond one single project. Investigating these dynamics can be used to identify successful and strongly connected organizations between consortia of different projects. This objective is tackled in Section 3.3.
• Visualizing and analyzing organizational collaborations within projects . Repeated collaboration in projects will create strong ties between organizations. Computing social network metrics for those connections will reveal the most important organizations currently involved in TEL research. This objective is dealt with in Section 3.4.
– 14 –
• Interactive visualization of geographical distribution of project consortia to complement the social network metric-based approaches with geographical map overlays, identifying hotspots in the European TEL landscape. This objective is dealt with in Section 3.5.
3.3 Project Consortium Progression
The project consortium progression graph contains projects and their relationships with , each other based on overlapping consortia. The = grap ( h will show projects as nodes and an edge between two nodes if there is any organization that has participated in both projects, i.e. , and =
= , ∶ , ∈ ∧ ≠ ∧∃ ∈ ∶ , ∧ , . can be modeled as a directed graph, which exposes the temporal progression of project consortia. Each edge in this graph represents a temporal relationship between two connected projects: the edge points from the project which started earlier to the project which started later.
3.3.1 FP7 Projects A visualization of for the 26 FP7 projects is shown in Figure 5. The size of each node in this visualization is proportional to the betweenness centrality [2] of that node, and the weight of the edge was determined by the number of partners that overlap between two project consortia. The betweenness centrality measure is an effective means of exposing nodes that act as “bridges” between otherwise distant nodes (or groups of nodes) by computing for each node the share of all shortest paths through the network that lead through the node.
COSPATIAL TERENCE INTELLEO
METAFORA MIROR
TEL-MAP
MIRROR ITEC
80DAYS GALA STELLAR NEXT-TELL DYNALEARN GRAPPLE LTFLL
XDELIA
IMREAL ROLE ECUTE
TARGET IDSPACE
MATURE
SIREN SCY ALICE
ARISTOTELE
Figure 5: FP7 TEL projects graph visualization.
– 15 –
The visualization of project connections in Figure 5 exposes one node that could be labeled as the current “epicenter” of TEL projects in FP7. This node represents GALA, the network of excellence on serious games [29]. There are two main factors why this project is such a strong connector:
1. the consortium is extraordinarily large with 31 participating organizations 6, and
2. the project has started only recently in October 2010, following the most recently closed TEL call in FP7 (see the projects timeline in Appendix A) .
Obviously, a project which starts later than other projects has a higher chance of having organizations in its consortium which were already part of previous project consortia. Other projects that carried on multiple consortium members to the GALA consortium are TARGET, GRAPPLE, and STELLAR. Another strong, currently running project is ROLE, which is a harbor for project consortium partnerships from previous projects, and also has overlaps with succeeding project consortia. If we had computed the betweenness centrality of the projects taking into account the direction of the edges, ROLE, STELLAR and MIRROR would be the most betweenness-central projects. Such a computation would, however, statistically favor projects that have started in the middle between the begin date of FP7 and the current date, since in this time window projects are more likely to have outgoing consortium connections in addition the incoming ones.
3.3.2 All TEL Projects – FP6, FP7, and eContentplus A graph of all TEL projects funded in FP6, FP7, and eContentplus is given in Figure 6. The graph includes all 77 projects and a total of 712 connections between those projects. KALEIDOSCOPE is by far the largest node, which can be attributed to the fact that this project had an extremely large consortium of 83 partner organizations, which is more than five times the typical consortium size. It is also evident in this visualization that in addition to strong ties between FP6 and FP7 projects, the eContentplus projects have very strong connections to both FP6 and FP7. This can probably be explained by the fact that eContentplus filled a “funding gap” in 2007 when FP6 funding was stalling following the last FP6 projects launched in 2006, while FP7 funding was kicked off with the first TEL projects starting in 2008. In fact, in 2007 only eContentplus projects were launched with EC funding in our data set (compare also the dynamic network analysis in Section 3.4.3, in particular Figure 13d).
This kind of gap filling by eContentplus, where a large share of organizations funded under FP6 and FP7 engaged in e-content focused R&D projects, could be interpreted as evidence for a “research follows money” attitude of researchers involved in TEL. That is, if there had not been funding from eContentplus, organizations would likely have looked for funding opportunities in TEL-related programmes with different focus between 2006 and 2008.
A table with all projects displayed in Figure 6 along with their SNA metrics (and ranks) is given in Appendix B.
6 See http://learningfrontiers.eu/?q=story/tel-project-landscape&proj=GALA and http://www.learningfrontiers.eu/?q=tel_project/GALA
– 16 –
FP7
ALICE SIREN MATURE ARISTOTELE NEXT-TELL IMREAL ECUTE TARGET MIROR 80DAYS COSPATIAL
GALA METAFORA INTELLEO MIRROR DYNALEARN GRAPPLE XDELIA IDSPACE SCY ROLE
TERENCE
STELLAR
ITEC TEL-MAP eContentplus LTFLL LiLa eViP
FP6
I-MAESTRO mEducator ECIRCUS MACE
EdReNe KeyToNature APOSDLE
UNFOLD OpenScout ASPECT COOPER JEM Math-Bridge
TELCERT RE.MATH iCOPER EUROGENE
PROLEARN MELT KALEIDOSCOPE CONNECT
EduTubePlus Intergeo MGBL ARISE COSMOS ARGUNAUT ELEKTRA PROLIX Organic.Edunet
TENCOMPETENCE OpenScienceResources CITER E-LEGI UNITE LEACTIVEMATH
PALETTE LT4EL ICLASS
VEMUS ICAMP ELU KP-LAB
L2C LEAD
LOGOS CALIBRATE ATGENTIVE EMAPPS.COM
Figure 6: Project consortium progression between FP6, FP7, and eContentplus projects.
3.3.3 Identifying Project Clusters The project consortium progression graph was subjected to cluster analysis using the Louvain method described in [1]. This method first divides the nodes into local clusters, and then collapses each clusters’ nodes into a single node. These two steps are applied repeatedly until the final set of clusters is reached.
There are 6 resulting clusters of projects as listed in Table 3:
• Cluster C0 includes mostly FP7 projects, with some FP6 and eContentplus projects, which focus on learning , development , research and technology as evident form the word cloud extract from these projects’ descriptions.
• Cluster C1 exposes the strongest thematic focus on learning (and education ) of all clusters; there are no other terms that really stand out. The cluster includes a mix of all funding programmes.
• Cluster C2 shows a strong topical emphasis on content , collaboration , knowledge and support ; this cluster is well represented by projects from all funding schemes.
– 17 –
• Cluster C3 includes projects related development , content , competence , tools and testing . In this cluster there is the smallest gap between frequency of occurrence of learning and other terms.
• Cluster C4 has a strong focus on science and education , and also school is a term that stands out.
• Cluster C5 emphasizes mostly on content , development and technology . It has the strongest focus on content of all clusters; yet it includes not only eContentplus projects.
It is evident that eContentplus projects are spread over all clusters, indicating that this funding programme (a) did not disrupt collaboration structures in TEL and (b) was definitely relevant for a topic focus on educational content. Moreover, projects of all funding schemes are represented in all clusters, indicating a coherent research agenda since the first FP6 projects.
Table 3: TEL project clusters in FP6, FP7, and eContentplus (ECP) and the word clouds of their project descriptions.
ALICE [FP7], APOSDLE [FP6], COSPATIAL [FP7], ECIRCUS [FP6], ECUTE [FP7], eViP [ECP], GALA [FP7], I-MAESTRO [FP6], C0 IMREAL [FP7], KALEIDOSCOPE [FP6], MATURE [FP7], MIRROR [FP7], NEXT- TELL [FP7], SIREN [FP7], TARGET [FP7] 80DAYS [FP7], CITER [ECP], DYNALEARN [FP7], EduTubePlus [ECP], ELEKTRA [FP6], ICLASS [FP6], Intergeo [ECP], C1 LEACTIVEMATH [FP6], LiLa [ECP], LOGOS [FP6], METAFORA [FP7], MIROR [FP7], PALETTE [FP6], PROLEARN [FP6], PROLIX [FP6], RE.MATH [FP6] ATGENTIVE [FP6], E -LEGI [FP6], EUROGENE [ECP], ICAMP [FP6], iCOPER [ECP], INTELLEO [FP7], JEM [ECP], KP- C2 LAB [FP6], L2C [FP6], LEAD [FP6], LT4EL [FP6], LTFLL [FP7], mEducator [ECP], OpenScout [ECP], ROLE [FP7], STELLAR [FP7], TEL-MAP [FP7], XDELIA [FP7]
COOPER [FP6], GRAPPLE [FP7], IDSPACE [FP7], MACE [ECP], Math-Bridge C3 [ECP], TELCERT [FP6], TENCOMPETENCE [FP6], UNFOLD [FP6]
ARGUNAUT [FP6], ARISE [FP6], ARISTOTELE [FP7], CONNECT [FP6], C4 COSMOS [ECP], OpenScienceResources [ECP], Organic.Edunet [ECP], SCY [FP7], UNITE [FP6], VEMUS [FP6]
ASPECT [ECP], CALIBRATE [FP6], EdReNe [ECP], ELU [FP6], EMAPPS.COM C5 [FP6], ITEC [FP7], KeyToNature [ECP], MELT [ECP], MGBL [FP6], TERENCE [FP7]
– 18 –
3.4 Organizational Collaborations In addition to the project consortium progression network presented in the previous section, TEL projects can be viewed from another angle: the organizational collaboration graph , contains organizations and their collaborations in the project consortia. This graph = ( shows organizations as nodes and an edge between two nodes if there is any project where both organizations have participated in, i.e. and =
= , ∶ , ∈ ∧ ≠ ∧∃ ∈ ∶ , ∧ , .
3.4.1 Collaborations in FP7 projects
In the collaboration graph for FP7 TEL projects, , the number of participating organizations | | is 211, and the number of collaborations between those organizations amount to 1,983. The | | visualization of is depicted in Figure 7. The size of each node was determined by the betweenness centrality of the node, while edge weight was determined by the number of projects in which two organizations collaborate or had collaborated. The node arrangement was computed using the yEd graph editor’s [28] organic layout algorithm, which tends toward a symmetric and clustered distribution of nodes [27].
UNIVERSIDAD POLITECNICA DE MADRID
FUNDAÇÃO UNIVERSIDADE DE BRASILIA UNIVERSITAET FUER BODENKULTUR WIEN
CENTRAL LABORATORY OF GENERAL ECOLOGY - ZENTRALNA LABORATORIYA PO OBSCHTA EKOLOGIYA
TEL AVIV UNIVERSITY
UNIVERSITEIT VAN AMSTERDAM UNIVERSITY OF HULL
NATIONAL UNIVERSITY CORPORATION, KYOTO UNIVERSITY NATIONAL TECHNICAL UNIVERSITY OF ATHENS EXTRIM MINTIA SOLOUSIONS ETAIREIA PLIROFORIKIS KAI TILEPIKOINONION ETAIREIA PERIORISMENIS EFTHINIS
GAKO HOJIN SEIKEI GAKUEN
THE REGENTS OF THE UNIVERSITY OF CALIFORNIA UNIVERSITAET AUGSBURG
IT UNIVERSITY OF COPENHAGEN
WAGENINGEN UNIVERSITEIT UNIVERSITY OF BATH MAKASH - ADVANCING CMC APPLICATIONS IN EDUCATION, CULTURE AND SCIENCE
EDUCA.CH INSTITUT SUISSSE DES MEDIAS POUR LA FORMATION ET LA CULTURE UNIVERSITE PIERRE MENDES FRANCE JACOBS UNIVERSITY BREMEN GGMBH
UNIVERSITY OF SUNDERLAND SVIETIMO INFORMACINIU TECHNOLOGIJU CENTRAS SPACE APPLICATIONS SERVICES NV KUNGLIGA TEKNISKA HOEGSKOLAN
FACULTES UNIVERSITAIRES NOTRE-DAME DE LA PAIX ASBL BUNDESMINISTERIUM FÜR UNTERRICHT, KUNST UND KULTUR INSTITUTE FOR PARALLEL PROCESSING OF THE BULGARIAN ACADEMY OF SCIENCES JYVASKYLAN YLIOPISTO UNIVERSITATEA POLITEHNICA DIN BUCURESTI BRUNEL UNIVERSITY HOGSKOLEN I OSLO SMART TECHNOLOGIES (GERMANY) GMBH THE UNIVERSITY OF MANCHESTER NATIONAL MINISTRY OF EDUCATION MORPHEUS SOFTWARE VOF GIUNTI LABS S.R.L.
LANDESINITIATIVE NEUE KOMMUNIKATIONSWEGE MECKLENBURG-VORPOMMERN E.V.STIFTUNG UNIVERSITAT HILDESHEIM SERIOUS GAMES INTERACTIVE EUN PARTNERSHIP AISBL
ELFA S.R.O.
INESC ID - INSTITUTO DE ENGENHARIA DE SISTEMAS E COMPUTADORES, INVESTIGACAO E DESENVOLVIMENTO EM LISBOA TIIGRIHUPPE SIHTASUTUS THE MANCHESTER METROPOLITAN UNIVERSITY AURUS KENNIS- EN TRAININGSSYSTEMEN B.V. HERIOT-WATT UNIVERSITY
THE UNIVERSITY OF BOLTON MINISTERIO DA EDUCACAO EDUCATIO PUBLIC SERVICES NON-PROFIT LLC
UNIVERSITY OF PIRAEUS RESEARCH CENTER RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHENUNIVERSIDAD COMPLUTENSE DE MADRID UNI-C, DANMARKS EDB-CENTER FOR FORSKNING OG UDDANNELSE FESTO LERNZENTRUM SAAR GMBH UNIVERSITY OF THE WEST OF SCOTLAND BIT MEDIA E-LEARNING SOLUTION GMBH AND CO KG PLAYGEN LTD CYNTELIX CORPORATION BV PROMETHEAN LIMITED ICODEON LIMITED EBERHARD KARLS UNIVERSITAET TUEBINGEN UNIVERSITEIT UTRECHT
KNOWLEDGE MARKETS CONSULTING GMBH WIRTSCHAFTSUNIVERSITAET WIEN FUNDACION ESADE AALTO-KORKEAKOULUSAATIO UNIVERSIDAD DE VIGO AALBORG UNIVERSITET TTY-SAATIO SHANGHAI JIAO TONG UNIVERSITY INDIRE ISTITUTO NAZIONALE DI DOCUMENTAZIONE PER L'INNOVAZIONE E LA RICERCA EDUCATIVA UNIVERSITY OF CYPRUS MINISTERUL APARARII NATIONALE EDUBIT VZW THE GOVERNING COUNCIL OF THE UNIVERSITY OF TORONTO THE BRITISH INSTITUTE FOR LEARNINGAND DEVELOPMENT LBG UNIVERSITE PAUL SABATIER TOULOUSE III INSTITUTO DE EDUCAÇÃO DA UNIVERSIDADE DE LISBOA UNIVERSITETET I OSLO BIBA - BREMER INSTITUT FUER PRODUKTION UND LOGISTIK GMBH FUTURELAB EDUCATION U&I LEARNING NV ORT FRANCENATO UNDERSEA RESEARCH CENTRE ZENTRUM FUER SOZIALE INNOVATION SENTER FOR IKT I UTDANNINGEN POLITECNICO DI MILANO UNIVERSITAET KOBLENZ-LANDAU CENTRE NATIONAL DE DOCUMENTATION PEDAGOGIQUE CONSIGLIO NAZIONALE DELLE RICERCHE THE OPEN UNIVERSITY FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V KATHOLIEKE UNIVERSITEIT LEUVENUNIVERSITA DEGLI STUDI DI GENOVA TARTU ULIKOOL OPEN UNIVERSITEIT NEDERLAND CENTRE EUROPEEN D'EDUCATION PERMANENTE
VIRTECH LTD LEAN ENTERPRISE INSTITUTE POLSKA SPOLKA Z OGRANICZONA ODPOWIEDZIALNOSCUPPSALA IA UNIVERSITET STICHTING TECHNASIUM ENOVATE AS UNIVERSITY OF LEICESTER CYNTELIX CORPORATION LIMITED COVENTRY UNIVERSITY EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH VOLKSWAGEN AG
UNIVERSITAET GRAZ INSTITUT FÜR ANGEWANDTE SYSTEMTECHNIK BREMEN GMBH UNIVERSITY COLLEGE LONDON ALMA MATER STUDIORUM-UNIVERSITA DI BOLOGNA NOKIA OYJ THE UNIVERSITY OF NOTTINGHAM TAKOMAT JOHNE, SCHNATMANN, SCHWARZ GBR FACULTY OF ORGANIZATIONAL SCIENCES, UNIVERSITY OF BELGRADE UNIVERSITETET I BERGEN TECHNISCHE UNIVERSITEIT DELFT UNIVERSITEIT TWENTE ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE EESTI OPETAJATE LIIT INI DOO DE PRAKTIJK, NATUURWETENSCHAPPELIJK ONDERWIJS V.O.F. IMC INFORMATION MULTIMEDIA COMMUNICATION AG
INOVACIJSKO-RAZVOJNI INSTITUT UNIVERZE V LJUBLJANISIEMENS AKTIENGESELLSCHAFT OESTERREICH ATHABASCA UNIVERSITY ATOS ORIGIN SOCIEDAD ANONIMA ESPANOLA SONY FRANCE S.A. TALLINNA ULIKOOL UNIVERSITE JOSEPH FOURIER GRENOBLESTIFTELSEN 1 SINTEF TECHNOLOGY & SOCIETY ERASMUS UNIVERSITEIT ROTTERDAM UNIVERSITAET DUISBURG-ESSEN NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET NTNU
FRIEDRICH-ALEXANDER UNIVERSITAET ERLANGEN - NUERNBERG
UNIVERSITY OF BRISTOL ZENTRUM FUER GRAPHISCHE DATENVERARBEITUNG E.V. GOETEBORGS UNIVERSITET COMPEDIA SOFTWARE & HARDWARE DEVELOPMENT LTD MOMA SPA TECHNISCHE UNIVERSITAET GRAZ THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF CAMBRIDGE SAXO BANK AS CLEAR COMMUNICATION ASSOCIATES LIMITED - CCA SCIENTER SCRL THE UNIVERSITY OF EXETER AMIS DRUZBA ZA TELEKOMUNIKACIJE D.O.O. FUNDACIO PER A LA UNIVERSITAT OBERTA DE CATALUNYA VRIJE UNIVERSITEIT BRUSSELTECHNISCHE UNIVERSITEIT EINDHOVEN BLEKINGE TEKNISKA HOEGSKOLA UNIVERSITY OF SUSSEX
CENTRO DI RICERCA IN MATEMATICA PURA ED APPLICATA - CONSORZIO GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET HANNOVER NATIONAL AND KAPODISTRIAN UNIVERSITY OF ATHENS ALBERT-LUDWIGS-UNIVERSITAET FREIBURG TESTALUNA SRL LUDWIG-MAXIMILIANS-UNIVERSITAET MUENCHEN UNIVERSITA DEGLI STUDI DI PADOVA HEALTHWARE SPA - PHI EXACT LEARNING SOLUTIONS SPA UNIVERSITE CATHOLIQUE DE LOUVAIN LIBERA UNIVERSITA DI BOLZANO
KOMPETENZZENTRUM FUER WISSENSBASIERTE ANWENDUNGEN UND SYSTEME FORSCHUNGS - UND ENTWICKLUNGS GMBH UNIVERSITY OF LEEDS UNIVERSITAET PADERBORN EMPOWERTHEUSER LTD
FONDAZIONE BRUNO KESSLER THE UNIVERSITY OF WARWICK UNIVERSITA DELLA SVIZZERA ITALIANA CENTRE INTERNACIONAL DE METODES NUMERICS EN ENGINYERIA MEDIEN IN DER BILDUNG ENGINEERING - INGEGNERIA INFORMATICA SPA UNIVERSITAET INNSBRUCK BOC INFORMATION TECHNOLOGIES CONSULTING SP. Z.O.O.
FORSCHUNGSZENTRUM INFORMATIK AN DER UNIVERSITAET KARLSRUHE REGOLA SRL UNIVERSITA DEGLI STUDI DI MILANO IMAGINARY SRL THE PROVOST FELLOWS AND SCHOLARS OF THE COLLEGE OF THE HOLY AND UNDIVIDED TRINITY OF QUEEN ELIZABETH NEAR DUBLIN UNIVERSITA DEGLI STUDI DI VERONA INFOMAN AG
NEUROLOGISCHE KLINIK GMBH BAD NEUSTADT UNIVERSITA DEGLI STUDI DELL'AQUILA SIVECO ROMANIA SA
SAP AG BAR ILAN UNIVERSITY DEUTSCHES FORSCHUNGSZENTRUM FUER KUENSTLICHE INTELLIGENZ GMBH RUHR-UNIVERSITAET BOCHUM THE CITY UNIVERSITY TRACOIN QUALITY BV UNIVERSITY OF HAIFA PONTYDYSGU LTD
BOC ASSET MANAGEMENT GMBH UNIVERSIDAD DE SALAMANCA FACHHOCHSCHULE NORDWESTSCHWEIZ VEREIN OFFENES LERNEN
SOLUCIONES INTEGRALES DE FORMACION Y GESTION STRUCTURALIA, S.A THE UNIVERSITY OF BIRMINGHAM MOHOLY-NAGY MUVESZETI EGYETEM THE HEBREW UNIVERSITY OF JERUSALEM
REGISTERED NURSING HOME ASSOCIATION LIMITED
UNI RESEARCH AS INSTITUTE OF EDUCATION, UNIVERSITY OF LONDONAMNIN D.O.O CENTR ZA ZNANSTVENO VIZUALIZACIJO COPENHAGEN BUSINESS SCHOOL BRITISH TELECOMMUNICATIONS PUBLIC LIMITED COMPANY* LONDON METROPOLITAN UNIVERSITY
KATHOLISCHE UNIVERSITAT EICHSTATT-INGOLSTADT
MTO PSYCHOLOGISCHE FORSCHUNG UND BERATUNG GMBH
JOANNEUM RESEARCH FORSCHUNGSGESELLSCHAFT MBH
Figure 7: Visualization of the FP7 collaboration graph.
– 19 –
A cut-out of the center region of the FP7 collaboration graph with legible node labels is shown in Figure 8.
INESC ID - INSTITUTO DE ENGENHARIA DE SISTEMAS E COMPUTADORES, INVESTIGACAO E DESENVOLVIMENTO EM LISBOA TIIGRIHUPPE SIHTASUTUS
AURUS KENNIS- EN TRAININGSSYSTEMEN B.V. HERIOT-WATT UNIVERSITY
THE UNIVERSITY OF BOLTON
RHEINISCH-WESTFAELISCHE TECHNISCHE HOCHSCHULE AACHEUNIVERSIDADN COMPLUTENSE DE MADRID
FESTO LERNZENTRUM SAAR GMBH UNIVERSITY OF THE WEST OF SCOTLAND PLAYGEN LTD CYNTELIX CORPORATION BV
UNIVERSITEIT UTRECHT
KNOWLEDGE MARKETS CONSULTING GMBH WIRTSCHAFTSUNIVERSITAET WIEN FUNDACION ESADE AALTO-KORKEAKOULUSAATIO
AALBORG UNIVERSITET TTY-SAATIO SHANGHAI JIAO TONG UNIVERSITY
MINISTERUL APARARII NATIONALE THE BRITISH INSTITUTE FOR LEARNINGAND DEVELOPMENT LBG UNIVERSITE PAUL SABATIER TOULOUSE III
BIBA - BREMER INSTITUT FUER PRODUKTION UND LOGISTIK GMBH U&I LEARNING NV ORT FRANCENATO UNDERSEA RESEARCH CENTRE ZENTRUM FUER SOZIALE INNOVATION POLITECNICO DI MILANO UNIVERSITAET KOBLENZ-LANDAU
CONSIGLIO NAZIONALE DELLE RICERCHE THE OPEN UNIVERSITY
KATHOLIEKE UNIVERSITEIT LEUVENUNIVERSITA DEGLI STUDI DI GENOVA OPEN UNIVERSITEIT NEDERLAND CENTRE EUROPEEN D'EDUCATION PERMANENTE
VIRTECH LTD UPPSALA UNIVERSITET
UNIVERSITY OF LEICESTER CYNTELIX CORPORATION LIMITED COVENTRY UNIVERSITY EIDGENOESSISCHE TECHNISCHE HOCHSCHULE ZUERICH UNIVERSITAET GRAZ UNIVERSITY COLLEGE LONDON NOKIA OYJ THE UNIVERSITY OF NOTTINGHAM
TECHNISCHE UNIVERSITEIT DELFT ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE IMC INFORMATION MULTIMEDIA COMMUNICATION AG
SIEMENS AKTIENGESELLSCHAFT OESTERREICH
ATOS ORIGIN SOCIEDAD ANONIMA ESPANOLA
STIFTELSEN SINTEF TECHNOLOGY & SOCIETY
NORGES TEKNISK-NATURVITENSKAPELIGE UNIVERSITET NTNU
UNIVERSITY OF BRISTOL
TECHNISCHE UNIVERSITAET GRAZ THE CHANCELLOR, MASTERS AND SCHOLARS OF THE UNIVERSITY OF CAMBRIDGE CLEAR COMMUNICATION ASSOCIATES LIMITED - CCA SCIENTER SCRL
VRIJE UNIVERSITEIT BRUSSELTECHNISCHE UNIVERSITEIT EINDHOVEN
GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET HANNOVER
Figure 8: Center region cut-out of the FP7 collaboration graph.
The visualizations show one core sub-network in the center, featuring organizations that have strong ties due to several shared projects. It also shows a strong sub-network towards the top-right of the center. The other sub-networks in the periphery of the visualization tend to expose consortia of projects whose members are not involved in multiple projects. The bonds between those sub-networks are established by organizations that are involved in collaboration clusters in both the periphery and the center of the network. These nodes represent connectors in the network. The strongest ties, of course, are built between organizations that collaborate in many different projects and/or that collaborate in projects with large consortia. The five top-ranked among those in FP7 are:
1. Technische Universität Graz, Austria (82 distinct connections in 7 projects)
– 20 –
2. Open Universiteit Nederland, Netherlands (67 / 5)
3. Aalto-Korkeakoulusaatio, Finland (66 / 3)
4. Katholieke Universiteit Leuven, Belgium (63 / 4).
5. ATOS Origin Sociedad Anonima Espanola, Spain (59 / 4)
This is actually a “multi-cultural” top-five list since the top-ranked organization is represented by a psychology department at a university of technology, the second-ranked is an open university, the third-ranked is a multidisciplinary research unit, the fourth-ranked is a hypermedia and databases department at a Catholic university, and the fifth-ranked is a research and IT services company. This is a confirmative signal that TEL is an interdisciplinary research area which needs strong partners from different ends of the spectrum, both in terms of research area and organizational background.
A visualization like this can, among other purposes, be used to identify the most prolific collaborators as an entry point to more detailed investigation. For instance, Technische Universität Graz, although being a university of technology, is primarily involved as a provider of the psychological expertise in the projects. The word cloud of all 26 project descriptions in Figure 9—which displays the 20 word stems with the highest frequency in the descriptions—clearly shows that TEL projects have a strong focus on the human(istic) aspects of TEL, e.g. learn , knowledg[e], design , adapt[ation], collabor[ation], and so forth. A strong and well-connected department of psychology, like that of Technische Universität Graz, has an excellent standing in such a landscape; this indicates that partners providing expertise in non-technological, “cross-cutting” themes have great opportunities in getting a piece of the TEL funding cake.
Figure 9: Word cloud of the 20 word stems with highest frequency in the FP7 project descriptions
3.4.2 Collaborations in all TEL Projects: FP6, FP7, and eContentplus A collaboration graph for all TEL projects in FP6, FP7, and eContentplus programmes, , is given in Figure 10. It includes 604 partner organizations and 9,330 distinct pairs of partner collaboration. Apart from| | the= core network, there are no strongly| | = connected sub-networks, which means that in every project there exists one partner who is involved in at least one other project.
– 21 –
Figure 10: Partner collaborations spanning FP6, FP7, and eContentplus projects.
We calculated SNA metrics and funding statistics for each participant in ; the resulting table of the top 30 organizations is given in Table 4. The table is ordered by PageRank, a metric that not only takes into account the number of edges of each node, but also the “importance” of the neighboring adjacent nodes. This means, an organization’s importance depends on the number of collaborations with other organizations and on the importance of the organization’s collaborators.
The ranking clearly shows that there is a strong relationship between the most “profane” ranking (i.e. funding) and most of the other network metrics. One notable exception is the local clustering coefficient, which has an apparent negative correlation with all other metrics. We recall that clustering of a node in a network refers to the connectedness of the node’s neighborhood. A organizational collaboration network like has some noteworthy properties in this respect: an organization which is involved in only one project will have a high clustering coefficient, since the neighborhood of that organization is strongly connected through the project. So this node will have low scores in centrality metrics and a higher score on clustering. On the other hand, an organization that is involved in several projects will likely connect several sub-networks made up of organizations involved in fewer projects. The organization will thus have a lower clustering coefficient, but likely higher values for the centrality metrics, like betweenness centrality. To illustrate this effect, Figure 13a plots the local clustering coefficient against the PageRank for each organization. The plot reveals and interesting characteristic of the collaboration network, namely that there are two quite clearly separated sets of organizations in the plot. One set of organizations manages to have a significantly higher PageRank than the other set
– 22 – at a similar level of local clustering. This separation is even more evident when plotting the degree (number of connections) against the local clustering coefficient (see Figure 13b).
Table 4: Top 30 organizations involved in TEL projects by PageRank. The numbers in square brackets next to the values represent the rank of that value among all 604 organizations.
Organization PR ▼ BC LC DC CC Funding* THE OPEN UNIVERSITY, UNITED KINGDOM .0125 [1] .1185 [1] .2151 [601] 219 [1] .6012 [1] 3.55 [3] KATHOLIEKE UNIVERSITEIT LEUVEN, BELGIUM .0090 [2] .0752 [2] .1716 [604] 148 [3] .5462 [5] 2.56 [6] OPEN UNIVERSITEIT NEDERLAND, NETHER LANDS .0086 [3] .0414 [6] .2161 [600] 133 [7] .5442 [6] 3.45 [4] JYVASKYLAN YLIOPISTO, FINLAND .0080 [4] .0667 [3] .3170 [588] 170 [2] .5657 [2] 1.26 [39] FRAUNHOFER -GESELLSCHAFT ZUR FOERDERUNG DER .0068 [5] .0529 [4] .1833 [603] 111 [22] .5294 [18] 3.40 [5] ANGEWANDTEN FORSCHUNG E.V, GERMANY DEUTSCHES FORSCHUNGSZENTRUM FUER .0066 [6] .0390 [7] .1916 [602] 106 [27] .5280 [21] 3.68 [1] KUENSTLICHE INTELLIGENZ GMBH, GERMANY ATOS ORIGIN SOCIEDAD ANONIMA ESPANOLA, SPAIN .0064 [7] .0236 [15] .4316 [565] 142 [5] .5482 [4] 1.33 [33] UNIVERSITAET GRAZ, AUSTRIA .0064 [8] .0230 [18] .4016 [573] 148 [3] .5552 [3] 2.03 [10] UNIVERSITEIT UTRECHT, NETHERLANDS .0061 [9] .0203 [23] .4323 [564] 139 [6] .5336 [11] 1.62 [19] INESC ID - INSTITUTO DE ENGENHARIA DE SISTEMA S E .0061 [10] .0368 [8] .4741 [552] 130 [8] .5280 [21] 1.68 [16] COMPUTADORES, INVESTIGACAO E DESENVOLVIMENTO EM LISBOA, PORTUGAL THE UNIVERSITY OF WARWICK, UNITED KINGDOM .0058 [11] .0333 [9] .4754 [551] 129 [10] .5374 [10] 1.68 [17] UNIVERSITY OF CYPR US, CYPRUS .0057 [12] .0176 [26] .4668 [554] 130 [8] .5394 [7] 1.48 [25] THE UNIVERSITY OF NOTTINGHAM, UNITED KINGDOM .0057 [13] .0198 [24] .4902 [542] 129 [10] .5327 [13] 1.52 [22] IMC INFORMATION MULTIMEDIA COMMUNICATION AG, .0056 [14] .0102 [4 6] .3124 [590] 86 [57] .5071 [47] 2.35 [8] GERMANY UNIVERSITAET DUISBURG -ESSEN, GERMANY .0056 [15] .0258 [12] .4773 [547] 125 [12] .5285 [20] 1.82 [14] GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET .0054 [16] .0186 [25] .2860 [592] 87 [54] .5115 [44] 2.41 [7] HANNOVER, GERMANY ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE, .0053 [17] .0111 [41] .5647 [506] 118 [18] .5322 [15] 1.66 [18] SWITZERLAND EUN PARTNERSHIP AISBL, BELGIUM .0052 [18] .0231 [16] .2617 [595] 87 [54] .4863 [97] 1.83 [13] KUNGLIGA TEKNISKA HOEGSKOLAN, SW EDEN .0052 [19] .0242 [14] .5335 [527] 121 [14] .5336 [11] .98 [69] THE PROVOST FELLOWS AND SCHOLARS OF THE .0051 [20] .0211 [20] .5344 [526] 119 [16] .5379 [9] 2.31 [9] COLLEGE OF THE HOLY AND UNDIVIDED TRINITY OF QUEEN ELIZABETH NEAR DUBLIN, IRELAND TECHNISCHE UNIVERSITAET GRAZ, AUSTRIA .0051 [21] .0257 [13] .2577 [598] 90 [50] .4971 [60] 3.56 [2] GIUNTI LABS S.R.L., ITALY .0051 [22] .0207 [21] .2644 [594] 82 [62] .5054 [49] 1.95 [12] CONSIGLIO NAZIONALE DELLE RICERCHE, ITALY .0050 [23] .0074 [58] .5 687 [505] 119 [16] .5303 [17] 1.08 [60] RESEARCH ACADEMIC COMPUTER TECHNOLOGY .0050 [24] .0263 [11] .5512 [517] 114 [20] .5132 [43] 1.26 [40] INSTITUTE, GREECE EOTVOS LORAND TUDOMANYEGYETEM, HUNGARY .0049 [25] .0230 [17] .4986 [538] 124 [13] .5394 [7] .98 [68] UNIVERSITETET I OSLO, NORWAY .0049 [26] .0175 [27] .5171 [533] 121 [14] .5294 [18] 1.32 [34] UNIVERZA V LJUBLJANI, SLOVENIA .0046 [27] .0212 [19] .2468 [599] 78 [93] .4778 [102] 1.16 [50] ELLINOGERMANIKI AGOGI SCHOLI PANAGEA SAVVA AE, .00 46 [28] .0118 [39] .2587 [597] 69 [96] .4483 [130] 1.11 [54] GREECE MEDIEN IN DER BILDUNG, GERMANY .0045 [29] .0122 [36] .6132 [494] 110 [23] .5262 [23] 1.39 [27] CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE, .0045 [30] .0117 [40] .6033 [498] 110 [23] .5 239 [24] 1.37 [30] FRANCE
PR = PageRank | BC = Betweenness centrality | LC = Local clustering coefficient | DC = Degree centrality | CC = Closeness centrality (all metrics were calculated using unweighted edges) * … Funding is stated in million Euro (EC contribution to the project cost). Note that CORDIS states the total funding for each project. The funding per consortium member for each project was computed by dividing the total EC contribution to that project by the number of consortium members. This should give a good estimate.
– 23 –
.014 250
.012 200 .010
150 .008 PageRank .006 Degree 100
.004 50 .002
.000 0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Local clustering cofficient Local clustering coefficient
(a) (b) Figure 11: Local clustering of organizations plotted against (a) PageRank and (b) degree.
Clearly, it is favorable to belong to the set of organizations that achieve a higher PageRank or degree at a given clustering value. This “bump” in PageRank seems to happen when an organization reaches a PageRank value range above .003. For degree, the bump seems to happen when an organization has connections to at least ~80 other organizations. To achieve these values, an organization needs to either connect to new organizations in new projects and/or by connecting to other organizations with a high PageRank.
A degree threshold of at least 80 was achieved by 92 organizations in the data set. Those projects in FP6, FP7 and eContentplus where at least half of the consortium is made up by these organizations are: KALEIDOSCOPE (100%), RE.MATH (88%), STELLAR (81%), LEAD (67%), GRAPPLE, ALICE (60% each), SCY (58%), PROLEARN (54%), and TEL-Map (50%). This short list includes three of the four TEL networks of excellence. GALA; the fourth TEL network of excellence is found at rank 25 with 29%. This can be interpreted as evidence that participation in a network of excellence is highly beneficial to the centrality of an organization in the collaboration network.
The top partnership bonds across all TEL projects are displayed in Table 5. The table shows the 22 pairs of organizations that have collaborated in at least 4 TEL projects. Assuming partnership is only continued from successful projects, we can conjecture that those projects where the listed organizations were involved can be flagged as having lasting impact, at least in terms of continuity in research collaborations. The most important of these projects, ordered by frequency of partnership in Table 5, are: PROLEARN (FP6; 16 pairs), ICOPER (eContentplus; 10 pairs), OpenScout (eContentplus; 9 pairs), GRAPPLE (FP7; 8 pairs), STELLAR, ROLE (FP7; 5 pairs), and PROLIX (FP6, 5 pairs). It is evident that the PROLEARN network of excellence that co-kicked off FP6 succeeded in creating and sustaining strong partnerships, while the KALEIDOSCOPE network of excellence, which started at the same time as PROLEARN, failed to achieve this despite having a much larger consortium.
– 24 –
Table 5: Strongest partnership bonds over all TEL projects in FP6, FP7 and eContentplus.
Rank Partnership # GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET 1. OPEN UNIVERSITEIT NEDERLAND with 6 HANNOVER - THE OPEN UNIVERSITY with KATHOLIEKE UNIVERSITEIT LEUVEN 6 3. THE OPEN UNIVERSITY with OPEN UNIVERSITEIT NEDERLAND 5 IMC INFORMATION MULTIMEDIA COMMUNICATION - THE OPEN UNIVERSITY with 5 AG - JYVASKYLAN YLIOPISTO with THE OPEN UNIVERSITY 5 6. OPEN UNIVERSITEIT NEDERLAND with KATHOLIEKE UNIVERSITEIT LEUVEN 4 DEUTSCHES FORSCHUNGSZENTRUM FUER - OPEN UNIVERSITEIT NEDERLAND with 4 KUENSTLICHE INTELLIGENZ GMBH IMC INFORMATION MULTIMEDIA COMMUNICATION GOTTFRIED WILHELM LEIBNIZ UNIVERSITAET - with 4 AG HANNOVER IMC INFORMATION MULTIMEDIA COMMUNICATION DEUTSCH ES FORSCHUNGSZENTRUM FUER - with 4 AG KUENSTLICHE INTELLIGENZ GMBH IMC INFORMATION MULTIMEDIA COMMUNICATION - with OPEN UNIVERSITEIT NEDERLAND 4 AG - ATOS ORIGIN SOCIEDAD ANONIMA ESPANOLA with UNIVERSITAET GRAZ 4 FRAUNHOFER -GESELLSCHAFT ZUR FOERDERUNG - with OPEN UNIVERSITEIT NEDERLAND 4 DER ANGEWANDTEN FORSCHUNG E.V - POLITECNICO DI MILANO with OPEN UNIVERSITEIT NEDERLAND 4 IMC INFORMATION MULTIMEDIA COMMUNICATION - GIUNTI LABS S.R.L. with 4 AG - ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE with THE OPEN UNIVERSITY 4 IMC INFORMATION MULTIMEDIA COMMUNICATION - WIRTSCHAFTSUNIVERSITAET WIEN with 4 AG - WIRTSCHAFTSUNIVERSITAET WIEN with THE OPEN UNIVERSITY 4 NATIONAL CENTRE FOR SCIENTIFIC RESEARCH IMC INFORMATION MULTIMEDIA COMMUNICATION - with 4 DEMOKRITOS AG DEUTSCHES FORSCHUNGSZENTRUM FUER - TECHNISCHE UNIVERSITEIT EINDHOVEN with 4 KUENSTLICHE INTELLIGENZ GMBH - EUN PARTNERSHIP AISBL with UNIVERZA V LJUBLJANI 4 - EUN PARTNERSHIP AISBL with TIIGRIHUPPE SIHTASUTUS 4 THE PROVOST FELLOWS AND SCHOLARS OF THE - COLL EGE OF THE HOLY AND UNDIVIDED TRINITY with UNIVERSITAET GRAZ 4 OF QUEEN ELIZABETH NEAR DUBLIN
3.4.3 Dynamic SNA of the TEL Project Landscape The previous figures all took the current status of collaborations and projects as a basis for calculating SNA metrics and graph characteristics. To understand the dynamics of the projects and their consortium collaborations we now take a look at the development of SNA metrics of the collaboration network on a yearly basis, starting from 2004 when the first FP6 projects were launched, up to the year 2010 (inclusive). Year 2011 was omitted from the analyses since the year is not yet over, and no new TEL projects were launched thus far.
Figure 12 shows that in FP6 the first set of (eight) projects launched in 2004 introduced 4,199 distinct collaboration connections among 157 organizations in the TEL landscape (Figure 12a). This massive entry number is mainly due to the KALEIDOSCOPE network of excellence, which was launched in that year with an extremely large consortium of 83 partners (the average consortium size in FP6 was 14.5 partners). In Figure 12b we see that the diameter of the network—i.e. the longest shortest path through the network—has reached its peak in 2006, after only 2 years; in 2010, the diameter shrunk to a value of 4, which means that one or more projects have introduced direct connections between previously
– 25 – distant partners. It also shows that the average path length has been stable at a value of around 2.5 since 2006. This means that each organization is on average connected to each other organization by only two intermediate organizations. This indicates that the collaboration network is extremely tightly knit. One correlate of being tightly knit is that such a network tends to be hard for outsiders to join; this can inhibit the diffusion of information and innovation across a network boundary.
1,000 9,330 10,000 6 8,234 5 5 5 5 5 800 7,409 8,000 6,607 4 4 6,047 4 600 4,987 6,000 3 4,199 604 3 2.46 2.52 2.51 2.50 2.50 542 2.18 400 476 4,000 1.74 423 2 371 200 2,000 Diameter 257 Organizations 1 Collaborations Avg. Path Length 157 0 0 0 2004 2005 2006 2007 2008 2009 2010 2004 2005 2006 2007 2008 2009 2010 (a) (b) Figure 12: Overall development of collaboration network since 2004.
Until the end of 2010, the number of organizations involved in TEL projects almost quadrupled (3.9- fold), while the number of project-based collaboration ties between those organizations slightly more than doubled (2.2-fold) during the same time window (cf. Figure 12a). This gap can partly be explained by Figure 13a, which shows that although there has been a steady flow of new projects, these projects have added fewer and fewer new organizations to the picture, exposing a drop from 8.1 new organizations per new project in 2006 to 4.8 new organizations per new project in 2010.
25 20 600 525 New Projects New Projects New Organizations per New Project 500 20 New Collaborations per New Project 15 19.6 14 14 13 400 15 12 14 14 13 10 300 12 9 10 8 7 9 8 7 200 8.1 5 89 84 5 7.1 7.4 56 76 80 69 5.9 100 5.5 4.8 0 0 0 2004 2005 2006 2007 2008 2009 2010 2004 2005 2006 2007 2008 2009 2010
(a) (b)
Ratio of New Organizations in Consortia of New Projects New projects per programme per year 15 Avg. Consortium Size of New Projects FP7 100% 50 14 85% 13 FP6 90% 45 12 eContentplus 80% 40 11 66% 67% 70% 62% 35 10 9 10 60% 30 8 7 44% 50% 42% 40% 25 7 14 6 13 40% 20 6 23.0 5 30% 15 4 8 3 7 20% 14.1 10 12.1 12.0 12.6 11.9 2 4 5 10% 10.9 5 1 3 0% 0 0 2004 2005 2006 2007 2008 2009 2010 2004 2005 2006 2007 2008 2009 2010
(c) (d) Figure 13: Impact of newly launched projects the collaboration network.
Figure 13c demonstrates that the average size of the consortia of newly launched projects has been relatively stable since 2005, ranging between 10.9 and 14.1. In contrast, the average share of newly introduced organizations per launched project has dropped from 66% in 2005 to 40% in 2010. The
– 26 – sharpest drop is evident for projects that started in the year 2008 (from 62% to 42%); this is the year when the first 6 FP7 projects plus 3 new eContentplus projects were launched. So it appears that at the transition from FP6 to FP7 and eContentplus, the project consortia resorted to building on an established core of members.
In Figure 14 we see that the average number of new collaboration ties created by each organization making its debut in TEL projects has, after an initial fall between 2004 and 2005, increased from 7.9 in 2005 to 17.7 in 2010. Hence, starting to participate in TEL projects has an increasingly positive effect in terms of new collaborations with other organizations involved in TEL.
30 26.7
25
20 17.7 15.1 15 12.5 10.8 9.3 10 7.9
5 New Collaborations per New Organization 0 2004 2005 2006 2007 2008 2009 2010
Figure 14: Impact of organizations on collaboration.
The project participation data shows that of the 34 TEL projects launched between 2008 and 2010, 20% were coordinated by organizations which had not participated in any previous (or at that time running) TEL project. The development of this percentage over time is plotted in Figure 15. The sharp increase in 2007 is likely due to eContentplus, where the focus shifted to e-content and metadata, and thus new organizations were introduced. The data shows that even for complete “newbie organizations” in TEL it is absolutely feasible to write a successful project proposal in the coordinator role.
100% 88% 86% 90% 80% 70% 60% 50% 40% 50% 43% 30% 20% Ratio of Projects with Novice 25% 23% 10% Participants as Coordinator 11% 0% 2004 2005 2006 2007 2008 2009 2010
Figure 15: Development of the ratio of projects coordinated by novice organizations
However, the tendency evident in most of the figures in this section points in another direction; it appears that there is less and less demand for new organizations on the TEL market. One the one hand, this is understandable: if an organization launches a new project it is likely to resort to partners it has already successfully collaborated with, particularly as more organizations are entering the “market” every year. On the other hand, it shows that project consortia and collaboration ties between organizations behave like an inertial mass, which impedes the involvement of new and fresh organizations, and likely also new ideas and research foci.
– 27 –
3.5 Geo-Mapping TEL Projects Another interactive visualization we developed based on the geographically enhanced CORDIS data shows the TEL hotspots in Europe (see Figure 16). It allows selection of (multiple) FP7 TEL projects, which will result in the display of
(a) the partners involved in those projects on a Google Map overlay, and
(b) a word cloud of the descriptions of all selected projects.
Clicking on a marker in the map will show a popup window with the name of the organization, the list of projects in which the organization is or was involved, and a link “Show partners”, which, upon being clicked will display all the consortium members of all projects where this organization is or was involved. An interactive online version of this is available at http://learningfrontiers.eu/?q=story/tel- project-landscape .
Figure 16: Google Map overlay with organizations involved in TEL projects.
– 28 –
In combination with the centrality and degree metrics obtained from the analysis of this visualization enables in-depth understanding of collaboration structures and geographical agglomeration of TEL organizations across Europe. The map exposes a strong north-south axis starting in Scotland, via England, the Netherlands, Belgium, and West Germany, to northern Italy with strong “outposts” in Switzerland and Austria.
4 Analysis of TEL Publication Outlets
In general, peer reviewed publications are one major scientific indicator of the impact of research and development work. The same holds true for TEL, which is an emerging research area in several established disciplines like psychology or computer science. Particularly in computer science, conferences have a dominant role in communication of research, with many conference venues having the same or even higher impact than well-ranked journals.
The major gathering of European TEL researchers is the EC-TEL conference, an international refereed conference that was launched in the scope of the FP6 network of excellence PROLEARN. According to Microsoft Academic Search, there are 57 conferences, compared to 19 journals in the computer education category. Such domination raises questions about understanding the communities of conferences and their development pattern in order to have an overview on the current research work of the TEL area. For researchers, understanding the community means getting to know the research environment, which leads to self-adaptation and (hopefully) improvement in the field. For conference organizers and other stakeholders, an overview of their communities is important for maintaining, cultivating and promoting the conferences and their communities.
The structure of scientific collaboration can be researched in great detail by SNA of two distinct data sets: the co-authorship graph and the citation graph. The co-authorship graph reveals the contribution structures of a scientific community by disclosing who has collaborated with whom in terms of co- authoring of papers. The citation graph discloses the influencing areas, conferences, and journals of a conference in terms of cited papers. Together, the two graphs allow a detailed analysis of the knowledge structure and flows within the scientific community but also the knowledge flows between adjacent scientific communities. SNA of TEL communities is only available on the level of single conferences [8, 15, 21] or from the perspective of a project [26]. A systematic comparison of key features of scientific community shows that, depending on the duration of existence, different conferences expose different development patterns.
4.1 Data Set The data set used in our study is the combination of DBLP and CiteSeerX digital libraries. We choose these two because they cover most of the relevant sub-disciplines. We retrieved the publication list of conferences from DBLP and used CiteSeerX to fill the citation list of publications in DBLP. This was achieved by using the canopy clustering technique [13]. Overall, the matching algorithm gave us 864,097 pairs of matched publications, of which only a subset is relevant to TEL, of course.
The data was stored in the TEL-Map Mediabase in a relational database schema. The schema includes several dozen tables. The most important of those are displayed at a conceptual level in Figure 17. The central entity is paper . Each paper has relationships with its authors , the keywords , abstract , and the proceedings in which it appeared. Proceedings are published for an event (e.g. EC-TEL 2011) and each event belongs to an event series (e.g. EC-TEL 2011 is the 6 th event in the EC-TEL event series). To simplify storage and querying, the same concepts are used to represent journal publications. A journal is represented as an event series, each volume of the journal as an event, and each issue as a proceeding. Events can be related to other events, e.g. a workshop event held in conjunction with a conference event.
– 29 –
Keyword cite
N N N
has_keyword Paper authorship Author N N N 1 N N
has_classification has_abstract appear_in Event Series
1 N 1 1
Classification Abstract Proceeding belong_to
1 N N N N belongs_to publish Event related_to 1
N
Figure 17: Data model for TEL papers and events.
From these data sets we extracted the co-authorship and the citation networks for conferences and journals with a primary focus on TEL. The set of most relevant conferences was obtained by the following procedure:
• Starting with definitely relevant TEL journals indexed in DBLP like Educational Technology & Society (ETS), IEEE Transactions on Learning Technology (TLT), and Computers & Education (CE), we computed for authors who have published in these journals a list of conferences where those authors have published the most papers since 2005. A total of 1,135 conference series (repeat: not single conferences, but series) are in the candidate set, which shows that TEL researchers are active in a wide range of sub-disciplines.
• From this result, ordered by papers per event, we filtered the top five conferences whose primary focus is on TEL related topics. Conferences that did not meet the filter were CHI (Computer Human Interaction), HCI (Human Computer Interaction), ICCE (International Conference on Computers in Education), HICSS (Hawaii International Conference on System Sciences) and CRIWG (International Workshop on Groupware). Some of those, while somehow relevant to European TEL, were removed because (a) their main focus is not in TEL, e.g. HICSS, which is a multi-track systems science conference; or (b) because they have a strong regional focus, e.g. ICCE focuses mainly on the Asia-Pacific region.
The resulting set of our top five TEL conference series is listed in Table 6. The table clearly shows that the most relevant international conference is ICALT, which has published 768 papers by ET&S, CE and TLT authors since 2005. The most relevant European conference is ECTEL with 215 papers published by ET&S, CS and TLT authors since 2005. AIED and ITS are venues of primary relevance to work by the artificial intelligence community as related to TEL. That is, the three core TEL conferences on DBLP appear to be ICALT, ECTEL and ICWL, since these have their primary focus in the TEL area.
– 30 –
Table 6: Selection of conferences relevant to the TEL community.
Conference Series Acronym Series Events Relevancy* IEEE Internation al Conference on Advanced Learning Annually 2001 -2010 ICALT 768 Technologies (except 2002) Artificial Intelligence in Education AIED Bi -annually 2005 -2009 216 European Conference on Technology Enhanced Learning ECTEL Annually 2006 -2010 215 Bi -annually 1992 -2010 International Conference on Intelligent Tutoring Systems ITS 168 (except 1994) International Conference on Web -Based Learning ICWL Annually 2002 -2010 90
* … Total number of papers in the conference series since 2005, which were written by authors who have also published in ET&S, CE, or TLT. Note that we considered all proceedings associated on DBLP with the respective event series; this means e.g. that workshop proceedings associated with a particular conference event are included as part of the event series.
The five conferences have published a total 3,291 papers between 2005 and 2010. The word cloud of those word stems that are mentioned at least 100 times in the titles is displayed in Figure 18. It is evident that the core word stem is learn , with system , support , design , model , collabor , web , educ , tutor , adapt in the vicinity.
Figure 18: Word cloud of most frequent terms in TEL conference paper titles.
4.2 Social Network Analysis of TEL Venues and Papers We applied time series analysis on the networks to reveal the following SNA parameters from the networks over time: Densification law, clustering coefficient, maximum betweenness, largest connected component, diameter, and average path length [25]. These parameters enable us to explain the community building process that we proposed in [20], as depicted with adaptations in Figure 19. To interpret the shape of the community, one needs to use a combination of all of those parameters. We note that this is not the only model that explains the development pattern of every conference. We present it here because, as we found in our previous study [20], it is a typical model that describes the community building process of many conferences in different areas in computer science: Initially, there are few connections between authors. After some events, author groups become apparent in the network (“Bonding” in Figure 19), which are—in the best case—gradually integrated through publications that involve authors from more than one group (“Emergence” in Figure 19). Over time, a conference a conference then typically reaches a state of development that either represents an interdisciplinary, hierarchical or focused authorship network, the latter including a strongly connected core group of authors that is connected to other smaller groups.
– 31 –
Figure 19: Development model for conference communities.
4.3 Co-Authorship Network Analysis
4.3.1 Formal Foundations Co-authorships on papers can be modeled as a social network. Let be the set of papers, let be the set of outlets relevant to TEL (conferences, journals, books, etc.), and let be the set of authors. Function represents the authorship of papers by author(s) and is defined as follows: ∈ ∈ if is author of , ∈ ∈ ∶ → otherwise . , Function represents the appearance of papers in outlets and is defined as follows: ∈ ∈ if appeared in , ∈ ∈ ∶ → otherwise . , Assuming that and only contain relevant authors and outlets, respectively, we can define the co- authorship graph with and = ( , =
= ( , ∶ , ∈ ∧ ∧∃( , ∈ ∶ ( , ∧ ( , ∧ ( , . 4.3.2 Overview All five conference co-authorship networks are complex networks. In the years since 2003, these five conferences combined have published papers written by a relatively stable number of 1,350 (co- )authors each year. This is shown in Figure 20.
For illustration, Figure 21 displays the current co-author network for each of the five conferences in thumbnail form. Each thumbnail represents the state of the co-authorship network as of end of year 2010.
– 32 –
1600
1400
1200
1000 ICWL AIED 800 IST ECTEL 600 ICALT
400
200
0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
Figure 20: Cumulative annual (co-)author figures of selected TEL conferences over the last 10 years.
(a) ICALT (b) ECTEL (c) AIED
(d) ITS (e) ICWL
Figure 21: Co-authorship network visualization for the TEL conferences.
– 33 –
4.3.3 Dynamic SNA In the light of a typical development pattern as introduced in Figure 19, these conferences expose a mixed picture. ICALT (a) and ECTEL (b) have well connected authors. For ECTEL, which is only half as old as ICALT, this is a remarkable achievement that is likely based in the origins of ECTEL: the conference was started as an initiative out of the PROLEARN project in 2006, and to this day remains a strongly EU TEL project focused presentation outlet and meeting venue. As we see from Figure 22d, ECTEL gathers about 30% of all authors in the largest connected sub-network, with several clearly distinguishable sub-networks in the periphery. ICWL (e) on the other hand is as old as ICALT and seems to struggle with managing the transition from stage b to stage c in the development pattern in Figure 19. Less than 15% of all ICWL authors form the core circle of this conference. The remaining two conferences AIED (c) and ITS (d) expose very mature author communities, which is likely due to the fact that these two conferences attract a strong core of artificial intelligence (AI) researchers. In that sense, they are probably difficult to compare with the other TEL conferences, since their core topic is AI rather than TEL.
The density, i.e. the ratio between the number of edges and the number of nodes increases over time with a coefficient larger than 1 and lower than 2 (Figure 22a). The clustering coefficient of all co- authorship networks is quite high and falling over the years (Figure 22b) but Figure 22d shows that AIED and ECTEL have quickly growing largest connecting components (i.e. the core sub-network of authors) indicating a faster scientific community building process than for ICALT and ICWL.
ITS has the largest core author group of all five conferences, but it needed longer to develop. Also the maximum betweenness of ITS is the biggest indicating that there are many active key members (i.e. those authors that connect different author communities through co-authoring of papers) contributing to the conference and the community development. ICALT and ICWL have not such a clear pattern, while AIED and ECTEL are developing very fast. Fast development of the community typically indicates that the conference has a tighter focus and/or the authors publishing at those conferences already had strong ties among each other. ECTEL, for example, is a European conference, so the community is by definition smaller than that of ICALT or ICWL, which address TEL communities worldwide.
For interdisciplinary conferences, it is common that there are several strongly connected clusters representing the different disciplines, while there are connecting authors between those. Some TEL conferences under analysis here expose characteristics of interdisciplinary networks (evident in the ICWL network), while there are also cases of apparently focused authorship networks e.g. for ICALT and ECTEL. Still, the largest connected components (the core author group) for TEL conferences tend to include about one third of the authors. For strongly focused and “grown-up” conferences, e.g. SIGMOD or VLDB in computer science (which are beyond their 30 th anniversary), the largest component includes roughly two thirds of all authors who have published there.
All diameters of the co-authorship networks are still growing; this indicates that the development of the communities of these conferences is not finished yet (Figure 22e). The diameter represents the length of the longest shortest path through the network, so a peak in diameter growth would indicate lack of integrating new author communities into the conference community. Also the average path length is still growing indicating again that the overall TEL author network is still growing.
– 34 –
4 (a) Densification law (b) Clustering Coefficient 10 1
0.95
3 10 0.9 ICALT: 0.34889*x1.1976 ICALT 1.0544 0.85 2 ICWL: 1.1149*x 10 ICWL ECTEL: 0.40338*x 1.2415 Number Number of edges ECTEL
1.3817 Clustering coefficient ITS: 0.15818*x 0.8 ITS AIED: 1.0128*x1.1197 AIED 1 10 0.75 1 2 3 4 10 10 10 10 1 2 3 4 5 6 7 8 9 Number of nodes Age (c) Maximum Betweenness (d) Largest connected component 0.08 0.7 ICALT ICALT ICWL 0.6 ICWL 0.06 ECTEL 0.5 ECTEL ITS ITS AIED 0.4 AIED 0.04 0.3
0.02 0.2
Maximum betweenness 0.1 Largest connected component 0 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Age Age (e) Diameter (f) Average Path Length 20 8 ICALT ICALT ICWL ICWL 15 6 ECTEL ECTEL ITS ITS 10 AIED 4 AIED Diameter 5 2 Average Average path length
0 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Age Age
Figure 22: Co-authorship network measures of five conferences in TEL.
4.3.4 Most Prolific Authors and Their Topics The 15 most prolific authors in terms of published papers at the conferences and journals are displayed in Table 7. The ranking considers papers published at venues with a core TEL scope—that is, papers at ICALT, ECTEL and ICWL conferences, as well as ET&S, TLT, and CE journals. To remove bias introduced by conferences and journals with a longer publishing record only papers published since the year 2000 were considered.
The table shows that by far the most prolific author has been Kinshuk, who is with Athabasca University, Canada. The most prolific European author is Demetrios G. Sampson from University of Piraeus, Greece. The ranking clearly shows that, although the conferences have visited all continents and the journals are independent of author and venue location, TEL research (in terms of published outcomes) seems mainly based in Europe and Asia. Only one author accounts for North America, i.e. Kinshuk. However, until 2006 Kinshuk was with Massey University in New Zealand, making the ranking an almost exclusively “Eurasian” list.
– 35 –
Table 7: Fifteen most prolific authors at conferences and journals with a broad TEL scope. Names marked with an asterisk (*) indicate authors currently based in Europe.
Rank Author Name # Papers 1. Kinshuk 59 2. Toshio Okamoto 38 Demetrios G. Sampson* 38 4. Rob Koper* 34 5. Alexandra I. Cristea* 33 6. Nian -Shing Chen 31 7. Maria Grigoriado u* 25 Dragan Gasevic 25 Erkki Sutinen* 25 Ralf Klamma* 25 Chin -Chung Tsai 25 12. Erik Duval* 23 Tak -Wai Chan 23 Gwo -Dong Chen 23 Wolfgang Nejdl* 23
Top topics in top authors’ papers. Assuming that the most important TEL authors according to this list also have an influence in the current focus and future direction of TEL topics, it is worthwhile to analyze the topics of the papers these authors have published. There are 21 paper title term stems 7 that have had a rising frequency between 2008 and 2010 8 (with no intermediate fall). The top-10 of those by ordered by absolute frequency in 2010 are mobil , comput , hypermedia , virtual , learners , engin , cognit , multimedia , study , style , and project . The ten most frequent term stems between 2007 and 2010 without taking into account the frequency tendency are learn , adapt , web , model , educ , system , support , semant , design , and collabor .
The most frequent term stems in 2010 that have not dropped in frequency compared to 2009 are adapt , semant , design , mobil , author , awar , manag , person , and knowledg (compare Figure 23). This word cloud represents the most recent upcoming work of the top authors published in 2010, and should therefore be a good indicator of the current direction. It clearly indicates a way towards adaptive, personalized and mobile learning, with an emphasis on semantics, management, authoring, and awareness.
7 Word stemming was performed using Jon Abernathy’s PHP implementation of the Porter Stemming Algorithm, which can be found at http://www.chuggnutt.com/stemmer-source.php . Prior to that, a stop-word list was applied. A few overlapping word stems resulting from this algorithm were then merged manually. 8 The paper data was last updated in October 2010, so the 2010 frequencies were multiplied by 4/3 to account for the missing three months in 2010. This will tend towards values that underestimate the real frequency, since the DBLP data is usually updated some time (sometimes days, sometimes weeks, sometimes months) after the publication date.
– 36 –
Figure 23: Most frequent terms in papers of top TEL authors in 2010.
4.3.5 Overall TEL Co-authorship Network Looking at the combined co-authorship graph of the core TEL venues (ECTEL, ICALT, ICWL, TLT, ETS and CE), we obtained 14,689 distinct co-authorship relationships among 7,802 authors. Some noteworthy facts about this graph are:
• The largest connected component—i.e. the inner circle of co-authors—consists of 2,249 authors (6,251 co-authorship relations), which corresponds to 28.8% of all authors. Having almost one third of all authors in the inner co-authorship circle indicates that this core TEL author community (displayed in Figure 25) is tightly knit. However, as evident in Figure 24 there are 1,401 connected components in total in the graph, which means that there are a huge number of disconnected co-authorship circles that do not “integrate” with the core.
Figure 24: Complete co-authorship network in the core TEL venues.
– 37 –
• The average degree centrality of the authors is 3.765, which means that each author has been writing papers with almost 4 different colleagues.
• The clustering coefficient of the graph has a value of .718, which indicates a mature co- authorship network.
• The average shortest path length is at 7.6, which is typical for a small-world network, in which most nodes can be reached from every other by a small number of steps, although the network may be very large (cf. Milgram’s small world experiment [14]). The diameter, i.e. the most distant connection between any two authors spans 28 nodes, which is quite a large extent.
Figure 25: Co-authorship network of the “inner circle” of authors in the core TEL venues.
4.3.6 Central Authors in the Co-Authorship Network The top 15 authors in the TEL co-authorship network are listed in Table 8. The table is ordered by betweenness centrality, since this measure is a very good indicator whether an author has integrated different communities. In addition, the table lists the PageRank value as well as clustering coefficient and degree centrality for each author. According to this list the top author is Kinshuk, who weighs in with 72 different co-authors and who is part of more than 3% of all shortest paths through the network. He has by far the highest betweenness and PageRank values, indicating his central position in the network. The most important European authors in this list are R. Koper, who is with Open Universiteit Nederland (a central organization in the European TEL project landscape; see Table 4 in
– 38 –
Section 3.4.2) and A. Cristea who has an affiliation history with University of Warwick and TU Eindhoven, which are also both frequent collaborators in TEL projects.
Table 8: Top 15 TEL authors by betweenness centrality 9.
Betweenness PageRank Clustering Degree Rank Author Centrality ×10 -2 Coefficient Centrality 1. Kinshuk .0315 .2011 .0411 72 2. Rob Koper * .0235 .1191 .0968 63 3. Alexa ndra I. Cristea * .0164 .0869 .1028 32 4. Toshio Okamoto .0116 .1092 .0796 39 5. Yongwu Miao * .0113 .0221 .2051 13 6. Heinz Ulrich Hoppe * .0112 .0765 .1351 32 7. Hugh C. Davis * .0109 .0854 .1336 37 8. Daniel Burgos * .0092 .0260 .2053 20 9. Stephen J. H. Yang .0088 .0439 .1810 15 10. Shian -Shyong Tseng .0074 .0401 .2762 15 11. Chin -Chung Tsai .0073 .0868 .0746 32 12. Davinia Hernández Leo * .0073 .0597 .2080 27 13. Ralf Klamma * .0069 .0870 .1817 41 14. Nian -Shing Chen .0064 .1108 .0692 40 15. Jin -Tan Yang .0061 .0535 .1527 29
The top-10 co-authorship pairs in the core TEL venues are listed in Table 9. Nine of those involve European authors, demonstrating the strong collaboration ties within and with European TEL.
Table 9: Top ten co-author pairs in core TEL venues.
Rank Author Pair Papers 1. Jelena Jovanovic * with Dragan Gasevic 16 2. Juan I. Asensio -Pér ez * with Yannis A. Dimitriadis * 14 Jon A. Elorriaga * with Ana Arruarte Lasa * 14 4. Kinshuk with Nian -Shing Chen 13 Ralf Steinmetz * with Christoph Rensing * 13 6. Manuel Caeiro * with Luis E. Anido -Rifón * 12 Pablo Moreno -Ger * with Baltasar Fernández -Manjón * 12 José Luis Sierra * with Baltasar Fernández -Manjón * 12 Yao -Ting Sung with Kuo -En Chang 12 10. Ignacio Aedo * with Paloma Díaz * 11 Ralf Klamma * with Marc Spaniol * 11
4.4 Structural-Semantic Analysis: SNA and Topic Mining Combined Methodology. In D4.1, “Report on Weak Signals Collection” [23], a simple approach to mining conference abstracts for candidate weak signals was described along with the findings and initial interpretation of the possible meaning of those findings. By combining this—i.e. the “semantic
9 The metrics were computed using the igraph package in R. The betweenness centrality was calculated using unweighted co-authorship connections. An asterisk (*) next to an author’s name indicates that this author’s currently main affiliation is with a European organization.
– 39 – analysis”—with social network analysis of the co-authorship network—i.e. the “structural analysis”— we believe it is possible to gain a better understanding of the significance of candidate weak signals.
Author betweenness centrality was used to identify abstracts from the set of papers found to have unexpected increases in topic prominence as judged by the increase in single-term occurrence (or newly appearing terms) in D4.1. This centrality measure is expected to be a useful indicator of weak signal significance since it shows the author’s importance in chains of communication between otherwise separate (or more-separated) groups of researchers. This can be viewed in two (reciprocal) ways:
1. An author with high betweenness centrality is likely to have more influence over the spread of the idea that appears in a conference paper.
2. An author with high betweenness centrality can be expected to be exposed to more diverse ideas and to identify their significance through a conference paper.
For the purposes of the structural-semantic analysis, the same set of conferences considered in D4.1 was used; betweenness centrality measures were therefore calculated for ICALT, ICWL and ECTEL only, in contrast to previous sections. About two thirds of the top 15 authors ranked by betweenness centrality occur in both sets. The most central author was found to be Chen-Chung Liu (betweenness .00441). The ranked list of authors was inspected down to a betweenness of .00044, i.e. a centrality of 1/10 th relative to the top-ranked author.
Key Authorships. Table 10 shows the authors of papers identified in D4.1 as potentially containing weak signals. It is remarkable that the most central author within this set of papers is at rank 36 and with a betweenness of approximately 1/6th of the most central author (Kinshuk, with a betweenness centrality of .026). While it would be premature to draw conclusions from this observation, it does appear to call into question our approach of using betweenness centrality as a measure of weak signal significance. Consequently, rather than understanding this measure to give us an absolute indicator of weak signal significance, we only use it to indicate the relative significance among the set of candidate weak signals.
Table 10: Betweenness centrality of authors of papers identified in D4.1.
Betweenness Relative # Papers Co -auth. Rank Author Centrality Centrality (D4.1) Group 36 . Chen -Chung Liu .00441 1.0 2 A 39 . Marcus Specht .00398 0.9 1 B 79 . Gary B. Wills .00214 0.5 2 C 95 . Marco Kalz .00168 0.4 1 B 109 . Marcelo Milrad .00134 0.3 1 - 127 . Yvonne Margaret Howard .00122 0.3 1 - 130 . Chin -Yeh Wang .00116 0.3 1 A 144 . Gwo -Dong Chen .00103 0.2 1 A 176 . Hui -Chun Chuang .00068 0.2 1 A 195 . Lester Gilbert .00060 0.1 2 C 220 . Baw -Jhiune Liu .00048 0.1 1 A
The table also shows co-authorship groups within the selected papers. Although it is clear from inspection of the full data set that individuals in each of these co-authorship groups are part of different networks, it would be unwise to assume that the paper “Design and Evaluation of an Affective Interface of the E-learning Systems” by Hui-Chun Chuang (.0007), Chin-Yeh Wang (.0012), Gwo-
– 40 –
Dong Chen (.0010), Chen-Chung Liu (.0044), Baw-Jhiune Liu (.0005) published at ICALT 2010, has a significance according to the sum of centrality measures.
On the basis of this data, and recognizing that this is a qualitative judgment, we might nominate the following as “ researchers to pay attention to ”:
• Chen-Chung Liu
• Marcus Specht
• Gary B Wills
Key Weak Signals. Four principle themes were synthesized in D4.1 on the basis of around 8 prominent terms and for each, there is at least one related paper written by at least one of the authors identified in the previous Table 10. The intersection between the themes and author centrality is summarized in the following Table 11, in which underlining is used to highlight “researchers to pay attention to”.
Table 11: Summary of structural-semantic analysis: themes and matching papers.
Theme and Comment Papers Matching Criter ia “Affect” is a relatively diffuse topic, which was represented through “Design and Evaluation of an Affective Interfac e of the E -learning terms such as “girl”, “emotion”, “skin” and “negative”. Only one Systems” (Hui-Chun Chuang, Chin-Yeh Wang, Gwo-Dong Chen, paper matches the SNA criterion although all of the authors match Chen-Chung Liu, Baw-Jhiune Liu, ICALT 2010) it. “e -Assessment” and its principle technical counterpart, “QTI” also “A Formative eAssessment Co -Design Case Study” (David A. has one paper matching the structural-semantic criteria, this time Bacigalupo, W. I. Warburton, E. A. Draffan, Pei Zhang, Lester with two listed authors. Gilbert, Gary B. Wills, ICALT 2010) “Risk” was highlighted in the context of evidence -based design of “Towards an Ergonomics of Knowledge Systems: Improving the learning environments. Only two papers were identified in D4.1 and Design of Technology Enhanced Learning” (David E. Millard and both also match the SNA criterion, which matched three out of the Yvonne Howard, ECTEL 2010 ) five authors (all three belong to the same department). “Towards a Competence Based System for Recommending Study Materials (CBSR)” (Athitaya Nitchot, Lester Gilbert, Gary B. Wills, ICALT2010) “Authentic learning” is manifested in differ ent ways between the five “An Audio Book Platform for Early EFL Oral Reading Fluency” (Kuo - papers identified in D4.1. Three of these also match the SNA Ping Liu, Cheng-Chung Liu, Chih-Hsin Huang, Kuo-Chun Hung, criterion with four distinct authors from our list. Chia-Jung Chang, ICALT 2010)
“Ambient Displays and Game Design Patterns” (Sebastian Kelle, Dirk Börner, Marco Kalz and Marcus Specht, ECTEL 2010)
“Exploring the Benefits of Open Standard Initiatives for Supporting Inquiry-Based Science Learning” (Bahtijar Vogel, Arianit Kurti, Daniel Spikol and Marcelo Milrad, ECTEL 2010)
From the point of view of a weak signals analysis, the weakest theme appears to be “e-Assessment”. In D4.1 the rise of interest in this theme was ascribed to a main-streaming of e-assessment leading to papers reflecting e-assessment in practice. This is somewhat in contrast to the areas of research that are grouped under the other three themes, which are more aspirational.
On the basis of the structural-semantic analysis, there seems no clear reason to give particular prominence to any of the other three themes; none of them has a markedly more influential set of authors than the others. This leads us to synthesize a vision of the future, latent in the chosen abstracts of the TEL research community, with aims such as:
• to increase the degree to which TEL practice uses tools with evidence-based and principled design methodologies;
– 41 –
• to increase the support for authentic rather than overtly-educational activities through TEL;
• to increase the degree to which affect and emotion are taken account of in learner interactions with TEL systems.
This represents a vision that is progressive rather than conservative and grounded in pedagogic concerns rather than technical or managerial matters.
The specific weak signals falling within the three themes are best identified by reference to the subject matter of the papers, here expressed through the abstracts:
• “Design and Evaluation of an Affective Interface of the E-learning Systems” Students' affections in learning have a significant impact on engagement and learning outcomes. When students have negative emotions, they usually do not learn well. But current e-learning systems often lack many features of profound affection, and fail to provide suitable emotional interaction. In this paper, we evaluate some studies of affective interaction e-learning systems. We also proposed our approach to develop an emotionally interactive learning system.
• “Towards an Ergonomics of Knowledge Systems: Improving the Design of Technology Enhanced Learning” As Technology Enhanced Learning (TEL) systems become more essential to education there is an increasing need for their creators to reduce risk and to design for success. We argue that by taking an ergonomic perspective it is possible to better understand why TEL systems succeed or fail, as it becomes possible to analyze how well they are aligned with their users and environment. We present three TEL case studies that demonstrate these ideas, and show how an ergonomic analysis can help frame the problems faced in a useful way. In particular we propose using a variant of ergonomics that emphasizes the expression, communication and use of knowledge within the system, we call this approach Knowledge System Ergonomics.
• “Towards a Competence Based System for Recommending Study Materials (CBSR)” Most e-learning systems require intervention from a teacher. The development of adaptive hypermedia systems, such as intelligent tutoring systems, aimed to reduce the teachers' task. However, such systems are still at risk of inconsistently modelling the user when estimating a learner's knowledge level. We propose a system called CBSR (Competence based System for Recommending Study Materials) which recommends appropriate study materials from the Web without requiring teacher intervention, based upon a competency model. This has the benefit of an improved pedagogical approach to e-learning, and a more consistent profile of learners' competences which can persist though their life.
• “An Audio Book Platform for Early EFL Oral Reading Fluency” Oral reading fluency is essential to overall reading achievements and repeat reading has been found to be an effective strategy for oral reading fluency. Choral reading is the most authentic use of repeated readings in the EFL primary grades. However, teachers have neither sufficient time nor adequate expertise to deal with non-fluent readers. Hence, challenges with oral reading fluency and motivation have long been considered a common characteristic for teacher and students. Hence, this study proposed a one-to-one Audio Book Platform ...
• “Ambient Displays and Game Design Patterns” In this paper we describe a social learning game we implemented to evaluate various means of ubiquitous learning support. Making use of game design patterns it was possible to implement information channels in such a way that we could simulate ubiquitous learning support in an authentic situation. The result is a prototype game in which one person is chosen randomly to become “Mister X”, and the other players have to find clues and strategies to find out who is the wanted person. In our scenario we used 3 different information channels to provide clues and compared them with respect to user appreciation and effectiveness.
– 42 –
• “Exploring the Benefits of Open Standard Initiatives for Supporting Inquiry-Based Science Learning” Mobile devices combined with sensor technologies provide new possibilities for embedding inquiry- based science learning activities in authentic settings. These technologies rely on various standards for data exchange what makes the development of interoperable mobile and sensor-based applications a challenging task. In this paper, we present our technical efforts related to how to leverage data interoperability using open standards. To validate the potential benefits of this approach, we developed a prototype implementation and conducted a trial with high school students in the field of environmental science. The initial results indicate the potential benefits of using open standards for data exchange in order to support the integration of various technological resources and applications.
The key weak signals of recently-recognized R&D needs and associated capacities emerging are:
• Emotionally interactive learning systems;
• An ergonomic perspective to understanding why TEL systems succeed or fail;
• Intelligent tutoring (broad sense) systems based on alternatives to cognitive models (e.g. competence);
• Game design patterns for learning support using at-hand technologies;
• Exploitation of sensor technologies in mobile devices for authentic learning activities.
The European Commission demonstrated great sensibility for these weak signals, since they are well represented in the target outcomes of the TEL objective in the upcoming ICT Call 8, which include TEL systems endowed with capabilities of human tutors with an affective and personalized design, as well as tools for fostering creativity and non-standard thinking 10 .
4.5 Citation Network Analysis The citation networks of the TEL conferences ECTEL, ICALT, ICWL, ITS, and AIED—like the co- authorship networks—are complex networks with a ratio between number of edges and number of nodes still growing (greater 1 and less than 2 in Figure 26a). The clustering coefficients of all conferences are similar, with ICWL exposing a higher coefficient than the other four conferences (Figure 26b). But Figure 26d shows that the literature of ICWL and ICALT is much less connected than that of ITS, AIED and ECTEL, which indicates that the two former have a broader, more interdisciplinary scope than the three latter. This is supported by the maximum betweenness in Figure 26c showing the existence of more common core references in these scientific communities. The diameters of ECTEL and AIED are beginning to shrink very early indicating that the body of literature of these community is quite stable and the themes of the communities are found. Also the path length development is supporting this indicator.
10 See http://ec.europa.eu/research/participants/portal/page/cooperation?callIdentifier=FP7-ICT-2011-8
– 43 –
(a) Densification law (b) Clustering Coefficient 104 0.5 ICALT: 0.072098*x 1.3285 ICWL: 0.024549*x 1.612 0.4 3 10 ECTEL: 0.030923*x 1.4728 0.3 ITS:0. 10363*x 1.3992 ICALT AIED: 0.042131*x 1.5493 2 0.2 ICWL 10 ECTEL Number of edges
Clustering0.1 coefficient ITS AIED 1 10 0 1 2 3 4 10 10 10 10 1 2 3 4 5 6 7 8 9 Number of nodes Age (c) Maximum Betweenness (d) Largest connected component 0.1 1 ICALT ICALT 0.08 ICWL 0.8 ICWL ECTEL ECTEL 0.06 ITS 0.6 ITS AIED AIED 0.04 0.4
0.02 0.2 Maximum betweenness
0 Largest connected0 component 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Age Age (e) Diameter (f) Average Path Length 20 8
15 6
10 ICALT 4 ICALT
Dia me ter ICWL ICWL 5 ECTEL 2 ECTEL ITS Average path length ITS AIED AIED 0 0 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 Age Age
Figure 26: Citation network measures of five conferences in TEL.
5 Analysis of the TEL Social Web
Starting in the PROLEARN project 11 , RWTH has been hosting the collection of social web artifacts related to professional learning and technology enhanced learning from different sources, including blogs, news lists, podcasts and websites. TEL-Map intends to use the content of these social web artifacts for supporting weak signal analysis by identifying topics and topic shifts on the TEL social web. Today, probably the most relevant channels for researchers, practitioners, consumers, and commentators are blogs and micro-blogs (tweets). Since blogs are the richest data set in Mediabase, we have extended the blog sources in the original PROLEARN Mediabase with additional blogs and feeds relevant to TEL. These sources—blogs and feeds—were collected via community mobilization in the Learning Frontiers portal feed aggregator (see Section 2.2). All sources provided in the feed aggregator are periodically ingested into the TEL-Map Mediabase.
Sources in the TEL-Map Mediabase are visited each night by a web crawler process that visits all sources and looks for new entries in these sources, e.g. new blog posts. One problem with RSS/Atom feeds is that they usually contain only a limited number of recent entries, typically not more than 20. In many blogs, particularly those kept by frequent bloggers or blogger communities, the feed reflects
11 http://www.learningfrontiers.eu/?q=tel_project/PROLEARN
– 44 – only a tiny fraction of the full body of entries posted by the blogger(s). To remedy this issue for newly ingested sources, the crawler also parses the HTML sources of the blog web pages or blog post pages for links to older entries that are not listed in the most recent feed. This is possible since most blog hosting platforms display links to the blog or website archive in a side pane of the page. All new entries found in the feed and the archived entries found via hyperlinks are retrieved from the web and stored in the Mediabase both in original markup form and plain non-markup text. In addition a word set is extracted from the text and stored in the database.
The word sets are immediately processed by a burst detector process, which identifies bursty words (i.e. frequently occurring words) in the sources. Generally speaking, bursts refer to topics which appear, gain popularity, and then fade [11]. For instance, if a blogger is writing blog posts about a conference that she is attending, it is likely that during the conference the blogger’s entries expose bursty words related to the topic of the conference. Each word burst is associated with a burst power value that allows ranking and visualization of bursts, e.g. via a word cloud.
Another process extracts links (URLs) from the markup texts of the new entries found in the blogs and feeds. All links are collected in the database and recorded with their original source. All other existing sources are scanned whether their URL matches one of the links. All matches are recorded in the database. This way, we are able to maintain a structured, inter-linked representation of the artifacts in the TEL blogosphere and the TEL web as represented by TEL-related RSS/Atom feeds in the Mediabase.
5.1 Social Web Data Set Data Model. The data model of the indexed TEL blogosphere is presented in Figure 27. Its main entities are described as follows. Blogs and websites are represented as sources that may contain several entries (e.g. blog posts or news pages). Both sources and entries are a sub-concept of actor , which is an abstract concept introduced in the Mediabase to facilitate network analysis by representing social media artifacts. Actors, which may have community-provided tags, can be located via a URL . These URLs may be referenced by other entries.
has Comment N
1
Entry entry_urls URL N N
1 N 1
actor_url Tag is_a Burst source_entries N 1 N 1
1 Actor actor_tag N
Source is_a 1 1
Figure 27: Relational model of the TEL blogosphere.
Data Set. At the time of writing this report, the TEL Media database of TEL-Map Mediabase contains 804 TEL-related sources, which contain a total of 341,649 entries. Those entries point to more than 1.08 million distinct URLs in their hypertext. 58,460 of these URLs (ca. 5%) represent sources and
– 45 – entries indexed in the Mediabase; the rest refers to sources outside of the collected TEL sources. The 1.08 million URLs are hyperlinked 1.86 million times in total from the different entries. That is, each entry includes 5.5 hyperlinks on average.
Data Collection. Collection of blog sources started in 2006 in the context of the PROLEARN project. The distribution of the number of blogs entered since then as well as the blog entries indexed by the crawlers is displayed in Figure 28. Interestingly, the highest number of blog sources was added to the Mediabase after the PROLEARN project ended (445 sources in 2008). In 2009 the activity almost stalled, while after the launch of the TEL-Map project in 2010 the figures are rising again (71 sources in 2010 and 82 sources in 2011). The development of the number of yearly blog entries indexed developed similarly. The figure shows the entries indexed since the year 2000, although there are many entries indexed that were published earlier. There is a discrepancy between blogs added in a year and the entries in indexed in that year. The reason is that when a blog is added for instance in 2008, the crawler also indexes archived entries that were published before 2008, adding blog entries to previous years. So the current increase in added blog sources in the coming year will lead to a “delayed” increase of indexed blog entries that were actually published in 2011, but not indexed yet. The figure suggests a lag of 2-3 years when considering the sharp increase in indexed blog entries starting in 2004, although the indexing only began two years later in 2006.
Although the number of indexed entries is very large and a useful body to analyze, there are certainly more than 804 relevant sources out there on the web for TEL. One focus in future TEL-Map work therefore will lie in continuously extending the body of indexed sources.
80,000 500
70,000 450 400 60,000 350 50,000 300 40,000 250 Entries Blogs 30,000 200 150 20,000 100 10,000 50 0 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Figure 28: Number of blogs added to and blog entries indexed in the TEL-Map Mediabase.
5.2 Formal Foundations Reference networks between sources and entries on the TEL social web can be modeled as social networks. Let be the set of social web sources (e.g. blogs, web pages), let be the set of distinct entries (e.g. blog posts, news posts) contained within these sources, and let be the set of actors, representing a union of sources and their entries. Function maps the =entries ∪ to their ∈ containing sources ∈ and is defined as follows: , if ∈ is contained in s ∈ ∶ × → , otherwise . Function represents the links from entries ∈ to actors ∈ as follows: if links to , ∈ ∈ ∶ × → , otherwise .
– 46 –
Function represents a link structure similar to , however only links from entries to entries are considered, i.e. if links to , ∈ ∈ ∶ → otherwise . , Now we can define several graphs, of which we focus on the following two:
1. A directed link network graph representing links between social web entries = ( , , with = = ( , ∶ , ∈ ∧ ∧ ( , .
2. A directed link network graph representing links between social web sources = ( , , with an edge being established between any originating source and and any target source = , if any entry of source links directly to source , or indirectly via one of the entries contained in source : ( , ∶ , ∈ ∧ ∧∃ ∈ : ( , ∧ = . ∃ ∈ : = ∧ ( , ∨ ∃ ∈ : ( , ∧ ( , 5.3 Results
5.3.1 TEL Blog Network and Most Central Blogs
Hyperlink Graph. Graph , which is visualized in Figure 29, consists of 617 blogs (nodes) and 7,303 directed links (edges) between the nodes. As defined above, each edge represents a hyperlink appearing in the source blog that points to the target blog. The graph by definition only includes blogs
Figure 29: TEL blogs link network visualization, excluding self-references.
– 47 –
that link to another blog or that are being linked to by another blog, therefore not all sources indexed in the TEL-Map Mediabase are represented in the network. However, the network includes 77% of all Mediabase sources, which indicates a healthy, interlinked selection of sources for the Mediabase.
The network exposes one single weakly connected network, i.e. the largest connected component includes 100% of the network’s nodes; network analysis reveals that there are a total of 217 strongly connected components, that is, sub-networks where all nodes are immediately connected to each other through edges. The diameter of the network equals 9 and the average path length is 2.97, which indicates a highly integrated network, where nodes are close to each other.
Top Sources. Table 12 exposes the twenty most relevant blog sources in the TEL-Map Mediabase ordered by PageRank, which not only takes into account the weight of edges between nodes, but also the importance of the source node of a link. The “TEL” column indicates whether a blog is primarily about TEL related topics (e.g. e-learning, educational technology, etc.) It is evident that nodes with a high PageRank typically also have strong centrality characteristics in the network. The list includes some prominent blogs, like those of Stephen Downes (ranked 6 th and 8 th ) and CogDogBlog (9 th ), while it also includes references to sources that are not genuinely TEL related, but of high relevance to TEL, e.g. Creative Commos News (2 nd ) which deals with licensing issues, and Read/WriteWeb (1 st ) or Google Operating System (14 th ) which deal with (web) technology with no particular TEL focus. Since about half of the blogs in the table are not genuinely about TEL, it appears that TEL blogs heavily link to sources that discuss cutting-edge technology.
Table 12: Top twenty blog sources by PageRank. The number in square bracket indicates the blog’s overall rank for the respective metric.
Between - TEL Source PageRank In-Degree Authority Hub ness Centr. Read/WriteWeb – http://www.readwriteweb.com .0449 [1] 178 [1] .0228 [1] .0257 [1] .1238 [1] Creative Commons » CC News – http://creativecommons.org .0400 [2] 144 [3] .0185 [3] .0212 [2] .0386 [4] apophenia – http://www.zephoria.org/thoughts .0205 [3] 99 [7] .0127 [7] .01 52 [6] .0162 [13] X Weblogg -ed – http://weblogg -ed.com .0158 [4] 132 [4] .0169 [4] .0197 [3] .0384 [5] Joi Ito's Web – http://joi.ito.com .0149 [5] 19 [117] .0025 [117] .0029 [10 9] .0015 [151] X Stephen's Web – http://www.downes.ca .0143 [6] 150 [2] .0192 [2] .0000 [472] .0000 [390] X Cool Cat Teacher Blog – http://coolcatteacher.blogspot.com .0100 [7] 95 [8] .0122 [8] .0143 [7] .0092 [35] Half an Hour – http://halfanhour.blogspot.com .0098 [8] 112 [5] .0144 [5] .0163 [4] .0595 [3] CogDogBlog – http://co gdogblog.com .0098 [9] 78 [10] .0101 [10] .0117 [9] .0349 [6] X Moving at the speed of creativity – http://www.speedofcreativity.org .0093 [10] 86 [9] .0111 [9] .0129 [8] .0268 [9] X elearnspace – http://www.elearnspace.org/blog .0090 [11] 111 [6] .0143 [6] .0158 [5] .0798 [2] iterating toward openness – http://opencontent.org/blog .0085 [12] 45 [28] .0059 [28 ] .0069 [26] .0085 [37] X Ideas and Thoughts from an EdTech – http://ideasandthoughts.org .0084 [13] 68 [12] .0088 [12] .0105 [11] .0152 [14] Google Operating System – http://googlesystem.blogspot.com .0081 [14] 42 [32] .0055 [32] .0063 [31] .0026 [113] X D'Arcy Norman dot net – http://www.darcynorman.net .0080 [15] 56 [17] .0073 [17] .0085 [17] .0103 [29] Seth's B log – http://sethgodin.typepad.com .0079 [16] 68 [12] .0088 [12] .0095 [13] .0043 [76] X K12 Online Conference – http://k12onlineconference.org .0079 [17] 57 [16] .0074 [16] .0088 [15] .0010 [183] X The Rapid eLearning Blog – http://www.articulate.com/rapid -elearning .0078 [18] 43 [30] .0056 [30] .0062 [33] .0032 [97] Avant Game – http://avantgame.blogspot.com .0075 [19] 8 [232] .0011 [232] .0014 [195] .0029 [105] Michael Geist Blog – http://www.michaelgeist.ca .0074 [20] 30 [61] .0039 [61] .0046 [59] .0039 [81]
Terms in Top Sources. The frequency of word appearance in the entries contained in the blog listed in Table 12 is displayed as a word cloud in Figure 30. Apart from Google , frequently appearing terms include for instance: access , copyright , internet , time , course , Canadian , people , search , learn . Unfortunately this list does not provide conclusive hints to potentially trending topics, probably with the exception of access and copyright .
– 48 –
Figure 30: Top 100 word stems appearing in 2011 blog entries of the top 20 blogs.
5.3.2 TEL Blog Clusters The blog network includes six communities / clusters, which were identified using the Louvain method [1]. The clusters are listed in Table 13 along with frequency of word stems appearing in the entries of those blogs in 2011. In total, there are 16,371 blog entries indexed during the year 2011 to date 12 .
Clusters C1 and C2 have a negligible count of blog entries, and cluster C2 in particular is represented mostly by blogs in German language. This language-based clustering indicates that bloggers tend to point to sources in their own language.
Table 13: Clusters of TEL blogs indexed in Mediabase.
Color in # Blogs / Entries Top 20 Word Stems in 2011 Figure 31
160 blogs (19.9%) C0 Yellow 6,865 entries (41.9%)
13 blogs (1.6%) C1 Gray 30 entries (0%)
43 blogs (5.3%) C2 Pink (German stop-words only) 302 entries (1.8%)
133 blogs (16.5%) C3 Green 5,389 entries (32.9%)
117 blogs (14.6%) C4 Blue 1,770 entries (7.1%)
150 blogs (18.7%) C5 Brown 1,447 entries (8.8%)
12 Note that the blog network in Figure 29 does not include all indexed blogs, therefore the figures in Table 13 do not add up to 100%.
– 49 –
The largest cluster, C0, does not have any form of learning or education in its top 20 word stems, indicating that this cluster represents a set of general technology related blogs of some relevance to the
TEL community. Those clusters with a topic focus closely related to TEL appear to be C3, C4, and C5, accounting for roughly 50% of all blogs and blog entries. Frequent word stems appearing (apart from learning-related word stems) in these clusters include among others: technolog , mobil , social , design , develop , share , open , peopl , and content . One interpretation of these terms could be an emphasis on “designing social and mobile technologies for people.” Particularly the design and mobile aspects can also be found in prolific authors’ papers as discussed in Section 4.3.
Figure 31 presents the TEL blog network in a colored style, where each color represents one of the clusters, the nodes size corresponds to the PageRank of the node, and the edge thickness corresponds to the frequency of links from one to the other node. Each node was colored using the color of its cluster; each edge was colored using the color of the source node’s cluster. The node labels represent the node’s identifiers in the database so that interested readers can query the database for details using the TEL-Map Mediabase Query Widgets (see Section 6). The layout was generated using Gephi’s ForceAtlas2 layout algorithm. The visualization exposes a few nodes that stand out in terms of
PageRank, with most of the important nodes belonging to cluster C0, i.e. the technology cluster.
783
85 104 892 787 914 834 130 881 147 558 820 917 379 929 237 883 785 600 439 255 136 609 906 576 901 853 426 617 516 102 874 526 882 425 146 21 215 524 465 519 885 807 563 597 886 258 638 628 140 858 8 636 591 810 487 483 304 795 305 514 565 490 137 329 594 711 786 183 551 614 335 24 624 453 108 771 854 265 653 103 913 710 878 272 122 654 12 474 765 469 675 308 708 370 239 652 310 704 100 434 300 101 254 438 836 716 269 482 712 325 595 687 224 307 773 262 279 347 309 139 312 169 238 133 649 602 685 396 372 22 400 391 306 96 518 596 241 755 705 669 9 290 324 534 302 106 132 264 422 1 26 420 843 292 489 857 127 119 721 598 501 171 275 242 682 87 278 494 174 772 257 703 358 639 86 927 577 221 504 23 267 713 593 220 192 135 375 832 109 123 776 17 180 507 440 113 251 105 557 462 252 442 791 415 170 16 283 599 322 219 825 177 83 13 701 298 615 459 928 678 240 666 261 151 365 376 15 466 110 346 554 260 556 571 293 491 263 228 92 424 157 536 664 281 797 750 680 707 303 806 93 756 112 722 650 532 414 81 646 90 578 443 297 209 32 78 629 403 473 764 910 668 547 395 448 362 82 207 6 189 359 356 168 301 409 194 658 80 831 360 214 550 266 125 402 529 815 352 273 570 344 826 689 196 700 696 803 922 89 326 749 193 661 657 181 253 327 676 809 166 11 68 588 502 590 419 690 729 107 693 176 655 460 575 686 695 284 548 345 75 27 684 515 580 569 416 620 407 73 484 586 271 14 179 688 216 814456 585 579 229 198 204 601 38 447 446 412 458 643 244 702 706 374 512 508 217 384 39 366 268 540 341 871 662 718 184 282 581 475 259 497 339 218 382 248 496 505 161 399 389 545 227 478 566 582 296 561 185 393 91 288 154 549 451 560 294 397 608 77 715 367 651 343 564 640 4 663 759 589 510 697 461 371 535 485 291 380 559 751 665 225 210 330 930 660 357 354 411 418 642 404 613 436 337 607 699 348 493 383 525 506 60 647 758 95 517 468 498 387 634 641 671 250 150 464 406 299 121 363 435 40 280 542 143 25 681 839 230 222 353 388 531 674 583 152 351 452 373 338 511 546 428 572 530 69 398 544 28 203 433 405 153 552 683 445 385 837 539 417 333 835 714 775 522 495 287 467 454 486 503 74 610476 648 131 645 538 622 631 679 931 477 863 423 361 562 527 42 274 757 492 212 541 289 332 450 141 852 844 64 523 568 368 43 386 145 156 410 331 499 644 401 246 667 378 148 120 659 114 827 924 849 165 128 637 5 621 149 29 134 231 431 98 41 437 876 233 441 677 247 286 413 155 444 692 635 470 328 182 488 848 349 574 838
813
Figure 31: Colored TEL blog clusters.
– 50 –
5.3.3 Bursts We have analyzed the word bursts appearing in blog sources using a dynamic topic mining approach. As described in Section 5.1, a burst occurs when particular words are used very frequently by bloggers. The “power value” of the burst represents the word frequency. Burstiness can then be analyzed in the context of particular time windows. For this analysis, the word bursts for all sources were grouped by year and analyzed for tendencies of rising and falling frequency. Standard word stemming procedures and stop-word lists were applied.
There are 2,640 word stems that started to appear in 2011 and never had bursts before 2011. A word cloud of the top 100 of those is displayed in Figure 32. The words refer to highly diverse topics from various areas such as politics (e.g. gaddafi , mubarak ), tools (e.g. bitcoin , itwin ), hardware (e.g. chromebook ) conferences (e.g. acmmm11 , itsc11 ), websites (e.g. bestcollegesonline.com ), and so forth. Some bursts of relevance to TEL include, among others:
• Screencastcamp : a screencastcamp is “a gathering of screencasters and visual communication aficionados wanting to network, learn, and collaborate on the art of screencasting. The event relies on the passion and creativity of the attendees—all sessions, discussions, and demos are led by attendees sharing their knowledge” [ http://screencastcamp.com ].
• MobiMOOC : A MOOC is a massive open online course, i.e. it involves many participants and is open to anyone who wants to join. [ http://www.mooc.ca ]; MobiMOOC adopts this concept in a mobile learning context.
• Edcamp : The edcamp model is based on the […] BarCamp model, which is an open ad-hoc gathering for sharing and learning including demos, discussions, and a focus on interaction. In edcamps, sessions are not planned or scheduled until the morning of the event using a scheduling board on which attendees can place an index card with their session on it. [http://www.edutopia.org/blog/about-edcamp-unconference-history ]
• Studyboost : “provides [an] interactive social media studying platform to enable learning beyond the classroom. StudyBoost allows students to study by answering self- or teacher- prepared questions, at home or while on the go, via two of the most highly used technologies
Figure 32: Bursty terms appearing only in 2011.
– 51 –
amongst students: SMS text messaging and instant messaging. For teachers, StudyBoost empowers them to use popular technology to further engage their students beyond the classroom or to integrate technology for learning into their plans. For students, StudyBoost makes true mLearning a reality by leveraging technology they are intimately familiar with and that they carry around everywhere.” [ https://studyboost.com/about_us ]
These four bursts in 2011 all point in to a model of leveraging web technology (in particular: mobile technology) for an open and inclusive approach to education and learning.
In a further step we identified bursty words that have been rising in power over the last three years. 3,641 word stems match this criterion, of which the top one hundred are displayed as a word cloud in Figure 33. Again, the figure exposes many terms with no particular TEL provenance. However, there are mentions of technologies and trends that are definitely relevant to TEL (and appearing to point into a similar direction as the bursts exclusively appearing in 2011 as presented above). For instance:
• Screencastcamp (see above);
• ds106 (an open online course on digital storytelling; see http://ds106.us and transmedia (often used in conjunction with storytelling);
• mobilelearn , mlearncon , and tablet (as a mobile device).
Figure 33: Bursty terms with rising frequency over the last three years.
6 Embeddable Interactive Visualizations and Queries
As a typical result of interacting with information visualization artifacts, the exploration of the social- network view on TEL-Map Mediabase artifacts such as papers, blogs and projects as presented in previous sections will spawn more specific questions and exploratory tasks (cf. [5]). It may trigger wishes for “zooming into” the data by obtaining more detailed information on contained nodes and edges, e.g. a list of the top funded organizations or the projects with the highest rate of consortium progression to or from other projects, or all papers that have specific words in the abstract, etc.
– 52 –
To achieve interaction with the data based on specific queries arising during the exploratory process, we developed a web-based toolkit for interactive SQL query visualization, which was developed specifically for end-users.
The toolkit was implemented as a set of inter-communicating Google Gadgets, allowing users to connect with TEL-Map Mediabase databases and query these databases using SQL. The result of a query is immediately visualized by simply choosing one of the predefined types of visualization, i.e. tables (the typical result presentation in database query applications), bar and pie charts, timelines, and graphs. There are several configuration options for the more complex visualizations, particularly for graphs.
For instance, the screenshot in Figure 34 shows two different visualizations of a query that obtains an ordered list of pairs of organizations that are involved in the same FP7 project consortia, restricted to pairs that appeared together in more than one project consortium—i.e. a subset of the partnership graph introduced in Section 3.4. The left side of the figure shows the query results visualized as a table. With two mouse clicks the same query results can be visualized as a graph. For constructing the graph on the right-hand side of Figure 34 the first column of the query result is interpreted as the source node, the second column as the target node, and the third column as the edge weight. The layout algorithm for the graph visualization, the meaning of columns, and other parameters can be configured easily using dropdown boxes (the configuration portion was clipped from the screenshots in Figure 34.
Figure 34: Visualization of the same SQL query as a table (left) and as a graph (right).
Figure 35 shows an example of a timeline-based visualization. The first column of the result is automatically interpreted as a timestamp, and all other result columns are plotted against that value. The top-right corner displays the values for the current point in time, where the user hovers with the mouse cursor. The visualization in Figure 35 shows a per-year breakdown of average and maximum funding per project in FP6 and FP7 as well as sum of funding and number of projects which started in that year.
– 53 –
Figure 35: SQL query visualization as an annotated timeline.
Furthermore, it is possible to formulate queries with certain filter parameters and have the visualization react to changed filter values. For example one might only be interested in a sub-network of that restricts the resulting visualization to immediate collaborators of one particular partner. On selection of a partner, the gadget refreshes its visualization based on the new query and its results.
Once the query author decides to share the current query and its visualization with the public, one click (on the “Send to Gadget Creator” button) will generate a custom gadget: A web service in the toolkit’s backend generates the complete code for a Google Gadget which will display the query result visualization. Once published the resulting gadget can be embedded into any web page, e.g. into the personal iGoogle homepage. By combining multiple gadgets produced with this toolkit, it is then possible to arrange complete interactive web-based data dashboards, where stakeholders can have a real-time visual presentation of data that is interesting and relevant to them.
Currently we have links to the query widgets available on the TEL-Map website (see the D4.3 resource page at http://telmap.org/?q=content/d4.3 ), allowing either to open the widgets in a normal browser window or to embed them in the personal iGoogle homepage. To remove SQL-related barriers and simplify the setup of such “intelligence dashboards” for different stakeholders in TEL to facilitate the stakeholders in observing different variables of interest at a glance, we are currently working on enhanced visualizations and stakeholder-tailored sets of predefined and refinable queries on the different data sources represented in the TEL-Map Mediabase. These are planned to be offered via the Learning Frontiers portal as embeddable widgets.
– 54 –
7 Key Findings for Weak Signals
In this section we present a list the key findings drawn from the analyses and results reported mainly in Sections 3–5. Those key findings that are related to or have implications for European Commission policy in TEL are highlighted with a bounding box. The key findings are grouped by the three data sources in the TEL-Map Mediabase.
7.1 TEL Projects “Multi-Culturality” The list of the most prolific organizations in FP7 exposes the diversity of top organizations both in structure and location. The top-five list includes a distance university, a research company, a traditional university, topped by a psychology department at a technical university, all located in different countries. The topic distribution also shows that particularly in FP7 there are many aspects in the project descriptions that are both genuinely related to human needs, with strong supportability by technology (e.g. adaptation).
North-South Axis. The more than 200 organizations that have participated or are participating in TEL projects in FP7 to date are mostly aligned on a North-South axis across Europe starting in Scotland/England and the Nordic countries, via Netherlands, Belgium, West Germany, Switzerland and Austria, and practically terminating in Northern Italy. The Southern and Eastern regions of Europe and some prominent countries like France appear underrepresented. There is a huge development potential for the countries in these regions, and for the European Commission as well, in terms of funding and contribution in future TEL calls. eContentplus as Gap Filler. While there are strong ties between FP6 and FP7 in terms of participating organizations, it was demonstrated that eContentplus acted as a broker between FP6 and FP7 project consortia. Particularly some Best Practice Networks like ASPECT or ICOPER, and also Targeted Projects like OpenScout, have many strong consortium overlaps with both preceding FP6 projects and succeeding FP7 projects. This pattern is probably simply due to the fact that in 2007 there were neither new project launches in FP6 nor in FP7. On the other hand it could also be attributed to a plain “research follows money” attitude. That is, if there had not been funding from eContentplus, organizations would likely have looked for funding opportunities in TEL-related programmes with different focus between 2006 and 2008. Anyway, eContentplus apparently was supportive and non- disruptive for the organizational collaboration network in European TEL, since eContentplus projects are found in all identified project clusters, despite having a quite “narrow” topic focus around e- content and metadata issues.
Role of Project Type. In the social network analysis of TEL projects it was revealed that Integrated Projects (IPs), Networks of Excellence (NoEs) and eContentplus Best Practice Networks (BPNs) are the most central projects, whereby this cannot solely be ascribed to the typically larger size of the consortia of these projects compared to e.g. STREPs. For instance, these projects typically also include pairs of organizations that appear in list of most frequent collaborators. Also these projects have a significant share of their consortium made of organizations that have a highly favorable centrality vs. clustering ratio. This indicates that IPs and NoEs are very important not only for shaping the research agenda, but also for creating the strong and sustained collaboration ties between TEL organizations.
The TEL Family. With every new TEL project, relatively fewer organizations are penetrating the existing overall collaboration network in TEL projects. Over the last three years, an average of 40% of the consortia of new projects was not previously involved in any TEL projects. The sharpest drop in this number occurred for projects that started in the year 2008 (from 62% to 42%), when the first FP7 TEL projects were launched. It appears that at the transition to FP7, the project consortia—and the European Commission—resorted to building on and funding an established core of organizations, thus strengthening existing collaboration bonds; this will eventually lead to a tightly knit “family” of TEL
– 55 – organizations, an inertial mass that can impede the involvement of new organizations, and likely also new ideas and research foci. This is strengthened by the fact that of the 34 launched TEL projects since 2008, 4 out of 5 are being coordinated by organizations that have already participated in at least one previous TEL project. Of course, from the EC’s viewpoint it seems reasonable to fund projects where a large share of the consortium have previous experience in EC-funded TEL projects. Still, this appears to be a policy issue that requires attention.
7.2 TEL Papers Solid European Base. European researchers are extremely well represented in the most important TEL conferences and journals, as well as by social network metrics in their co-authorship network. For instance, of the 15 most prolific authors in TEL outlets 8 are currently based in Europe; the same ratio applies to the list of 15 most betweenness-central authors in the co-authorship network of TEL outlets.
“Under Construction.” The five TEL-related conferences, which were subject to scrutiny in Section 4, have developed constantly, although at a different pace. Comparing this pattern with that of established conferences in a field—e.g. in sub-disciplines of computer science like databases, data mining, etc.—we found that the TEL conferences expose a development pattern that is typical of “young” and interdisciplinary conferences. This means that while there is significant core of 20-30% of the authors in the “inner circle”, there are also dozens or even hundreds of very small sub-networks within the co-authorship network that are disconnected from each other. Nevertheless, we see that conferences and journals in TEL are building their community in a way that shapes a clear core. In this sense, maintaining and promoting key members who play the role of gate keepers is important.
EC-TEL. The EC-TEL conference series and its (collocated) events can be considered as a highly successful community in terms of development pattern of the co-authorship network and citation network. EC-TEL managed to achieve a tight collaboration network and stable citation network after only few years, while other TEL related conferences are struggling with coherence in their community. This rapid community development is certainly being propelled by the strong ties in EU TEL projects, and it is certainly a strong piece of evidence for a high-impact initiative that has its roots in the FP6 network of excellence, PROLEARN.
Interdisciplinarity. TEL is an interdisciplinary field of research, which is both evident from the project descriptions, core organizations involved in the projects, and also from the publications. For instance, the authors who published in important TEL journals Educational Technology & Society, Transactions on Learning Technology, and Computers & Education, also have papers published in more than one thousand different conference series since 2005 in our data set. That is, the authors are coming from a multitude of different (sub-)disciplines. In terms of community development, interdisciplinarity has pros and cons. On the one hand, it attracts researchers from different areas to a conference. On the other hand, it slows down the process of building the core, as we saw in the development pattern of more focused conferences like AIED, ITS and ECTEL, versus conferences like ICWL and ICALT, which have a broader topic focus. The most important conference series for TEL authors on a global scale authors is probably ICALT: in the events since 2005, ICALT has published a large number of papers, and it is also an important venue for authors who frequently publish in TEL- related journals.
Intelligent Systems. The key weak signals that emerged from analysis of papers by central authors at three main TEL conferences include emotionally interactive learning systems, intelligent tutoring and exploitation of sensor technologies and mobile devices for authentic learning activities. As said above in a different context, the European Commission demonstrated great sensibility for these weak signals, since they are well represented in the target outcomes of the TEL objective in the upcoming ICT Call 8, which include “TEL systems endowed with capabilities of human tutors” in an affective and personalized way, as well as “tools for fostering creativity and non-standard thinking”.
– 56 –
7.3 TEL Social Web Technology Sources as Authorities. From the analysis of the blog network, it became evident that the largest cluster of blogs being pointed to from entries in the TEL blogosphere are related to technology news and trends in general and not to TEL in particular; TEL related blogs can be found in three separate clusters, with one focusing on schools, and the other two focusing on learning, education, and technology aspects. Two of the TEL related clusters are rather hard to distinguish in terms of content; more detailed analyses will have to be performed to identify what differentiates these clusters.
Open and Inclusive Education. The word bursts that have appeared only in 2011, as well as those with a rising frequency of occurrence in the last three years, indicate a clear emphasis in the most central blog sources on leveraging web technology—in particular mobile technology—for open and inclusive approaches to education and learning, e.g. through MOOCs: massive open online courses. This emphasis could be an interesting thread to follow up in European R&D policy.
8 Conclusion
TEL-Map aims at providing the European Commission and other TEL stakeholders with detailed information (trend-indicators, statistics, analyses, policy implications) on the current and future TEL landscape. This deliverable reported first analyses and results obtained in WP4 involving the application of social network analysis and topic mining in several TEL data sources. For each source, the formal foundation for social network analysis was laid to enable a graph-based and network-based analysis of the current TEL landscape. In addition, stakeholders are facilitated in interacting with the TEL data sets and analysis results using intuitive, embeddable tools and flexible analyses. This will support stakeholders in perceiving, understanding and reasoning about complex data sources in TEL, which is one of the main goals of visual analytics [22].
There are several limitations in the current data sources and the analyses, which will have to be addressed in forthcoming WP4 work:
Firstly, the scientific publications indexed in the TEL Papers database are not fully representative of current trends and current work. Also many highly successful innovative technologies and practices do not necessarily come with scientific papers. For instance, there was probably no scientific publication that introduced micro-blogging in education before Twitter was taken up by early-adopting teachers. In addition to that, our main data source is DBLP, which presents itself as a computer science bibliography. Although many important TEL venues are represented in DBLP, it only represents a small subset of potentially relevant venues. In future work we plan to include additional sources. However, many conference and workshop publications are not indexed in any publicly accessible bibliography, therefore presenting a serious obstacle to maintaining a comprehensive TEL publications database.
Secondly, the blogosphere indexed in the TEL Social Media database have been collected with direct involvement of the community (e.g. through the Mediabase Commander plugin for Firefox); this database does not constitute a centrally controlled set of sources. Therefore there are many sources that are not explicitly or immediately related to TEL. For upcoming work we plan to come up with a data cleansing and sources filtering strategy to increase the focus on TEL.
Finally, the TEL projects dataset exclusively contains FP6, FP7 and eContentplus projects relevant to TEL. There are many additional sources and projects that could be included, e.g. the Lifelong Learning Programme, additional projects from the EC Policy Support Programme, the UK JISC funded projects, and many more. Also, we currently have descriptive metadata on the projects only; that is, we do not have project deliverables in the database (which would significantly increase the text corpus for data
– 57 – mining), nor do we have a unified, integrated view on persons as paper authors, as bloggers, and as members of organizations participating in TEL projects.
By next year several new TEL projects will be funded in FP7 ICT call 8, thousands of new TEL-related papers will be published, and the blogosphere indexed in the TEL-Map Mediabase will continue to grow to reflect those changes and subsequent changes. There is a dedicated page on the TEL-Map website [ http://telmap.org/?q=content/d4.3 ] where the up-to-date data can be accessed, obtained and queried using embeddable query visualization widgets. In addition, the core results of the first analyses reported in this deliverable, as well as upcoming analyses with extended and enhanced data sets and results will be published as stories on the Learning Frontiers portal and accessible via the TEL-Map website.
References
1. Blondel, V.D., Guillaume, J.-L., Lambiotte, R., Lefebvre, E.: Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment 2008, P10008 (2008)
2. Brandes, U., Erlebach, T. (eds.): Network Analysis: Methodological Foundations. Springer (2005)
3. European Commission: FP7 in Brief. http://is.gd/fp7brief (2007)
4. European Commission: Community Research and Development Information Service (CORDIS). http://cordis.europa.eu/home_en.html
5. Fekete, J.-D., van Wijk, J., Stasko, J., North, C.: The Value of Information Visualization. In: Kerren, A., Stasko, J., Fekete, J.-D., North, C. (eds.) Information Visualization, pp. 1-18. Springer Berlin / Heidelberg (2008)
6. Hiltunen, E.: Good Sources of Weak Signals: A Global Study of Where Futurists Look For Weak Signals. Journal of Futures Studies 12(4), 21–43 (2008)
7. Keim, D., Andrienko, G., Fekete, J.-D., Görg, C., Kohlhammer, J., Melançon, G.: Visual Analytics: Definition, Process, and Challenges. In: Kerren, A., Stasko, J., Fekete, J.-D., North, C. (eds.) Information Visualization, pp. 154-175. Springer Berlin / Heidelberg (2008)
8. Kienle, A., Wessner, M.: Analysing and cultivating scientific communities of practice. International Journal of Web Based Communities 2(4), 377 - 393 (2006)
9. Klamma, R., Cao, Y., Spaniol, M.: Watching the Blogosphere: Knowledge Sharing in the Web 2.0. In: Nicolov, N., Glance, N., Adar, E., Hurst, M., Liberman, M., Martin, J.H., Salvetti, F. (eds.) Proceedings of the 1st International Conference on Weblogs and Social Media, Boulder, Colorado, USA, March 26-28, 2007, pp. 105 – 112 (2007)
10. Klamma, R., Spaniol, M., Cao, Y., Jarke, M.: Pattern-Based Cross Media Social Network Analysis for Technology Enhanced Learning in Europe. In: Nejdl, W., Tochtermann, K. (eds.) Innovative Approaches for Learning and Knowledge Sharing, pp. 242-256. Springer Berlin Heidelberg, Berlin, Heidelberg (2006)
11. Klamma, R., Spaniol, M., Denev, D.: PALADIN: A Pattern Based Approach to Knowledge Discovery in Digital Social Networks. Proceedings of I-KNOW ’06, 6th International Conference on Knowledge Management, pp. 457-464. Graz, Austria (2006)
12. Kleinberg, J.M.: Hubs, authorities, and communities. ACM Computing Surveys 31, 5-es (1999)
13. McCallum, A., Nigam, K., Ungar, L.H.: Efficient clustering of high-dimensional data sets with application to reference matching. Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining - KDD ’00, pp. 169-178. Boston, Massachusetts, United States (2000)
14. Milgram, S.: The small world problem. Psychology Today 2, 60-67 (1967)
– 58 –
15. Ochoa, X., Méndez, G., Duval, E.: Who We Are: Analysis of 10 Years of the ED-MEDIA Conference. World Conference on Educational Multimedia, Hypermedia and Telecommunications 2009, pp. 189-200 (2009)
16. Olivier, B. (ed.): TEL-Map Deliverable D1.2: Conceptual Framework for Dynamic Roadmapping. http://is.gd/telmapD12 (2011)
17. Page, L., Brin, S., Motwani, R., Winograd, T.: The PageRank Citation Ranking: Bringing Order to the Web. http://ilpubs.stanford.edu:8090/422/ (1999)
18. Petrushyna, Z., Klamma, R.: No Guru, No Method, No Teacher: Self-classification and Self- modelling of E-Learning Communities. In: Dillenbourg, P., Specht, M. (eds.) Times of Convergence. Technologies Across Learning Contexts, pp. 354-365. Springer Berlin Heidelberg, Berlin, Heidelberg
19. Pham, M.C., Klamma, R.: The Structure of the Computer Science Knowledge Network. 2010 International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 17-24. IEEE (2010)
20. Pham, M.C., Klamma, R., Jarke, M.: Development of Computer Science Disciplines - A Social Network Analysis Approach. http://arxiv.org/abs/1103.1977 (2011)
21. Reinhardt, W., Meier, C., Drachsler, H., Sloep, P.: Analyzing 5 years of EC-TEL proceedings - Who we are and what we publish. Proceedings of Sixth European Conference on Technology Enhanced Learning. Springer LNCS, Palermo, Italy (2011)
22. Thomas, J.J., Cook, K.A. (eds.): Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Computer Society (2005)
23. Voigt, C., Pirkkalainen, H. (eds.): TEL-Map Deliverable D4.1: Report on Weak Signals Collection. http://telmap.org/?q=content/deliverables (2011)
24. Wang, X., Jin, X., Chen, M., Zhang, K., Shen, D.: Topic Mining over Asynchronous Text Sequences. IEEE Transactions on Knowledge and Data Engineering PrePrint, 1-15 (2010)
25. Wasserman, S., Faust, K.: Social Network Analysis: Methods and Applications. Cambridge University Press (1994)
26. Wild, F., Haley, D., Bülow, K.: CONSPECT: Monitoring Conceptual Development. In: Luo, X., Spaniol, M., Wang, L., Li, Q., Nejdl, W., Zhang, W. (eds.) Advances in Web-Based Learning – ICWL 2010, pp. 299-308. Springer Berlin Heidelberg, Berlin, Heidelberg (2010)
27. yWorks: Organic Layout Style. http://is.gd/yedorganic
28. yWorks: yEd Graph Editor. http://yworks.com/yed
29. GaLA Home Page. http://www.galanoe.eu/
– 59 –
Appendix A: TEL Projects — Timeline
This figure shows a timeline of TEL projects funded under FP6, FP7, and eContentplus.
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
KALEIDOSCOPE (FP6) PROLEARN (FP6) TELCERT (FP6) ICLASS (FP6) UNFOLD (FP6) LEACTIVEMATH (FP6) CONNECT (FP6) E-LEGI (FP6) EMAPPS.COM (FP6) ICAMP (FP6) I-MAESTRO (FP6) MGBL (FP6) VEMUS (FP6) CALIBRATE (FP6) ARGUNAUT (FP6) ATGENTIVE (FP6) COOPER (FP6) LEAD (FP6) LT4EL (FP6) PROLIX (FP6) TENCOMPETENCE (FP6) RE.MATH (FP6) ARISE (FP6) ELU (FP6) KP-LAB (FP6) PALETTE (FP6) UNITE (FP6) LOGOS (FP6) APOSDLE (FP6) ECIRCUS (FP6) ELEKTRA (FP6) L2C (FP6) JEM (eContentplus) MACE (eContentplus) MELT (eContentplus) CITER (eContentplus) EdReNe (eContentplus) eViP (eContentplus) KeyToNature (eContentplus) Organic.Edunet (eContentplus) Intergeo (eContentplus) EUROGENE (eContentplus) COSMOS (eContentplus) GRAPPLE (FP7) LTFLL (FP7) SCY (FP7) MATURE (FP7) 80DAYS (FP7) IDSPACE (FP7) EduTubePlus (eContentplus) iCOPER (eContentplus) ASPECT (eContentplus) TARGET (FP7) INTELLEO (FP7) STELLAR (FP7) DYNALEARN (FP7) ROLE (FP7) COSPATIAL (FP7) XDELIA (FP7) mEducator (eContentplus) Math-Bridge (eContentplus) LiLa (eContentplus) OpenScienceResources (eContentplus) OpenScout (eContentplus) ALICE (FP7) ARISTOTELE (FP7) MIRROR (FP7) METAFORA (FP7) MIROR (FP7) NEXT-TELL (FP7) ITEC (FP7) ECUTE (FP7) SIREN (FP7) IMREAL (FP7) GALA (FP7) TERENCE (FP7) TEL-MAP (FP7)
– 60 –
Appendix B: TEL Projects — SNA Metrics
This table lists the 77 projects in our data set including SNA metrics (ordered by project acronym).
Progr- Start Page- Weighted Closeness Betweenness Local Clust- PROJECT Cluster Authority Hub Degree amme Year Rank Degree Centrality Centrality ering Coeff. 80DAYS FP7 2008 1 .0115 [28] .0121 [26] .0087 [52] 13 [51] 18 [49] .5315 [48] .0016 [52] .6667 [20] ALICE FP7 2010 0 .0102 [37] .0106 [32] .0076 [57] 11 [53] 14 [58] .5241 [52] .0015 [54] .7091 [18] APOSDLE FP6 2006 0 .0115 [28] .0106 [32] .0129 [26] 22 [22] 26 [29] .5672 [26] .0208 [13] .5238 [50] ARGUNAUT FP6 2005 4 .0089 [40] .0091 [37] .0123 [29] 22 [22] 27 [28] .5758 [23] .0082 [25] .5455 [46] ARISE FP6 2006 4 .0064 [53] .0076 [44] .0060 [63] 11 [53] 11 [61] .5067 [62] .0004 [66] .8545 [10] ARISTOTELE FP7 2010 4 .0115 [28] .0000 [75] .0059 [64] 8 [65] 10 [63] .4967 [67] .0008 [63] .5357 [48] ASPECT eContentplus 2008 5 .0281 [8] .0302 [7] .0246 [9] 30 [12] 59 [9] .6179 [12] .0284 [8] .5034 [53] ATGENTIVE FP6 2005 2 .0000 [73] .0015 [69] .0031 [73] 3 [72] 3 [72] .4153 [73] .0000 [72] 1.0000 [1] CALIBRATE FP6 2005 5 .0064 [53] .0060 [51] .0135 [23] 17 [34] 30 [23] .5507 [33] .0064 [31] .4485 [66] CITER eContentplus 2006 1 .0102 [37] .0106 [32] .0055 [68] 9 [62] 9 [67] .5033 [66] .0015 [53] .5000 [56] CONNECT FP6 2004 4 .0051 [57] .0045 [55] .0145 [20] 20 [26] 32 [21] .5630 [27] .0081 [26] .5158 [51] COOPER FP6 2005 3 .0051 [57] .0060 [51] .0111 [37] 16 [38] 26 [29] .5352 [45] .0009 [62] .8250 [12] COSMOS eContentplus 2007 4 .0115 [28] .0121 [26] .0119 [31] 15 [44] 25 [32] .5429 [38] .0050 [37] .5429 [47] COSPATIAL FP7 2009 0 .0089 [40] .0091 [37] .0058 [65] 9 [62] 10 [63] .5067 [62] .0011 [59] .6389 [27] DYNALEARN FP7 2009 1 .0089 [40] .0091 [37] .0056 [67] 8 [65] 9 [67] .5067 [62] .0020 [50] .4286 [70] ECIRCUS FP6 2006 0 .0038 [62] .0030 [61] .0087 [51] 10 [58] 15 [55] .5101 [60] .0017 [51] .6444 [25] ECUTE FP7 2010 0 .0064 [53] .0060 [51] .0065 [60] 6 [69] 10 [63] .4872 [69] .0003 [69] .8000 [14] EdReNe eContentplus 2007 5 .0000 [73] .0015 [69] .0027 [77] 2 [74] 2 [74] .3938 [74] .0000 [72] 1.0000 [1] EduTubePlus eContentplus 2008 1 .0077 [48] .0076 [44] .0089 [49] 12 [52] 18 [49] .5278 [51] .0041 [41] .4697 [61] E-LEGI FP6 2004 2 .0051 [57] .0045 [55] .0184 [13] 34 [9] 42 [13] .6441 [8] .0365 [5] .4385 [68] ELEKTRA FP6 2006 1 .0077 [48] .0076 [44] .0082 [55] 11 [53] 16 [54] .5170 [56] .0014 [55] .6364 [29] ELU FP6 2006 5 .0102 [37] .0106 [32] .0104 [40] 18 [31] 22 [37] .5588 [30] .0091 [24] .4444 [67] EMAPPS.COM FP6 2005 5 .0026 [69] .0030 [61] .0076 [58] 7 [68] 15 [55] .4634 [71] .0001 [71] .8571 [8] EUROGENE eContentplus 2007 2 .0153 [23] .0166 [21] .0126 [28] 22 [22] 29 [24] .5802 [21] .0058 [32] .6667 [20] eViP eContentplus 2007 0 .0077 [48] .0076 [44] .0055 [69] 9 [62] 9 [67] .5101 [60] .0011 [58] .6389 [27] GALA FP7 2010 0 .0536 [1] .0619 [1] .0332 [4] 42 [3] 79 [5] .6847 [4] .0580 [3] .3449 [75] GRAPPLE FP7 2008 3 .0217 [14] .0242 [11] .0278 [7] 35 [8] 69 [7] .6333 [9] .0220 [11] .4571 [64] ICAMP FP6 2005 2 .0038 [62] .0045 [55] .0110 [38] 15 [44] 25 [32] .5315 [48] .0030 [43] .5524 [43] ICLASS FP6 2004 1 .0038 [62] .0030 [61] .0155 [17] 24 [19] 35 [17] .5846 [20] .0151 [19] .4783 [59] iCOPER eContentplus 2008 2 .0332 [6] .0378 [5] .0347 [3] 39 [5] 91 [3] .6667 [7] .0218 [12] .4791 [58] IDSPACE FP7 2008 3 .0153 [23] .0166 [21] .0094 [46] 17 [34] 21 [40] .5507 [33] .0010 [61] .8235 [13] I-MAESTRO FP6 2005 0 .0000 [73] .0015 [69] .0028 [75] 2 [74] 2 [74] .3838 [76] .0000 [72] 1.0000 [1] IMREAL FP7 2010 0 .0230 [13] .0242 [11] .0106 [39] 18 [31] 20 [43] .5547 [32] .0152 [18] .5294 [49] INTELLEO FP7 2009 2 .0089 [40] .0091 [37] .0062 [61] 10 [58] 12 [60] .5170 [56] .0004 [67] .8000 [14] Intergeo eContentplus 2007 1 .0115 [28] .0136 [23] .0091 [48] 14 [49] 18 [49] .5241 [52] .0053 [35] .5934 [36] ITEC FP7 2010 5 .0255 [9] .0287 [8] .0164 [16] 22 [22] 37 [16] .5758 [23] .0174 [14] .5065 [52] JEM eContentplus 2006 2 .0026 [69] .0030 [61] .0028 [76] 2 [74] 2 [74] .3858 [75] .0000 [72] 1.0000 [1] KALEIDOSCOPE FP6 2004 0 .0000 [73] .0015 [69] .0612 [1] 61 [1] 150 [1] .8352 [1] .1630 [1] .2732 [76] Key ToNature eContentplus 2007 5 .0128 [25] .0121 [26] .0116 [33] 20 [26] 22 [37] .5758 [23] .0287 [7] .6474 [24] KP-LAB FP6 2006 2 .0089 [40] .0091 [37] .0113 [35] 16 [38] 22 [37] .5429 [38] .0136 [23] .4000 [73] L2C FP6 2006 2 .0089 [40] .0076 [44] .0085 [53] 16 [38] 17 [53] .5507 [33] .0046 [39] .8333 [11] LEACTIVEMATH FP6 2004 1 .0051 [57] .0045 [55] .0100 [41] 15 [44] 21 [40] .5352 [45] .0025 [46] .6952 [19] LEAD FP6 2005 2 .0038 [62] .0030 [61] .0072 [59] 10 [58] 14 [58] .5135 [58] .0012 [57] .6667 [20] LiLa eContentplus 2009 1 .0115 [28] .0121 [26] .0061 [62] 10 [58] 11 [61] .5205 [54] .0014 [56] .6444 [25] LOGOS FP6 2006 1 .0026 [69] .0030 [61] .0031 [72] 3 [72] 3 [72] .4294 [72] .0000 [72] 1.0000 [1] LT4EL FP6 2005 2 .0064 [53] .0060 [51] .0114 [34] 19 [30] 26 [29] .5630 [27] .0024 [48] .7485 [16] LTFLL FP7 2008 2 .0217 [14] .0242 [11] .0167 [15] 27 [15] 41 [15] .5984 [15] .0066 [28] .6325 [30] MACE eContentplus 2006 3 .0179 [18] .0211 [14] .0195 [12] 33 [10] 48 [12] .6333 [9] .0158 [16] .5000 [56] Math-Bridge eContentplus 2009 3 .0293 [7] .0332 [6] .0153 [18] 26 [16] 35 [17] .5891 [16] .0159 [15] .5508 [45] MATURE FP7 2008 0 .0089 [40] .0091 [37] .0097 [43] 16 [38] 19 [45] .5390 [41] .0041 [42] .5750 [39] mEducator eContentplus 2009 2 .0255 [9] .0287 [8] .0133 [24] 24 [19] 28 [27] .5891 [16] .0232 [10] .5652 [41] MELT eContentplus 2006 5 .0128 [25] .0136 [23] .0139 [22] 20 [26] 32 [21] .5588 [30] .0052 [36] .5842 [37] METAFORA FP7 2010 1 .0191 [16] .0211 [14] .0095 [45] 16 [38] 19 [45] .5390 [41] .0047 [38] .5833 [38] MGBL FP6 2005 5 .0000 [73] .0015 [69] .0028 [74] 1 [77] 2 [74] .3671 [77] .0000 [72] .0000 [77] MIROR FP7 2010 1 .0077 [48] .0076 [44] .0051 [70] 6 [69] 8 [70] .4967 [67] .0005 [65] .6667 [20] MIRROR FP7 2010 0 .0255 [9] .0287 [8] .0127 [27] 24 [19] 29 [24] .5802 [21] .0057 [33] .6123 [34] NEXT-TELL FP7 2010 0 .0179 [18] .0196 [17] .0099 [42] 15 [44] 20 [43] .5315 [48] .0030 [44] .6000 [35] OpenScienceResources eContentplus 2009 4 .0166 [21] .0181 [19] .0117 [32] 14 [49] 25 [32] .5390 [41] .0023 [49] .6154 [33] OpenScout eContentplus 2009 2 .0459 [2] .0514 [2] .0293 [6] 39 [5] 75 [6] .6726 [5] .0347 [6] .4656 [62] Organic.Edunet eContentplus 2007 4 .0128 [25] .0136 [23] .0119 [30] 17 [34] 25 [32] .5390 [41] .0055 [34] .4779 [60] PALETTE FP6 2006 1 .0089 [40] .0091 [37] .0077 [56] 11 [53] 15 [55] .5205 [54] .0024 [47] .4182 [71] PROLEARN FP6 2004 1 .0026 [69] .0015 [69] .0440 [2] 50 [2] 114 [2] .7451 [2] .0641 [2] .3682 [74] PROLIX FP6 2005 1 .0115 [28] .0121 [26] .0213 [10] 29 [13] 53 [10] .6080 [13] .0146 [20] .5025 [55] RE.MATH FP6 2005 1 .0038 [62] .0030 [61] .0093 [47] 11 [53] 19 [45] .5135 [58] .0030 [45] .4364 [69] ROLE FP7 2009 2 .0370 [5] .0423 [4] .0272 [8] 39 [5] 69 [7] .6726 [5] .0279 [9] .4521 [65] SCY FP7 2008 4 .0179 [18] .0196 [17] .0129 [25] 20 [26] 29 [24] .5630 [27] .0066 [27] .5579 [42] SIREN FP7 2010 0 .0077 [48] .0076 [44] .0049 [71] 6 [69] 7 [71] .4872 [69] .0006 [64] .7333 [17] STELLAR FP7 2009 2 .0395 [4] .0453 [3] .0318 [5] 42 [3] 81 [4] .6909 [3] .0385 [4] .4146 [72] TARGET FP7 2009 0 .0115 [28] .0121 [26] .0112 [36] 17 [34] 23 [36] .5468 [37] .0065 [29] .5515 [44] TELCERT FP6 2004 3 .0038 [62] .0030 [61] .0142 [21] 25 [17] 34 [20] .5891 [16] .0065 [30] .6267 [31] TEL-MAP FP7 2010 2 .0408 [3] .0000 [75] .0204 [11] 31 [11] 51 [11] .6230 [11] .0144 [21] .5032 [54] TENCOMPETENCE FP6 2005 3 .0115 [28] .0106 [32] .0176 [14] 25 [17] 42 [13] .5891 [16] .0141 [22] .5700 [40] TERENCE FP7 2010 5 .0242 [12] .0000 [75] .0096 [44] 18 [31] 21 [40] .5429 [38] .0043 [40] .6209 [32] UNFOLD FP6 2004 3 .0038 [62] .0045 [55] .0083 [54] 15 [44] 18 [49] .5352 [45] .0004 [68] .9048 [6] UNITE FP6 2006 4 .0191 [16] .0211 [14] .0152 [19] 28 [14] 35 [17] .6080 [13] .0157 [17] .4630 [63] VEMUS FP6 2005 4 .0051 [57] .0045 [55] .0057 [66] 8 [65] 10 [63] .5067 [62] .0001 [70] .8571 [8] XDELIA FP7 2009 2 .0166 [21] .0181 [19] .0088 [50] 16 [38] 19 [45] .5507 [33] .0010 [60] .8583 [7]
– 61 –