2009:180 CIV EXAMENSARBETE
OpenStreetMap - Crowd sourcing client
Mattias Grape
Luleå tekniska universitet Civilingenjörsprogrammet Medieteknik Institutionen för Systemteknik Avdelningen för Medieteknik
2009:180 CIV - ISSN: 1402-1617 - ISRN: LTU-EX--09/180--SE Abstract Location based mobile services/applications [1] are becoming more and more popular both on the web and in the mobile phones, according to research firm Gartner. The demand for accurate maps is rapidly growing and the possibility for the user to contribute to add details to the maps. In many parts of the world maps cost a lot of money to get and each time you want to update the maps there is a fee to update them. OpenStreetMap (OSM) is an open source project that provides free wiki based maps. The OpenStreetMap has an engaged and active community of mappers that constantly improve the map data. One way is to record GPS tracks and upload them to the OSM web site. On the site there is a drawing tool that can be used to draw and tag objects like road, fields, houses, rivers etc. Also Points Of Interests (POI) can be added to the map like bus stops, train stations, stores, restaurants, tourist objects etc. This thesis work shows the process on how to contribute to OSM and presents different techniques used for collecting useful data that is used to build the maps. Also the process on how the Java ME client application was made for crowd sourcing contribution to the OSM. The client “Mobile OSM Creator” let’s you easily record GPS track points, take waypoints and record audio clips. Each of these features is saved to a file that’s in GPS eXchange Format (GPX) format, which can then later be uploaded to your OSM account or imported to JOSM or any other map editor. Points Of Interests can be uploaded directly to OSM from the mobile phone by logging in with your OSM account. The client can be customized by the user in a lot of different ways. All in all the “Mobile OSM Creator” is a feature rich application where you among many things can set GPS tracking frequency, offline/online maps and render styles for the map. This client can be downloaded from Ericsson Labs (https://labs.ericsson.com/applications ). Preface This thesis is a part of the Master of Science in media technology education at Luleå University of Technology (LTU). The purpose of the project was to make an application to the Ericsson Labs (https://labs.ericsson.com/ ) collections of applications and make it open source to let the community carry on the project after my thesis work is done. The project was developed at Ericsson Research in Luleå. I would like to thank my supervisor at LTU Kåre Synnes. I would also like to thank the employees at Ericsson Research, especially my supervisor Staffan Larsson E, Sebastien Pierrel for valuable feedback from an experienced OSM contributor, and Tor Björn Minde for giving me this opportunity to do this work. Finally I would like to thank everyone that have tested the program, especially Rikard Nilsson or read parts of the thesis report and provided comments, making the work better then it would have otherwise been. Contents Introduction ...... 1 1.1 Objectives ...... 1 1.2 Limitations ...... 1 1.3 Thesis structure...... 2 Background ...... 3 2.1 What is OpenStreetMap? ...... 3 2.1.1 How can a project like this create accurate maps? ...... 4 2.1.2 Why is the data sometimes inconsistent? ...... 4 2.1.3 Why use OSM? ...... 4 2.2 How to contribute to OSM community...... 5 2.2.1 Public domain map...... 5 2.2.2 OpenStreetBugs...... 5 2.2.3 Simple edits...... 5 2.2.4 Creating maps from scratch...... 5 2.2.5 Collaborating with others ...... 5 2.3 How is OSM build up? ...... 6 2.4 OSM community...... 7 2.5 Why is Sony Ericsson interested in OSM?...... 8 2.6 Ericsson Labs API ...... 8 Why to use open source as development method...... 10 3.1 License...... 10 3.2 Input data ...... 10 Method ...... 11 4.1 Literature Studies ...... 11 4.2 Studies of Existing Applications...... 11 4.3 Analysis of mapping techniques ...... 11 4.4 Identify and priority features for a Java ME client tool...... 11 4.5 Discussion...... 11 4.6 Iterative implementation according to these priorities ...... 11 4.7 Evaluate each implementation phase ...... 12 4.8 Beta testing ...... 12 Map Making...... 13 5.1 Mapping techniques ...... 13 5.1.1 GPS track logs ...... 13 5.1.2 Satellite/Arial Imagery...... 13 5.1.3 GPS Waypoints ...... 14 5.1.4 Photo mapping ...... 14 5.1.5 Hand drawn maps of city streets...... 15 5.1.6 Audio mapping ...... 15 5.1.7 Video mapping ...... 16 5.1.8 Dictacam ...... 16 5.1.9 Focus on one type of street ...... 16 5.1.10 Always turn right...... 17 5.1.11 Set borders...... 17 5.2 Mapping style ...... 17 5.2.1 One way mapping ...... 17 5.2.2 Two way mapping ...... 17 5.3 Transportation method ...... 18 5.3.1 Walk ...... 18 5.3.2 Bicycle/ Inline skates ...... 18 5.3.3 Car ...... 18 5.3.4 Rail...... 18 5.4 Upload data...... 18 5.4.1 Save to GPX...... 18 5.4.2 Uploading to OSM ...... 19 5.5 Editing ...... 19 5.5.1 General tips for adding roads or other ways ...... 20 5.5.2 Tagging ...... 21 5.5.3 Changeset...... 22 5.5.4 Potlatch online editor in OSM webpage ...... 22 5.5.5 JOSM offline editor...... 26 5.6 Render maps...... 28 Features discussed at the start of the project...... 29 6.1 Core Features ...... 29 6.1.1 Support different connections of GPS...... 29 6.1.2 Map ...... 29 6.1.3 Create GPX files...... 29 6.2 Upload GPX track from phone to OSM account...... 29 6.3 Download latest OSM map...... 30 6.4 Edit map on fly when tracking...... 30 6.5 Audio features ...... 30 6.6 Photo features...... 30 6.7 Accelerometer ...... 30 6.8 Overlay map...... 30 6.9 Recognition intelligence...... 30 6.10 Automatic building of roads ...... 31 Features included in final version...... 32 7.1 Startup screen...... 32 7.2 GPS . .32 7.3 Map...... 32 7.3.1 Map tiles...... 33 7.4 Settings ...... 33 7.5 User feedback ...... 33 7.6 Send SMS ...... 34 7.7 Record audio...... 34 7.8 GPX track log ...... 34 7.9 Upload GPX track and edit work...... 35 7.10 Basic map editing on the phone...... 35 7.10.1 Points of interest...... 35 7.10.2 Build way...... 35 7.11 Render styles ...... 36 7.12 Waypoints...... 37 7.13 Setup menu...... 37 Implementation...... 39 8.1 Work flow ...... 39 8.1.1 Optimization ...... 39 8.1.2 User experience ...... 40 8.1.3 Redesign of source code...... 40 8.1.4 Obfuscation ...... 40 8.1.5 Progress...... 40 8.1.6 Debugging...... 41 8.2 Testing ...... 41 8.3 Permissions...... 41 8.4 Beta testing ...... 41 Future work...... 42 9.1 Audio mapping with bike...... 42 9.2 Overlay map...... 42 9.3 Map...... 42 9.4 Pre download maps on PC...... 42 9.5 Change GUI to LWUIT ...... 42 9.6 Optimization ...... 43 9.7 Preprocessing ...... 43 9.8 Split up the GUI in layers...... 43 Discussion and Conclusions...... 44 10.1 Map ...... 44 10.2 Mapping techniques ...... 44 10.3 Sending larger sized information to OSM ...... 45 10.4 Difference between java platforms...... 46 10.5 GPS...... 46 10.6 What happens next...... 47 10.7 Conclusions...... 47 References...... 48 Appendix...... 51 Abbreviations API Application Programming Interface Exif Exchangeable image file format GPS Global Positioning System GPX GPS eXchange Format HDOP Horizontal Dilution of Precision JAD Java Application Descriptor JAR Java ARchive Java ME/J2ME Java Micro Edition/ (Java 2 platform Micro Edition old name) JP Java Platform JSR Java Specification Request JOSM Java OpenStreetMap Editor LCDUI Limited Connected Device User Interface LWUIT The Lightweight User Interface Toolkit NMEA National Marine Electronics Association OSM OpenStreetMap PDA Personal Digital Assistant PNA Personal Navigation Assistant POI Points of interest WGS 84 World Geodetic System 1984 XML eXtensible Markup Language
Chapter 1 Introduction Location based mobile services/applications [1] are becoming more and more popular both on the web and in the mobile phone, according to research firm Gartner. These services needs a map that’s up to date and contain a lot of details e.g. Points Of Interests (POI) such as bus stops, train stations, stores, restaurants, and tourist objects etc. The quality of the maps is a deal breaker whether these types of services will be a commercial success or not. The OpenStreetMap (OSM) is an open source project that was started in August 9, 2004 that tries to build maps that have good coverage and are up to date. OSM is not built from a company like most traditional maps, instead OSM is like Wikipedia for maps, people that want to contribute or use the map is free to do it. In this thesis it will be presented how the process of building up the OSM map looks like. Identifying what features are used, what features are missing and creating a crowd sourcing client in Java Micro Edition (Java ME) which includes the features that was found to be useful.
1.1 Objectives The goal of this master thesis is to make a tool to the persons collecting data and editing the map (OpenStreetMap). The application should be easy to use, and in the same time give more advanced options for the experienced user. A clear intuitive GUI is also desired. The thesis’ main objectives are: • Study OpenStreetMap crowd sourcing functionalities and map drawing tools. • Identify features and possibilities with a Java ME client tool. • Identify and describe a possible technical solution. • Implement a Java ME client solution. • Focus on making the application easy to extend with new features.
1.2 Limitations Because a thesis work has limited time frame some limitations has to be made. You can take Global Positioning System (GPS) track with software that is on several types of devices, like dedicated GPS, laptops, Personal Digital Assistant (PDA), Personal Navigation Assistant (PNA), Mobile phones and homebrew like Sony PSP. The focus in this thesis will be on mobile phones. Mobile phones can have an internal GPS built inside the phone, use an external GPS that you connect to the phone with a cable or use a Bluetooth GPS. GPS sends out information using either National Marine Electronics Association (NMEA) or SiRF (company name) protocol. I choose to only implement support for Bluetooth GPS, external cable GPS and NMEA protocol, because not every phone has an inbuilt GPS and it’s cheaper to buy and add GPS support using another small device than to buy a new phone with inbuilt GPS. Sony Ericsson provides 26 mobile phone models with inbuilt GPS. From the feasibility study it was found that NMEA
1 protocol was used by most similar software’s. Because of this NMEA protocol was selected as the communicating protocol between the GPS and the application. Every phone model has different support of libraries with their own structure. Building an application that is working well on all mobile phones is a big project. The application is tested on a few Sony Ericsson phones that have SE Java Platform 7 (JP 7) support or better, which is a set of profiles describing particular Sony Ericsson Java implementation. Some features are harder to realize than others, which mean that some features had to be balanced with respect to the other desired features. Thus work had to be limited by restrict functionality of features or excluding others. For instance the feature to upload your work directly to your account proved to be hard to implement. This feature then got limited functionality, the feature works on JP 8.5 or later Sony Ericsson phones, and to give time to add other desired features. The application should currently be considered as a prototype which will be further refined, since the envisioned final version lies well beyond the scope of this thesis, hence the focus on making the application easy to extend. One of the focuses in the project was to implement as many features as possible to give a better hint of what the application is capable of in its final revision, instead of making an application work on all mobile phones with few features included.
1.3 Thesis structure The structure of the report is: • Chapter 1 introduces the thesis, its objectives and which limitations were made. • Chapter 2 describes the background of the OpenStreetMap project. • Chapter 3 discussion about open source. • Chapter 4 provide the methodology used. • Chapter 5 goes through the process of map making. • Chapter 6 list which features that were discussed at the start of the project. • Chapter 7 describes which features that are included in the final version. • Chapter 8 is about how the implementation process was done. • Chapter 9 possible future work. • Chapter 10 discussion and conclusions.
2 Chapter 2 Background Information about OpenStreetMap, ideas of how to collect data and edit the map presented in this report is a compilation of information that can be found in OSM wiki page http://wiki.openstreetmap.org/wiki/Main_Page [2]. 2.1 What is OpenStreetMap? OpenStreetMap (OSM) is like Wikipedia for maps. OSM creates and provides free geographic data such as street maps to anyone who wants them. The project was started because most maps you think of as free actually have legal or technical restrictions on their use, holding back people from using them in creative, productive, or unexpected ways.
Figure 1: A map over Luleå using OSM data. Contributors to OSM take handheld Global Positioning System (GPS) devices with them on journeys, or go out specifically to record GPS tracks. They record street names, village names and other features using notebooks, digital cameras, and voice recorders. Back at the computer, contributors upload those GPS logs showing where they traveled, and trace out the roads on the OpenStreetMap's collaborative database. Using their notes, contributors add the street names, information such as the type of road or path, and the connections between roads. That data is then
3 processed to produce detailed street level maps, which can be published freely on sites such as Wikipedia, used to create handheld or in car navigation devices, or printed and copied without restriction. The resulting free map of the world can be viewed at http://www.openstreetmap.org , a picture of how the maps can look is shown in Figure 1.
2.1.1 How can a project like this create accurate maps? The wiki style process gives no guarantee that the map will be accurate. But do proprietary maps carry no errors? In fact commercial map providers have for years used “Copyright Easter Eggs [3]” that is intentional errors in the map e.g. imaginary names and wrong streets. If the owner of the proprietary map finds these errors on another map they can be used as proof that they have violated the copyright. Maps from different providers are very similar to each other and a way to protect your maps is to use Easter Eggs. Proving that a plagiarized piece is either a copy or unlawful derivative of another's effort is often very hard. It’s always a possibility the map could just be another unique work based on the same facts, making it something that is completely protected. Easter Eggs that is used in digital maps can be used in a way that is not obvious to a user of the map. The lower bits of the geographic coordinates can be mangled in some systematic way e.g. all the coordinates have a remainder of 3 when divided by 7, or all coordinates are displaced from their real position by some small but real offset. All users gain in having accurate data. If few persons put in inaccurate data, maliciously or accidentally, the other 99.9% of the people can correct the errors made by these few persons that puts in inaccurate data. This is one of the corner stones in the wiki style process. Since April 21, 2009, a full editing history is stored for each user. When errors is found you can check the editing history from that person and if you find that user deliberately puts in inaccurate data, it’s easy to find all the changes that user made and fix them. The Wikipedia project has shown that a large amount of good quality data can be collected but it can be difficult to weed out all the inevitable errors.
2.1.2 Why is the data sometimes inconsistent? OpenStreetMap is a free editable map that is made by its users. The map is as consistent as the people that use the map contribute to it. The database will always be subject to the whims, experimentation, and mistakes of the community. But it’s also one of the strength of OSM because it allows the data to quickly accommodate changes in the physical world.
2.1.3 Why use OSM? Dirk Snauwaert [4] says in the article that about 15% of the roads changes every year like, new roads, one way restrictions and new speed limits. It’s also mentioned that it’s not economically profitable to update everything that often or even track small roads. This leads to outdated maps and missing coverage for the user. For each time you want to update your maps you need to pay for it. Even if you buy maps from companies that provide maps you are not free to do whatever you want with it. You can't correct a street name, or add a pub/bar into the maps. What about sending it to a friend, enclosing it in an invitation or posting it on a notice board? A lot of these are less legal than you might think. With OSM you just correct errors and add missing features you find and everyone is the owner of the map and can use it freely.
4 2.2 How to contribute to OSM community There are different ways to contribute to the OpenStreetMap.
2.2.1 Public domain map People that have access to public domain data can add it to OSM’s database. It’s wise to contact the community before adding any public domain data to make sure there is no licensing issues or if it’s a large amount of data.
2.2.2 OpenStreetBugs People that feel like it’s too difficult to edit the map themselves can report errors in the maps with the easy to use http://openstreetbugs.schokokeks.org/ [5] website to mark errors they see on OpenStreetMap data.
2.2.3 Simple edits It’s not that hard to learn how to do simple edits. All that needs to be done is to create an account on the OSM website and check the Map features [6] page to get an idea of how to tag nodes. Choose an editor among the several choices that exist and start adding things like pubs, fast food restaurants and shops, that haven’t been added to the map in your neighborhood. Correct any mistakes you see, e.g. misspelled road names.
2.2.4 Creating maps from scratch In areas that have no coverage in the map it’s a good start to learn different mapping techniques such as recording GPS tracks and audio mapping, which is discussed in Chapter 5.1 and the process of map making that’s whole chapter 5. People can edit from tracks others have uploaded or use Yahoo! Aerial Imagery [7], if they have no access to a GPS device.
2.2.5 Collaborating with others Mapping together with others is more fun and by working together effort can be focused and formidable milestones become achievable. 3 common ways to do mapping together are. • Mapping Projects: People with common goals and interests collaborating together in goal oriented activities, such as cycle networks or places. These projects have a long time span. • Mapping party: Meetings where experienced mappers help novices on how to do the mapping. The people that attend receive instructions in how to use the GPS receiver, take waypoints with it and write down notes of the waypoints to a paper. The place that’s going to be mapped is divided up among the participants. When the newcomers return back to the venue, the experienced mappers help them out to transfer the waypoints and GPS track to an OSM editor. The waypoints gets converted to POIs and gets appropriate tags from the hand written information the participants have collected and roads are built from the GPS track. The data then gets uploaded to the map. It's a very social event where people can meet up and talk (usually at a pub) in between mapping sessions. Mapping parties often last for a weekend.
5 • Micro mapping party: Are well defined mapping objectives where a small group of mappers meet on a casual basis. London has used this method successfully where several mappers live close to each other. These parties last from a few hours to one day. If you want to organize your own mapping party for the first time this guide [8] can be helpful.
2.3 How is OSM build up? Figure 2 [9] is a technical overview of logical system/components of OpenStreetMap.
Figure 2: High level overview of OSM. Figure 2 gives a hint of how OSM components connect to each other. The database holds a table of the current representation of the map and another table with all revisions that the map has had. The database is using the PostgreSQL software. The Application Programming Interface (API) 0.6 is a protocol that communicates between clients like map editing software and the database server. The API is fetching and saving raw geodata from/to the OpenStreetMap database and stores user account information. The API is based on the ideas of the Representational state transfer (RESTful) API. For instance Potlatch is sending request in the form of HTTP GET, PUT, POST, and DELETE messages to the OSM database server. Requests to modify the database are authorized using HTTP Basic Authorization. Planet dumps are made each week of the current representation of the map on Wednesday and released on Thursday morning.
6 Map tiles that the main website uses to display the map are rendered from 2 different render processes called Mapnik and Osmarender. The application that is made in this thesis work uses a few of these components. API 0.6 is used to communicate with the OSM database. Map tiles are downloaded with the API. Map tiles are OSM data that are rendered, which are used to build the map. Features in the application are made in way that can easily be used by Potlatch and Java OpenStreetMap Editor (JOSM). If you want to know more details about what the components do visit the Component overview [10] page.
2.4 OSM community The OSM project was started in 2004. The last 18 months, the user base has grown a lot and the map gets better and bigger for each new contributor. In the figure 3 you can see this growth in registered users and track points over time from OSM database statistics [11].
Figure 3: Users and User GPX uploads diagram. The OpenStreetMap community is large, and spread across many locations, speaking different languages, and focusing on different interest areas. There are several different channels for the community to communicate with each other. A forum, live chat at IRC channel, blog for announcements and interesting developments, wiki page used for reference and over 20 mailing lists. It’s in the mailing lists most of the discussions take place and decisions are made. GPS is not an accurate tool. Each time it’s used the results it gives can be several meters from each other. When you start from scratch and building a map and use GPS as a tool to build it up,
7 it’s good to have a big community that all can edit the map and change it when they find errors in the map. In the Figure 4 you can see which program that is used to create changesets that is uploaded to the OSM database, the data is from April 21, 2009 to November 25, 2009. The data comes from the changeset dump that is taken every week on OSM homepage. Every upload you do to the OSM database needs to be wrapped inside a changesets, you can read more about changesets in section 5.5.3. In the table you see the number of changesets uploaded by each program, “with_bboxes” is the number of changesets that have a bounding box and users are the number of distinct (public) user ids.
creator | Changesets | with_bboxes | users ------+------+------+------Potlatch | 964387 | 718768 | 54823 JOSM | 1064183 | 1015659 | 12741 Merkaartor | 121672 | 114819 | 2057 BigTinCan Upload Script | 245 | 201 | 109 osm2go | 898 | 865 | 98 iLOE | 940 | 850 | 84 Osmose Raw Editor | 714 | 330 | 78 bulk_upload | 59740 | 55051 | 58 osmtools | 7914 | 7622 | 56 Vespucci | 542 | 359 | 48 Mapzen Alpha | 310 | 164 | 37 andnav | 207 | 191 | 31 Mapzen POI Collector | 738 | 709 | 27 QGIS OSM v | 114 | 98 | 23 PythonOsmApi | 935 | 753 | 18 upload | 35026 | 33431 | 14 KMLManager | 17443 | 17313 | 9 Figure 4: Table of which program that is used to create changesets. The online editor Potlatch that you can find on the main web page is used by most users and JOSM have most created changesets. These 2 editors are used the most by a large margin and therefore they will be evaluated in this thesis work and the crowd sourcing client application should integrate well with them.
2.5 Why is Sony Ericsson interested in OSM? To reach a larger user base by providing several maps for example Tele Atlas and OSM. Many of the users of OSM are people that Sony Ericsson has identified as potential users of “Ericsson Labs” and by providing OSM they might be more interested in “Ericsson Labs”. Since Ericsson is doing research on crowd sourcing and using OSM in their services, they want to give something back to the community, hence the OSM crowd sourcing client.
2.6 Ericsson Labs API At the start of the project Ericsson had an idea that this work should use one of the APIs that is provided at “Ericsson labs” the Mobile Maps API. Mobile Maps API provides the functionality of downloading, displaying, and interacting with maps to your applications. It also includes
8 access to the map data itself. The maps data are streamed from a server and rendered in the mobile phone, which enables the possibility to use effective compression and transfer algorithms. Most maps today are rendered in the server and the rendered map tiles are sent to the mobile phone application, a side effect of this is that the map has fixed scale levels. Mobile Maps does not have fixed scale levels.
9 Chapter 3 Why to use open source as development method Open source involves a broad body of collaborators, typically volunteers, whose every contribution builds on those before. Just as important, the product of this collaboration is freely available to all comers and the source code is freely shared. Open source harnesses the distributive powers of the Internet, parcels the work out to thousands, and uses their piecework to build a better whole putting informal networks of volunteer coders in direct competition with big corporations. It works like an ant colony, where the collective intelligence of the network supersedes any single contributor. 3.1 License Open source software doesn't just mean access to the source code. The distribution terms of open source software must follow some definitions [12]. A program is open source only if it uses one of the approved licenses [13], these licenses follow the definitions set by Open Source Initiative (OSI). Some examples of approved licenses are GNU General Public License, GNU Lesser General Public License and BSD license. OpenStreetMap data is published under an open content license, with the intention of promoting free use and re distribution of the data both commercial and non commercial. The license used is currently the Creative Commons [14] Attribution Share Alike 2.0 license (CC BY SA), however legal investigation work and community consultation is under way to re license the project under the Open Database License (ODbL) from Open Data Commons [15] (ODC) which is more suitable for a map dataset. OpenStreetMap software is licensed under the GNU GPL [16] if not stated with another license. What this means to you (and this isn't legal advice) is basically you can do what you like with the data, so long as you mention the original creator and the license and anyone else can do the same with anything you produce based on it. A good example on how to credit your work that is based on OSM can look like “Data/Maps Copyright 2009 OpenStreetMap Contributors | License: Creative Commons BY SA 2.0”.
3.2 Input data All data added to the project needs to have a license compatible with the Creative Commons Attribution Share Alike license. This can include out of copyright information, public domain or other licenses. All contributors must register an account to OSM site to be able to upload data and by creating an account; you agree that all data you submit to the OpenStreetMap project is to be (non exclusively) licensed under Creative Commons Attribution Share Alike license. Use of copyright data is an especially severe problem for maps, as the only way to prove that no data came from copyright sources would be to rollback the whole area in question to before the addition of the questionable data.
10 Chapter 4 Method This is the method that is used to identify which features that should be included in the Java Me crowd sourcing client.
4.1 Literature Studies The first step is to get to know what the OSM community is about. OSM has a wiki page with a lot of information about what OSM project is, how to start mapping and which software that exist etc.
4.2 Studies of Existing Applications After establishing background from literature studies, the next step was to test out how mapping is done today with available software for mobile phones and how to edit the collected data. The purpose of this was to identify what can be done and find drawbacks and missing features that can be implemented.
4.3 Analysis of mapping techniques In OSM wiki page they list several mapping techniques of how to collect data. An analysis of these techniques provides valuable knowledge of what features that will be liked by the OSM community.
4.4 Identify and priority features for a Java ME client tool After testing available applications which features they provide and read the mapping techniques of how to use these features. Its time to prioritize the features and make a list for in which order these features should be implemented in the Java ME client and identify new features that could be added.
4.5 Discussion The priority list of features is discussed in a Brainstorm meeting with 3 employees from Ericsson research in Luleå and 1 from Kista. Sebastien Pierrel from Kista has been out mapping and has some knowledge of what can be done to make his work easier.
4.6 Iterative implementation according to these priorities Iterative and incremental development [17] is a cyclic software development process. Key steps in the process are to start with a simple implementation of a subset of the software requirements and iteratively enhance the evolving sequence of versions until the full system is implemented. For every iteration of the application development time, design modifications are made and new functional capabilities are added.
11 The project was divided into 3 iterative implementation phases: • First phase to make Core functions to collect data and a map you can zoom and pan around in. • Second phase to add editing tools. POI, waypoints and upload data to OSM. • Third phase to add more features and robustness.
4.7 Evaluate each implementation phase After each implementation phase there is an evaluation of what’s been done and problems that have come up. The feature that was in the original plan is evaluated and a new priority list is made of features to implement in next implementation phase.
4.8 Beta testing When the last implementation phase was done a few selected persons, some with experience with other similar software, was used to beta test the application to find weakness and strengths of the program. The comments made by the beta testers gave invaluable feedback on how they use the application to make final adjustments to the program before it’s released to “Ericsson Labs” webpage.
12 Chapter 5 Map Making The process of adding data to OpenStreetMap is done in 4 steps • Collect data. Chapter 5.1 5.3 • Upload data. Chapter 5.4 • Edit data and add tags. Chapter 5.5 • Render maps. Chapter 5.6
5.1 Mapping techniques When mapping you need to know where you are and what are at that position. There are different types of techniques to collect data that gives this information.
5.1.1 GPS track logs The basic technique for mapping roads, footpaths, ferry crossings, ski routes, etc. is to use a GPS and let it record a track log. If the GPS takes the timestamps in World Geodetic System 1984 (WGS 84) datum you can upload the GPS eXchange Format (GPX) track log directly to your OSM account. Some applications let you record GPX tracks with your mobile phone, laptop or PDA, if a GPS receiver is connected. Make sure that your GPS unit has the 'Lock on road' or 'Snap to road' option switched off or else you will be recording a direct copy of the map loaded into the GPS unit. This is a copyright infringement. Turn off wrapping function if your GPS has one, to avoid that the first points will be removed when the memory is full. Set the GPS to record points as densely as possible and necessary. Two different recording rule sets are used either time or distance. With the distance rule set every point is recorded after a threshold distance “x” is reached. The good thing is that no points will get recorded while standing still. The bad things is that, unnecessary points will be recorded when traveling on straight roads and it can miss the subtleties of curves and direction changes if “x” is too large. With time rule set a points is recorded every “x” seconds. The bad thing is that when standing still, points will be saved that is not needed and when moving slowly more points will be saved then necessary. The good thing are that relative speed can be judged by the spacing, There are more points when you slow down for curves or turns and less on straight ways where you need fewer points.
5.1.2 Satellite/Arial Imagery Many cities have high resolution aerial photos available which can be used to aid mapping. Due to copyright concerns of the derived data, only public domain or CC imagery should be used. Yahoo’s aerial imagery the editing applet can also be used, and in fact this aids building the map later if you're using the online Potlatch editor. Select an area you want to map from Yahoo! Maps. Use the satellite tab to get the map which you have legal rights to use. Do not print out the street map. Print out the map and go out mapping. Make notes of street names, street types and POIs etc. A good use of Yahoo Aerial maps is to
13 find areas of interest to be mapped. When you get home, open up the editing applet and draw everything as you've noted down.
5.1.3 GPS Waypoints Waypoints are single points you can log separately with your GPS receiver that mark geographical positions of relevant objects, such as pubs, ATMs, bus stops, restaurants and many more. To take waypoints with a GPS is usually just a button push. Many GPS receivers number each waypoint. Use a notepad or sound recorder to record what that waypoint corresponds to. While moving and saving waypoint be aware of that, a GPS receiver usually sends position information every second. Saving a waypoint can happen anywhere from exactly after you have received new position information up to just before the next information, hence almost 1 second. This is a big factor of inaccuracy when taking waypoints while moving. In one second, you move 10 m at 36 km/h, 20 m at 72 km/h, 30 m at 108 km/h etc.
5.1.4 Photo mapping Digital cameras are useful for recording street names, village names, and other features. Most digital cameras have a timestamp of when a photo was taken. Provided the camera clock was reasonably in sync with the GPS logger, you can get an accurate position of where the photo was shot. This information can be put to the Exchangeable image file format (Exif) header inside the picture file and thus geotagging it. JOSM includes support for reading geocoded photographs A geocoded photograph is a photo which is associated with a geographical location. The image can be associated with latitude, longitude, or to a street address. Some methods to create geocoded photographs are • Automatic using a connected GPS: With a digital camera or camera phones that support GPS. The GPS can be built into the camera or be connected to an external GPS receiver. With full supported geocoding when a photo is taken relevant GPS data will automatically be put into the photo’s Exif header tag. • Synchronizing with a separate GPS: Most cameras sold today do not contain a built in GPS receiver; however, an external location aware device, such as a hand held GPS logger, can still be used with a non GPS digital camera for geocoding. The photo is taken without geographical information and is processed later using software in conjunction with the GPS data. Timestamps made by the camera can be compared to timestamps in the recorded GPS information, provided that the clocks in the separate devices can be synchronized. The resulting coordinates can then be added to the Exif header information of the photo. • Manually geocoding : can also be used even if no GPS device was present when the photo was shot. Latitude and longitude can be entered directly to the Exif header tag or by selecting a location from a map using software tool. A less precise method is to add descriptive data of the location to an image. A tip is to take a photo of your GPS screen/display showing the time (on the GPS) when you start mapping. You can then later on sync the time by checking what time the photo was taken and the time shown in the picture.
14 5.1.5 Hand drawn maps of city streets One big problem when making a map is to get the names of every street. Draw how streets intersect and write down names for them. The sketches are then compared with your GPX track. The quality of the sketches has to be good enough for you to spot where on the GPX track you are, to be of any use. This can be done by using the Yahoo Aerial Imagery that you print out (this map has to have no tags or names to not brake the copyright) or OSM map of streets that is already mapped but not named. Then the sketches are already done and you only have to name the streets.
5.1.6 Audio mapping Some of the benefits of using audio mapping technique are that it’s faster than pen & paper or typing in meaningful names for waypoints. It’s the only meta data technique that is suitable while driving a car and practical to use with bike (because you might not need your hands for recording). Give details that the GPS logs can’t give, street types and names, access restrictions, POIs, linear features parallel to your track, like rivers. But it can be confusing when the sound can have several different spellings and it’s easy to miss apostrophes. Remember to spell names that differ from the usual/common spelling when they have an equivalent pronunciation, e.g. "M O R E" is better than “MORE” so you don’t accidentally add "MOOR" to the OSM database. If your area is already mapped, or if you have good satellite images, you might use audio mapping on its own, but in most cases, you will use a GPS at the same time. When you are on the road, just say what you see. When you are audio mapping start you’re recording by saying “Mark” when you enter a new road, while using continuous recording method described later on. This is done to know exactly where a road starts. An example on how this can look like is “Mark residential road called midsommarvägen speed limit 30”. Your audio data will be worthless if you cannot tell what recording belongs to which location. Because the GPS track and the audio recording might not have been started simultaneously you might have to synchronize them. You may also have paused the recording part way through, so you would need to re synchronize at that point too. There ways to do the synchronization are: • Implicit synchronization: If the same hard and software is used for audio recording and GPS, they will be synchronized automatically. • Time synchronizing: Uses the timestamp of the audio recording and looks up the GPS data to know what location you where on that time. If the sync is as accurate as a delay of 1 second between audio recorder and GPS, you can correlate voice and position as accurately as mapping can be. Record “Mark cinema at right side, called Filmstaden”. The keyword “Mark” is used exactly when you pass the entrance of the cinema, to know where to add the POI. If the timestamps between the audio recorder and GPS is too large you have to use spatial synchronizing first to get an accurate synchronization. • Spatial synchronizing: Gives you second accuracy even if the clock lags or the device does not take timestamp. Start the GPS and wait for a fix with full signal strength before doing the spatial synchronizing. Choose a place to do the sync that is clearly identifiable (for instance a line that is painted on the ground) and pass that point at the same velocity as when you are mapping. When passing that point you record “Mark” and pass the same
15 point from another direction and record “Mark” again at the exact moment that you pass it. Along with the synchronizing problem goes the question, do you record one big audio file or make a new audio file for every clip you take. To not get a lot of noise when making one big file, pause the recording when there is nothing to say. There is a huge range of variety in audio mapping techniques 3 of them are. • Continuous audio with GPS waypoints: Make waypoints with the GPS that saves its location, time and its number/name. At the same time you dictate onto a continuous audio recording what each waypoint represents e.g. POI or street name. Synchronization merely helps to select correct part in the audio track for each waypoint. It’s easy to locate the audio description for each POI. But you have to do some button pressing on the GPS, which can be cumbersome when moving. • Continuous audio with contextual waypoints: Instead of entering waypoints into the GPS you make audio notes for each waypoint. If you speak out every turn you make, you might follow where you are by looking at your track log. Synchronization with this method is critical. No button pressing is needed while you are moving. But you have to be clever while you make the audio recording to not get lost in the track. • Audio clips with waypoints : For every waypoint you make record a separate audio file. The name of the audio clip is saved as a element in the corresponding waypoint into the GPX file, to know which audio clip that belongs to each waypoint. No synchronization is required. If your GPS can do the linking automatically and create the GPX file this method will be easy to use. Otherwise you have to use another software that does the linking.
5.1.7 Video mapping With Video mapping you don’t have to do anything while you’re out and collecting data beside mount the video capturing device on your car and start the video and GPS recording. The disadvantages are that, processing of the collected data will take more time and other mapping techniques; you can only map what’s captured in the video and takes up a lot of storage space. This method is rarely used by OSM mappers. Commercial companies use it and more advanced cameras to get 3D street maps.
5.1.8 Dictacam Dictacam are a combination of audio mapping and photo mapping where you take a photo and record a voice tag to that photo. When you’re audio mapping it’s hard to correlate the audio clip with a point in the GPS track. With a picture the correct point is easier to find. Dictacam use the strengths of both photo and audio mapping which makes it suitable for POI and street names recording.
5.1.9 Focus on one type of street To get less information to keep track on while mapping you can focus on one type of street for every recording you take. If you decide before you start tracking to follow for example a cycleway you don’t have to take notes of when you change to another type of street. If you are on a trip and just collecting data of where roads are and not taking any notes, you can at least tag
16 every road you edit with the street type. Now you have less information to collect for the second trip in that area and you can easier choose a suitable transportation method for that trip because you know the street type.
5.1.10 Always turn right When recording tracks in urban areas, a good way to cover convolved housing developments is to always turn right (or, always turn left in the countries that have left side traffic). It is easier to cover all side roads when using this technique. It also avoids making turns across oncoming traffic which can be both time consuming and dangerous. This means that you will catch the side roads of one side of the main street on your journey outward and on your journey back to the starting place the roads on the other side of the main street will be covered. When you arrive at a junction you always choose to take the street that is the right side of where you stand and if you have already been to the street at the right side, go straight. If that is also covered, go left. If all turns are already covered you make a u turn and go back the way you came.
5.1.11 Set borders To map a city, or a big area, start by driving all the main streets with the car to divide the area into chunks. Now you can choose a chunk and use the main streets as your borders while mapping. Use the “always turn right” technique within the confines of the borders.
5.2 Mapping style Mapping is done in two steps: First, you need to know “where” things are, mainly the streets and roads. Then you need to know “what” there is, street names, POIs or which type of road it is. You can do these one after another, or both at the same time, but you can hardly do” what” before the “where”. Between either collecting both where and what in one trip or split it up in two trips where you first focus on where things are and later on make a second trip to map what is at that place. This section will discuss which mapping techniques are good to use for the two mapping styles.
5.2.1 One-way mapping Combinations of the mapping techniques GPS track logs, audio mapping, GPS waypoints and transportation method walking/bicycle are good to use when mapping in one trip. Because you will make many stops while making notes of waypoints and audio clip, walking is preferred. Bicycle can also be used which are easy to start and stop if you are planning to make a longer trip.
5.2.2 Two-way mapping The first trip you make the GPX track logs of where the roads are and take few GPS waypoints to note which type of road you are mapping. For this trip you want to use the fastest transportation method that is suitable for the type of road you are mapping. For the second trip you focus on taking notes of what detail that is in that area. It’s always recommended to take GPX track logs for every trip you take to get more tracks to use from when building roads.
17 5.3 Transportation method Which transportation method you use make different mapping techniques appropriate. Depending on the type of tracks which you intend to create, some transportation means are better suited than others.
5.3.1 Walk For short distance tracks that have a lot of features to map, hand drawn maps of city streets technique can be used, because you have both hands free and no vehicle that’s in your way when you stop frequently to take notes.
5.3.2 Bicycle/ Inline skates A bicycle is good to use for medium distance tracks when you want to make several stops to take notes. Inline skates are more suited for tracking residential areas with dense features to map.
5.3.3 Car A car is good to track all the length of a motorway, where little special features appear. If there are a lot of special features along the motorway that you want to map, 2 persons are needed, one that is the dedicated driver and the other one does the mapping work. If you are recording in time intervals, drive slower (if possible) while turning to get more densely points where the road is curving.
5.3.4 Rail While taking a rail trip, get a good fix before boarding. Once you got the fix, the GPS receiver can get a new fix very fast within tens of milliseconds for a period of 2 4 hours. This is valuable in harsh receiver conditions such as rail cabs. It can also help (though to a lesser extent) when traveling by bus or car. If possible place the GPS receiver near a window in the same height as the window to easier get the signals from the satellites.
5.4 Upload data Before you can upload data and edit the map you need to create an account on the OSM website and login. Your GPX file must consist of track points with valid timestamps. Tracks will fail to upload if data formats are not correct. For example GPS setup occasionally throws in a NaN as an elevation value. Deleting those lines from the file fixes the upload. GPX 1.0 and 1.1 schema is accepted, example of how they look can be seen in appendix B. OSM uses the WGS 84 latitude/longitude datum exclusively. All uploaded tracks and edits should always be in WGS 84, the default datum for most GPS receivers.
5.4.1 Save to GPX If you have collected your track data with a GPS receiver you need to save the file in GPX format. Check the manual of your GPS receiver or software to learn how to save the track. If GPX is available, save it as GPX file. If GPX is unavailable, save the track in another format and use a converter program to change it to GPX format. One such converter program that can convert from most formats is GPSBabel.
18 5.4.2 Uploading to OSM Once you have a GPX file, upload it to the web site by clicking the “GPS traces” tab. After uploading your GPX file you can download the data from the editor JOSM, along with all other data in that area. Alternatively, you can open it into the editor Potlatch, by clicking the “edit” link to the right of the trace (not the one at the top). 5.4.3 Always upload your tracks The accuracy of the points in a single GPS track may be out by several meters. This depends on lots of factors such as the positions of the satellites when the track was taken, tree cover, proximity to nearby buildings, position of the GPS unit relative to the centre of the road etc. If many traces are taken for the same road, then the effect of errors in any one trace will have a much smaller impact on the average position of these tracks. To help with this, it's useful if you upload all your GPS tracks to the server, even if they cover roads already in the database. This allows others to use your tracks to average out the errors and should eventually result in lower position errors.
5.5 Editing After you have collected the required information it’s time to edit the map with your new data. OpenStreetMap's data is made up from the elements, node, way and relations. • ode: A node is used to mark points of interests or as segments to build up a way. A node consists of latitude and longitude that tells where the node is located. Standalone nodes should have at least one tag that describes what point of interest it is referring to e.g. (amenity=pub). • Way: An ordered list of nodes of at least 2 nodes and up to 2000 nodes, displayed as connected by line segments. They are used to describe roads, paths etc. If a way is over the upper limit it will not get accepted. Just divide it up in smaller segments and group the ways in a relation of necessary. Nodes can be a part of several ways. • Closed Way: Is a special case of the element way, which is a way that goes in a complete loop, which is used to describe areas like parks, lakes or islands. • Relation: From the data model's perspective, relations are basically groups of objects in which each object may take on a specific role. Relations may also have tags. Typically you would expect the 'type' tag to be set, specifying what type of relation it is. Relations can be used for representing things like, traffic enforcement devices; speed cameras, red light cameras, weight checks or destination signs at or before intersections.
Figure 5: The elements that make up the OSM data.
19 All these elements must have tags like describing name or type of road etc. You have a choice of several editors for adding elements and tags, 2 of them are Potlatch and Java OpenStreetMap Editor (JOSM). As a very rough rule of thumb, Potlatch is easier for quick edits, and better for edits which range over a wide area (e.g. cycle routes). JOSM is better for heavy duty editing in a concentrated area (e.g. housing estates) and offers facilities for audio and photo surveying. But everyone will have their own preferences.
5.5.1 General tips for adding roads or other ways The length of a way segment can vary greatly. If a road is extremely curvy then it needs a lot of segments/nodes. If it's dead straight, with no adjoining ways, then 1 segment is sufficient for the entire distance. Note the straight bit in Figure 6 at the center of the way requires only one segment.
Figure 6: Example on how to create a road. If there are multiple tracks you should put the way down the center of them all, as in Figure 7, image 1. But you should consider that tracks that comes from routes that was done on foot or bicycle is more likely to be accurate than routes that are done in a car. Because the slower means of transportation has had more time for the GPS to get signals from the satellites. It's best to rely on this data to be more accurate. GPS data is far from a perfect drawing of the path. If the dots are all over the place, draw a smooth way down the center (in Figure 7, image 2 is incorrect, image 3 is correct. The average route has been drawn cleanly down the centre). If you do the editing shortly after you have collected the GPX track it’s easier to remember how the road looks. If the dots are really scattered you should consider if you should use that track at all.
Figure 7: Example on how to use the GPS track points.
20 The more GPS tracks you have over the same road and average between those the more likely it’s that you have an accurate road. The points that you see from all the GPS tracks is used as a guideline of where you will put your nodes that builds up the road. When creating and editing dual carriageways, try to keep the nodes and segments aligned and parallel, as in Figure 8, image 1. If one of the carriageways curves off then nodes don't need to be wasted on the straight carriageway, as in image 2.
Figure 8: Example on how to create dual carriageways. When connecting two roads together they must both link to a shared node this is showed in Figure 9, image 1. A common beginner’s mistake is to place nodes very close to one another, but not form a connection Figure 9, image 2, or overlap 2 ways and presume a connection, Figure 9, image 3. Only image 1 will work. To check if both ways have a shared node, move the assumed shared node and then undo it, if both ways move you have done a successful connection.
Figure 9: How connecting 2 roads should be done. To learn more on how to edit the maps visit Editing Standards and Conventions [18]. Many questions can be answered by looking at the work of others. Download pieces that are very well mapped and have a look. Note that some countries have different standards; preferably look at your own country.
5.5.2 Tagging Once you have a way, you need to add tags to the way in order to see them rendered on the map. Each element may have an arbitrary number of tags which are Key Value pairs (e.g. amenity=theatre). All tags need to be in lower case, except for names. On the Map Features [6] page you can learn more on how to tag correctly. Many of the examples on the map features page, particularly those regarding roads, are specific to the U.K. Users in other countries will want to look at Tagging [19] for guidance in applying the tagging scheme to their country. Anyone can propose tags and it's not a requirement to have them "approved" before you can use them, but it’s unlikely the tag will show up with an icon in the maps before it is approved. The best advice is always this: • Check the map features or any other tagging related pages on the OSM wiki webpage to see if there is a logical tag that fits your needs. • Check the Proposed features [20] pages to pick up the tagging ideas of others.
21 • If you find tags from one of the above pages that match your criteria and you believe they are logical then use those. • If what you want to tag doesn't fit any of the existing documented tags or proposed tags then make up your own using a logical process. Use your new tag(s) and add a page on the OSM wiki webpage and explain what the tag is. No tag needs approval or to become "official" before use. Any tag key or value is valid in OSM and the approval process simply offers a mechanism to discuss tags as they become used.
5.5.3 Changeset Every modification of one or more of the elements has to be wrapped in a changeset. Changesets can be up to 50 000 edits and alive for up to 24 hours. The changeset needs to be open before anything can be uploaded to the OSM database. Closing a changeset can be done manually or automatically if nothing is uploaded for a certain time (1 hour currently). A changeset can have tags just like the other elements. A recommended tag for changesets is to give a short description of what you have modified in the map made in that changeset. Changesets are used for rollbacks of the OSM database. If one person is committing something that other persons of the community is questioning and thinks is wrong it’s easier to identify the changes made by that person if it’s wrapped inside a changeset instead of rolling back a whole region. It can be a very complex process to rollback a changeset therefore support for direct rollback is not available in the API. Instead rollbacks need to be made manually by reverse merging, examine the changes and undo them to obtain the desired results.
5.5.4 Potlatch online editor in OSM webpage Potlatch is an online editor for OpenStreetMap intended as an easy to learn editing environment for adding new roads, railways, rivers or other features to the map and to change existing ones. A clear, uncluttered user interface is paramount for this audience. Potlatch uses Adobe Flash, so you will need a Flash Player to use it. It is running in your standard web browser. When you go to the website OpenStreetMap [21] zoom into the streets you want to edit, and then start Potlatch with the edit tab of the main map, a screen will pop up as shown in Figure 10. You will get a choice to “Edit with save” where the update of your work happens after you save it, or “Edit live” where your changes will be uploaded directly after you make them. Edit live is good to use at mapping parties. Edit with save is recommended for beginners.
Figure 10: Login screen to Potlatch.
22 An increasing number of experienced mappers use Potlatch as a supplement to JOSM for instant editing, and a few use it as their principal editor. There is no need to wait for the track to be imported into the database before you start working with a track you have uploaded. In Sunday evenings uploading things to the database usually takes longer when everyone is uploading their weekend’s mapping. One of the more powerful features is to directly convert a GPS track to a road. On the “GPS Traces” tab on the site, click the “edit” link to the right of your track as shown in Figure 11.
Figure 11: GPS Traces tab in OSM webpage, click edit convert GPS trace to road. When you open Potlatch in this way, wait until the GPS track has loaded and a light blue line will appear when this is done. Mark “Convert GPS track to ways” box to do the automatic conversion as in Figure 12. After a few seconds, a new, bright red way will appear.
23 Figure 12: Login screen to Potlatch when opened with edit button. Your waypoints will also be brought in as POI that is locked, so they do not get automatically uploaded to the database if you use “Edit live” Figure 13.
Figure 13: Example on how a waypoint from a GPX file can look when opened in Potlatch. If you do want to upload the POI, tag it up properly and click the little padlock to the right of the POI number. When you deselect the POI, it will be uploaded.
Figure 14: How to unlock the POI. Like the waypoints the way will also be locked. Do the normal edit work of splitting up the way, join it to existing ways and tag it. To unlock the way click the little padlock to the right of the way number. As usual, deselect to upload. If the way has more than 200 points unlocking the way will not work, split it up more before unlocking it.
24 Figure 15: Example on how an converted GPS trace can look like. Potlatch is using Douglas Peucker algorithm to simplify your GPS track to remove unnecessary points on a long straight road. If you find the automatic conversion of the GPS track to coarse, you can hold Shift when clicking “Edit with save” or “Edit live”. This will run a finer simplification over the track. Before you start mapping and want to use the automatic conversion feature you need to know the pitfalls you can fall in. You can’t cross the same road twice because then you will get double ways of one way. This feature is not suited for mapping a housing estate because it’s hard to avoid crossing the same road several times. But if you are mapping a normal trip from a point A to point B e.g. you record your journey from your home to the town. This feature can save you an awful lot of tracing. Pros It's right there on the 'Edit' tab. It allows quick, easy and immediate editing. Map changes can be sent back to the server as you make them. Yahoo! Aerial Imagery is supported without installing a plug in. GPS traces are available as a separate layer. Automatic conversion of uploaded GPS tracks.
25 Cons It's not intended for power users or those who want the speed of a desktop client. Zooming and panning prompts a map fetch. Photo and audio mapping and other file accessing features are not possible. Flash needs to be installed. It is not possible to work fully offline.
5.5.5 JOSM offline editor JOSM is a feature rich editor for the experienced OSM mappers. It requires a little installation and configuration efforts. You need to download the editor, java 1.5 and quite a few plugins [22] exist that you can install. In the preferences enter your username and password to OSM account. Start by importing the GPX file then JOSM will automatically fit the workspace to the bounding box of your GPX track. Click on the green down arrow to fetch the data from openstreetmap.org. In the following dialog (Figure 16) ensure that the checkbox to get OSM data is enabled and press the OK button. There is also a checkbox to download every raw GPS data for that bounding box. If the track is too long, the server will refuse to send you the OSM data for that area. Zoom in a bit to restrict the request to a reasonable size.
Figure 16: Download screen in JOSM for OSM data and GPS tracks.
26 An alternative way to start if you don’t have a GPX file is to click and zoom in to the area you want to work on. Ensure that the checkbox to get OSM data and raw GPS data is enabled and press OK button.
Figure 17: Example on how the OSM data can look like in JOSM. Once you have drawn your line segments as in Figure 17, its time to upload your changes to OSM. Follow this procedure. • Re download the area you have edited. This checks that the area you have been working on has not been altered by somebody else in the time you have been editing. Click on the green down arrow . • If there are conflicts, compare them and pick which one you think is correct. Conflicts can be viewed in the side bar section, opened with alt c. You should be informed when conflicts occur.
• Upload your changes. Click on the green up arrow . • JOSM will give you an overview on the elements it's about to upload. Enter a comment that describes what you have edited.
27 The longer you spread uploads apart, the greater the chances are that somebody else will edit the same area. It is best to upload changes every few minutes. After you have committed your changes to the openstreetmap.org database, it will take some time until the new ways will appear in the main map. If you want to shorten this process, you want to get familiar with the Tiles@home [23] project. JOSM have several more features that can be used to make the editing work easier some of them are, photo and audio mapping. Aerial satellite imagery from various sources can be traced in JOSM. This requires the installation of the WMSPlugin that allows you to fetch images from any WMS source, and display them as a background image within JOSM. High resolution imagery for major cities is available from Yahoo! Pros The panning and zooming of the map is fast and fluid. You can zoom in to near infinite level for super precise mapping. JOSM can work offline using downloaded data files, and can work with local photo and GPX files. Advanced editing functionality e.g. makes nodes of a roundabout into a circle. JOSM is highly configurable and extendible via plugins [22]. Cons Must be downloaded before it can be used and it requires Java 1.5 to work. The finer points of the interface take a while to learn.
5.6 Render maps You have now collected data, created ways, tagged it and uploaded the changes to OSM and want to see the new map with your changes. But because map images take some time to render, the map tiles are cached, and only updated on a periodic basis, rather than immediately after you edit it. On the other hand you can download uploaded changes to OSM database immediately using for example the JOSM editor. The default map Mapnik is updated with hourly diffs so that most data changes should get rendered within an hour. When you try to view the map for that area, it will flag the area to the rendering software. Please keep in mind that "tiles" at particular zoom levels are not updated all at once. Therefore during the update period you may see your changes at some zoom level, while at another they will be visible after a while. Alternatively you can choose the Osmarender layer. Select this using the + icon at the top right of the map. The Osmarender tiles are updated by many OSMers using their own computers Tiles@home [23], which has results in different response speeds usually less than one hour. If this does not work or you like to see quicker update, you can go to informationfreeway.org [24] web page, go to zoom level 12 and press ctrl+click on the tiles to be updated. It will then be added to the list for Tiles@home computers to render. If your changes does not appear even after some time has gone by or you have manually requested a new render you might have a tagging problem. Check that. • All your ways are tagged with something appropriate that will be rendered • Tags are in lower case: HIGHWAY and Highway will not work. • You have underscore “_” and not a space for tags such as highway=bus_stop.
28 Chapter 6 Features discussed at the start of the project This chapter discusses which features the persons that are mapping could have use of in a crowd sourcing client. This was discussed in a brainstorm meeting in the beginning of the project. Several features that were discussed at the start of the project will be presented and assigned a priority when that feature should be implemented.
6.1 Core Features Some features have to be included to be able to have a track log of the path you traveled. Those features belong to the core features and will be implemented first.
6.1.1 Support different connections of GPS There exist many types of GPS devices. Some are connected with Bluetooth; others are serial connected to an external GPS unit for example Sony Ericsson HGE 100. JSR 179 API produces information about the device's present physical location to Java applications. You can use the internal GPS in the mobile phones, which support JSR 179. Mobile phones which support JSR 179 are far fever than JSR 82 which is an API for Bluetooth communication. This means that the most important GPS connection to implement is support for JSR 82 and therefore be implemented first.
6.1.2 Map A map that you can zoom, pan and center the map on your position that is built from OSM data. To get any context of your GPS track nodes it’s a nice feature to be able to see them plotted against a map over the area you are located in, then you get confidence that the GPS works. The map can also be used to make POI.
6.1.3 Create GPX files Each point must have a timestamp or else the OSM database will not accept the point. All other information about the point is optional to include like altitude of the points or signal strength from the satellites. GPX, or GPS eXchange Format is an XML schema designed for describing GPS data between software applications. Waypoints can be added to the GPX file. When all points and waypoints are collected that should be included, a GPX file is created and stored in the file system.
6.2 Upload GPX track from phone to OSM account Instead of the normal way of either storing your GPX file in the mobile phone internal or card memory, transfer the GPX file to the computer from the mobile phones memory and upload it to your OSM account through the OSM website. You could upload your GPX file directly to your OSM account from the phone. Then you get fewer steps to do when uploading your collected data. This is one of the main features Ericsson wanted to be implemented and have high priority when the core functions are implemented.
29 6.3 Download latest OSM map To not get conflicts when uploading changes and to not do work that is already done. It’s practical to be able to download the latest OSM map directly to your mobile phone.
6.4 Edit map on fly when tracking Basic editing work like adding POIs, which type of street it is and giving name to objects can be a useful feature while mapping. Then you don’t have to use any mapping techniques to remember where your are and what to map. This requires that you can see the latest OSM map on your mobile phone and the ability to upload the edited maps to OSM database to be convenient. This is another main feature Ericsson was asking for and will be done after the core functions.
6.5 Audio features It can be cumbersome to type in long messages in a mobile phone or use a notebook; instead you could make audio clips and mark them with a waypoint of where the audio clip was recorded. In the editor you select the waypoint to play the audio clip. This feature will be implemented when all core and main features are made.
6.6 Photo features Taking photos is very space heavy, a mobile phone might have limited storage memory and the use of this approach to make notes of what to map, is not suited for a crowd sourcing client for the mobile phone. This feature has low priority.
6.7 Accelerometer The Accelerometer can be used in the mobile phones that support it, to record vibrations of the mobile phone. This can be used to give a suggestion of what type of street you are on e.g. paved streets or off road street. Another feature is to feel the direction the Mobile phone is directed to. This can help you remember which way of the street you took a picture. Accelerometer is not supported by all mobile phones. Features that use Accelerometer can’t be core parts if you don’t make the same feature which has no requirement of Accelerometer. Accelerometer requires detection works to be implemented that’s not trivial to add. This belongs more to a future work when you have the other features already working on its own.
6.8 Overlay map The overlay map may be used as a guide of where GPS track needs to be recorded. The most powerful use of Overlay map is to use a Yahoo Aerial Imagery over OSM map to spot areas which have not yet covered by OSM. Otherwise you have to guess if there is something around the corner that needs to be tracked. Yahoo has given OSM community legal rights to use their aerial imagery to build up OSM, or else this approach would be breaking the copyrights. This feature is handy but not required and will therefore be implemented if there is time left.
6.9 Recognition intelligence Software reads information from audio clips or photos and interprets it. You take a photo of a speed sign and the software sees that it’s a 30 km/h speed sign and make a note of it (key = maxspeed and value = 30). This feature is a big project on its own and will be skipped in this thesis work.
30 6.10 Automatic building of roads While tracking you can see the GPS track logs continually get updated on your map with the path you have traveled. These points of where you were located on a specific time can be used to create roads automatically. When starting your GPS tracking you decide which type of road to track, for instance key = highway and value = cycle way. Then you follow only that type of road and if you want to track another road type you make a new GPS track log. This is done to make it easier for the software to know what type of road that is tracked. While this is a good feature to have, it has to make correct decisions all the time or else it will add roads that are not correct. This feature is better suited for areas that do not have any coverage in the OSM database already, because then the algorithm does not have to download every node from the OSM database and make assumptions if a node should be connected to the road that is being made while tracking. If this is not done by the algorithm, then you have to manually edit the road created by the application in an editor and the point of the feature to avoid edit work is lost. That makes this feature time consuming and hard to do right and low priority is given to this feature.
31 Chapter 7 Features included in final version In this chapter it will be presented which features that are included in the final version at the end of this thesis’ work and how to use them.
7.1 Startup screen Before the application can be used you have to accept the terms of use of the application, this is shown only the first time the program is used, if accepted. The first screen that is shown when the application is started gives the user a choice of 3 different modes for GPS tracking frequency, car, bicycle and walk, each of these modes have their own GPX settings. The map will start from different zoom levels depending which mode you have selected. The user can also change the GPX settings for every mode. There is a button that is named “Online maps” or “Offline maps” that will change the behavior of the map to be either an online map or an offline map. You can choose to start the application without starting the GPS with the button that is named “Without GPS”. Final button is “New GPS” that will give the user the ability to change which GPS they use.
7.2 GPS Start your GPS and wait for it to get a fix before starting the application. There are usually some lights blinking on the GPS when it is trying to get a fix from the satellites that leave an indication when the GPS has got a fix, when this happens the program is ready to be used. For example with Sony Ericsson HGE 100 you need to plug the GPS unit into the phone, leave it there for a good few minutes until the blinking green light turns solid green to get a satellites fix. Make sure you are outside and that the antenna is positioned with the writing facing upwards. When you start the application for the first time you will get a choice to select which GPS type you want to use. The available GPS types that are Bluetooth GPS in NMEA 0183 format or external GPS connected with a cable using NMEA 0183 format. The rest of the times the application is started the program will try to setup a connection to the GPS that was used the last time. After selecting which type of GPS that is going to be used, the application will try to setup a connection between the GPS and the application. It will then check if the program can read any data containing the information about your position which is sent by the GPS. If data is found a map will be presented that shows that location. If no data is found this step will be repeated 4 more times and if no data is found after that the last known position will be used.
7.3 Map When starting the application the map shows the current position and if the GPS is not turned on or a satellite fix takes to long to receive, the map will show the last known position. You can zoom out and in by pressing “1” and “3” button. Pan around in the map by pressing joystick or “2, 4, 6, 8” buttons. Centering the map on your position if GPS is turned on, or last known position otherwise, by pressing “5” button. The map will follow the path you travel. After using a pan button the map will stop following your travel path until you press “5” again.
32 When a map tile is used it’s stored in the cache memory for fast access. The cache size is 10 map tiles, hence only the last 10 used map tiles is stored in the cache memory, this is done because the mobile phone have limited memory. The user can choose if downloaded map tiles should be stored in the file system in order to get faster access to the map tiles. This also minimizes the data traffic. In “Setup menu” there are options to make the program ignore map tiles that are stored in the file system or to clear all map tiles in a folder. Be aware that the clear operation can take a long time if many map tiles exist in that folder. You can also turn the map to an offline map that does not download map tiles from the OSM server by selecting “Offline map” in either “Setup menu” or at the startup screen. In offline map mode a completely white map tile will be displayed instead of downloading a map tile from OSM server.
7.3.1 Map tiles The Map is built up from tiles that are 256*256 pixel PNG files.
0 1 tile covers whole world
1 2 × 2 tiles
2 4 × 4 tiles
n 2n × 2 n tiles Figure 18: Map tiles per zoom level. When the painter needs a new tile the program will search the cache first and look if it’s there and otherwise check if it’s in the file system. If no map tile is found, the program will download the map tile from the OSM server if the map is in “Online mode”. For each request map call, it’s calculated in which tile you are located. The mathematics for that can be seen in appendix D. The application will then check where in the map tile you are, to know which nearby tiles that also needs to be used to fill the screen. If you are located in right top corner of the map tile you also need, tilex + 1 tiley, tilex tiley + 1, tilex + 1 tiley + 1. When all tiles that are needed to fill the screen are known, the printer is called to paint the screen. Last it checks if you entered a new tile and pre downloads the tiles that are 1 step further away and one zoom level more zoomed in and out.
7.4 Settings Information needs to be remembered to the next time the program is used; to do this a record store was implemented. The Java ME record management system (RMS) provides a mechanism through which MIDlets can persistently store data and retrieve it later. Many settings that change how the program behaves can be changed in “Setup menu” including where files will be saved.
7.5 User feedback User feedback is very important in a client based program. Without any feedback on key presses the user doesn’t know if the action was successful, unsuccessful or if the program has been
33 frozen. User feedbacks that are supported in the application are; menus that change after preformed action like “Start Tracking” becomes “Stop Tracking”, pop ups messages that explain what happened or why the action can’t be used, loading screen while waiting for something to be done. There is also status information at the bottom right corner for the GPS, if the GPS is not turned on, if the GPS has not found a fix yet or how many satellites the GPS has connection with. The signal strength (Horizontal Dilution of Precision [25]) HDOP is also shown. HDOP is a value that tells the additional multiplicative affect of GPS precision, hence low HDOP value are greater signal strength. If you got a value over 5 HDOP, it’s not recommended to use that track to build up roads, because if your signal strength is this low your points will have to bad accuracy. After you have activated “Start Tracking”, information about how many points that have been recorded will be displayed in the bottom right corner.
7.6 Send SMS You can send an SMS with your location in latitude and longitude. This can be used when you have mapping parties and want to give your partner the location where you have found a good place to meet up. You can enter the location you got from the other person by pressing “#” for 1 second and enter longitude and latitude to move the map to that location.
7.7 Record audio The program uses the “Audio clips with waypoints” method from section 5.1.6 to save recordings. To record audio clips your mobile phones need to support encoding in WAV format. The reason for choosing this format to record the audio is that JOSM (Java OpenStreetMap, An editor program) can understand it. If you record in AMR format another application needs to be used on your computer to convert the AMR to WAV format if you want to use the audio mapping feature with JOSM. A message will tell you if you have no support for WAV encoding when trying to record. Press “Record Audio” in menu or “9” button to start recording audio. Press “Stop RecordAudio” in menu or “9” button to stop the recording. A waypoint will be saved into the GPX file where the audio clip was recorded. A link tag will be added to the waypoint that tells which audio clip that belongs to the waypoint. To listen to the audio clips in JOSM, transfer the GPX file to your computer and copy the audio folder from the mobile phone and put it in the same folder as the GPX file. Then start JOSM and import the GPX file. GPS tracks and waypoints will be displayed as normal but now JOSM creates Audio Marker for each waypoint that contains an audio clip. JOSM knows which audio clip to play when you click on the marker through looking on the description tag.
7.8 GPX track log The application will need to get signals from at least three satellites before a position can be determined and tracking can be initiated. The application saves the current signal strength (HDOP) to every point that is recorded. Many experienced mappers use a script that automatically discards points in the GPX file that was taken under bad signal strength from the satellites by looking at HDOP value for each point. Track points are saved after either a distance threshold is reached or after a time interval. In setup menu “GPX settings” you can choose to use distance or time as the rule set of how often points are saved and the threshold distance or time
34 interval can be changed. Waypoints and audio recordings waypoints are saved to the GPX file. You can display your GPX tracks by pressing “Manage GPX” in the menu or pressing the “*” button for 1 second. The map will be centered on the first point of the GPX track. Pressing “7” will toggle between showing the GPX track in the display or not.
7.9 Upload GPX track and edit work Adding POIs and paths to the OSM database is supported. Also uploading of GPX tracks to your OSM account are supported. But the application does not support uploads that are larger than 2kb in size on Sony Ericsson phones that have the JP 8.4 platform or less. JP 8.5 platform version will make upload work regardless of the size of the upload. This is an OSM server problem where they don’t support HTTP POST data sent in chunked mode, even when required by the HTTP/1.1 specifications . In appendix c you can read a post made in the Sony Ericsson developer forum about this change made in JP 8.5 platform. Two kb correspond to about 7 nodes, which is not enough to make the upload of GPX files have any practical use.
7.10 Basic map editing on the phone In the application there is support for basic map editing directly to the OSM database.
7.10.1 Points of interest Points of interests can be added to the map by the normal way of taking a waypoint and take note of what POI it is. Back on your computer you tag the waypoint with an editor and upload it to the database. With this application you have another way to do this directly on your mobile phone. Press the “0” button or “Add POI” in the menu to get a preset list of common POIs. Choose one of the POIs in the list or none if you want to add some POI that is not in the predefined list. Add all appropriate tags to the POI, by pressing “Add tag” in menu. Enter the key and value of the tag e.g. to get a tag name=Hard Rock Cafe enter “name” as key and “Hard Rock Café” as value. All tags need to be in lowercase except name tag to get accepted by OSM database. After all tags are set press “Upload to OSM” to upload the POI. If no changeset is made you will get a textbox to add a comment to describe what type of update you are doing to the OSM database.
7.10.2 Build way “Edit mode” in menu will make the map pan in smaller steps, for better precision when adding nodes. Press “9” to add new node and press “7” to remove last node. Create a way of the added nodes, by pressing “*”. It’s important that all the nodes are added in an ordered sequence when making a road.
Figure 19: Nodes that builds up the way have to be in an ordered sequence. The same procedure is used to add additional tags and upload to OSM as for POI.
35 7.11 Render styles Four different render styles are supported in the application. The default map is “Mapnik” which is used when starting the application the first time. “Osmarender” which can be used instead has another look that is similar to “Mapnik”. “Cycle Map” highlights the cycle ways. “NoName” is the last option and highlights roads with no name tag with a red casing at higher zoom levels, or as thin red lines at lower zoom levels.
(a) Mapnik (b) Osmarender
(c) Cycle Map (d) o ame Figure 20: Example on how each render style can look like. The pictures in Figure 20 shows a part of Luleå in zoom level 12. Each render style use different parameters for how to build the map, for example Osmarender shows cycle ways in this zoom level and “Mapnik” starts to display the cycle ways at zoom level 13. The user can choose which of these render styles that they like most, by going to the “Setup – Map render type” and select which of these 4 render styles he/she wants to use.
36 7.12 Waypoints There are several different ways supported to create a waypoint. First one is to press “5” for 1 second or in the menu select “Save Waypoint” to save the waypoint on your position or pressing “#” to save the waypoint on the position of the cross in the middle of the screen. Now you will get a form where you have a text field to enter a waypoint. There is a list of the last 9 different waypoints made, which are numbered from 1 9. If you enter one of these numbers first in the text field, the waypoint associate with that number will be added. Another way is to double press a key to add a predefined waypoint. For example selecting Add Speed Limits” in the menu will get you a list of waypoints that belongs to speed limits and pressing “3” will add a waypoint maxspeed=30 km/h. That is useful while tracking in a car. Third is to make your own predefined waypoint key bindings. Make a TXT document with predefined waypoints in the format showed in Figure 21, image a. every waypoint ends with a “,” sign that is used to parse the file. Figure 21, image b shows to which button on the mobile phone it’s mapped to. amenity=restaurant, Button 1 amenity=pub, Button 2 amenity=food_court, Button 3 amenity=fast_food, Button 4 amenity=drinking_water, Button 5 amenity=bbq, Button 6 amenity=biergarten, Button 7 amenity=cafe, Button 8 amenity=cinema, Button 9 amenity=nightclub, Button * amenity=studio, Button 0 amenity=theatre, Button # (a) Text document (b) Key binding on phone Figure 21: Table that shows which button a waypoint is mapped to. Put the .txt document in the mobile phones file system. In the setup menu you can chose which TXT document to use. The name of the TXT document will appear in the menu, select “Add (name)” and the predefined waypoints for double press will change. Waypoints are always saved on the location where the GPS is.
7.13 Setup menu In the setup menu you can configure some settings in the program: • “DiscoverGPS”, Choose which GPS type to use from Bluetooth GPS or serial connected GPS.
37 • “OSM account”, the user can save his/her OSM account username, password and web address to OSM server. • “File path settings”, choose where files will be stored in the file system. • “GPX settings”, change parameters on how GPX file is created. • “Waypoint key bindings”, change which user created TXT file that are used for add waypoints by double button press. • “POI list”, change user created .txt file, which are used for quick access to predefined POI. • “Online/Offline maps”, will turn on/off the ability to download new map tiles. • “Map render type”, there is 4 different render techniques to choose from “Mapnik”, “Osmarender”, “Cycle Map” and “NoName”. • “About”, gives credit to OSM contributors that have helped build the map and the license for OSM data. • “Help”, show what each key bindings is mapped to.
38 Chapter 8 Implementation The main goal with the design of the architecture was to make it easy to extend with new features. Another goal was to make as many settings as possible in the application user configurable. First task was to arrange a working environment by: • Setting up a Bluetooth connection between mobile phone and GPS receiver. • Implementing a thread that fetches information from the GPS receiver in regular intervals. • Constructing a parser that interpret the data that the GPS receiver is sending and store that information. • Using the information about latitude and longitude to calculate which map tile you are in. • Setting up an internet connection to the OSM API to download that map tile. • Creating a GUI that displays the map. • Making a setup menu for settings of parameters in the program. • Adding functionality to change what is displayed in the screen. With these functions we have a working environment that can be used for testing. All that is left is just adding new feature to the program and you can instantly test if it is working as intended, thus we begin the next phase of the implementation which is the work flow.
8.1 Work flow The philosophy for the work flow was to add small updates in the form of a new feature. Most added features went through the following steps: • Design: In the design phase, select a feature and plan how it will be implemented. • Implementation: In the implementation phase, you realize the design and write the source code. • Testing: In the testing phase, you check the implementation. Because the application is using online information by requesting map tiles that are not in the cache memory or file system, all the testing was done on a real device. As told in chapter 4 an iterative implementation cycle was used in 3 phases. At the end of each of these 3 phases it was devoted time for optimizations and adding user feedback functions.
8.1.1 Optimization When you implement and test your prototype, you usually discover shortcomings that need to be fixed in the optimization phase. Typical shortcomings include the following: encountered bugs on some devices, but not on others and the performance are not as good as expected.
39 Typical real world scenarios were tested and the perceived performance was analyzed. If you don’t get frustrated on the slow speed of the application it’s a good indicator the performance is good enough. For the first implementation of a feature the goal was to make it work and if the performance was to poor another solution was tested. One example of optimization that was done was with the problem that after tracking starts the points were delayed 20 seconds before they were saved. At this point several threads were used that was put to sleep when they had nothing to do, instead of this a single main thread was made that did the same work as the other threads did together. With that high level design change the delay was improved a little more than 1 second but made the program crash frequently. The program crashed from synchronization issues. After fixing that issue the program stopped crashing but the fix made it little slower.
8.1.2 User experience Next step was to add user feedback to the features. Letting the users know about the progress of an ongoing task is a good investment, because it increases the users’ patience dramatically. When the application was evaluated by other persons most of their comments was to change words of buttons or to use fewer buttons to get something done. Thus in this phase the goal was to make the tasks complete in fewer user confirmed steps and add more automatic behavior from the application.
8.1.3 Redesign of source code After the last implementation phase was finished, the architecture of the whole application was analyzed and redone to minimize the application size. After this work the Java ARchive (JAR) file of the compiled application was reduced in size from 209kb to 126kb. At the same time debugging testing was done to polish the program even more.
8.1.4 Obfuscation When compiling the source code to Java Application Descriptor (JAD) and JAR files obfuscation [26] was used to get better security, smaller footprint, and improved runtime performance. After obfuscation the size of the JAR file was reduced in size from 126kb to 76kb. In general, an obfuscator changes the byte code (the code stored in your .class files). The obfuscator removes the package, method, class, field, and variable names that is unused and changes the remaining ones to meaningless strings that are as short as possible. It also removes any comments or debugging information. To further improve the security, obfuscators change the actual control flow (loops, conditionals, etc.) and actual data structures are altered. It’s easy to decompile and reverse engineer your application; hence it’s important to use methods like obfuscation to make the code that comes out from decompilers nearly impossible to understand.
8.1.5 Progress To get a feeling that the application was moving forward each day, and not get stuck on a problem for a long time when nothing gets added to the program, hard to implement features was mixed with easy to implement features each day, instead of just focusing on one feature at a time.
40 The high flexibility of the software development process model used, where you can add any new feature and test it in the working environment made it possible to work on several things at the same time independently of each other.
8.1.6 Debugging The debugging method was to test the program in the mobile phone and see what happened with the newest added code by writing what is done in the code to a log form. In the log you can see if the latest code does what it was supposed to do.
8.2 Testing The application has been tested to work on a few Sony Ericsson mobile phones. Some features can’t be used by all mobile phones. In the bullet list that follows the restriction that applies to the bullets before also apply to current one. • K850, G502 and C702 no upload over 2kb. • K810 no audio recording. • K800 needs to be signed to work without messages that pop ups repeatedly. The program was tested on one Nokia mobile phone 6267. Nokia 6267 needs to be signed and given permission to access necessary resources to work. No audio recording is supported but you can upload a file that is as large as the memory heap allows. Uploading a GPX file with 2000 points took about 2 minutes. Depending on which phone brand that is used reading information from a GPX file with 700 points takes 3 seconds on Sony Ericsson G502 and 40 seconds on Nokia 6267. The program has been tested with 2 different Bluetooth GPS (Wayfinder and Holux M-1000 ) and Sony Ericsson HGE 100 for serial connected GPS, all tested GPS’s are working.
8.3 Permissions The application is required to have permissions to “Read user data”, “Write user data”, “Internet access” and “Local connectivity”. “Multimedia” is optional and used to give permission to record audio. If your mobile phone does not have any option to set these permissions to “Ask once” or preferably “Never ask” the application will not be fun to use because it will ask you for permission to do these tasks regularly and often. These settings are set on your mobile phone. If you can set the permissions to “Ask once” the program will ask if it’s allowed to access them once when starting the application. With “Never ask” permission the program will never ask.
8.4 Beta testing Up to this point almost all testing has been done by the author and some limited testing by a few employees at Ericsson Research. Now the application was at a stage that was ready to be tested by a selected few people that are the target group for this application. The beta testers were used to, test the stability of the program, and give feedback of what things that are good/bad in the application, discuss ideas on how to improve the program and get an understanding on how they use the application. This feedback was used to make some last adjustments to the crowd sourcing client application.
41 Chapter 9 Future work A requirement for widespread usage of this application is to make it work with more mobile phone models and brands. It should also have support for internal GPS. Existing features can be improved upon in several areas and there are a number of new features which would be of benefit to the general usability of the application. Some ideas and suggestions are mentioned below.
9.1 Audio mapping with bike Support for accessories like Sony Ericsson HGE 100, which has a GPS, some buttons and a microphone. The GPS can be used instead of a Bluetooth GPS and the buttons can be used for easy access to useful commandos like start tracking, save track and record audio. The ability to record audio in this manner gives an incredible powerful tool while trekking with a bike. Then you can just put your mobile phone in your pocket. Lock the buttons on the phone to prevent that they are accidentally pushed and use the buttons on the HGE 100 to record audio through its microphone. If you connect earphones you could have different types of sound notifications while you push a button to keep track on what happens e.g. start record audio and stop record audio.
9.2 Overlay map Of the planned features that are presented in chapter 6 that did not get included in the final version in this thesis work, next to implement should be overlay map section 6.8. With the overlay map it’s easier to find places where OSM has not been mapped already.
9.3 Map Implementing a feature that checks if the stored map tiles are old, and download new map tiles if they are obsolete, would be a good addition. Because checking for new map tiles is a time consuming task. One approach to fix this is to do an update feature where you press a button to start looking if the map tiles that are stored in the file system need to be downloaded again to be up to date. The advantage in doing this before you go out and mapping is that you will get a better responsive map. When you do this the map will work as an offline map for the places you have been to before and the new places will work as an online map. For the default Mapnik render style map, every map tile has a status field that can be checked to see when it was last rendered. To see this information just add “/status” to the URL address for the map tile.
9.4 Pre download maps on PC Make an application that downloads map tiles on your pc that can be transferred to the file system on the mobile phone. This is useful for the people that do not have “flat rate” subscription for data and needs to pay for all downloaded data. These users use the offline maps feature when absolutely latest maps are not needed and know which area he/she will visit in advance.
9.5 Change GUI to LWUIT The Lightweight UI Toolkit [27] (LWUIT) use little, if any device specific code which makes it more portable between mobile phones models and brands than standard library LCDUI can offer. The real strength and power of LWUIT comes in how much more control you have of layout and
42 display characteristics (color, font, and so on). If there is a missing feature or component, you can create your own and plug it in your code. All LWUIT components support touch events. No special coding needed for a LWUIT application to run on a touch enabled device.
9.6 Optimization You could optimize the performance by closing operations when they are not needed. For example when you upload your GPX track to your OSM account you could stop listening for messages sent by the GPS receiver.
9.7 Preprocessing In the preprocessing phase, the source code is changed before it is compiled. You can utilize a whole armada of preprocessing directives for customizing your application to different environments. Each device defines capabilities and symbols that you can use in the preprocessing step. With preprocessing you can adapt your application to different handsets without creating incompatibilities. This is needed because of the fragmentation in the mobile phone market where every device can use a different API. If you don’t use the device’s own API you will not get optimal performance or worse. “Some devices do not accept any applications that refer to unknown classes, even when those classes are loaded only via Java's reflection mechanism [28]”.
9.8 Split up the GUI in layers Split up the GUI in several layers to avoid that everything needs to be repainted when the map needs to be updated. One layer with the map tiles, another layer with info about signal strength etc. and a layer for GPS track points. With these layers you could optimize the application a lot. For example when you pan the map you could also pan the layer that is holding the GPS track points instead of rewriting the points, which can be quite many after 1 hours of tracking. Then you would only need to repaint the track points for zoom operations, which are a lot less used, compared to the pan operation. When you have the info in a layer on its own, that layer could be updated every second when new information is fetched from the GPS receiver, instead of every 10 second as it is now without any effect on the performance. This feature should have high priority to implement if this work is continued.
43 Chapter 10 Discussion and Conclusions Many maps today collect new map data over a time period and in regular intervals update the maps. In a brainstorm meeting it was decided to not use the Mobile Maps API and instead use a direct link to the OSM database to guarantee that the maps you download are the latest rendered maps. For this application it’s very important that you have the latest map available to minimize conflicts when uploading your edit work with other people’s work that is done in the same area. For example if you go out and map in a place you think are unmapped and tries to upload your edit work a problem will appear. If your map is 2 days old and another person already have mapped that place, then you need to spend time to do unnecessary work to fix the conflicts that could have been used to add new features to the map instead if you would have had the latest map from the beginning. Another advantage is that your work can be seen faster when you work directly with the OSM database, usually the maps are rendered with new changes within an hour.
10.1 Map To use an online map poses a challenge with loading times to fetch the map tiles that builds up the map in contrast to an offline map where the map is pre downloaded before it’s used. To avoid that the application downloads new map tiles for every request map call, each used map tile is put into the cache memory. There is an option to use the map as a mix of both online and offline map, where the map tiles are stored in the file system when they are downloaded. This will make the map work as an online map the first time you visit a place and the following times as an offline map because then the map tiles will be fetched from the file system instead of downloaded from the OSM database. The advantage of an online map is that the map is always up to date, which will avoid conflicts with double work of the same area. The advantage of an offline map is that the responsiveness of the map is much faster to zoom and pan operations. The best of both worlds would be to only download map tiles when they are updated, but that introduce additional computations to check if a new map tile exist in the OSM database and the gain of this approach needs to be measured to be able to declare if it is any faster than an online map in the average case. In this thesis work only the perceived loading times for zoom and pan operations of the map has been analyzed and the conclusion from the testing’s are that the speed is good when fetching map tiles from the cache memory or the file system, but slow when downloading from the OSM database. The speed is good enough to download map tiles for the main tasks with the application which is to use it as a tool for crowd sourcing to see the path you travel and check nearby map tiles of where to go. But if you want to zoom and pan around in the map a lot, it’s too slow. An idea to improve the map is presented in section 9.3 and 9.8.
10.2 Mapping techniques Of the mapping techniques mentioned in chapter 5, “audio mapping” is one of the most powerful features to have in a mobile phone crowd sourcing client. With a mobile phone you want to type in text as little as possible because it’s a time consuming task and to build a good map you need a lot of field information, hence use “audio mapping”.
44 Of the problems that exist with “audio mapping” almost none is a real problem when using a mobile phone. Synchronizing, linking waypoints to corresponding audio file and to put the waypoint into the GPX file are done automatically. Interpreting the audio clips can be overcome by being careful and aware of the problem while recording sound clips. The real problem lays in that audio clips take a lot of space in the memory which is a limited factor in a mobile phone, especially when recording is done in WAV encoding to make the audio clips work with JOSM editor directly. In section 9.1 an idea is presented on how to make the “audio mapping” feature work well in a mobile phone solution while mapping with a bike. To make Waypoints is also very useful if you only have to press a button or 2 to get one because typing in a full length waypoint name/description takes to long time. To achieve this goal of a fast way to make specific waypoints, this application support the user to do their own predefined waypoint list of up to 12 waypoints per list and the list can be change in runtime while tracking between 3 predefined waypoint lists. To add a waypoint you double press a key fast with the right list activated.
10.3 Sending larger sized information to OSM The problem that took most time to solve was one of the main features that Ericsson Research wanted me to do, uploading GPX tracks from phone to OSM account, discussed in detail in section 6.2. When you use an external server that you have no access to study what really is sent by the client. It is hard to know if the error was on the application’s part or the OSM server. Many different parameters when sending HTTP POST messages to the server was tested and none affected the problem that the server refused to handle upload larger than 2kb correctly. Then a proxy server was setup to see exactly what the application on the mobile phone was sending and discovered that the client was sending in different ways for uploads larger than 2kb. When sending uploads smaller than 2kb the length parameter in the header for the HTTP POST message is used by the server to see how many bytes that the server should expect to get. While sending uploads larger than 2kb the mobile phone did the transfer encoding in chunked mode. In chunked mode the phone ignores the length parameter in the header and instead sends a message that tells how many bytes that will be sent and thereafter sends the real data, in chunked mode the client relies that the server is following the HTTP 1.1 specifications and can assemble this. OSM server can’t do this; hence the problem to send files over 2kb. Then a Nokia phone was tested to see if it has the same problem which it didn’t. The Nokia phone could send data larger than 2kb fine until the upload was larger than the memory could handle. Now the root of the problem was narrowed down to not being caused by the application and the OSM server could handle uploads that was not using transfer encoding in chunked mode. Later on, an announcement made by Sony Ericsson developers was put in their forum that the next java platform JP 8.5 will use the HTTP POST header to specify length of message regardless of the size of the upload. After that it was decided with my supervisor Staffan E Larsson to focus on other features when the cause of the problem was found instead of finding a workaround to this problem.
45 10.4 Difference between java platforms To make a good looking GUI for a Java ME application that looks the same for every mobile phone model and brand is not an easy task. This also applies to several other areas where each mobile phone device can have their own device specific architecture that solves the problem in a different way, thus making cross handset friendly code hard work. The built in UI components in Java ME Limited Connected Device User Interface (LCDUI) are generally hideous; you have very little control over their appearance. What looks pretty on one handset looked awful and out of place on another. For example prioritizing components in which order they will be put on the screen by giving a value for each component seemed fine in theory, but some phones build up the components from left side to the right side and other phones from the opposite direction, hence putting a choice to exit the application by pressing yes or no that has yes on left side was not always possible. It's very difficult to write cross handset friendly code using LCDUI and it’s recommend to use another solution. After searching for other options “J2ME Polish [29]” was found. J2ME Polish is really useful when developing J2ME Applications specially a generic one. J2ME Polish is a framework, for building your application for various devices. J2ME Polish is distributed under the GNU General Public License as well as commercial licenses. You can use the GPL license for projects which are licensed under the GNU General Public License without limitations. Another option to use for GUI is The Lightweight UI Toolkit [27] (LWUIT). LWUIT is a UI library that is bundled together with applications and helps content developers in creating compelling and consistent Java ME applications. LWUIT supports visual components and other UI goodies such as layout management, transitions, animation and more. The current version of the library works on top of CLDC 1.1/MIDP 2.0 and Connected Device Configuration (CDC) platforms. LWUIT helps to eliminate a major source of device fragmentation issues that occur at the graphical user interface level. The LWUIT JAR is about 225KB, while not insignificant, the power LWUIT packs makes it well worth the space allocation and it works similar to Swing/AWT that is available for desktop applications. It is reasonably licensed. The GPL+CLE (classpath linking exception) makes it available to closed source projects. LWUIT comes with a pretty good set of documentation. The Javadoc API documentation is not thick (many methods contain a single line of documentation), but it is complete. For more details, take a look at the Developer's Guide, which is distributed with the download ZIP file and provide more explanation about LWUIT features and often provides example code for how to use LWUIT classes.
10.5 GPS A couple of things about the GPS units: • They vary in accuracy by brand and model. Units that are 10 years old or more have less accuracy. • The more satellites the unit is receiving, the more accurate the points get. • Trees and buildings or any metal above the GPS can block signals.
46 • With state of the art receivers (sirfstar III or similar) the precision of your track points can reach from 3 meters (free sight to the sky) to 10 meters (dense forest) or even 30 meters and more (inner city with high buildings around). It’s good to make a few traces along the path, preferably at different times of the day and/or different days. This will give you readings from different satellites from different angles, and then average your traces. If you bring up previously loaded traces from others, you will see quite often the traces are wobbly but if there are enough traces, you can see a fairly accurate line from the average of all the traces. Some tips on how to improve the track qualities is to hold the GPS in front of yourself rather than having it in a pocket. It’s good to give the device a few minutes whilst standing stationary with free sight to the sky, to get a good lock from the satellites before you start tracking. When tracking a street, walk down one side of a street and back up the other side. If there’s a glitch in the GPS it’s usually obvious to spot them if you walked the same street twice and to waypoint certain objects several times to get an idea of accuracy/drift.
10.6 What happens next What happens with the application after this thesis´ work is finished is that it will be released as an open source project; an investigation has been started to see if this is possible. The goal is that the application will continue to be developed by everyone that gets interested in the project, hence why this thesis have had a focus on making it easy to extend with new features.
10.7 Conclusions This study has explored how the OSM maps are built up. A number of possible mapping techniques has been identified and analyzed. This thesis work documents the first phase of developing a crowd sourcing client application. Currently the application is working on a limited number of combinations of mobile phones and GPS devices. The ambition, within the scope of the thesis, was to add as many features as the time allowed with functionality good enough to properly handle its task without getting bad user experience. The final version of the application in this thesis work can be found in Ericsson Labs homepage at the application section called Mobile OSM Creator (https://labs.ericsson.com/applications ). With the fast growing of services/applications that are location based, more and more people understand the needs for maps that are easy to update, free of technical restrictions and cost free to use. The idea to make the maps open source is an interesting approach. Everyone benefits from having large and accurate maps, hence let the community update it. For many people, mapping can be highly addictive.
47 References [1] 2009: The Year of LBS (Location Based Services), 2009 [Online], Available: http://www.readwriteweb.com/archives/2009_the_year_of_lbs_location based_services.php Last accessed 8 December 2009. [2] OpenStreetMap wiki website, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Main_Page Last accessed 7 October 2009. [3] Copyright Easter Eggs, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Copyright_Easter_Eggs Last accessed 20 October 2009. [4] IGN.se Tele Atlas: Framtidens gps karta uppdateras hela tiden, 2007 12 03 [Online], Available: http://m3.idg.se/2.1022/1.133998 Last accessed 2 October 2009. [5] OpenStreetBugs, 2009 [Online], Available: http://openstreetbugs.schokokeks.org/ Last accessed 7 October 2009. [6] Map features, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Map_features Last accessed 7 October 2009. [7] Yahoo! Aerial Imagery, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Yahoo%21_Aerial_Imagery Last accessed 7 December 2009. [8] Mapping Weekend Howto, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Mapping_Weekend_Howto Last accessed 7 October 2009. [9] OSM_Components, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Image:OSM_Components.png Last accessed 2 December 2009. [10] Component overview, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Component_overview Last accessed 7 October 2009. [11] Stats, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Statistics Last accessed 23 November 2009.
48 [12] The Open Source Definition, 2009 [Online], Available: http://www.opensource.org/docs/osd Last accessed 7 October 2009. [13] Open Source Licenses, 2009 [Online], Available: http://www.opensource.org/licenses Last accessed 3 December 2009. [14] Creative Commons, 2009 [Online], Available: http://en.wikipedia.org/wiki/Creative_Commons Last accessed 10 November 2009. [15] Open Data, 2009 [Online], Available: http://en.wikipedia.org/wiki/Open_Data_Commons Last accessed 10 November 2009. [16] GNU General Public License, 2009 [Online], Available: http://www.gnu.org/copyleft/gpl.html Last accessed [17] Iterative and incremental development, 2009 [Online], Available: http://en.wikipedia.org/wiki/Iterative_and_incremental_development Last accessed 18 November. [18] Editing Standards and Conventions, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Editing_Standards_and_Conventions Last accessed 21 October 2009. [19] Tagging, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Tagging Last accessed 7 October 2009. [20] Proposed features, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Proposed_features Last accessed 21 October 2009. [21] OpenStreetMap, 2009 [Online], Available: http://www.openstreetmap.org/ Last accessed 7 October 2009. [22] Plugins, 2009 [Online], Available: http://josm.openstreetmap.de/wiki/Plugins Last accessed 7 October 2009.
49 [23] Tiles@home, 2009 [Online], Available: http://wiki.openstreetmap.org/wiki/Tiles@home Last accessed 7 October 2009. [24] informationfreeway.org, 2009 [Online], Available: http://www.informationfreeway.org/ Last accessed 7 October 2009. [25] Dilution of precision (GPS), 2009 [Online], Available: http://en.wikipedia.org/wiki/HDOP Last accessed 7 December 2009. [26] Use Obfuscation to Improve the Size, Performance, and Security of Your J2ME Applications August 17, 2005 [Online], Available: http://www.devx.com/wireless/Article/28989/0/page/1 Last accessed 12 November 2009. [27] Java ME Technology – LWUIT, 2009 [Online], Available: http://java.sun.com/javame/technology/lwuit/ Last accessed 4 November 2009. [28] Robert Virkus, Pro J2ME Polish Open Source Wireless Java Tolls Suite, 2005, p. 101. [29] J2ME Polish, 2009 [Online], Available: http://www.j2mepolish.org/cms/topsection/home.html Last accessed 21 October 2009. [30] Important HTTP POST related change in JP 8.5, 2009 [Online], Available: http://developer.sonyericsson.com/community/docs/DOC 3217 Last accessed 4 December 2009.
50
Appendix A) User guide Startup • Before the first time you use the application set the permissions to “Read user data”, “Write user data”, “Internet access”, “Local connectivity” and “Multimedia” on the application in your mobile phone to “Ask once” or preferably “Never ask”. • Turn on the GPS before starting the application. Keep in mind that it can take a while for the GPS to get contact with the satellites before you can get a position. • The first time the application is used it will search nearby Bluetooth devices to setup connection with if you select “Bluetooth GPS”, the rest of the times the application is used, it will use the last selected Bluetooth GPS at startup. The same procedure applies if you choose “Serial” for GPS connected with cable. o If you want to change the GPS device, it can be done in the Setup menu or when you start the application the next time select “New GPS” at startup screen. • The startup screen lets you choose from 3 different modes (Car, Bicycle, Walk), that have their own settings for how often GPS points will be recorded. These settings can be changed in the “Setup menu – GPX settings”. • To prevent bad points in the start of your track stand still with a free sight to the sky until you have a good fix from the satellites. A good idea is to wait another minute or so for everything to settle down. o In the bottom right corner there is a value called horizontal dilution of precision (HDOP) that tells how good fix you have 1.0 HDOP is ideal fix. It is recommended to have below 5.0 in HDOP to have confidence your position detection is enough accurate. • Before you can Upload GPX files, POI and ways to the OSM database you have to set your OSM account details in the setup menu. • If you don’t want to download anything from the internet and have an offline map select “Offline map” in the startup screen, this can also be done in the “Setup menu”. Only map tiles that are stored in the file system will be used and when no map tile is there a completely blank map tile will be displayed instead. • If you want to start the application without starting the GPS select “Without GPS” at startup screen.
51 Tracking • While tracking, the red dots mark your travel path, the blue dots are saved waypoints and green dots are saved audio clips. • Saving waypoints can be made in 3 different ways: o First is to use ”save waypoint” either in the menu or hold down the ”5” button for a second. You get a textbox where you enter your note. The last 9 waypoints added is remembered, these can be used by typing in a number 1 9 in the textbox. The location of the waypoint is fetched from the position the GPS is giving. o Second is the same as first choice but the location of the waypoint is fetched from the position of the cross that is in the middle of the screen. Use the “#” button to add a waypoint with this method. o Third is to make a TXT file where you can make up to 12 predefined notes, where every note ends with ”,”. In “setup” you can load this list. The name of the TXT file will now be an option in the menu. When selecting it, the keys will be mapped to the 12 predefined notes. The first one is mapped to button 1 and second to button 2 etc. To create a waypoint you double press a button. For example double press on button “2” to add a waypoint “lanes=2”. Two predefined waypoint list already exist in the application that can be used which are “Add Speed Limit” and “Highway Note”. In your mobile phone there is an example TXT file that shows how to make your own predefined waypoint list, that can be found in “e:/Videos/Key bindings/” or “c:/Videos/Key bindings/”. This TXT file will be created after accepting the terms of use. Remember to end each waypoint note with “,” as shown in the example TXT file to make your list work. • To save the GPX track logs, select ”save track” or press button ”*”. • To record audio you select "Record Audio” in menu or press the ”9” key. To stop the audio clip select "Stop RecordAudio” in menu or ”9”. A waypoint will be added where this audio clip was recorded. Point of interest (POI) • Add POI by selecting “Add POI” in menu or press the ”0” key. You will get a preset list of common POI that is supported with icons by OSM. After choosing a POI you can add more tags for the node, like adding a name to the POI by adding a tag with key ”name” and value ”Ericsson”. Remember that all tags need to be in lowercase, except the “name” tag. When all tags are set, you upload the POI by selecting "Upload to OSM”. If no changeset is created you will get a form to make one. Edit • Copy the GPX file to the PC and the audio folder to the same folder where the GPX track is placed. • Start Java OpenStreetMap Editor (JOSM), download the map over the area you have tracked from OSM server and import the GPX file to start edit the map. If you have recorded some audio clips they will now appears as a marker in JOSM that you can click to play up the sound you have recorded.
52 Tip • It’s recommended to always upload your GPX file to your account so others can see where your source comes from. • If you are using the mapping technique “two way mapping” that is described in section 5.2, a tip is to change the render style in “Setup – Map render type” to NoName. Now you will get a map that highlights roads that have no name tag, which gives you a hint of where additional information should be collected. • A tip on how to improve the track qualities is to hold the GPS in front of yourself rather than having it in a pocket. It’s good to give the device a few minutes whilst standing stationary with free sight to the sky, to get a good lock from the satellites before you start tracking. When tracking a street, walk down one side of a street and back up the other side. If there’s a glitch in the GPS it’s usually obvious to spot them if you walked the same street twice and to waypoint certain objects several times to get an idea of accuracy/drift.
53 B) GPX 1.0 schema
54 GPX 1.1 schema
Figure 23: Sample GPX file using GPX 1.1 schema with one point.
55 C) Problem sending larger files Important HTTP POST related change in JP 8.5 at http://developer.sonyericsson.com/community/docs/DOC 3217 [30] page. “On our platforms prior to JP 8.5 with HTTP POST we used chunks to send files larger than 2 kb to servers. Since there are servers that do not have support for this, although it is a required HTTP/1.1 feature, we made changes in JP 8.5”. “In new implementation you just have to set correct Content Length header and HTTP POST will work with all servers”. “HttpConnection#setRequestProperty("Content-Length", sizeOfRequestDataInBytes) . This guarantees that the transfer encoding will be non chunked”.
56 D) Reproject the coordinates to the Mercator projection:
x = longitude
y = log(tan( latitude ) + sec( latitude )) (Latitude and longitude are in radians) Transform range of x and y to 0 1 and shift origin to top left corner:
x = (1 + ( x / π)) / 2
y = (1 ( y / π)) / 2 Calculate the number of tiles across the map, n, using 2 zoom where “zoom” is the zoom level. Multiply x and y by n. Round results down to give tilex and tiley .
57