André Klapper <[email protected]> GUADEC 2007, Birmingham Where's my monitoring at? Some random figures about a software project called GNOME (and a pr etty untechy talk, compared to the rest here) Monitoring ● Measurement & numbers for decision making support & a more optimal use of resources ● continuous process on various levels, can end up in defining and better documenting of processes and assigning of responsibilities What is this about, dude? ● GNOME = not a company, but nevertheless professional? ● identifying problems with the help of some automatization – some examples what we have, and what we don't have ● end up in better overall quality and happier end customers by clearer internal processes? „Where are we, where do we wanna be?“ GNOME: Structure Here: ● Release Team ● Docs ● GTP/I18N ● Bugsquad ● („Hacking“ only w.r.t maintainers) Source: https://twiki.softwarelivre.org/bin/viewfile/Main/ Vice nteAguiar?rev=1;filename=OrganogramaGNOME2.4_.png by Vicente Aguiar Release Team (RT) ● Aim: Test & publish perfect(TM) releases [partially helped by GARNOME and the BuildBrigade (-> Thursday 19th, 16:00)] ● Tasks: Release Engineering, Schedule Planning, Nagging, module inclusion final decisions, assigning Release Notes ● Resources: 8 persons, see http://live.gnome.org/ReleasePlanning/Membership ● Stats: no, what for? :-) ● Problems: response times, manpower/slacking. RT: Break request amounts and response times ● Rules: http://live.gnome.org/ReleasePlanning/ RequestingFreezeBreaks ● ABI/API UI String Hardcode 2.13 Amount 2 2 24 21 ø Resp.Time >1 >1, >3 18× >1, (in days) 3× >2 2.17 Amount 1 1 5 10 ø Resp.Time >1 >1 2× >1, 2× >2, >1 (in days) 1× >4, 1× >9 RT: Wrong/missing announcements of maintainers ● miss of informing any of release-team, desktop- devel-list, gnome-i18n and gnome-doc-list (http://live.gnome.org/MaintainersCorner) ● total correct missed... r-t i18n d-d-ldoc all 2005/09/05 – 2006/03/12 (2.13) 41 23 12 1 3 6 0 2006/03/13 – 2006/09/03 (2.15) 46 20 19 2 12 11 0 2006/09/04 – 2007/03/11 (2.17) 54 26 19 2 16 8 1 ● Some modules do not have strings, therefore no announcements were sent to gnome-i18n@. Table results by searching for „branch“ in subject of the four mail ing lists) Maintainers' quick guide de g sk r no t el g m op ea no e -d se m -d e -t e o ve e -i1 c-l l-l Code- am 8 is is Time @ n@ t@ t@ Action base http://live.gnome.org/Schedule Branching % always inform inform inform inform String Change trunk Announcement Period inform inform trunk Freeze period request (accidentially forgotten stable to mark) inform inform stable ...else: inform request UI Change trunk Announcement Period inform trunk Freeze period request request stable always request request Code Change trunk Hardcode Freeze request RT: Release notes ● Resources: pure luck? ● Problems: last cycles: always new writers; different lenghts & styles => no consistency ● 2.18 release notes written in the last days, no volunteers ● 2.20: volunteers available, better planning and embedded into the schedule (timeframe: august 6th – September 10th) => looks good ;-) User Documentation ● http://live.gnome.org/DocumentationProject ● Aim: Provide documentation for applications ● Tasks: Write and update documentation ● Resources, Stats: ? ● Problems: not always up-to-date (only a few weeks between freeze and release) => manpower (?); no way of tracking actuality Documentation: How up to date? ● Outdated if UI changes a lot => developers have to inform the doc writers ● Docs that haven't been updated for a long time: svn log - -limi t 1 h ttp:/ /akla pper@ svn.g nome. or g/ svn/ $modu le/tr unk/{d oc,he lp,do cs}/C /$mod ule.x ml (excluded build fixes, migration to g-d-u) ● => oldest docs: bug-buddy; gnome-keyring-manager (2005-03-25), fast-user-switch-applet, zenity, gnome- applets\battstat (2005-12-11) ● http://l10n.gnome.org/module/: add „last change“ info? I18N/GTP ● Aim: Translate UI and Documentation to other languages ● Resources: 2 GTP spokepersons (http://live.gnome.org/TranslationProject/SpokesPersons), >100 language teams (http://developer.gnome.org/projects/gtp/ teams.html), each one with one coordinator and an unknown number of translators I18N/GTP ● Stats: damned-lies at http://l10n.gnome.org per language, module, team, and release set. Powerful. ● Tasks of GTP: String Freeze Break Approvals, Language Team Maintainership changes, new languages approval: gnome-18n@ for entry at l10n.gnome.org, bugzilla team for creation of L10N component, and later on accounts-team for svn permissions (only 3 1/2 people on accounts team) I18N Teams: Maintenance ● http://l10n.gnome.org/languages/ lists 133 langs ● Strings: 2.14 2.18 UI ~35000 36916 Docs ~18100 24094 ● (note: partially buggy stats due to http://bugzilla.gnome.org/416009) I18N Teams: UI Translation changes within the last 12 months Positive (>10%) Negative (>-7%) Lang 2.14 2.18 diff Lang 2.14 2.18 diff Bengali Indian 9 80 71 Nepali 97 77 -20 Malayalam 19 76 57 Albanian 98 84 -14 Slovenian 40 89 49 Indonesian 96 84 -12 Oriya 18 61 43 Czech 98 87 -11 Marathi 9 52 43 Croatian 65 54 -11 Malagasy 8 50 42 Romanian 93 83 -10 Arabic 57 99 42 Norwegian Nynorsk 80 70 -10 Tamil 67 85 18 Serbian Latvian 98 89 -9 Serbian 98 89 -9 Welsh 97 88 -9 Canadian English 96 87 -9 Latvian 97 89 -8 GNOME 2.18: 63 langs Georgian 66 58 -8 with UI translation >50% Slovak 61 54 -7 Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC I18N Teams: Docu Translation changes within the last 12 months Docs with >10% status Comparison Lang 2.14 2.18 diff Lang UI Docs Spanish 71 88 17 Arabic 99 0 French 34 85 51 Dzongkha 99 0 Swedish 24 70 46 Macedonian 99 0 Italian 41 44 3 Hungarian 99 2 Russian 14 40 26 Catalan 99 2 British English 0 34 34 Finnish 99 2 Ukrainian 33 32 -1 Lithuanian 99 0 Brazilian Portugu2e5se 27 2 Danish 99 0 Punjabi 5 18 13 Japanese 99 1 Portuguese 0 16 16 Vietnamese 99 0 Korean 0 14 14 German 0 12 12 Bulgarian 11 12 1 63 langs with UI >50%, but 13 langs with Docs >10%? Source: Stats as per http://l10n.gnome.org/releases/gnome-2-1* on 2007-07-10, 02:00 UTC I18N Teams: Maintainer changes and other non-“freeze requests“ ● data taken from gnome-18n@ mailinglist Maintainership change or drop actively by Complaint about Granting for SVN maintainer himself, or unresponsiveness, Access; webpage or after a complaint with and NO response by email address change, positive reaction of the team leader => or wrong maintainer in maintainer to change maintainership change b.g.o 2.13 9 1 8 2.15 12 6 7 2.17 9 0 6 ● response times? I18N Teams: Maintainer changes etc: Response times of GTP ● Number of „Trackable“ Median [in Average Version requests requests days] [in days] 2.13 17 16 1 7,06 2.15 23 20 2,5 7,75 2.17 13 8 0 2,38 ● some requests perhaps answered off-list, or really unanswered... GTP: Announcement amounts ● 2.14 (6 weeks) 2.18 (8 weeks) Total 163 69 Top 3 evolution (58) gedit (8) gnome-utils (13) eog (6) gnome-panel (12) evolution (6) ● not significant Bugsquad ● Aim: Quality Assurance (http://live.gnome.org/Bugsquad) ● Tasks: Keeping track of bugs, make major bugs not go unnoticed, user help and feedback. ● Resources: Technical bugmasters and varying number of triagers (2-8 „hardcore“ triagers, plus maintainers and some „normal“ triagers) ● Existing stats: b.g.o/weekly-bug-summary.cgi, b.g.o/duplicates.cgi, Patches: b.g.o/reports/patch-diligence-report.cgi, b.g.o/reports/patch-report.cgi (general) Bugsquad: Workload 2005 2006 2007 Amount of Days: 365 365 193 Current open reports, excludes enhance. req.: 18888 26942 30691 Opened in that period: 37845 67543 64952 +78% +81% Closed: 34196 59006 62413 #Bugs closed by top All-time: bug closer 3211 8371 6788 15842 Sources: <http://developer.gnome.org/projects/bugsquad/statistics/2005/> and /2006/>, <http://bugzilla.gnome.org/reports/weekly-bug- summary.cgi?days=194&products=0>, <http://mail.gnome.org/archives/desktop- devel-list/2006-January/msg00004.html>, Olav Top 10 Bug closers in 2007 Bugs without a response Open bugs Bugs without Product (incl. enhanc.)response Evolution 4259 1045 24,54% nautilus 2057 665 32,33% gtk+ 2015 406 20,15% doxygen 650 344 52,92% rhythmbox 818 100 12,22% gnome-vfs 456 120 26,32% epiphany 612 23 3,76% GnuCash 514 38 7,39% gnome-panel 436 69 15,83% gnome-applets 377 124 32,89% gimmie 296 233 78,72% Gnumeric 486 55 11,32% Evolution Exchange 288 117 40,63% GtkHtml 319 65 20,38% Gstreamer 446 56 12,56% Source: top 15 modules of the last 365 days (http://bugzilla.gnome.org/reports/weekly-bug- summary.cgi?days=365&products=30) as per 2007/07/13 UTC 02:45 Graphical Reports? Source: http://bugzilla.ximian.com/reports.cgi?product=Evolution& output=show_chart&datasets=NEW %3A&banner=1&quip=0 Bugsquad ● Problems: Amount of untriaged bugs, response rate and time; understaffed products (bugzilla mail ignored by maintainers and only up to triagers), quality of stacktraces and installing debug packages, hacked up Auto-reject feature, manpower, Patch nagging (-> Patchsquad BoF on Saturday 21st, 10:30) ● 2.20: Breakpad Support (see Fernando Herrera's „Managing bugs and crashes in GNOME“, Sa 21st July, 14:00) ● => Major aim: reduce w orkloud Bugsquad: Flood me, baby! Bugsquad: Auto-reject ● http://live.gnome.org/Bugsquad/AutoReject ● Bugsquad: Auto-reject Date 2007/05/08 2007/06/10 2007/07/10 Nautilus 8925 9347 9603 Evolution 10604 11190 11524 TOTAL 26236 27571 28619 Dead modules in SVN ● Problems: Identify dead modules (bit rott) => translators waste time; or new maintainership needed? ● "svn log --limit 5" to find modules without