| Time | Nick | Message |
|---|---|---|
| 12:06 | erikos | uff :) |
| 12:06 | mchua | cheers for erikos and meetbot |
| 12:07 | erikos | #TOPIC SugarLabs QA |
| 12:07 | mchua | Is http://sugarlabs.org/go/Testin[…]gendas/2008-12-10 still our agenda, erikos? That's what I'm working from |
| 12:07 | (for the purpose of meeting notes, here) | |
| 12:07 | erikos | mchua: yes |
| 12:08 | mchua | sweet, thanks! |
| 12:08 | erikos | so the first point is to define the bugsquad and it's purpose |
| 12:08 | because for me the QA as a whole has two groups: | |
| 12:08 | a bugsquad (mainly triaging) and a testing group | |
| 12:08 | do the others agree with that? | |
| 12:09 | unmadindu_ | agrees |
| 12:09 | mchua | agrees |
| 12:09 | tomeu1 | yup |
| 12:09 | marcopg | erikos: with QA you mean the team? |
| 12:10 | erikos | marcopg: yeah - because if we name it testing - this would be misleading for the bugsquad |
| 12:10 | marcopg | I guess I'm still not fully convinced about the name |
| 12:10 | but probably just because of the meaning I'm used to give to QA | |
| 12:11 | erikos | ok i am sure everyone has slightly other meanings here |
| 12:11 | cjl | erikos: If you write it up on the wiki, that makes it true. |
| 12:11 | erikos | do people agree with the definition of the bugsquad we have on the wiki (agenda) ? |
| 12:12 | mchua | copy-pastes |
| 12:12 | The BugSquad keeps track of current bugs in the sugar software and try to make sure that bugs are triaged correctly. You do not need any programming knowledge to be in the Bugsquad; in fact it is a great way to return something to the Sugar community if you cannot program. The squad is modeled on the gnome bugsquad: http://developer.gnome.org/projects/bugsquad/ | |
| 12:12 | I agree with this. | |
| 12:13 | marcopg | yup, it sounds good to me too |
| 12:13 | cjl | erikos: It does not capture the distinction you make a bout there also being a testing group under the QA umbrella |
| 12:13 | mchua | I think that at the very least, it is an excellent concrete beginning. I would like to set a date in the future (a month or two out, or maybe after the next release or interim release) to reevaluate whether this is what we need. |
| 12:13 | notes that she sound really formal today... | |
| 12:13 | tomeu1 | that's contagious :p |
| 12:14 | erikos | cjb: ok - the testing group - is a bit more vague at the moment - that one we need to agree here |
| 12:14 | damn cjl ^^^ | |
| 12:14 | marcopg | you sound like m_stone today ;) |
| 12:14 | mchua | yay! that's (usually, I think) a good thing. :) |
| 12:14 | erikos | maybe we don't need that QA umbrella |
| 12:15 | and just make a bugsquad and a testing team? | |
| 12:15 | cjl | erikos: not until it starts raining bugs :-) |
| 12:15 | marcopg | erikos: to me the testing is still very undefined |
| 12:15 | mchua | well, my thought is that the TestTeam should have two parts to it, right now: one is the bugsquad, the other is general enabling of non-SL testing of Sugar. |
| 12:15 | marcopg | erikos: so perhaps it makes sense to try to define it better, before deciding on a structure |
| 12:16 | mchua | Bug Squad has been defined pretty well by erikos on the wiki, I think. General enabling of non-SL testing of Sugar is what I see the rest of the agenda at http://sugarlabs.org/go/Testin[…]gendas/2008-12-10 being about figuring out. |
| 12:16 | marcopg | mchua: i.e. you don't think SL should be doing any testing directly? |
| 12:16 | mchua | marcopg: that's correct. |
| 12:16 | just as SL is not doing any deployments directly, or development directly, as I understand it. | |
| 12:16 | erikos | ok as we seem to agree on the bugsquad definition - lets decide on what the testing group is about |
| 12:16 | tomeu1 | right |
| 12:16 | mchua | SL enables the SL community to do that, is my understanding. |
| 12:17 | erikos | which - mel is right are the next items in the agenda |
| 12:17 | mchua | (Please correct me if I'm wrong.) |
| 12:17 | tomeu1 | marcopg: do you consider soas a SL project? |
| 12:17 | because that's a good chunk of integration work that is normally done by distros | |
| 12:17 | marcopg | mchua: I'm thinking of SL as *the* community lately. David posted about it too |
| 12:17 | cjl | mchua: slightly dissenting opinion expressed here. Maybe fuzzy doue to SL/SL community differentiation mchua is making. |
| 12:18 | http://sugarlabs.org/go/Talk:T[…]gendas/2008-12-10 | |
| 12:18 | marcopg | tomeu1: no, I consider SoaS an independent project, enabled by SL |
| 12:18 | tomeu1 | marcopg: fine |
| 12:18 | then I see testing of distros as out from SL | |
| 12:18 | because they can do it better | |
| 12:18 | marcopg | yeah, but if SL doesn't test distros, what does it test? |
| 12:19 | tomeu1 | nothing ;) |
| 12:19 | marcopg | I guess I'm thinking to mchua "SL does not do any testing" in a different way |
| 12:19 | tomeu1 | testing is done on finished products, like a distro or soas are |
| 12:20 | marcopg | i.e. SL enables distros testing |
| 12:20 | tomeu1 | and is better done by those closer to those products |
| 12:20 | mchua | erikos: maybe a way to do this is to bring up some cases of testing that might be done that we know about or think will come up soon, and go through the list and say "that's SL TestTeam," or "That's not SL TestTeam." |
| 12:20 | tomeu1 | marcopg: how enables that other than providing the code to test? |
| 12:20 | erikos | mchua: sounds good |
| 12:21 | marcopg | tomeu1: it provides an upstream tracking system, for example |
| 12:21 | tomeu1: and defines the policies to use it | |
| 12:21 | mchua | I'll throw out the one we're talking about now, then: Sugar-on-<distro> testing. SL TestTeam or not? |
| 12:21 | (and if not, what does SL TestTeam do to help it, if anything?) | |
| 12:21 | marcopg | mchua: Sugar core or Sugar with activities? |
| 12:22 | mchua | marcopg: Ah, right. Thanks. Sugar-core-on-<distro> testing, no activities. |
| 12:22 | cjb | oh, hey, it's an IRC conversation about what I was thinking about before I opened the IRC window :) |
| 12:22 | marcopg | hey cjb |
| 12:23 | my current opinion about that, is that exclusively distro should take care of that | |
| 12:23 | (mostly because of the points tomeu made above) | |
| 12:23 | mchua | I vote not SL TestTeam, with marcopg's suggestion that distros should do it and SL TestTeam should provide them with an upstream tracking system to those distros and define the policies to use it. |
| 12:23 | marcopg | SL provides a bug tracker, policies and a triaging team which mediates distro testers <-> developers |
| 12:23 | mchua | marcopg + 1 |
| 12:24 | erikos | well the policies and the tracker are handled by the bugsquad |
| 12:25 | tomeu1 | sounds good |
| 12:25 | erikos | my scenario about testing was: |
| 12:25 | that last time, the testing happened late in the cycle which had direct impact on our roadmap | |
| 12:26 | we need some way of doing testing during the release cycle as well | |
| 12:26 | cjl | any role for SL in helping distros find testers (community building, lists, etc.)? |
| 12:26 | erikos | not testing of distros - but just the sugar core |
| 12:27 | maybe with a reference distribution | |
| 12:27 | soas or fedora or whatever | |
| 12:27 | marcopg | ok |
| 12:27 | that's another scenario, I think | |
| 12:27 | erikos | i am still not sure how we can handle that case best |
| 12:27 | mchua | Yeah, I guess one of my questions is "what is 'canonical sugar'? What do we use in order to say "Sugar Works," independent of how it runs on any distro?" |
| 12:28 | (i.e. if Sugar-on-Fedora works but Sugar-on-Debian has a lot of bugs, does Sugar-core work?) | |
| 12:28 | (but that can wait until another time, since I know it wasn't in the original scope of this meeting's agenda.) | |
| 12:28 | erikos | mchua: does depend on the bugs i would say ;p |
| 12:28 | marcopg | I think it's almost impossible to separate the Sugar core from the distro it runs on |
| 12:29 | that's the point I try to make about Activities on the agenda | |
| 12:29 | mchua | so we can't say "Sugar works," but we can only say "Sugar-on-<distro>" works? |
| 12:29 | erikos: btw, what time is this meeting supposed to end? I originally blocked 1630-1730 UTC for it, and have to run in ~10min, alas | |
| 12:30 | marcopg | if we can ensure a consistent Sugar core on each distro, then we can say if an Activity works or not, independently from the distro it runs on |
| 12:30 | erikos | mchua: we waited for you :) |
| 12:30 | mchua: since others were late as well | |
| 12:31 | marcopg: what would be your solution - for the activities case? | |
| 12:31 | marcopg | mchua: I don't think we can say for sure. But we can probably provide guidelines to guess if the problem is in the distro or in sugar |
| 12:31 | mchua: stupid example. If a bug is present in only one distro, then Sugar works and that specific distro is broken | |
| 12:32 | mchua: not absolute criteria though, just guidelines. | |
| 12:33 | erikos: ensure that activities, or at least a set of them, can run unmodified on any distro | |
| 12:33 | erikos: that requires a well defined platform, and automated testing to verify platform consistency | |
| 12:33 | mchua | ok, so sugar-core and the distro can't be tested independently, but we *do* have the concept of something called "sugar-core" that we are focused on the quality of. |
| 12:34 | marcopg | mchua: yeah, that's correct |
| 12:34 | mchua | since we can't test sugar-core independently of sugar-on-<distro>, the only gauge we have for the "health" of sugar-core is the health of the various sugar-on-<distro>s. |
| 12:34 | erikos | hmmm ok |
| 12:34 | marcopg | yeah |
| 12:35 | one of these "distro" being compiled code on developers laptops | |
| 12:36 | erikos was posing the problem of how we do testing of the core-sugar during unstable development | |
| 12:36 | erikos | i just thought about it a bit |
| 12:36 | marcopg | I think we can use some of the distros to provide this testing |
| 12:36 | erikos | yup |
| 12:36 | marcopg | SoaS is shipping 0.83 already for example |
| 12:37 | if there are people in the community that would like to test unstable sugar, they have a way to | |
| 12:37 | tomeu1 | yeah, we shouldn't wait to have stable releases to do packaging |
| 12:37 | erikos | and if we make sure that we have one easy to use way to test sugar (soas) we cna announce it when releasing and ask for testing |
| 12:37 | we don't need an organized team for that | |
| 12:37 | marcopg | erikos: yup, we can announce all the distros which are/will be shipping the release |
| 12:38 | erikos | yes - the important part is that we work with the feedback |
| 12:38 | -> bugsquad | |
| 12:38 | marcopg | right! |
| 12:38 | sugar-core testing sounds like a solved problem to me ;) | |
| 12:38 | anyone sees other issues? | |
| 12:38 | erikos | marcopg: you brought up the point of automated testing |
| 12:39 | no sounds good to me | |
| 12:39 | mchua | I've got to run, do you guys need anything else from me before I go? |
| 12:39 | I'll read the logs after. | |
| 12:39 | erikos | mchua: no - i think we made good progress |
| 12:40 | marcopg | mchua: would like to discuss a couple of points on the agenda with you... |
| 12:40 | mchua: but we can do by mail or in another meeting | |
| 12:40 | mchua: thanks for coming! | |
| 12:40 | erikos | marcopg: points you would like to see already discussed now? |
| 12:41 | marcopg | well the points are 1 activities testing 2 OLPC |
| 12:41 | mchua | marcopg: likewise - erikos, if you're going to email minutes/logs to the iaep list, I'll comment on those for my reply, how does that sound? |
| 12:41 | marcopg | we can discuss 1 now... |
| 12:41 | 2 we need mchua ;) | |
| 12:41 | erikos | mchua: sure |
| 12:42 | marcopg: the floor for point 1 is yours | |
| 12:43 | marcopg | mchua: sure! |
| 12:43 | erikos: so the question is on the agenda | |
| 12:43 | who tests activities? | |
| 12:43 | mchua gave an answer there | |
| 12:43 | what do the others think? | |
| 12:44 | also cjl made the point about fructose, which I tend to agree with | |
| 12:44 | cjl | One thought is that SL has decreasing responsibility to test Activities from Glucose > Fructose > Honey. Must test Glucose activities, should cooperate closely with OLPC (and activity upstreams, e.g. AbiWord, evince, xulrunner) on testing Fructose activities, should encourage and empower and participate in community testing of Honey activities. |
| 12:44 | marcopg | for Honey I think mchua approach is fine |
| 12:44 | erikos | honey? |
| 12:45 | activities not in Frcutose? | |
| 12:45 | marcopg | erikos: extra activities, which are not in fructose |
| 12:45 | erikos | marcopg: the question is a bit as well how we package activities |
| 12:45 | cjl | ertamtam playgo, measure, memorizer, moon, etc. https://dev.laptop.org/translate/af/honey/ |
| 12:46 | marcopg | erikos: right, that's what I'm talking about in the agenda... |
| 12:46 | I'm more and more convinced we need to support .xo at some level | |
| 12:46 | erikos | hmmm |
| 12:46 | mchua | I really have to run - can you guys ping me on email if you need input from me on something for #2 in particular? I'll read the logs too, when I get back this evening |
| 12:47 | marcopg | i.e. packages which works independently from the distribution |
| 12:47 | mchua | thanks guys! |
| 12:47 | out | |
| 12:48 | erikos | what kind of support is that? |
| 12:49 | marcopg | erikos: what do you mean with "support"? |
| 12:49 | erikos | marcopg: you said that ;p |
| 12:49 | <marcopg> I'm more and more convinced we need to support .xo at some level | |
| 12:49 | marcopg | oh |
| 12:49 | erikos | what exactly do you mean here |
| 12:49 | cjl | CRC check fail. |
| 12:49 | erikos | encourage authors to use it |
| 12:49 | marcopg | I mean you should be able to install a .xo into any of the Sugar distributions |
| 12:49 | and have it working | |
| 12:50 | erikos | and that we don't currently? |
| 12:50 | marcopg | I don't want Sugar to dictate a policy between self-contained-no-dependencies-bundles and standard packages |
| 12:51 | cjb | erikos: not to the extent that they contain CPU-compiled code, at least |
| 12:51 | marcopg | distributors and users will use what better fit their needs |
| 12:51 | erikos | cjb: oh yeah i see |
| 12:51 | marcopg | erikos: in theory, there are a bunch of problems with it though |
| 12:51 | cjb | (and to the extent that they rely on system dependencies) |
| 12:51 | marcopg | compiled code |
| 12:51 | but also we dont' have a consistent platform | |
| 12:51 | so yeah dependencies | |
| 12:52 | erikos | well we need either an interface for install/update (that handles deps) or make them self contained |
| 12:53 | i guess | |
| 12:53 | cjl | you can't really make it so it know to yum here and apt-get there |
| 12:53 | marcopg | consistent platform is a solvable problem, compiled code much less, but we might just decide to punt on that, for example |
| 12:54 | erikos: it's not just about installing, it's also about distributing | |
| 12:54 | erikos: being able to just create a .xo and give it to everyone | |
| 12:54 | erikos: is different than having to get your application in a distribution | |
| 12:54 | erikos: same for modifications | |
| 12:54 | this is getting way out of topic now :) | |
| 12:54 | erikos | marcopg: oh and we want to solve all that :) |
| 12:55 | yeah we can organize a conference for that subject | |
| 12:56 | marcopg | we are somewhat out of time |
| 12:56 | erikos | it comes to action items now |
| 12:56 | marcopg | should we come with some action and go through the other problems in the next meeting? |
| 12:56 | erikos | yup |
| 12:57 | so i would argue that we call the testing team - bugsquad | |
| 12:57 | from now on - to get rid of all the irritations | |
| 12:57 | and clean up the wiki | |
| 12:58 | marcopg | I think it make sense |
| 12:58 | if we find that we need to actually do some testing | |
| 12:58 | erikos | we can add another team |
| 12:58 | marcopg | we can always create another team or something |
| 12:58 | I guess I'm wondering which team deals with trac and policies | |
| 12:58 | perhaps dev | |
| 12:59 | or maybe triage | |
| 12:59 | erikos | triage i would say |
| 12:59 | so bugsquad | |
| 12:59 | maybe in coordination with the dev team | |
| 12:59 | marcopg | yeah |
| 13:00 | Bugsquad is sort of a odd case about team naming | |
| 13:00 | but I guess it's fine | |
| 13:00 | BugTeam would seriously suck ;) | |
| 13:00 | erikos | #ACTION make BugSquad the team - clean up wiki (erikos) |
| 13:01 | marcopg: you need at least one odity | |
| 13:01 | marcopg | hehe |
| 13:01 | cjl | re: automated testing check out the links off here, hmmmm http://www.testingfaqs.org/ |
| 13:01 | marcopg | I guess that we try to reschedule the two points above for discussion next week |
| 13:02 | and we try to get mchua to participate | |
| 13:02 | erikos | which is activity testing, olpc, automated testing |
| 13:02 | marcopg | should be an action? |
| 13:02 | oh automated testing | |
| 13:02 | yeah | |
| 13:02 | that one too | |
| 13:02 | erikos | #ACTION items for next week meeting: activity testing, olpc, automated testing |
| 13:03 | how do we call next week meeting then? | |
| 13:03 | it is not a bugsquad meeting | |
| 13:03 | lets use testing i guess | |
| 13:03 | marcopg | well |
| 13:03 | bugsquad deals with policies | |
| 13:04 | erikos | oh yeah the olpc case |
| 13:04 | marcopg | and activities testing might fit in them, depending on the outcome of the discussion :) |
| 13:04 | automated testing is between dev and bugsquad I think | |
| 13:04 | erikos | eieiei |
| 13:04 | ok - lets call it a bugsquad meeting | |
| 13:04 | marcopg | it sort of fits in bugsquad I think |
| 13:05 | erikos | even if we find out that we reject the items |
| 13:05 | marcopg | other actions? |
| 13:05 | erikos | not that i am aware of |
| 13:05 | marcopg | we should try to get people interested in doing bugsquad work in next meeting |
| 13:06 | so that we can come up with a TODO there | |
| 13:06 | cjl | see also http://www.aptest.com/resources.html for pleasure reading about testing |
| 13:07 | erikos | ok |
| 13:07 | so lets end the meeting here | |
| 13:07 | 5,4 | |
| 13:07 | marcopg | I'm sort of worried that if we do activities, olpc and automated we will run out of time, though |
| 13:07 | erikos | 3 |
| 13:07 | 2 | |
| 13:07 | marcopg | and do no real bugsquad |
| 13:07 | erikos | 2 |
| 13:07 | 2 | |
| 13:07 | marcopg | hehe |
| 13:07 | feel free to close | |
| 13:07 | this is next meeting org ;) | |
| 13:07 | erikos | 1.5 |
| 13:07 | 1 | |
| 13:07 | 0 | |
| 13:08 | #ending | |
| 13:08 | #close | |
| 13:08 | #endmeeting |